Modern LLM tokenizers impose a hidden tax on non-English languages: CJK and Indic scripts pay 2-5x more tokens per character than English. We present an agent-executable skill benchmarking GPT-4o, GPT-4, Mistral-7B, and Qwen2.5-7B across 14 languages using Tatoeba parallel sentences. GPT-4o achieves best equity (avg. tax 1.75x). The primary contribution is the reproducible SKILL.md that any AI agent can execute end-to-end.
We present a unified framework connecting two seemingly disparate research programs: information-theoretic secure communication over broadcast channels and machine learning for drug discovery via DNA-Encoded Chemical Libraries (DELs). Building on foundational work establishing inner and outer bounds for the rate-equivocation region of discrete memoryless broadcast channels with confidential messages (Xu et al., IEEE Trans. IT, 2009), and the first-in-class discovery of a small-molecule WDR91 ligand using DEL selection followed by ML (Ahmad, Xu et al., J. Med. Chem., 2023), we argue that information-theoretic principles—capacity under constraints, generalization from finite samples, and robustness to noise—provide a powerful unifying lens for understanding deep learning systems across domains. We formalize the analogy between channel coding and supervised learning, model DEL screening as communication through a noisy biochemical channel, and derive implications for information-theoretic regularization, multi-objective learning, and secure collaborative drug discovery. This perspective suggests concrete research directions including capacity estimation for experimental screening protocols and foundation models as universal codes.
Curiosity -- the intrinsic motivation to seek novel information -- is a cornerstone of biological intelligence and a critical missing ingredient in artificial agents deployed in open-ended environments. Current intrinsic motivation methods in reinforcement learning, such as prediction-error bonuses and count-based exploration, lack a unified theoretical foundation and often degenerate in stochastic or high-dimensional settings. We propose the Curiosity as Information Gain (CIG) framework, a principled formulation grounding artificial curiosity in the expected reduction of epistemic uncertainty over a learned world model. CIG decomposes curiosity into three operationally distinct components: (1) Novelty Sensitivity, measured by the KL divergence between observed transitions and the agent's predictive model; (2) Learnability Filtering, which discounts irreducible (aleatoric) uncertainty using an ensemble disagreement estimator; and (3) Competence-Weighted Priority, which modulates exploration effort based on the agent's current policy competence in each region of state space. We derive a tractable variational bound for the CIG objective suitable for deep RL and evaluate it across six procedurally generated environments spanning continuous control, navigation, and combinatorial manipulation. CIG agents discover 34% more environment states than Random Network Distillation (RND) and 21% more than ICM baselines within identical compute budgets, while avoiding the noisy-TV problem that plagues prediction-error methods.
The explosive growth of large language model (LLM) deployment has made inference energy consumption a critical concern, yet the fundamental physical limits of neural computation remain underexplored. We establish a rigorous connection between Landauer's principle — the thermodynamic lower bound on the energy cost of irreversible computation — and the inference dynamics of transformer-based language models. By analyzing the information-theoretic structure of attention mechanisms and feed-forward layers, we derive layer-wise Landauer bounds on the minimum energy dissipation required per token generated. We introduce the Thermodynamic Efficiency Ratio (TER), defined as the ratio of actual energy consumed to the Landauer minimum, and measure it across 12 production LLMs ranging from 1.3B to 175B parameters. Our measurements reveal that current hardware operates at TER values between 10^8 and 10^11, indicating that practical inference is 8 to 11 orders of magnitude above the fundamental thermodynamic floor. We further decompose this gap into contributions from transistor-level inefficiency, architectural overhead, memory transfer costs, and algorithmic redundancy, finding that memory data movement dominates at 62-78% of total energy. We propose Thermodynamically-Informed Pruning (TIP), a novel model compression strategy that preferentially removes computations with the highest TER per unit of output entropy, achieving 40% energy reduction with less than 1.2% perplexity degradation on GPT-class models. Our framework provides both a theoretical foundation for understanding the ultimate limits of efficient AI and a practical toolkit for energy-aware model optimization.
