Introduction

Benford's Law[benford1938] states that in many naturally occurring collections of numbers, the leading significant digit $d$ ($1 \leq d \leq 9$) appears with probability $$P(d) = \log_{10}\left(1 + \frac{1}{d}\right),$$ yielding approximately 30.1% for digit 1, decreasing to 4.6% for digit 9. This law applies to diverse datasets including population figures, physical constants, and financial data.

Recent work has established a connection between Benford's Law and neural network weights. Sahu[sahu2021] introduced Model Enthalpy (MLH), measuring the closeness of weight distributions to Benford's Law, and demonstrated a strong correlation with generalization across architectures from AlexNet to Transformers. Toosi[toosi2025] confirmed these findings in RNNs and LSTMs, showing that higher-performing models exhibit stronger Benford conformity.

However, prior studies focused on large models requiring GPUs, limiting reproducibility. We contribute an agent-executable study using tiny MLPs trainable on CPU in about two minutes on our machine, with rigorous statistical testing (chi-squared, MAD with Nigrini's thresholds[nigrini2012], and bootstrap uncertainty bands).

Methodology

Tasks and Models

We train two-hidden-layer MLPs (ReLU activations) on two tasks:

- **Modular arithmetic:** Predict <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mi>a</mi><mo>+</mo><mi>b</mi><mo stretchy="false">)</mo><mtext> </mtext><mo lspace="0.22em" rspace="0.22em"><mrow><mi mathvariant="normal">m</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">d</mi></mrow></mo><mtext> </mtext><mn>97</mn></mrow><annotation encoding="application/x-tex">(a + b) \bmod 97</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal">a</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">b</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.0556em;"></span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin"><span class="mord"><span class="mord mathrm">mod</span></span></span><span class="mspace" style="margin-right:0.0556em;"></span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">97</span></span></span></span> from normalized inputs <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mi>a</mi><mi mathvariant="normal">/</mi><mo stretchy="false">(</mo><mi>p</mi><mo lspace="0em" rspace="0em">−</mo><mn>1</mn><mo stretchy="false">)</mo><mo separator="true">,</mo><mi>b</mi><mi mathvariant="normal">/</mi><mo stretchy="false">(</mo><mi>p</mi><mo lspace="0em" rspace="0em">−</mo><mn>1</mn><mo stretchy="false">)</mo><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">(a/(p{-}1),  b/(p{-}1))</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal">a</span><span class="mord">/</span><span class="mopen">(</span><span class="mord mathnormal">p</span><span class="mord"><span class="mord">−</span></span><span class="mord">1</span><span class="mclose">)</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">b</span><span class="mord">/</span><span class="mopen">(</span><span class="mord mathnormal">p</span><span class="mord"><span class="mord">−</span></span><span class="mord">1</span><span class="mclose">))</span></span></span></span>. Classification with 97 output classes. Known to exhibit "grokking"[power2022].
- **Sine regression:** Predict <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>sin</mi><mo>⁡</mo><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\sin(x)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mop">sin</span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span></span></span></span> for <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi><mtext> </mtext><mi>U</mi><mo stretchy="false">(</mo><mn>0</mn><mo separator="true">,</mo><mn>2</mn><mi>π</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">x ~ U(0, 2\pi)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">x</span><span class="mspace nobreak"> </span><span class="mord mathnormal" style="margin-right:0.109em;">U</span><span class="mopen">(</span><span class="mord">0</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">2</span><span class="mord mathnormal" style="margin-right:0.0359em;">π</span><span class="mclose">)</span></span></span></span>. A smooth function approximation task.

For each task, we train models with hidden dimensions $h \in {64, 128}$, yielding four configurations. All models use Adam (lr=$10^{-3}$) for 5,000 epochs with seed 42.

Benford Analysis

At each snapshot epoch ${0, 100, 500, 1000, 2000, 5000}$, we:

- Extract all weight values (excluding biases), take absolute values, discard values <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>&lt;</mo><msup><mn>10</mn><mrow><mo>−</mo><mn>10</mn></mrow></msup></mrow><annotation encoding="application/x-tex">&lt; 10^{-10}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5782em;vertical-align:-0.0391em;"></span><span class="mrel">&lt;</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.8141em;"></span><span class="mord">1</span><span class="mord"><span class="mord">0</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">10</span></span></span></span></span></span></span></span></span></span></span></span>.
- Compute leading significant digit via <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>d</mi><mo>=</mo><mo stretchy="false">⌊</mo><msup><mn>10</mn><mrow><msub><mrow><mi>log</mi><mo>⁡</mo></mrow><mn>10</mn></msub><mi mathvariant="normal">∣</mi><mi>w</mi><mi mathvariant="normal">∣</mi><mo>−</mo><mo stretchy="false">⌊</mo><msub><mrow><mi>log</mi><mo>⁡</mo></mrow><mn>10</mn></msub><mi mathvariant="normal">∣</mi><mi>w</mi><mi mathvariant="normal">∣</mi><mo stretchy="false">⌋</mo></mrow></msup><mo stretchy="false">⌋</mo></mrow><annotation encoding="application/x-tex">d = \lfloor 10^{\log_{10}|w| - \lfloor\log_{10}|w|\rfloor}\rfloor</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal">d</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.138em;vertical-align:-0.25em;"></span><span class="mopen">⌊</span><span class="mord">1</span><span class="mord"><span class="mord">0</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.888em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mop mtight"><span class="mop mtight"><span class="mtight">l</span><span class="mtight">o</span><span class="mtight" style="margin-right:0.0139em;">g</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1944em;"><span style="top:-2.2341em;margin-right:0.0714em;"><span class="pstrut" style="height:2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">10</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2659em;"><span></span></span></span></span></span></span><span class="mspace mtight" style="margin-right:0.1952em;"></span><span class="mord mtight">∣</span><span class="mord mathnormal mtight" style="margin-right:0.0269em;">w</span><span class="mord mtight">∣</span><span class="mbin mtight">−</span><span class="mopen mtight">⌊</span><span class="mop mtight"><span class="mop mtight"><span class="mtight">l</span><span class="mtight">o</span><span class="mtight" style="margin-right:0.0139em;">g</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1944em;"><span style="top:-2.2341em;margin-right:0.0714em;"><span class="pstrut" style="height:2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">10</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2659em;"><span></span></span></span></span></span></span><span class="mspace mtight" style="margin-right:0.1952em;"></span><span class="mord mtight">∣</span><span class="mord mathnormal mtight" style="margin-right:0.0269em;">w</span><span class="mord mtight">∣</span><span class="mclose mtight">⌋</span></span></span></span></span></span></span></span></span><span class="mclose">⌋</span></span></span></span>.
- Compare observed digit distribution to Benford's expected distribution.

Statistical Tests

Chi-squared test: $\chi^2 = \sum_{d=1}^{9} \frac{(O_d - E_d)^2}{E_d}$ with 8 degrees of freedom, where $O_d = n \cdot f_d^{\text{obs}}$ and $E_d = n \cdot P(d)$.

Mean Absolute Deviation (MAD): $\text{MAD} = \frac{1}{9}\sum_{d=1}^{9} |f_d^{\text{obs}} - P(d)|$, classified per Nigrini[nigrini2012]: $< 0.006$ = close conformity, $0.006$--$0.012$ = acceptable, $0.012$--$0.015$ = marginal, $> 0.015$ = nonconformity.

Uncertainty quantification: For each row in the aggregate/per-layer/control tables, we estimate a 95% confidence interval for MAD by multinomial bootstrap (1,000 resamples) using the observed digit frequencies and sample size.

Controls

We generate 10,000 values from three distributions: Uniform $U(-1,1)$, Normal $N(0, 0.01)$, and Kaiming Uniform (simulating PyTorch default initialization). None are expected to conform to Benford's Law.

Results

Training Dynamics

Table shows that MAD falls substantially from initialization across all configurations, with the largest gains appearing early in training and small later fluctuations, indicating that gradient-based optimization drives weight distributions toward Benford conformity overall.

MAD from Benford's Law over training epochs. All models start in nonconformity and finish below their initialization MAD. Values are from the deterministic seed-42 run reproduced by run.py; repeated reruns with the same seed in our environment produced identical MAD values.

The mod97_h64 model shows the strongest trajectory, with MAD dropping by $~$60% from initialization to its best value by epoch 500 and remaining near the "marginal conformity" threshold through epoch 5,000. The smaller model's stronger conformity suggests that the ratio of training signal to parameter count influences how structured weight distributions become.

Per-Layer Analysis

Table shows per-layer MAD at epoch 5,000 for the mod97_h64 model, revealing that output-adjacent layers tend to conform more closely than input-adjacent layers.

*Per-layer MAD at epoch 5,*000 (mod97_h64). Output layer shows best conformity.}

The input layer's poor conformity is partly explained by its small size (128 weights), but the monotonic improvement from input to output is consistent across models. Note that layer size varies considerably, and the chi-squared test is sensitive to sample size[nigrini2012]; MAD provides a more robust comparison across layers.

Controls

Control distributions: all show nonconformity, as expected.

All control distributions show nonconformity (Table). Notably, the Normal distribution has lower MAD than Uniform or Kaiming, consistent with log-normal-like distributions partially approximating Benford's Law.

Discussion

Training drives Benford conformity. Across all four model configurations, MAD from Benford's Law is lower at epoch 5,000 than at initialization, with reductions of 18--60%. The largest gains occur early in training for the modular task, followed by modest later fluctuations. This corroborates Sahu[sahu2021]'s findings on large models, now demonstrated in a minimal, CPU-reproducible setting.

Task and size effects. The modular arithmetic task with $h=64$ achieves the strongest conformity, approaching the marginal/acceptable boundary. Larger models ($h=128$) show higher MAD throughout, possibly because they have more parameters relative to the training signal, leading to less structured weight distributions. The sine regression task shows weaker conformity overall, suggesting that task complexity influences how strongly Benford's Law emerges.

Layer depth matters. Output-adjacent layers consistently show better Benford conformity than input-adjacent layers. This may reflect the gradient structure: output layers receive more direct error signal, potentially imposing more structure on their weight distributions.

Determinism and scope. Repeated reruns with the fixed seed produced identical MAD trajectories and control statistics in our environment; only wall-clock runtime varied. The generated results.json records software versions (Python, PyTorch, NumPy, SciPy, Matplotlib) to support environment-level reproducibility audits. We did not run a multi-seed sweep in this submission, so our claims are limited to the seed-42 configuration documented in run.py.

Limitations. Our models are deliberately tiny (4K--29K parameters) for reproducibility. Larger models may show stronger effects. We analyze only weight matrices, not biases. The MAD thresholds (Nigrini) were developed for financial forensics and may not directly apply to neural network weights. The chi-squared test is known to be overly sensitive for large sample sizes[nigrini2012], which is why we emphasize MAD.

The skill as contribution. This analysis runs entirely on CPU with no internet access, completing in roughly two minutes for the full profile and a few seconds for an optional quick profile. The SKILL.md enables any AI agent to reproduce all results, demonstrating that meaningful scientific analysis of neural network properties can be conducted in minimal, fully reproducible settings.

Conclusion

We presented an agent-executable study showing that training moves neural network weight distributions toward Benford's Law conformity overall. The mod-97 model with $h=64$ reduces MAD by ${~}60$ over 5,000 epochs, approaching marginal conformity. Output-adjacent layers conform more strongly than input-adjacent layers, and smaller models show better conformity than larger ones. These findings extend prior work on Benford's Law in neural networks to a minimal, CPU-reproducible setting, with all results reproducible via a single SKILL.md file.

References

• [benford1938] F. Benford, "The law of anomalous numbers," Proceedings of the American Philosophical Society, vol. 78, no. 4, pp. 551--572, 1938.

• [sahu2021] S. K. Sahu, "Rethinking Neural Networks with Benford's Law," in NeurIPS Workshop on Machine Learning and the Physical Sciences, 2021.

• [toosi2025] R. Toosi et al., "Benford's Law in Basic RNN and Long Short-Term Memory and Their Associations," Applied AI Letters, 2025.

• [nigrini2012] M. J. Nigrini, Benford's Law: Applications for Forensic Accounting, Auditing, and Fraud Detection. Wiley, 2012.

• [power2022] A. Power et al., "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets," in ICLR Workshop on Mathematics of Deep Learning, 2022.