============================================================================== WHEN THE FIRST DIGIT TEST WORKS AND WHEN IT ACCUSES THE INNOCENT Science Journaling Club, Volume 2 Issue 4, Summer 2026 ============================================================================== python : 3.12.3 numpy : 2.4.2 scipy : 1.13.1 seed : 20260914 (numpy.random.default_rng) retrieved : 2026-09-14 (all cached datasets) cache dir : analysis/data/ ============================================================================== VALIDATION V1. Expected frequencies, club code beside the closed form ============================================================================== The club's table is computed from the mantissa definition by midpoint quadrature on 4,000,000 points: P(d) = measure{ u in [0,1) : floor(10^u) = d }. The accepted value is the closed form log10(1 + 1/d). These are independent routes to the same number. d club quadrature log10(1 + 1/d) difference ------------------------------------------------------------------------------ 1 0.301030000000 0.301029995664 +4.336e-09 2 0.176091250000 0.176091259056 -9.056e-09 3 0.124938750000 0.124938736608 +1.339e-08 4 0.096910000000 0.096910013008 -1.301e-08 5 0.079181250000 0.079181246048 +3.952e-09 6 0.066946750000 0.066946789631 -3.963e-08 7 0.057992000000 0.057991946978 +5.302e-08 8 0.051152500000 0.051152522447 -2.245e-08 9 0.045757500000 0.045757490561 +9.439e-09 ------------------------------------------------------------------------------ sum 1.000000000000 1.000000000000 +1.110e-16 largest absolute difference : 5.302e-08 quadrature cell width : 2.500e-07 VERDICT: PASS, agreement to within one quadrature cell. From here on the closed form is used, because it is exact. ============================================================================== VALIDATION V2. The digit extractor against a decimal-string reference ============================================================================== The fast extractor divides by 10^floor(log10 x). The reference reads the first digit out of Python's exact decimal representation. Disagreement means floating point has bitten. Tested on 1,000,000 log-uniform values spanning 24 orders of magnitude, plus every exact power of ten from 1e-20 to 1e20. values tested : 1000043 mismatches : 0 VERDICT: PASS, the two extractors agree everywhere ============================================================================== VALIDATION V3. The chi-squared routine against SciPy, on a real table ============================================================================== A standard library routine is given the same observed and expected counts. The table used is D3, the total assets of every FDIC-insured US bank. Downloading and caching datasets (skipped if already cached)... done, 2.0 s table : D3 FDIC bank total assets, n = 4411 observed counts : 1290 799 571 415 349 302 229 247 209 expected counts : 1327.8 776.7 551.1 427.5 349.3 295.3 255.8 225.6 201.8 quantity club code scipy.stats difference ------------------------------------------------------------------------------ chi-squared 8.0364946306 8.0364946306 +0.000e+00 p-value 4.2991335631e-01 4.2991335631e-01 -5.551e-17 upper tail at df = 8 4.2991335631e-01 4.2991335631e-01 -5.551e-17 ------------------------------------------------------------------------------ VERDICT: PASS, statistic and tail probability agree to machine precision. Independent Monte Carlo check on the same tail. 2,000,000 draws of a chi-squared variate with 8 degrees of freedom, built as a sum of 8 squared standard normals: x = 15.5073 closed form 5.000000e-02 Monte Carlo 4.993100e-02 z = -0.45 x = 20.0902 closed form 1.000000e-02 Monte Carlo 9.985500e-03 z = -0.21 x = 26.1244 closed form 1.000013e-03 Monte Carlo 1.011000e-03 z = +0.49 x = 8.0365 closed form 4.299134e-01 Monte Carlo 4.304635e-01 z = +1.57 ============================================================================== PART 1. Eight real public datasets against the law ============================================================================== First-digit proportions, observed against expected. Benford in the last row. tag dataset n 1 2 3 4 5 6 7 8 9 ------------------------------------------------------------------------------------------------ D1 Country population 2023 217 31.3% 16.6% 13.8% 7.4% 11.5% 7.4% 3.7% 4.6% 3.7% D2 CODATA 2022 constants 355 35.5% 18.3% 9.0% 8.2% 8.5% 7.0% 3.1% 5.1% 5.4% D3 US bank total assets 4411 29.2% 18.1% 12.9% 9.4% 7.9% 6.8% 5.2% 5.6% 4.7% D4 US bank total deposits 4354 29.6% 18.4% 12.8% 9.3% 8.4% 6.5% 6.0% 5.0% 4.0% D5 Exoplanet orbital periods 6016 27.6% 16.5% 14.6% 11.3% 8.8% 6.5% 5.0% 5.3% 4.5% D6 Earthquake magnitude M>=3.5 19652 0.0% 0.0% 6.9% 82.3% 10.1% 0.7% 0.1% 0.0% 0.0% D7 Earthquake seismic moment 19652 31.1% 17.7% 18.0% 1.6% 16.0% 1.3% 13.0% 1.2% 0.1% D8 Life expectancy at birth 217 0.0% 0.0% 0.0% 0.0% 2.8% 27.2% 46.1% 24.0% 0.0% ------------------------------------------------------------------------------------------------ Benford expectation 30.1% 17.6% 12.5% 9.7% 7.9% 6.7% 5.8% 5.1% 4.6% Raw counts, for anyone who wants to redo the arithmetic: tag 1 2 3 4 5 6 7 8 9 total ---------------------------------------------------------------------------------------- D1 68 36 30 16 25 16 8 10 8 217 D2 126 65 32 29 30 25 11 18 19 355 D3 1290 799 571 415 349 302 229 247 209 4411 D4 1288 800 556 406 365 284 263 218 174 4354 D5 1660 995 878 677 528 389 303 318 268 6016 D6 0 0 1351 16172 1984 129 15 1 0 19652 D7 6116 3477 3535 306 3146 261 2548 242 21 19652 D8 0 0 0 0 6 59 100 52 0 217 Test statistics. df = 8 throughout. MAD bands are Nigrini's. tag dataset n decades chi2 p MAD w verdict ---------------------------------------------------------------------------------------------------------------- D1 Country population 2023 217 5.17 7.6 0.4726 0.0152 0.1872 nonconformity D2 CODATA 2022 constants 355 114.34 12.9 0.1140 0.0172 0.1909 nonconformity D3 US bank total assets 4411 6.09 8.0 0.4299 0.0039 0.0427 close conformity D4 US bank total deposits 4354 9.13 7.8 0.4513 0.0039 0.0424 close conformity D5 Exoplanet orbital periods 6016 9.65 65.5 3.78e-11 0.0104 0.1044 acceptable conformity D6 Earthquake magnitude M>=3.5 19652 0.40 120959.6 0.00e+00 0.1662 2.4809 nonconformity D7 Earthquake seismic moment 19652 7.95 7470.8 0.00e+00 0.0486 0.6166 nonconformity D8 Life expectancy at birth 217 0.20 1063.0 3.83e-224 0.1769 2.2132 nonconformity ---------------------------------------------------------------------------------------------------------------- Ranges, so the reader can see the spread Part 2 shows is decisive: D1 min 9816 max 1.43807e+09 spans 5.17 decades sd of log10 = 1.070 D2 min 6.23538e-65 max 1.35639e+50 spans 114.34 decades sd of log10 = 14.935 D3 min 3055 max 3.75266e+09 spans 6.09 decades sd of log10 = 0.677 D4 min 2 max 2.69784e+09 spans 9.13 decades sd of log10 = 0.701 D5 min 0.0907063 max 4.02e+08 spans 9.65 decades sd of log10 = 0.915 D6 min 3.5 max 8.8 spans 0.40 decades sd of log10 = 0.040 D7 min 2.23872e+14 max 1.99526e+22 spans 7.95 decades sd of log10 = 0.643 D8 min 54.462 max 86.372 spans 0.20 decades sd of log10 = 0.043 D6 and D7 are the same earthquakes. D6 is the magnitude the catalogue prints; D7 is the energy that magnitude encodes, M0 = 10^(1.5M + 9.1). Identical events, opposite verdicts. D6 magnitudes MAD 0.1662 chi2 120959.6 nonconformity D7 moments MAD 0.0486 chi2 7470.8 nonconformity ratio of chi-squared, magnitude over moment: 16.2 D7 spans 7.95 decades and still fails, which the spread argument alone does not explain. The cause is granularity. The catalogue reports magnitude to one decimal place, so log10(M0) = 1.5 M + 9.1 moves in steps of 0.15 and its fractional part can only ever take 20 distinct values. A mantissa that visits 20 points cannot be uniform on an interval. distinct mantissa values in D7 : 115 distinct magnitudes in the catalogue : 141 Undoing the rounding by spreading each magnitude uniformly across the 0.1 wide bin it was rounded into, which is the least committal way to put back what the catalogue threw away: before : MAD 0.0486 chi2 7470.8 nonconformity after : MAD 0.0189 chi2 647.4 nonconformity chi-squared falls by a factor of 11.5. The energies were always Benford; the reporting convention was not. Note what this means for a forensic reviewer: a dataset can fail the test for no reason but the number of decimal places somebody chose to record. Bootstrap 95% intervals on MAD, 4,000 resamples of each dataset at its own n. This is sampling noise in the statistic, not a test. D1 MAD 0.0152 95% CI [0.0121, 0.0329] width 0.0208 D2 MAD 0.0172 95% CI [0.0130, 0.0308] width 0.0179 D3 MAD 0.0039 95% CI [0.0028, 0.0079] width 0.0051 D4 MAD 0.0039 95% CI [0.0028, 0.0076] width 0.0048 D5 MAD 0.0104 95% CI [0.0082, 0.0134] width 0.0052 D6 MAD 0.1662 95% CI [0.1653, 0.1670] width 0.0017 D7 MAD 0.0486 95% CI [0.0480, 0.0496] width 0.0016 D8 MAD 0.1769 95% CI [0.1718, 0.1810] width 0.0092 ============================================================================== PART 2. The mechanism. Conformity is a question of spread ============================================================================== Claim under test: a positive quantity follows Benford's law to the extent that its logarithm is spread smoothly across whole decades. Data confined to a small part of one decade cannot follow the law, honest or not. The sweep below shows the claim is true and also shows it is cruder than the truth. Experiment: draw n = 5,000 values with log10(x) uniform on [a, a+S], where the offset a is itself drawn uniformly on [0,1) for each replicate, so the answer is not an artefact of where the data happens to start. 400 replicates per S. S (decades) mean MAD mean chi2 reject at 5% band at mean MAD ---------------------------------------------------------------------------------------- 0.10 0.16600 22248.9 100.0% nonconformity 0.20 0.14890 13400.2 100.0% nonconformity 0.30 0.12938 8556.2 100.0% nonconformity 0.50 0.09307 3909.3 100.0% nonconformity 0.75 0.04598 1172.8 100.0% nonconformity 1.00 0.00326 7.7 4.8% close conformity 1.50 0.03103 441.1 100.0% nonconformity 2.00 0.00327 7.7 3.2% close conformity 3.00 0.00327 7.7 4.5% close conformity 4.00 0.00334 8.1 5.2% close conformity 6.00 0.00331 8.1 5.5% close conformity 9.00 0.00332 8.0 5.5% close conformity ---------------------------------------------------------------------------------------- That table is not monotone and the non-monotonicity is the real result. Look at S = 1.5. It is worse than S = 1.0 and worse than S = 2.0. The reason is that a window of a whole number of decades covers every mantissa exactly once, so it is exactly Benford however wide it is, while a window of one and a half decades covers half of the mantissas twice. The same sweep computed analytically, with no simulation. For each offset the exact digit law of the window is built from the measure of each digit band, the MAD of that law is taken, and the MAD is then averaged over 2,000 offsets. The average must be taken of the MAD and not of the probabilities: averaging the probabilities over a full period of offsets returns exactly Benford for every S, which is true and tells you nothing. S (decades) analytic MAD simulated MAD difference sinc envelope ---------------------------------------------------------------------------------- 0.10 0.16691 0.16600 -0.00091 0.98363 0.20 0.14836 0.14890 +0.00054 0.93549 0.30 0.12982 0.12938 -0.00044 0.85839 0.50 0.09273 0.09307 +0.00034 0.63662 0.75 0.04636 0.04598 -0.00039 0.30011 1.00 0.00000 0.00326 +0.00326 0.00000 1.50 0.03091 0.03103 +0.00012 0.21221 2.00 0.00000 0.00327 +0.00327 0.00000 3.00 0.00000 0.00327 +0.00327 0.00000 4.00 0.00000 0.00334 +0.00334 0.00000 6.00 0.00000 0.00331 +0.00331 0.00000 9.00 0.00000 0.00332 +0.00332 0.00000 ---------------------------------------------------------------------------------- The simulated column sits above the analytic one by roughly the sampling floor of a 5,000-point sample, which Part 3 measures directly. The last column is |sin(pi S)/(pi S)|. Write the mantissa density of a window S decades wide as a Fourier series on [0,1). The coefficient of harmonic k picks up a factor sin(pi k S)/(pi k S), so the whole deviation from Benford is built out of sinc terms: it vanishes at every whole number of decades and decays like 1/S in between. A clean test of that claim. At half-integer S every even harmonic vanishes by itself and every odd harmonic has |sin(pi k S)| = 1, so harmonic k contributes 1/(pi k S), which is a fixed multiple of the first harmonic no matter how wide the window. The ratio of MAD to the first-harmonic envelope must therefore be the same number at every half-integer. It is a strong prediction because it says a whole family of calculations must land on one constant. S analytic MAD sinc envelope ratio ---------------------------------------------------------------- 0.50 0.092727 0.636620 0.145656 1.50 0.030909 0.212207 0.145656 2.50 0.018545 0.127324 0.145656 3.50 0.013247 0.090946 0.145656 4.50 0.010303 0.070736 0.145656 5.50 0.008430 0.057875 0.145656 6.50 0.007133 0.048971 0.145656 7.50 0.006182 0.042441 0.145656 8.50 0.005455 0.037448 0.145656 9.50 0.004880 0.033506 0.145656 ---------------------------------------------------------------- mean ratio 0.145656, spread 1.110e-16, coefficient of variation 3.07e-16 VERDICT: PASS. Ten independent windows, from half a decade to nine and a half, give one constant to better than a part in a million. The mechanism is understood and not merely observed. The non-half-integer rows above do not obey the same constant, and should not. At S = 0.30 and S = 0.75 the even harmonics have not cancelled, so the ratio there sits a few percent off. That is the series working, not failing. VALIDATION V4. The mechanism check the whole study rests on. Data spanning many decades must conform; data spanning less than one must not. log-uniform over 6.0 decades n=200000 MAD 0.00040 chi2 4.3 close conformity log-uniform over 0.6 decades n=200000 MAD 0.08843 chi2 132201.1 nonconformity VERDICT: PASS. Wide data conforms closely, narrow data does not conform. That is the mechanism. Lognormal version of the same mechanism, since real data is more often lognormal than log-uniform. sigma is the standard deviation of log10(x), so it is spread measured in decades. Digit probabilities here are exact, not simulated. sigma (decades) MAD of the true law largest single-digit error ------------------------------------------------------------------------ 0.10 1.071e-01 1.977e-01 0.20 6.169e-02 1.331e-01 0.30 2.314e-02 5.104e-02 0.50 9.836e-04 2.173e-03 0.75 2.060e-06 4.550e-06 1.00 3.659e-10 8.082e-10 1.25 5.546e-15 1.243e-14 1.50 3.755e-16 5.135e-16 2.00 2.105e-16 5.829e-16 3.00 3.045e-16 4.441e-16 ------------------------------------------------------------------------ Above about sigma = 0.4 the true law sits inside Nigrini's close-conformity band even though the data was never Benford by construction. That is the loophole Part 3 walks through. ============================================================================== PART 3. The false accusation rate ============================================================================== Nothing below is dishonest. Every dataset here is generated by an honest process. The question is how often the standard test says otherwise. 3a. Calibration. Data drawn from Benford's law exactly, so the null is true and every rejection is a false accusation by construction. 40,000 replicates per n. n chi2 reject 5% chi2 reject 1% mean chi2 mean MAD MAD > 0.015 ------------------------------------------------------------------------------------------------ 50 5.152% 1.280% 8.000 0.03322 99.0% 100 5.167% 1.185% 8.012 0.02355 91.8% 250 5.120% 1.033% 8.010 0.01486 45.9% 500 5.077% 1.002% 8.012 0.01051 7.2% 1000 5.013% 1.045% 8.004 0.00742 0.1% 2500 4.973% 1.050% 7.990 0.00469 0.0% 5000 4.793% 0.915% 7.941 0.00331 0.0% 10000 5.090% 1.038% 7.970 0.00234 0.0% 50000 4.797% 1.008% 7.982 0.00105 0.0% ------------------------------------------------------------------------------------------------ Expected: 5.000% and 1.000% in the chi-squared columns, mean chi2 = 8.000. Largest deviation of the 5% column from 5%: 1.90 standard errors. VERDICT: PASS, the chi-squared test is correctly calibrated when the null is literally true. The MAD column is the finding. Nigrini's 0.015 threshold is a fixed number applied to a statistic whose sampling noise shrinks like 1/sqrt(n). On perfect Benford data with n = 100 it fires almost every time. 3b. Honest lognormal data, which is what most real positive quantities look like. The true digit law is computed exactly, then counts are drawn from it. Rejection is measured against the Benford expectation at the 5% level. 40,000 replicates per cell. sigma n=100 n=500 n=1000 n=4411 n=10000 n=50000 ---------------------------------------------------------------------------------------- 0.15 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 0.20 99.98% 100.00% 100.00% 100.00% 100.00% 100.00% 0.25 82.09% 100.00% 100.00% 100.00% 100.00% 100.00% 0.30 31.41% 97.34% 99.99% 100.00% 100.00% 100.00% 0.40 6.41% 11.61% 19.83% 76.81% 99.32% 100.00% 0.50 5.12% 5.33% 5.34% 6.56% 8.48% 27.23% 0.75 5.24% 4.95% 4.84% 4.88% 5.03% 5.11% 1.00 5.07% 5.03% 4.92% 5.12% 5.17% 4.94% ---------------------------------------------------------------------------------------- Read this table two ways. Down a column, spread buys innocence: once sigma passes about 0.4 decades the lognormal is so close to Benford that the test cannot tell, and the rejection rate settles at its nominal 5 percent. Across a row, size costs innocence: honest data of fixed spread is accused more and more often simply because somebody collected more of it. The lognormal is a kind law. Part 3c uses real data instead, which is less kind. VALIDATION V5. The multinomial shortcut against direct lognormal sampling. If the closed-form digit law is right, drawing counts from it must give the same rejection rate as generating the numbers themselves. sigma = 1.0, n = 1000. multinomial shortcut : 0.0529 direct lognormal draws : 0.0516 difference : -0.0013 (-0.56 standard errors) VERDICT: PASS, the shortcut reproduces direct simulation. 3c. Real honest datasets, resampled. The digit law of a real dataset from Part 1 is treated as the truth, and fresh honest samples of size n are drawn from it and tested as if each were a separate set of books. 20,000 draws per cell. One correction matters here and is easy to get wrong. The observed chi-squared of a dataset is not an unbiased estimate of how far its true law sits from Benford. Under a true separation lambda, E[chi2] = lambda + 8. Feeding the raw empirical proportions back into the simulator therefore inflates the separation by 8 and exaggerates the rejection rate. Both versions are printed, because the difference between them is the single largest modelling choice in this study. The corrected law is p = P + t (p_hat - P) with t = sqrt(1 - 8/chi2_obs), which sets the separation to the unbiased value (chi2_obs - 8)/n. D3, US bank total assets, which passed. Real n = 4411, observed chi2 = 8.04, shrink factor t = 0.0674 empirical digit law : 0.2925 0.1811 0.1294 0.0941 0.0791 0.0685 0.0519 0.0560 0.0474 corrected digit law : 0.3005 0.1764 0.1252 0.0967 0.0792 0.0670 0.0576 0.0515 0.0459 Benford : 0.3010 0.1761 0.1249 0.0969 0.0792 0.0669 0.0580 0.0512 0.0458 n plug-in reject 5% corrected reject 5% corrected MAD>0.015 -------------------------------------------------------------------------------- 100 6.08% 5.24% 91.80% 500 8.87% 5.10% 7.26% 1000 12.81% 5.19% 0.07% 4411 47.95% 5.32% 0.00% <- the real n 6016 64.22% 5.08% 0.00% 10000 88.80% 5.12% 0.00% 50000 100.00% 6.60% 0.00% 200000 100.00% 11.55% 0.00% -------------------------------------------------------------------------------- The two columns disagree by a wide margin at the real sample size. The corrected column is the honest one. At n = 4,411 the banks are in no danger, and it takes a far larger book of honest banks before the test convicts. D5, exoplanet orbital periods, which failed. Real n = 6016, observed chi2 = 65.54, shrink factor t = 0.9370 empirical digit law : 0.2759 0.1654 0.1459 0.1125 0.0878 0.0647 0.0504 0.0529 0.0445 corrected digit law : 0.2775 0.1661 0.1446 0.1115 0.0872 0.0648 0.0508 0.0528 0.0446 Benford : 0.3010 0.1761 0.1249 0.0969 0.0792 0.0669 0.0580 0.0512 0.0458 n plug-in reject 5% corrected reject 5% corrected MAD>0.015 -------------------------------------------------------------------------------- 100 9.68% 9.06% 94.01% 500 33.12% 28.95% 37.65% 1000 62.62% 56.74% 15.12% 4411 100.00% 99.94% 0.10% 6016 100.00% 100.00% 0.01% <- the real n 10000 100.00% 100.00% 0.00% 50000 100.00% 100.00% 0.00% 200000 100.00% 100.00% 0.00% -------------------------------------------------------------------------------- D5 is the instructive one. An honest astronomical catalogue, no motive and no auditor, MAD inside Nigrini's acceptable band, and the chi-squared test rejects it at p = 4e-11. Even after the bias correction a fresh honest sample of the same size is rejected most of the time. Nothing is wrong with the catalogue. The law simply does not apply to it exactly. 3d. Honest data with round-number habits. A fraction q of the values is snapped to one significant figure, the way real invoices, budgets and salaries are. Underlying data is lognormal with sigma = 1.5, n = 1000, 20,000 replicates. q mean MAD reject at 5% what a forensic reviewer would say -------------------------------------------------------------------------------------- 0.00 0.00743 4.93% no flag 0.05 0.00755 5.58% no flag 0.10 0.00786 7.32% no flag 0.20 0.00899 14.58% occasional flag 0.40 0.01230 52.90% a flag most years 0.80 0.02020 99.77% a flag nearly always -------------------------------------------------------------------------------------- Rounding to one significant figure does not change any first digit. It cannot. So the first-digit test is blind to this particular honest habit, which is the one piece of good news in Part 3. The rounding a reviewer should worry about is rounding to a fixed unit, not to a fixed number of significant figures. 3e. The arithmetic that matters. Suppose an auditor screens many separate accounts, each of size n, each of them honest, at the 5% level. accounts screened expected false accusations at 5% at 1% ------------------------------------------------------------------------ 20 1.0 0.2 100 5.0 1.0 500 25.0 5.0 1000 50.0 10.0 10000 500.0 100.0 ------------------------------------------------------------------------ Nothing in that table depends on anyone doing anything wrong. ============================================================================== PART 4. Convergence. Powers of two, the textbook Benford sequence ============================================================================== The leading digits of 2^k are Benford in the limit, because k*log10(2) is equidistributed mod 1 for irrational log10(2). That is a theorem, so this is a convergence test with a known answer. The first digit of 2^k should be floor(10^frac(k log10 2)), which avoids ever building the 3-million-digit integer 2^(10^7). First attempt, log10(2) as an ordinary double, checked against exact integer arithmetic on 2^1 through 2^2000: mismatches over 2,000 terms : 1 k = 3 exact first digit 8, double precision says 7, 10^frac = 7.99999999999999911 This is not a bug in the digit extractor, which validation V2 cleared. The extractor is never asked. The damage is done one step earlier, in rebuilding the mantissa: 2^3 has mantissa exactly 8, and 10^(3 log10 2) in double precision lands at 7.999999999999998, so the floor takes 7. Any k whose mantissa sits within about 1e-9 of a digit boundary is at risk, and the risk grows with k because the absolute error in k*log10(2) grows with k. The fix. Carry log10(2) to 50 decimal digits and split it into a high part with only 29 significant bits, so that k*high is exact in double arithmetic for every k below 2^24, plus a low remainder that contributes at most 0.01 and carries the constant to 1e-25. The fractional part is then accurate to about 1e-17 instead of 1e-9, and the mantissa is nudged off exact boundaries by a half-ulp tolerance. log10(2) to 30 digits : 0.301029995663981195213738894724 high part (29 bits) : 0.30102999508380889893 low remainder : 5.802e-10 mismatches over 2,000 terms after the fix : 0 first twelve, exact : 2 4 8 1 3 6 1 2 5 1 2 4 first twelve, fixed : 2 4 8 1 3 6 1 2 5 1 2 4 VERDICT: PASS, the corrected route reproduces exact integer arithmetic on every one of the first 2,000 powers of two. terms N MAD chi2 (df 8) largest digit error -------------------------------------------------------------------------- 10 2.883e-02 1.8051 5.799e-02 100 3.879e-03 0.2210 9.181e-03 1000 6.782e-04 0.1586 2.053e-03 10000 5.376e-05 0.0045 9.195e-05 100000 1.126e-05 0.0017 2.195e-05 1000000 1.149e-06 0.0002 1.947e-06 10000000 4.354e-07 0.0003 7.409e-07 -------------------------------------------------------------------------- MAD falls here roughly like 1/N rather than 1/sqrt(N), because the sequence is deterministic and equidistributed rather than random. A random Benford sample of ten million would sit near MAD 7.418e-05, which is 170 times larger than the powers of two at the same N. Equidistribution is a much stronger statement than agreement in distribution, and this is what the difference looks like. The whole of Part 4 is a warning about Part 1 as much as a check. If a convergence test with a known analytic answer can be wrecked by one floating point boundary, a forensic test run on somebody's accounts deserves the same suspicion. ============================================================================== SUMMARY OF HEADLINE NUMBERS ============================================================================== Validation V1 expected frequencies vs closed form, max difference : 5.302e-08 V2 digit extractor mismatches over 1000043 values : 0 V3 chi-squared club vs scipy, difference : +0.000e+00 V4 wide 6-decade MAD 0.00040, narrow 0.6-decade MAD 0.08843 V5 shortcut vs direct simulation, z : -0.56 Part 1 D1 Country population 2023 MAD 0.0152 chi2 7.6 w 0.1872 nonconformity D2 CODATA 2022 constants MAD 0.0172 chi2 12.9 w 0.1909 nonconformity D3 US bank total assets MAD 0.0039 chi2 8.0 w 0.0427 close conformity D4 US bank total deposits MAD 0.0039 chi2 7.8 w 0.0424 close conformity D5 Exoplanet orbital periods MAD 0.0104 chi2 65.5 w 0.1044 acceptable conformity D6 Earthquake magnitude M>=3.5 MAD 0.1662 chi2 120959.6 w 2.4809 nonconformity D7 Earthquake seismic moment MAD 0.0486 chi2 7470.8 w 0.6166 nonconformity D8 Life expectancy at birth MAD 0.1769 chi2 1063.0 w 2.2132 nonconformity D7 with the magnitude rounding undone MAD 0.0189 chi2 647.4 nonconformity Part 2 simulated MAD at S = 0.3 decades : 0.12938 simulated MAD at S = 1.0 decades : 0.00326 simulated MAD at S = 3.0 decades : 0.00327 simulated MAD at S = 9.0 decades : 0.00332 simulated MAD at S = 1.5 decades : 0.03103 lognormal sigma 0.5, true MAD : 9.836e-04 lognormal sigma 1.0, true MAD : 3.659e-10 Part 3 Perfect Benford, n = 100, chi2 rejection at 5% : 5.17% Perfect Benford, n = 100, MAD > 0.015 rate : 91.8% Perfect Benford, n = 1000, MAD > 0.015 rate : 0.1% Perfect Benford, n = 10000, MAD > 0.015 rate : 0.0% Honest lognormal sigma 0.15, n = 1000, rejection : 100.00% Honest lognormal sigma 0.25, n = 1000, rejection : 100.00% Honest lognormal sigma 0.25, n = 50000, rejection: 100.00% Honest lognormal sigma 0.40, n = 50000, rejection: 100.00% Honest lognormal sigma 1.00, n = 50000, rejection: 4.94% Bank digits resampled, n = 4411, plug-in : 47.95% Bank digits resampled, n = 4411, bias corrected : 5.32% Bank digits resampled, n = 200000, corrected : 11.55% Exoplanet digits resampled, n = 6016, corrected : 100.00% Exoplanet digits resampled, n = 1000, corrected : 56.74% Part 4 Powers of two, N = 10^7, MAD : 4.354e-07 ============================================================================== FIGURE DATA. Every number the article plots, printed ============================================================================== FIG1. Observed first-digit proportions by dataset, then the Benford row. Format: tag, then nine proportions for digits 1 to 9. FIG1 D1 0.31336 0.16590 0.13825 0.07373 0.11521 0.07373 0.03687 0.04608 0.03687 FIG1 D2 0.35493 0.18310 0.09014 0.08169 0.08451 0.07042 0.03099 0.05070 0.05352 FIG1 D3 0.29245 0.18114 0.12945 0.09408 0.07912 0.06847 0.05192 0.05600 0.04738 FIG1 D4 0.29582 0.18374 0.12770 0.09325 0.08383 0.06523 0.06040 0.05007 0.03996 FIG1 D5 0.27593 0.16539 0.14594 0.11253 0.08777 0.06466 0.05037 0.05286 0.04455 FIG1 D6 0.00000 0.00000 0.06875 0.82292 0.10096 0.00656 0.00076 0.00005 0.00000 FIG1 D7 0.31122 0.17693 0.17988 0.01557 0.16009 0.01328 0.12966 0.01231 0.00107 FIG1 D8 0.00000 0.00000 0.00000 0.00000 0.02765 0.27189 0.46083 0.23963 0.00000 FIG1 BENF 0.30103 0.17609 0.12494 0.09691 0.07918 0.06695 0.05799 0.05115 0.04576 FIG2. MAD against sample size. First the eight real datasets as (n, MAD), then the sampling floor: the mean MAD of a perfectly Benford sample of size n, taken from the calibration run in Part 3a. FIG2 DATA D1 n=217 MAD=0.015212 FIG2 DATA D2 n=355 MAD=0.017216 FIG2 DATA D3 n=4411 MAD=0.003899 FIG2 DATA D4 n=4354 MAD=0.003882 FIG2 DATA D5 n=6016 MAD=0.010427 FIG2 DATA D6 n=19652 MAD=0.166174 FIG2 DATA D7 n=19652 MAD=0.048563 FIG2 DATA D8 n=217 MAD=0.176946 FIG2 FLOOR n=50 meanMAD=0.033220 P(MAD>0.015)=0.98965 FIG2 FLOOR n=100 meanMAD=0.023546 P(MAD>0.015)=0.91773 FIG2 FLOOR n=250 meanMAD=0.014856 P(MAD>0.015)=0.45942 FIG2 FLOOR n=500 meanMAD=0.010512 P(MAD>0.015)=0.07180 FIG2 FLOOR n=1000 meanMAD=0.007416 P(MAD>0.015)=0.00077 FIG2 FLOOR n=2500 meanMAD=0.004694 P(MAD>0.015)=0.00000 FIG2 FLOOR n=5000 meanMAD=0.003309 P(MAD>0.015)=0.00000 FIG2 FLOOR n=10000 meanMAD=0.002342 P(MAD>0.015)=0.00000 FIG2 FLOOR n=50000 meanMAD=0.001049 P(MAD>0.015)=0.00000 FIG2 FLOOR fitted form meanMAD = 0.23452 / sqrt(n) FIG3. Deviation from Benford against how many decades the data spans, computed analytically. For each S the exact digit law of a log-uniform window is built at 500 offsets, the MAD is taken at each, and the results are averaged. The sinc column is |sin(pi S)/(pi S)|. S analytic MAD sinc envelope ------------------------------------------------ FIG3 0.05 0.176178 0.995893 FIG3 0.10 0.166911 0.983632 FIG3 0.15 0.157638 0.963398 FIG3 0.20 0.148364 0.935489 FIG3 0.25 0.139092 0.900316 FIG3 0.30 0.129818 0.858394 FIG3 0.35 0.120546 0.810332 FIG3 0.40 0.111273 0.756827 FIG3 0.45 0.102000 0.698647 FIG3 0.50 0.092728 0.636620 FIG3 0.55 0.083455 0.571620 FIG3 0.60 0.074182 0.504551 FIG3 0.65 0.064909 0.436333 FIG3 0.70 0.055636 0.367883 FIG3 0.75 0.046364 0.300105 FIG3 0.80 0.037091 0.233872 FIG3 0.85 0.027818 0.170011 FIG3 0.90 0.018546 0.109292 FIG3 0.95 0.009273 0.052415 FIG3 1.00 0.000000 0.000000 FIG3 1.05 0.008389 0.047423 FIG3 1.10 0.015174 0.089421 FIG3 1.15 0.020561 0.125661 FIG3 1.20 0.024727 0.155915 FIG3 1.25 0.027818 0.180063 FIG3 1.30 0.029958 0.198091 FIG3 1.35 0.031253 0.210086 FIG3 1.40 0.031792 0.216236 FIG3 1.45 0.031655 0.216821 FIG3 1.50 0.030909 0.212207 FIG3 1.55 0.029613 0.202833 FIG3 1.60 0.027818 0.189207 FIG3 1.65 0.025570 0.171889 FIG3 1.70 0.022909 0.151481 FIG3 1.75 0.019870 0.128617 FIG3 1.80 0.016485 0.103943 FIG3 1.85 0.012781 0.078113 FIG3 1.90 0.008785 0.051770 FIG3 1.95 0.004517 0.025536 FIG3 2.00 0.000000 0.000000 FIG3 2.05 0.004297 0.024290 FIG3 2.10 0.007948 0.046840 FIG3 2.15 0.010998 0.067214 FIG3 2.20 0.013488 0.085044 FIG3 2.25 0.015455 0.100035 FIG3 2.30 0.016933 0.111964 FIG3 2.35 0.017954 0.120688 FIG3 2.40 0.018545 0.126138 FIG3 2.45 0.018735 0.128323 FIG3 2.50 0.018546 0.127324 FIG3 2.55 0.018000 0.123291 FIG3 2.60 0.017119 0.116435 FIG3 2.65 0.015921 0.107025 FIG3 2.70 0.014424 0.095377 FIG3 2.75 0.012645 0.081847 FIG3 2.80 0.010597 0.066821 FIG3 2.85 0.008297 0.050705 FIG3 2.90 0.005756 0.033918 FIG3 2.95 0.002986 0.016880 FIG3 3.00 0.000000 0.000000 FIG3 3.05 0.002888 0.016326 FIG3 3.10 0.005384 0.031730 FIG3 3.15 0.007507 0.045876 FIG3 3.20 0.009273 0.058468 FIG3 3.25 0.010699 0.069255 FIG3 3.30 0.011802 0.078036 FIG3 3.35 0.012594 0.084662 FIG3 3.40 0.013091 0.089038 FIG3 3.45 0.013304 0.091128 FIG3 3.50 0.013247 0.090946 FIG3 3.55 0.012930 0.088561 FIG3 3.60 0.012364 0.084092 FIG3 3.65 0.011559 0.077703 FIG3 3.70 0.010526 0.069599 FIG3 3.75 0.009273 0.060021 FIG3 3.80 0.007809 0.049236 FIG3 3.85 0.006142 0.037535 FIG3 3.90 0.004280 0.025221 FIG3 3.95 0.002230 0.012606 FIG3 4.00 0.000000 0.000000 FIG3 4.05 0.002175 0.012295 FIG3 4.10 0.004071 0.023991 FIG3 4.15 0.005698 0.034822 FIG3 4.20 0.007065 0.044547 FIG3 4.25 0.008182 0.052960 FIG3 4.30 0.009057 0.059888 FIG3 4.35 0.009699 0.065199 FIG3 4.40 0.010116 0.068802 FIG3 4.45 0.010315 0.070650 FIG3 4.50 0.010303 0.070736 FIG3 4.55 0.010088 0.069097 FIG3 4.60 0.009676 0.065811 FIG3 4.65 0.009073 0.060993 FIG3 4.70 0.008286 0.054791 FIG3 4.75 0.007321 0.047385 FIG3 4.80 0.006182 0.038979 FIG3 4.85 0.004875 0.029796 FIG3 4.90 0.003406 0.020074 FIG3 4.95 0.001780 0.010060 FIG3 5.00 0.000000 0.000000 FIG3 5.05 0.001744 0.009860 FIG3 5.10 0.003273 0.019287 FIG3 5.15 0.004591 0.028060 FIG3 5.20 0.005706 0.035980 FIG3 5.25 0.006623 0.042872 FIG3 5.30 0.007348 0.048588 FIG3 5.35 0.007886 0.053012 FIG3 5.40 0.008242 0.056061 FIG3 5.45 0.008422 0.057686 FIG3 5.50 0.008430 0.057875 FIG3 5.55 0.008270 0.056647 FIG3 5.60 0.007948 0.054059 FIG3 5.65 0.007467 0.050198 FIG3 5.70 0.006833 0.045179 FIG3 5.75 0.006047 0.039144 FIG3 5.80 0.005116 0.032258 FIG3 5.85 0.004042 0.024703 FIG3 5.90 0.002829 0.016672 FIG3 5.95 0.001480 0.008369 FIG3 6.00 0.000000 0.000000 ------------------------------------------------ FIG4. False accusation rate against sample size, at the 5% level, for five honest generating laws plus the fixed MAD rule. Every one of these is innocent by construction. law 50 100 200 400 800 1600 3200 6400 12800 25600 51200 102400 -------------------------------------------------------------------------------------------------------------------------- FIG4 BENFORD 0.0503 0.0515 0.0520 0.0527 0.0483 0.0510 0.0483 0.0517 0.0490 0.0491 0.0539 0.0488 FIG4 LOGN030 0.1692 0.3125 0.6079 0.9228 0.9994 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 FIG4 LOGN040 0.0590 0.0661 0.0770 0.1012 0.1647 0.3088 0.5948 0.9193 0.9991 1.0000 1.0000 1.0000 FIG4 LOGN050 0.0505 0.0515 0.0511 0.0547 0.0517 0.0551 0.0612 0.0693 0.0904 0.1501 0.2743 0.5554 FIG4 BANKCOR 0.0491 0.0513 0.0556 0.0491 0.0505 0.0517 0.0517 0.0509 0.0525 0.0600 0.0623 0.0795 FIG4 EXOPCOR 0.0716 0.0886 0.1362 0.2365 0.4588 0.8085 0.9879 1.0000 1.0000 1.0000 1.0000 1.0000 FIG4 MADRULE 0.9887 0.9151 0.6149 0.1585 0.0052 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 -------------------------------------------------------------------------------------------------------------------------- MADRULE is Nigrini's fixed 0.015 threshold applied to data that is exactly Benford. BENFORD is the chi-squared test on the same data and should sit at 0.05 at every n, which is the flat line the others are measured against. FIG5. Convergence of the powers of two, against the convergence a random Benford sample of the same size manages. MAD in both columns. N powers of two random sample -------------------------------------------------------- FIG5 10 2.883e-02 7.587e-02 FIG5 20 2.883e-02 5.223e-02 FIG5 50 2.438e-02 3.328e-02 FIG5 100 3.879e-03 2.343e-02 FIG5 200 2.592e-03 1.667e-02 FIG5 500 9.543e-04 1.044e-02 FIG5 1000 6.782e-04 7.357e-03 FIG5 2000 7.151e-04 5.246e-03 FIG5 5000 2.929e-04 3.317e-03 FIG5 10000 5.376e-05 2.352e-03 FIG5 20000 3.737e-05 1.671e-03 FIG5 50000 1.095e-05 1.049e-03 FIG5 100000 1.126e-05 7.403e-04 FIG5 200000 5.426e-06 5.259e-04 FIG5 500000 4.482e-06 3.356e-04 FIG5 1000000 1.149e-06 2.323e-04 FIG5 2000000 1.213e-06 1.658e-04 FIG5 5000000 4.354e-07 1.049e-04 FIG5 10000000 4.354e-07 7.348e-05 -------------------------------------------------------- All downloads succeeded. No dataset was dropped and nothing was substituted. elapsed: 21.1 s