Skip to content
Science Journaling Club

Volume 2, Issue 2December 2025

Winter 2026

Old evidence, read carefully, alongside two animals caught doing something nobody had recorded before: a bat killing and eating a bird entirely in flight, and killer whales manufacturing their own grooming tools out of kelp.

This issueSJC 2.2
Published
December 2025
Articles
8
Research
3
Field notes
5
With code
6

Contents

8 articles, in the order the club meant them to be read.

  1. Field noteExplainer

    What a P Value Actually Means, and the Four Things People Think It Means Instead

    A p value measures your data against a pretend world, the one where the effect you are hunting does not exist. The number describes data, given a hypothesis. Swap those two around and every misreading in this article follows. We define the thing properly, then take apart four beliefs the club has had to argue with at its own table: that p is the probability the hypothesis is wrong, that a smaller p means a bigger effect, that 0.049 and 0.051 are different animals, and that a non-significant result means nothing happened. Our own simulation of 200,000 experiments under a true null backs the definition, with the p values coming out flat to a Kolmogorov-Smirnov distance of 0.0014 from perfect uniformity. Another 400,000 simulate a whole literature. Now the headline. In a field where 10% of tested hypotheses are true and studies have 50% power, 47.4% of everything published as significant at p<0.05p<0.05 is false, with no fraud, no p-hacking, no hidden analyses and no arithmetic errors anywhere. Two honest studies of one real effect disagree half the time, and one line of arithmetic gives you both numbers. No results section reads the same way afterwards.

    StatisticsRead 29 minSections 13

  2. ResearchMethods

    The Treatment That Works Because You Picked the Worst Cases

    Take a population, measure everybody, keep the worst tenth, do absolutely nothing to them, and measure them again. In our simulation they improved by 0.8760 ± 0.0013 standard deviations. The instrument was a program the club wrote, so the computation is the experiment and no person, patient or pupil entered the study at any point. Across a grid of six selection thresholds (p=0.50p = 0.50 down to 0.010.01) and seven noise-to-signal ratios we simulated 372 million individuals, each measured twice, with a stable underlying trait and fresh noise on each occasion. Every cell was checked against the closed form. For a bivariate normal with correlation rr, a group selected on the first measurement has its mean shrunk toward the population mean by exactly rr, and our measured shrinkage matched rr in all 42 cells, the largest discrepancy being 2.53 standard errors against a Bonferroni threshold of 3.41; the pooled fit gave ∑z2=36.20\sum z^2 = 36.20 on 36 degrees of freedom, p=0.459p = 0.459. With zero noise, where r=1r = 1, the apparent improvement was identically zero at every threshold, which is the control that makes the rest of the grid mean anything. Adding a genuine treatment effect, a single-arm before-and-after design overstated a true 0.20 SD effect by 5.38× in our headline cell, attributing 81.5% of its result to an intervention that supplied none of it, while a randomised design run on the same simulated people recovered the truth to within 0.87 standard errors. At a true effect of zero the single-arm design reached p<0.05p < 0.05 in 100% of replicates. Set against the published figure of −0.24-0.24 for change from baseline in the untreated arms of 37 three-armed trials [15], regression alone accounts for the whole of it once entry reliability falls to 0.863 at a bottom-tenth cut. Seed 20260211.

    Winter 2026Read 31 minSections 12

  3. ResearchMethods

    Building a Dataset Where Every Group Trends Up and the Whole Trends Down

    Simpson's paradox is usually shown once, with one famous table, and then left alone as a curiosity. We treated it as a piece of engineering instead. The club wrote a generator that takes a target within-group slope and a target pooled slope of the opposite sign and returns a dataset hitting both, by solving for the required between-group slope rather than searching for one. Across eight specifications the worst deviation between the fitted slopes and the requested ones was 4.44×10−16. Nothing was measured in the world; the computation is the experiment. Then we ran the machine backwards and asked how often the paradox arrives uninvited. With four groups of sixty points, a within-group slope of +1+1, and a group-level confounder as strong as the within-group spread in xx, 9.70% ± 0.21% of 20,000 simulated datasets showed every group sloping one way and the pooled fit sloping the other, and 6.04% ± 0.17% did so with every slope involved clearing a two-sided tt test at α=0.05\alpha = 0.05. Turning the confounder off gave 0 reversals in 40,000 datasets, which it must, because the pooled slope is then equal to the within-group slope as an algebraic identity. Group-size imbalance turned out to work the opposite way from the folklore, cutting the rate from 9.70% to 4.86% at a 30:1 ratio, and coupling size to the confounder cut it further to 2.43%, a difference of 13.0 standard errors. In the continuous case with a single lurking variable the Monte Carlo matched a closed-form integral to within 1.60 standard errors at every strength. As an external check, our 3,000,000 random 2×2×2 tables reversed at 0.016734 ± 0.000074 against Pavlides and Perlman's exact 1/60 [10], a gap of 0.90 standard errors. Seed 20260214.

    Winter 2026Read 30 minSections 12

  4. ResearchOriginal Research

    How Many Reasonable Choices Does It Take to Find a Result That Is Not There

    Two groups of numbers drawn from the same distribution cannot differ. We drew them 50,000 times and then analysed them the way data actually gets analysed, making one defensible choice after another with the numbers already in front of us. With no choices at all the false positive rate came out at 0.0504 ± 0.0010, a 95% interval of 0.0486 to 0.0524 that covers the nominal 0.05, and that single number is the check which licenses everything else here. Five researcher degrees of freedom were then applied singly and in all 32 combinations: peeking four extra times while enrolment ran from 20 to 40 per group, testing three correlated outcome measures instead of one, splitting on an irrelevant binary attribute, adding or omitting an uninformative covariate, and excluding observations beyond two standard deviations. Singly the most expensive was outcome choice, at 0.1286 ± 0.0015. Together the five produced a significant result in 35,331 of 50,000 studies of pure noise, a rate of 0.7066 ± 0.0020, which is 14.1× the nominal level. The 180 analysis paths this permits behave like 23.9 ± 0.1 independent tests rather than 180, because they share data; 180 genuinely independent tests would give 0.99990. The club's t statistic agrees with scipy.stats.ttest_ind to 1.3×10−15 on identical data, and the covariate-adjusted statistic agrees with a LAPACK least-squares solve to 3.6×10−15. Optional stopping alone, with equally spaced looks, reproduces the 1969 inflation table of Armitage, McPherson and Rowe at every published entry, the largest deviation being 2.7 standard errors at twenty looks. Three situations from Table 1 of Simmons, Nelson and Simonsohn (2011) were rebuilt to that table's own specification and landed within 2.7 standard errors of the published values at all three significance levels. The headline depends on choices we made and could have made otherwise: at outcome correlation 0.8 rather than 0.5 it falls to 0.613, and under a three standard deviation outlier rule rather than two it falls to 0.573. Seed 20260214.

    Winter 2026Read 34 minSections 12

  5. Field noteData Story

    An Empire's Air Pollution, Measured in Greenland Ice and in Roman Children's Blood

    Three Arctic ice cores hold a dated record of the smoke that European silver smelters put into the air two thousand years ago. In January 2025 a team led by Joseph McConnell at the Desert Research Institute ran that record backwards through an atmospheric transport model to recover annual emissions, then forwards again to map lead concentrations across Europe, then through two published epidemiological regressions to reach the blood of Roman children [1]. Their emission figure for the Pax Romana is roughly 3 to 4 kilotons of lead a year. Their arithmetic carries that to a childhood blood lead level about 2.4 micrograms per decilitre above background, and from there to an empire-wide cognitive cost of 2.5 to 3 IQ points. We rebuilt the chain. We replaced their dispersion model with a single well-mixed box over Europe and left the two published regressions untouched. Out came 0.514 nanograms of lead per cubic metre of air, a blood lead rise of 2.39 µg/dL, and a decrement of 2.61 IQ points. Close enough to examine. Not close enough to trust blindly. Run the whole chain across the plausible parameter space and something else appears. The atmospheric physics, the hard part, barely matters. A twelvefold change in emissions moves the final answer by 27 per cent. A one-standard-error wobble in a single epidemiological exponent moves it by 57 per cent.

    Environmental ArchaeologyRead 29 minSections 12

  6. Field notePaper Analysis

    A Volcano That Has Been Dead for 250,000 Years Is Still Breathing

    Uturuncu is a 6,008-metre dacite volcano in southwest Bolivia whose last eruption was 250,000 years ago, give or take five thousand. For three decades it has impersonated a mountain about to go off. The ground above it has been rising at roughly a centimetre a year over a broad dome, that dome sits inside a ring of slow subsidence reaching past 70 kilometres, and small earthquakes fire underneath it most days. In April 2025 a team led by Ying Liu and Mike Kendall published a joint seismological and petrophysical anatomy of its interior, built from more than 1,700 local earthquakes, and concluded that the unrest is gas and brine climbing a narrow chimney out of a much older magma body rather than fresh magma on its way up [1]. We rebuilt the deformation half of the argument ourselves. Working on a synthetic sombrero pattern of our own making, we show that no single pressurised point source at any depth can produce a ring of subsidence around a central uplift, because the vertical displacement of such a source carries the sign of its volume change at every radius without exception. The best single source leaves a misfit of 1.29 mm/yr, about six and a half times the noise we put in; a pair at 20.8 and 77.4 km gets to 0.17 mm/yr. Then we divide. The fitted inflation rate of 2.5 × 107 m³/yr is 192 times Uturuncu's own long-run magma output rate, and had it run steadily since the last eruption it would have delivered 6,360 km³, seventy-five times the volume of the entire mountain. Both calculations are the club's own and both are crude. The direction of the conclusion does not depend on either.

    VolcanologyRead 39 minSections 12

  7. Field notePaper Analysis

    Twenty-Three Minutes of Chewing, One Kilometre Up

    Carlos Ibáñez found the first clue in 2001: feathers, too many to be nesting material, packed into the guano beneath a greater noctule bat roost in southern Spain [2]. In 2007 his group ran the blood chemistry and got an isotope signature that, every autumn, sat inside the zone reserved for animals eating almost nothing but birds [3] — a gap of 3.21‰ in carbon and 3.15‰ in nitrogen from the bats’ ordinary insect diet, our own subtraction of the paper’s reported means. Neither result could say how a bird ends up eaten by something that echolocates. Caught alive, a kilometre up, by an animal the size of a fist? Or found already dead? Isotopes cannot tell a kill from a carcass; the molecules read the same either way. For eighteen more years nobody could resolve it, because nobody could follow a bat far enough into the dark to watch. In October 2025 a team led by Laura Stidsholt did, by making the bat carry the witness. Sound-and-movement tags built at Aarhus University, riding on Nyctalus lasiopterus over Spain’s Doñana reserve, recorded two hunts start to finish: a 30-second chase that failed, and a chase lasting nearly three minutes — more than a kilometre above the ground, in total darkness — that ended with 21 recorded distress calls from the bird, a robin, followed by 23 minutes of chewing sounds as the bat kept flying and ate it in the air [1]. This field note follows the case from the guano pile to the recording, and asks the question the tag actually answers: what does a hunting technique built for insects a few centimetres long look like when it is redirected at something built to survive a hawk?

    Animal BehaviourRead 16 minSections 13

  8. Field notePaper Analysis

    Killer Whales Are Manufacturing Grooming Tools, Mostly With Family

    In the spring and summer of 2024, a drone hovering over the central Salish Sea filmed something nobody had described before: a southern resident killer whale biting a short length of stipe off a whole bull kelp stalk, pressing the piece between its own body and a partner's, and rolling it back and forth for as long as 12 minutes and 4 seconds. Michael Weiss, Darren Croft and six coauthors call the behavior allokelping, and their June 2025 report in Current Biology documents 30 bouts of it across roughly 10 hours 21 minutes of footage, spanning all three southern resident pods [1]. The kelp itself is nothing special. What earns the paper its attention is the verb: on seven of eight clearly filmed acquisitions, the whale did not pick up a loose scrap, it bit one to length off a still-rooted stalk, the "detach" mode in the standard taxonomy of animal tool manufacture [2]. This field note works through why that distinction is the whole story, what a Bayesian model of who-grooms-whom in one pod actually found — maternal kinship predicts a pairing with posterior probability >0.99, similar age with probability 0.98 — and does the club's own long division on the paper's own reported effort and bout counts. We add no data of our own. We only divide what the authors already published, and we say so every time we do it.

    Animal BehaviorRead 20 minSections 13