INTERACTIVE MODELFIELD NOTEMETASCIENCE
The Sleeping-Side Bench
- Study
- Field note
- Review
- Peer-edited by the club review board
- Article
- 13 sectionsabout 18 min
- Files
- LaTeX source22 kB
Everything in this dispute comes down to one comparison: how many cats fell left, how many fell right, and how surprising that split is against a coin flip. Model 1 below loads the actual published counts from both papers and runs the same chi-square test on them live, so you can watch why one dataset comes out p < 0.001 and the other four don't. Model 2 is ours: flip your own virtual cats from a population you control, and see how often pure chance produces a result that looks, on paper, like a discovery.
Model 1. Run the real studies
Pick a dataset. The bars and the statistic below them are computed from the published left/right counts, not simulated — this is the actual arithmetic behind Figure 2 in the field note, laid out so you can check it yourself.
* The Instagram arm reports a mean and standard deviation across accounts (t = 0.836, p = .407) rather than a single pooled count; the bars shown for it use the reported mean percentage applied to n = 60 as an illustrative equivalent count, not the paper's own test statistic.
Model 2. Flip your own cats
This part is entirely the club's own simulation, not a re-analysis of either paper. Set a sample size and a true population-wide chance of landing left, then flip. Each click draws that many independent virtual coin flips and drops one mark on the strip below at the resulting percentage left. Marks are grey when the result isn't significant at the conventional 0.05 threshold, and coloured when it is — purely from the sampling noise you dialled in.
- Flips so far
- 0
- Flagged “significant” (p<.05)
- 0 (0.0%)
- Expected by chance at this threshold
- ≈ 5%, if the true % is exactly 50
Set true % left to 50 and sample size to 301, then flip a few dozen times. Almost none of your marks will land anywhere near the amber overlay line at 63.8% — which is the whole reason the original result reads as a real signal rather than noise. Now drop the sample size to 20 or 30 and watch how much wider the noise band gets on its own, with nothing about the underlying population changing at all.
What the two models are and aren't showing
Model 1 is arithmetic on real numbers: it will always agree with the field note, because it's running the same test on the same published counts. Model 2 is a coin, not a cat. It cannot tell you whether cats really lean left, why YouTube and Reddit and Instagram might sample different populations of photogenic cats, or what it means that only a repeated-observation design can measure an individual trait rather than a population snapshot. What it can show, honestly, is the part of the story that's easiest to lose sight of in the argument over whose result is “right”: sample size alone changes how much noise a true 50:50 population throws off, and a result's significance is never separable from how hard it was even possible to find one, one way or the other, at that n.