INTERACTIVE COMPANIONFIELD NOTEMETASCIENCE

The Editor Anomaly Bench

A live model accompanying “Forty-Five Editors, Thirty Percent of the Retractions”

Study: Field note
Review: Peer-edited by the club review board
Article: 14 sectionsabout 24 min
Files: LaTeX source29 kB

Richardson et al.'s actual test is an exact Poisson binomial run editor-by-editor, year-by-year, with the false discovery rate held at 0.05 across all 18,329 editors at once — not something you can drag a slider on. What follows is the club's own, much simpler stand-in: a one-sided normal approximation to a binomial test, so you can feel the shape of the question the paper actually asked. Treat every number below as illustrative, not as a reproduction of their statistics.

z = (k − 0.5 − n·p) / √(n·p·(1−p)) p-value = 1 − Φ(z)

The real numbers this bench is built around. PLOS ONE: 276,956 articles since 2006, 702 retracted (baseline rate 0.2535%), 18,329 editors who have accepted a submission. The 45 editors the paper flags handled about 1.3% of articles (≈3,600) and 30.2% of retractions (≈212) — the club's own division of those two percentages.

Model 1. Flag a single editor

Set how many articles an editor accepted, how many of those were eventually retracted, and the journal's baseline retraction rate. The bench tells you how anomalous that editor looks under the club's simplified test.

Articles this editor accepted200Of those, eventually retracted3Journal baseline retraction rate0.2535%

this editor's rate: 1.500%enrichment over baseline: 5.92×z-score: 1.47one-sided p-value: 0.0708

Move a slider to run the test.

This is a single normal-approximation test with no false-discovery correction. The paper runs the exact version of this test 18,329 times at once and only calls an editor flagged if it survives FDR ≤ 0.05 across all of them — a much harder bar than clearing p < 0.05 once.

Model 2. How little it takes to get caught, and how little some of them needed to hide

Here is the puzzle the paper's own numbers pose. The 45 flagged PLOS ONE editors handled about 80 articles each on average (3,600 divided across 45). If the group's excess of roughly 203 retractions above baseline expectation had been split evenly, each editor would carry about 4–5 extra retractions in that caseload — and Model 1 shows that is wildly significant. Yet the paper directly flagged only 22 of the 45 on retraction rate alone. The other 23 apparently carried close to none of it personally. Drag the slider below to find out how few retractions, out of a typical 80-article caseload, it actually takes to trip this test.

This editor's own retractions, out of ~80 accepted0

caseload: 80 articles (this ring's average)personal rate: 0.000%one-sided p-value: 1.0000

Start at 0 and drag up one retraction at a time.

On the club's approximation, an editor handling ~80 articles needs only 2 personal retractions to clear p < 0.05, because the journal-wide baseline (0.2535%) is so low that almost any personal excess reads as extreme. That low bar is exactly why roughly half the real 45 (22 of them) tripped it directly — and why the other half (23) plausibly carried 0 or 1 retraction of their own and needed a completely different signal, who they routed submissions to, to be caught at all.

What the two models are (and are not) telling you

Model 1 is the honest, boring case: most editors, with most workloads, never come near p = 0.05, because most editors are not doing anything wrong. Model 2 is the sharper case, and it is not the story we expected going in: we assumed a coordinated ring would hide by diluting its excess evenly across many people, each one individually unremarkable. The arithmetic says otherwise, at least for retraction counts at a journal this large: because the baseline rate is so rare, it takes shockingly few personal retractions to become statistically loud. What actually let roughly half the real cluster escape the retraction-rate test was not dilution across many people; it was that they apparently carried almost none of the group's excess personally. Their own paperwork looked ordinary. That is precisely why Richardson et al.'s real analysis never relies on the retraction-count test alone. It cross-checks it against PubPeer comments, turnaround time, and, decisively, against who these editors were routing submissions to — the signal that catches a member of a ring whose own numbers give away nothing at all.

Neither model reproduces the paper. The real test is an exact Poisson binomial, not a normal approximation; it runs per editor-year, not on a lifetime total; and it is only one of three signals the authors combine. What you have here is the shape of the underlying idea, with the arithmetic left where you can check it.