========================================================================== READING A PAPER -- HOW SCIENTIFIC WRITING IS SHAPED Science Journaling Club own calculation Run date 2026-09-13 seed 20260913 ========================================================================== VALIDATION 1 -- TOKENISER vs HAND COUNT --------------------------------------- passage: We measured the response of 12 samples. The mean was 4.2 units, ... quantity hand measured match sentences 5 5 yes word tokens 30 30 yes hedge tokens 3 3 yes V1 verdict: PASS CORPUS A -- THE CLUB'S OWN PUBLISHED ARTICLES --------------------------------------------- source : C:\Users\BCheng\.vscode\projects\scijournal\articles frame : the 27 slugs in CLUB_SLUGS, frozen 13 September 2026 snapshot : C:\Users\BCheng\AppData\Local\Temp\sjc-reading-a-paper-cache\club-corpus-2026-09-13.json articles parsed : 27 h2 sections : 298 (reference lists excluded) words of body prose : 115,159 words of abstract : 6,133 CORPUS B -- OPEN-ACCESS PAPERS FROM EUROPE PMC ---------------------------------------------- service : Europe PMC REST fullTextXML frame : CC-BY research articles, 2022-01-01 to 2024-12-31 journals : 8, twelve most-cited articles each candidates : 96 PMCIDs, listed in the source cache : C:\Users\BCheng\AppData\Local\Temp\sjc-reading-a-paper-cache eLife kept 12 of 12 PLoS Biology kept 6 of 12 Nature Communications kept 11 of 12 Scientific Reports kept 10 of 12 PeerJ kept 5 of 12 PLoS One kept 10 of 12 PLoS Computational Biology kept 9 of 12 Royal Society Open Science kept 4 of 12 retained (abstract + Results present) : 67 dropped (no parsable Results section) : 29 fetch errors : 0 words of Results text : 231,448 words of abstract text: 15,907 PART 1 -- SECTION LENGTH, IN WORDS (CORPUS B) --------------------------------------------- section n median mean p10 p90 max abstract 67 206 237 147 354 575 introduction 66 704 831 408 1538 2395 methods 61 1652 1841 585 3241 5152 results 67 3148 3454 967 5824 12675 discussion 55 1194 1224 438 2058 2630 COMPRESSION: median Results is 15.3x the median abstract. A reader who reads only the abstract sees 1 word in 16.3 of the evidence-bearing text, and none of the numbers behind it. For comparison, Corpus A (club sections): n=298 median 358 words, p10 222, p90 588. Our sections are deliberately short. PART 2 -- SENTENCE LENGTH, IN WORDS ----------------------------------- text n sent mean sd median p10 p90 B:abstract 694 22.9 10.8 22.0 12 36 B:results 9391 24.6 18.5 22.0 9 42 B:discussion 2498 26.9 12.8 25.0 13 42 B:methods 4503 24.9 18.4 21.0 10 42 A:club prose 7111 16.1 10.9 13.0 5 31 Histogram of sentence length, percent of sentences per bin bin (words) 0-7 8-11 12-15 16-19 20-23 24-27 28-31 32-39 40-59 60+ B:abstract 3.7 5.9 13.8 18.4 16.9 15.4 9.9 9.5 5.8 0.6 B:results 7.8 9.3 12.6 13.3 12.8 11.4 8.7 11.5 9.9 2.7 A:club prose 24.0 19.1 14.5 10.9 9.0 7.1 5.4 6.6 3.1 0.3 VALIDATION 2 -- BOOTSTRAP SE vs ANALYTIC s/sqrt(n) -------------------------------------------------- population : 9391 Results sentences sd of sentence len : 18.4525 words analytic bootstrap ratio SE of mean, resample size 200 1.30479 1.30659 1.0014 SE of mean, full sample 0.19041 0.19068 1.0014 V2 verdict: PASS (agreement within 5% required; off by 0.14%) PART 3 -- HEDGING DENSITY, HITS PER 1,000 WORDS ----------------------------------------------- lexicon: 39 hedge forms (may, might, could, suggest, appear, ...) text n mean sd median p90 B:abstract 67 5.16 5.80 3.98 12.17 B:introduction 66 5.58 3.46 5.37 9.39 B:methods 61 2.62 2.14 1.92 6.51 B:results 67 4.16 2.80 3.49 8.18 B:discussion 55 11.30 5.60 11.02 19.15 A:club prose 298 3.24 3.41 2.79 8.22 Ten commonest hedges across abstract + Results + Discussion: may 401 could 285 potential 162 likely 134 possible 124 might 119 suggest 89 suggests 57 potentially 55 relatively 53 PART 4 -- REPORTING LANGUAGE vs INTERPRETATION LANGUAGE ------------------------------------------------------- reporting lexicon : 41 forms (increased, observed, median, ...) interpretation lexicon: 40 forms (suggests, mechanism, therefore, ...) text n report interp ratio R:I B:abstract 67 9.13 7.19 0.75 B:introduction 66 9.07 6.18 1.07 B:methods 61 18.52 2.00 9.00 B:results 67 39.08 5.09 7.58 B:discussion 55 13.17 10.05 1.27 A:club prose 298 12.84 4.80 2.00 Read down the ratio column. Methods and Results are where the reporting vocabulary lives. Abstract and Discussion are where the interpretation vocabulary lives. The abstract is written in the register of the Discussion while claiming the authority of the Results. PART 5 -- CLAIM STRENGTH, ABSTRACT vs RESULTS, PAIRED BY PAPER -------------------------------------------------------------- graded lexicon: 81 forms, weights -2 (may, unclear) to +2 (demonstrate, show) abstract Results mean claim strength /1k 10.33 10.00 median 11.15 9.57 sd across papers 16.07 10.68 PAIRED DIFFERENCE (abstract minus Results), n = 67 papers mean difference : +0.33 per 1,000 words sd of difference : 17.32 standard error : 2.12 t (66 df) : 0.16 papers where the abstract is the more assertive text: 35 of 67 (52%) ASSERTIVE SHARE: of the claim-bearing words in a section, the fraction that assert rather than hedge. abstract : 0.706 (median 0.750) Results : 0.738 (median 0.773) gap : -0.033 Discussion sections, for context: n=55 mean -3.49 median -6.72 CORPUS A, same instrument, so we are scored too: club abstracts : n=27 mean 1.13 club body prose: n=298 mean 1.38 club gap : -0.25 PART 6 -- THE SPREAD, WHICH IS THE POINT ---------------------------------------- Five papers whose abstract is MOST inflated relative to Results: pmcid journal abs res gap PMC8913741 Nature Communications 42.68 -3.37 +46.06 PMC8846514 PLoS Biology 53.06 10.19 +42.87 PMC9090908 Nature Communications 28.57 -4.27 +32.85 PMC9448324 eLife 30.00 2.26 +27.74 PMC8748686 Nature Communications 36.84 10.53 +26.31 Five whose abstract is MORE cautious than their own Results: PMC8993826 Scientific Reports -37.50 -0.44 -37.06 PMC8976074 Nature Communications -21.16 13.46 -34.62 PMC9346371 Royal Society Open Science -14.18 18.50 -32.69 PMC10543445 Scientific Reports 12.99 40.68 -27.69 PMC9561184 Scientific Reports -16.13 10.00 -26.13 So the gap is a tendency across a population, not a verdict on any single paper. 32 of 67 abstracts are more cautious than the Results they summarise. Use the number as a prior, then check the paper. PART 7 -- NUMBERS USED IN THE ARTICLE FIGURES --------------------------------------------- { "fig1_section_median_words": { "abstract": 206, "introduction": 704, "methods": 1652, "results": 3148, "discussion": 1194 }, "fig2_sentence_hist_edges": [ 0, 8, 12, 16, 20, 24, 28, 32, 40, 60, 10000 ], "fig2_sentence_hist_pct": { "B:abstract": [ 3.7, 5.9, 13.8, 18.4, 16.9, 15.4, 9.9, 9.5, 5.8, 0.6 ], "B:results": [ 7.8, 9.3, 12.6, 13.3, 12.8, 11.4, 8.7, 11.5, 9.9, 2.7 ], "A:club prose": [ 24.0, 19.1, 14.5, 10.9, 9.0, 7.1, 5.4, 6.6, 3.1, 0.3 ] }, "fig2_sentence_mean": { "B:abstract": 22.9, "B:results": 24.6, "B:discussion": 26.9, "B:methods": 24.9, "A:club prose": 16.1 }, "fig3_pairs": [ [ -7.97, 10.61 ], [ -2.14, -0.73 ], [ -6.54, 9.93 ], [ 15.65, 19.24 ], [ 14.93, 13.84 ], [ 19.23, 0.68 ], [ 15.08, 2.75 ], [ 20.13, 13.08 ], [ 13.61, 19.83 ], [ 7.94, 11.28 ], [ 26.18, 8.2 ], [ 30.0, 2.26 ], [ 10.15, -1.85 ], [ 0.0, 11.09 ], [ 53.06, 10.19 ], [ 7.84, 16.83 ], [ 3.85, 6.2 ], [ 6.45, 2.19 ], [ 27.32, 19.34 ], [ 10.64, 23.87 ], [ 42.68, -3.37 ], [ 6.17, 11.51 ], [ 36.84, 10.53 ], [ -21.16, 13.46 ], [ 35.9, 21.55 ], [ 23.04, 41.04 ], [ -5.52, 8.5 ], [ 42.45, 16.78 ], [ 28.57, -4.27 ], [ -37.5, -0.44 ], [ 4.72, 3.99 ], [ -7.35, -9.27 ], [ -4.69, 14.9 ], [ 24.63, 12.29 ], [ 0.0, 7.16 ], [ 10.53, 30.23 ], [ -16.13, 10.0 ], [ 17.99, 11.23 ], [ 12.99, 40.68 ], [ 12.77, -7.89 ], [ 17.14, 16.19 ], [ 6.01, 4.41 ], [ 14.39, 35.0 ], [ 6.87, 11.39 ], [ 5.18, 0.0 ], [ 12.66, 9.04 ], [ 0.0, 7.63 ], [ -16.04, -1.32 ], [ 14.71, 5.79 ], [ 16.39, 9.04 ], [ 27.12, 17.39 ], [ 10.6, 9.57 ], [ 0.0, 19.39 ], [ 13.61, 0.0 ], [ 0.0, 4.08 ], [ 22.47, 6.93 ], [ 13.11, 13.66 ], [ 11.15, 0.85 ], [ 26.76, 8.73 ], [ 19.51, 6.35 ], [ 15.97, 31.83 ], [ 3.83, 5.01 ], [ 22.66, 8.7 ], [ -14.56, -4.52 ], [ -14.18, 18.5 ], [ -15.33, -15.52 ], [ 0.0, 14.61 ] ], "fig3_summary": { "abs_mean": 10.33, "res_mean": 10.0, "gap": 0.33, "se": 2.12, "t": 0.16, "n": 67, "pct_up": 52 }, "fig4_hedge_mean": { "B:abstract": 5.16, "B:introduction": 5.58, "B:methods": 2.62, "B:results": 4.16, "B:discussion": 11.3, "A:club prose": 3.24 }, "fig4_ratio_med": { "B:abstract": 0.75, "B:introduction": 1.07, "B:methods": 9.0, "B:results": 7.58, "B:discussion": 1.27, "A:club prose": 2.0 } } HEADLINE NUMBERS FOR THE ARTICLE -------------------------------- Corpus B papers analysed : 67 (of 96 candidates) Corpus A club sections analysed : 298 from 27 articles Median abstract / Results length: 206 / 3148 words Compression factor : 15.3x Mean sentence, abstract/Results : 22.9 / 24.6 words Mean sentence, club prose : 16.1 words Hedges per 1k, abstract/Results/Discussion : 5.16 / 4.16 / 11.30 R:I ratio, Results / abstract / Discussion : 7.58 / 0.75 / 1.27 Claim strength, abstract vs Results : 10.33 vs 10.00 per 1k Paired gap : +0.33, t = 0.16, n = 67 Abstracts more assertive than own Results : 52% of papers Assertive share, abstract vs Results : 0.71 vs 0.74 LIMITATIONS, STATED BY US ABOUT US ---------------------------------- 1. Lexicon counting is crude. It cannot read negation, so a sentence saying 'we did not show' scores as assertive. 2. The sample is most-cited open-access papers in eight journals, which over-represents methods and tools in the life sciences. Corpus B has 67 papers. It is not science as a whole. 3. Results sections legitimately use fewer claim verbs, because their job is reporting. Some of the measured gap is genre, not spin, and we cannot separate the two with word counts alone. 4. The club corpus is 100% written by the same small group, so it is a style sample of size one, not 27. ==========================================================================