============================================================================== THE INSECT DECLINE LITERATURE DISAGREES WITH ITSELF Science Journaling Club -- random-effects pooling and diagnostics Effect scale: rho = natural-log annual rate of change in total abundance or biomass. percent per year = 100*(exp(rho)-1). ============================================================================== PART 0 -- VALIDATING THE MACHINERY BEFORE POINTING IT AT ANYTHING ------------------------------------------------------------------------------ Check 0a. Recovery of a known effect from synthetic data. true mu = -0.025000 (-2.469% per year) true tau = 0.015000 k = 14 synthetic studies pooled RE estimate = -0.022530 95% CI [-0.030543, -0.014517] tau-hat = 0.013258 (I2 = 84.4%) truth inside the stated 95% interval? YES 2000-replicate coverage of the 95% interval = 91.3% 2000-replicate mean tau-hat = 0.014253 (true 0.015000) [DL is known to under-cover a little when k is small and tau is large. High 80s to mid 90s is the documented behaviour of this estimator, not a coding error.] Check 0b. Fixed and random effects must coincide when tau^2 = 0. k = 6, all effects identical, so Q must be exactly 0. Q = 0.000e+00 tau^2 = 0.000e+00 FE estimate = -0.020000000000 SE 0.002036633792 RE estimate = -0.020000000000 SE 0.002036633792 |FE - RE| = 0.000e+00 |SE_FE - SE_RE| = 0.000e+00 identical to machine precision? YES Same check with dispersion small but not zero, so that tau^2 is clamped to 0 by the max(0, .) rather than landing there by construction: Q = 0.0101 on 5 df -> tau^2 = 0.000e+00 |FE - RE| = 0.000e+00 identical to machine precision? YES PART 1 -- THE EXTRACTED STUDIES ------------------------------------------------------------------------------ study span rho SE %/yr Hallmann 2017 1989-2016 -0.06300 0.00200 -6.11 Shortall 2009 Hereford 1973-2002 -0.04340 0.00944 -4.25 Shortall 2009 Rothamsted 1973-2002 +0.01283 0.00942 +1.29 Shortall 2009 Starcross 1973-2002 -0.00500 0.00610 -0.50 Shortall 2009 Wye 1973-2002 -0.00297 0.00610 -0.30 Hallmann 2020 Kaaistoep moths 1997-2017 -0.04000 0.00600 -3.92 Hallmann 2020 Kaaistoep beetles 1997-2017 -0.04800 0.01000 -4.69 Hallmann 2020 Wijster carabids 1986-2016 -0.04400 0.00600 -4.30 Wepprich 2019 Ohio 1996-2016 -0.02000 0.00500 -1.98 Edwards 2025 contiguous US 2000-2020 -0.01309 0.00543 -1.30 Weiss 2024 NE Germany 1999-2022 -0.03149 0.01133 -3.10 Provenance of every number above: [hallmann17] PLoS ONE 12(10):e0185809, Results: 'annual trend coefficient = -0.063, sd = 0.002, i.e. 6.1% annual decline' [shortall_here] Insect Conserv. Divers. 2:251-260, Table 1 'Biomass Hereford': slope -0.01885, SE 0.00410 (log10 g per sample) [shortall_roth] Insect Conserv. Divers. 2:251-260, Table 1 'Biomass Rothamsted': slope +0.00557, SE 0.00409 (log10) [shortall_star] Insect Conserv. Divers. 2:251-260, Table 1 'Biomass Starcross': slope -0.00217, SE 0.00265 (log10) [shortall_wye] Insect Conserv. Divers. 2:251-260, Table 1 'Biomass Wye': slope -0.00129, SE 0.00265 (log10) [hallmann20_lep] Insect Conserv. Divers. 13:127-139, Table 2 Lepidoptera: rho -0.040 (0.006) [hallmann20_col] Insect Conserv. Divers. 13:127-139, Table 2 Coleoptera: rho -0.048 (0.010) [hallmann20_car] Insect Conserv. Divers. 13:127-139, Results: 'rho = -0.044, se = 0.006, P < 0.001, 4% decline per year' [wepprich19] PLoS ONE 14(7):e0216270, Results: 'declined at an annual rate of 2.0% (b1 = -0.020, std. err. 0.005, p < 0.001)' [edwards25] Science 387:1090-1094: 'a rate of 1.3% annually [95% confidence interval: -2.3%, -0.2%]' [weiss24] Ecography 2024:e07020, Abstract: 'significant linear declines in abundance and biomass with annual rates of -3.1% (95% CI [-5.3, -1]) and -4.9% (95% CI [-9.4, -1.6])' PART 2 -- POOLING ------------------------------------------------------------------------------ k = 11 effects Fixed-effect estimate = -0.04352 (-4.26%/yr) SE 0.00145 Cochran's Q = 269.96 on 10 df p = 3.407e-52 DerSimonian-Laird tau^2 = 0.000791 tau = 0.02813 I-squared = 96.3% H-squared (Q/df) = 27.00 RANDOM-EFFECTS POOLED ESTIMATE rho = -0.02717 95% CI [-0.04435, -0.00999] per year = -2.68% 95% CI [-4.34%, -0.99%] per decade = -23.8% 95% CI [-35.8%, -9.5%] z = -3.10, p = 1.94e-03 95% PREDICTION INTERVAL for the next comparable study rho in [-0.08492, +0.03057] = [-8.14%, +3.10%] per year [The confidence interval describes the mean. The prediction interval describes a study. They are different objects and the second is the honest one to quote at a reader who wants to know what their local woodland is doing.] Random-effects weights, percent of total: Hallmann 2017 9.66% Shortall 2009 Hereford 8.73% Shortall 2009 Rothamsted 8.73% Shortall 2009 Starcross 9.27% Shortall 2009 Wye 9.27% Hallmann 2020 Kaaistoep moths 9.29% Hallmann 2020 Kaaistoep beetles 8.62% Hallmann 2020 Wijster carabids 9.29% Wepprich 2019 Ohio 9.41% Edwards 2025 contiguous US 9.36% Weiss 2024 NE Germany 8.36% For contrast, the FIXED-effect weights, which is why nobody should be using a fixed-effect model on this literature: Hallmann 2017 52.79% Shortall 2009 Hereford 2.37% Shortall 2009 Rothamsted 2.38% Shortall 2009 Starcross 5.67% Shortall 2009 Wye 5.67% Hallmann 2020 Kaaistoep moths 5.87% Hallmann 2020 Kaaistoep beetles 2.11% Hallmann 2020 Wijster carabids 5.87% Wepprich 2019 Ohio 8.45% Edwards 2025 contiguous US 7.17% Weiss 2024 NE Germany 1.65% PART 3 -- SMALL-STUDY ASYMMETRY ------------------------------------------------------------------------------ Egger's regression intercept = +6.8747 (SE 2.1755) t = +3.160 on 9 df, p = 0.0115 regression slope = -0.07177 (-6.93%/yr): the effect this test extrapolates to a study of infinite precision. [A significant intercept says the funnel is lopsided. It does NOT say why. With k=11 and one study an order of magnitude more precise than the rest, this test has almost no power and almost no interpretability. We report it because we said we would, and then we do not lean on it.] Funnel coordinates, most precise first: study rho SE 1/SE Hallmann 2017 -0.06300 0.00200 500.0 Wepprich 2019 Ohio -0.02000 0.00500 200.0 Edwards 2025 contiguous US -0.01309 0.00543 184.3 Hallmann 2020 Kaaistoep moths -0.04000 0.00600 166.7 Hallmann 2020 Wijster carabids -0.04400 0.00600 166.7 Shortall 2009 Starcross -0.00500 0.00610 163.9 Shortall 2009 Wye -0.00297 0.00610 163.9 Shortall 2009 Rothamsted +0.01283 0.00942 106.2 Shortall 2009 Hereford -0.04340 0.00944 105.9 Hallmann 2020 Kaaistoep beetles -0.04800 0.01000 100.0 Weiss 2024 NE Germany -0.03149 0.01133 88.3 PART 4 -- LEAVE-ONE-OUT ------------------------------------------------------------------------------ omitted rho 95% CI I2 Hallmann 2017 -0.02313 [-0.03496,-0.01131] 87.0% Shortall 2009 Hereford -0.02561 [-0.04402,-0.00721] 96.7% Shortall 2009 Rothamsted -0.03099 [-0.04813,-0.01385] 96.1% Shortall 2009 Starcross -0.02945 [-0.04696,-0.01194] 96.0% Shortall 2009 Wye -0.02966 [-0.04700,-0.01232] 96.0% Hallmann 2020 Kaaistoep moths -0.02585 [-0.04490,-0.00681] 96.7% Hallmann 2020 Kaaistoep beetles -0.02520 [-0.04354,-0.00686] 96.7% Hallmann 2020 Wijster carabids -0.02544 [-0.04450,-0.00639] 96.7% Wepprich 2019 Ohio -0.02791 [-0.04657,-0.00925] 96.3% Edwards 2025 contiguous US -0.02863 [-0.04670,-0.01055] 96.2% Weiss 2024 NE Germany -0.02678 [-0.04498,-0.00857] 96.7% range of the pooled estimate across omissions: -0.03099 to -0.02313 i.e. -3.05%/yr to -2.29%/yr No single study moves the pooled estimate outside the interval reported in Part 2. PART 5 -- SENSITIVITY TO THE DEPENDENCE WE KNOW WE HAVE ------------------------------------------------------------------------------ Two papers contribute more than one effect. Shortall 2009 gives four traps; Hallmann 2020 gives three datasets from two localities. Re-pool keeping one effect per PAPER, then one per LOCALITY, then split by metric. one per paper (k= 6): rho -0.03529 [-0.05648,-0.01410] = -3.47%/yr I2 96.2% one per locality (k=10): rho -0.02520 [-0.04354,-0.00686] = -2.49%/yr I2 96.7% biomass only (k= 5): rho -0.02056 [-0.05532,+0.01420] = -2.04%/yr abundance only (k= 6): rho -0.03194 [-0.04394,-0.01994] = -3.14%/yr Europe only (k= 9): rho -0.02962 [-0.04929,-0.00995] = -2.92%/yr North America (k= 2): rho -0.01682 [-0.02403,-0.00962] = -1.67%/yr Excluding the single most precise study (Hallmann 2017, SE 0.0020, which carries 52.8% of the FIXED-effect weight and 9.7% of the random-effects weight): rho -0.02313 [-0.03496,-0.01131] = -2.29%/yr, I2 87.0% PART 6 -- HOW THE POOL COMPARES WITH THE PUBLISHED SYNTHESES ------------------------------------------------------------------------------ These are NOT in the pool. They are syntheses of overlapping primary data; including them would double-count. They are the external check on whether our number is sane. synthesis or benchmark %/yr %/decade van Klink 2020 terrestrial, as published -0.92 -8.8 van Klink 2020 terrestrial, erratum -1.11 -10.6 van Klink 2020 terrestrial, outliers removed -0.66 -6.4 van Klink 2020 terrestrial, North America out -0.49 -4.8 van Klink 2020 freshwater +1.08 +11.3 Haase 2023 European freshwater abundance +1.17 +12.3 Sockman 2025 subalpine Colorado, one meadow -6.60 -49.5 THIS POOL, k=11, random effects -2.68 -23.8 Difference between our pool and van Klink's corrected figure: -0.01601 in log units, a factor of 2.43 on the annual rate Over 30 years: 56% loss on our number versus 28% loss on theirs. Same literature. Different rooms. Is van Klink's corrected estimate inside our 95% CI? YES Is it inside our 95% PREDICTION interval? YES PART 7 -- SITE SELECTION BIAS WITH NO REAL TREND UNDERNEATH ------------------------------------------------------------------------------ 400 sites, 27 years, permanent site quality SD 1.0, annual noise SD 0.6, TRUE trend exactly zero. Sites are chosen on their first year's observed abundance. Mean fitted log-linear slope: kept n slope %/yr 27-yr total top 100.0 % 400 -0.00050 -0.05 -1.3% top 75.0 % 300 -0.00198 -0.20 -5.0% top 50.0 % 200 -0.00214 -0.21 -5.4% top 25.0 % 100 -0.00335 -0.33 -8.3% top 10.0 % 40 -0.00396 -0.40 -9.8% top 5.0 % 20 -0.00767 -0.76 -18.1% top 2.5 % 10 -0.00732 -0.73 -17.3% Repeat with a REAL decline of 1%/yr underneath, to see how much selection adds on top of a true signal: kept n slope %/yr 27-yr total top 100.0 % 400 -0.01055 -1.05 -24.0% top 50.0 % 200 -0.01219 -1.21 -27.2% top 10.0 % 40 -0.01401 -1.39 -30.5% top 2.5 % 10 -0.01737 -1.72 -36.3% And again with year-to-year noise turned down to SD 0.2, which is what you would have if traps were quiet and weather did not matter. Regression to the mean needs noise to work with: kept n slope %/yr top 100.0 % 400 -0.00017 -0.02 top 10.0 % 40 -0.00065 -0.07 top 2.5 % 10 -0.00142 -0.14 And once more with the noise turned UP to SD 1.0, which is what a light trap in a bad summer looks like: kept n slope %/yr 27-yr total top 100.0 % 400 -0.00083 -0.08 -2.1% top 50.0 % 200 -0.00582 -0.58 -14.0% top 10.0 % 40 -0.01070 -1.06 -24.3% top 2.5 % 10 -0.01917 -1.90 -39.3% Full grid, for the figure in the article. Rows are the fraction of sites kept; columns are the year-to-year noise SD. Entries are the mean fitted trend in percent per year, true trend zero. Each entry is averaged over 24 independent seeds, because a single run of 10 surviving sites is itself a noisy estimate and we are not going to draw a wobble and call it a mechanism. kept sd=0.2 sd=0.6 sd=1.0 100.0% -0.001 -0.003 -0.006 75.0% -0.016 -0.112 -0.239 50.0% -0.028 -0.194 -0.450 35.0% -0.031 -0.250 -0.556 25.0% -0.032 -0.285 -0.674 15.0% -0.041 -0.344 -0.827 10.0% -0.052 -0.391 -0.971 5.0% -0.058 -0.490 -1.136 2.5% -0.073 -0.551 -1.363 How much of OUR pooled estimate could this account for? pooled estimate -0.02717 (-2.68% per year) top-10% selection -0.00396 (-0.40% per year) share explained 14.6% [That share is an upper bound under a deliberately unkind model. It is not nothing, and it is not the whole thing.] ============================================================================== END. Every printed number is reproducible by running this file. ==============================================================================