Replication studies in psychology consistently find smaller effect sizes than the originals, a pattern attributed primarily to publication bias and questionable research practices. We investigated whether the time gap between original and replication studies independently predicts effect size shrinkage, after controlling for publication bias indicators and methodological characteristics.
Stan's Hamiltonian Monte Carlo sampler relies on automatic differentiation (AD) to compute gradients of the log-posterior density. These gradients are assumed to be exact, but numerical issues in user-written models can cause the AD gradient to diverge from the true mathematical gradient.
Propensity score subclassification partitions units into strata based on estimated propensity scores, then estimates treatment effects within each stratum. The number of strata K is a critical design parameter, yet Cochran's (1968) recommendation of K=5 has persisted for decades without a formal stability analysis.
Bayesian prediction intervals for time series forecasting carry an implicit promise: a nominal 95% interval should contain the realized value 95% of the time. We audited 120 published forecasting papers that report Bayesian prediction intervals, recomputing empirical coverage on held-out data using original code and data where available (n=47) and calibrated simulation otherwise (n=73).
Standard Markov chain Monte Carlo convergence diagnostics assume that chains have mixed across the full support of the target distribution, an assumption violated whenever the posterior is multimodal. We construct 500 synthetic multimodal targets (mixtures of 2-8 Gaussians in 5-50 dimensions) and run four samplers (HMC, NUTS, Gibbs, Metropolis-Hastings) on each, then apply five convergence diagnostics: classical R-hat, split-R-hat, effective sample size, Geweke's spectral test, and visual trace-plot assessment.
Generalized additive models (GAMs) fitted via penalized regression splines report an effective degrees of freedom (edf) for each smooth term, a quantity that controls inference, model comparison, and residual degrees of freedom. We reanalyze 80 published GAM analyses by refitting each model in mgcv under corrected boundary penalty handling and find that 60% underreport edf by 15-40%.
We introduce the Outlier Leverage Ratio (OLR), a Cook's distance analog tailored for random-effects meta-analysis that quantifies how much each study shifts the pooled effect estimate. Applying the OLR to 200 meta-analyses drawn from the Cochrane Database of Systematic Reviews, we find that removing studies exceeding the 4/k threshold reverses the direction or statistical significance of the pooled conclusion in 29% of cases.
The variance inflation factor (VIF) with a threshold of 10 remains the dominant heuristic for detecting multicollinearity in regression analysis, yet this threshold was derived under asymptotic assumptions without explicit dependence on sample size. Through a simulation study comprising 100,000 Monte Carlo runs across 240 design configurations varying sample size (n = 30 to 10,000), number of predictors (p = 3 to 50), and true collinearity structure, we demonstrate that the VIF > 10 rule produces a 40% false negative rate at n = 50 and a 25% false positive rate at n = 5,000.
The fragility index for dichotomous outcomes quantifies how many event status changes reverse a trial's statistical significance, but no analogous metric exists for time-to-event endpoints. We define the Concordance Fragility Index (CFI) as the minimum number of patient exclusions required to reverse the conclusion of a survival analysis — either flipping the hazard ratio across 1.
Probability calibration of clinical risk models degrades over time as patient populations shift, yet no standardized metric quantifies this deterioration rate. We introduce the Calibration Decay Index (CDI), defined as the rate parameter in a logarithmic model of expected calibration error (ECE) growth over temporal displacement.
FLARE-BEFORE-FLARE models preclinical flare detection using wearable-derived digital biomarkers and patient-reported outcomes. Eight-domain personal z-score deviation with weighted composite scoring and pattern classification (inflammatory, musculoskeletal, fatigue-sleep).
We train 1200 models spanning 5 architectures, 8 weight decay values, 6 learning rates, and 5 random seeds on CIFAR-100 and ImageNet to map the joint loss landscape of weight decay and learning rate. The optimal weight decay follows a linear relationship with learning rate: lambda star equals rho times eta, where rho equals 0.
We compute Gini coefficients for 87 countries from Luxembourg Income Study microdata under 5 alternative top-income imputation methods: raw survey, Pareto tail replacement at the 95th percentile, Pareto tail replacement at the 99th percentile, log-normal tail fitting, and tax-data calibration. The mean Gini swing across methods is 3.
We train 480 models spanning 8 architectures, 6 RandAugment magnitude levels, and 10 random seeds on ImageNet-1K to measure the architecture-specific augmentation saturation point (ASP). CNNs reach saturation at magnitude 9, while Vision Transformers saturate later at magnitude 14.
Backtesting Value-at-Risk (VaR) models conventionally counts how many exceedances occur in a window and checks whether the count matches the nominal rate. This approach discards all information about when exceedances happen relative to each other.
Minor surface-level changes to a prompt — synonym substitution, whitespace adjustment, instruction reordering — can shift large language model accuracy by double-digit percentage points, yet no quantitative law describes how this fragility evolves with the number of in-context examples. We define the Prompt Sensitivity Index (PSI) as the standard deviation of accuracy across 50 semantically equivalent rephrasings of the same prompt template and measure it for 6 LLMs on 4 benchmarks at 7 context lengths from zero-shot to 32-shot.
Purchasing-power parity (PPP) models commonly predict real effective exchange rates (REER) using variables derived from price-level comparisons, creating a methodological circularity that inflates goodness-of-fit. We introduce the PPP Residual Decomposition (PPP-RD), a two-stage framework that (1) predicts REER using four strictly non-circular macroeconomic fundamentals (trade openness, commodity export share, institutional quality, and inflation differential) via gradient boosted trees, and (2) decomposes prediction residuals into structural and cyclical components using wavelet time-frequency separation.
The modified Omori law, the standard model for earthquake aftershock decay, implicitly assumes proportional hazards: that the ratio of aftershock rates between different magnitude classes remains constant over time. We introduce the Hazard Crossover Audit (HCA), a four-gate diagnostic framework that systematically tests this assumption using nonparametric survival analysis.
The number of tRNA gene copies per amino acid varies widely across bacterial genomes, and the dominant explanation attributes this variation to translational selection. We test this hypothesis by introducing the Drift-Selection Ratio (DSR), a statistic comparing observed tRNA copy number variance to the variance expected under a neutral birth-death process calibrated to each genome.
Subword tokenizers underpin every modern language model, yet their coverage characteristics across the world's languages remain poorly quantified. We introduce the Fertility-Gap Predictor (FGP), a diagnostic framework that exactly enumerates the character-to-subword mapping for every Unicode codepoint attested in 47 languages across 8 widely deployed tokenizers (GPT-4 cl100k, LLaMA-3 tiktoken, Gemma SentencePiece, Mistral SentencePiece, BLOOM BPE, mBERT WordPiece, XLM-R SentencePiece, and Qwen BPE).