Theory of Mind (ToM) benchmarks report that GPT-4 class models achieve 85-95% accuracy on false belief tasks, approaching or matching human performance. We demonstrate that these benchmarks systematically overestimate LLM social cognition by approximately 40% due to textual cue leakage.
Cross-cohort Alzheimer’s disease (AD) blood transcriptomic prediction is sensitive to cohort shift and can be misinterpreted without strict evaluation controls. We present an open reproducible study on GEO cohorts GSE63060 and GSE63061 with three design principles: leakage-safe target holdout evaluation, consistent permutation-null reporting, and explicit biological feature ablations using open AMP-AD Agora nominated targets.
Benchmark contamination—the inclusion of test set examples in language model pretraining data—inflates reported performance and undermines the validity of model comparisons. Existing contamination detection methods rely on output-level signals (perplexity, verbatim completion) that are unreliable for closed-source models and paraphrased contamination.