From Gene Lists to Durable Signals: A Self-Verifying Bioinformatics Pipeline for Longevity Transcriptomic State Triage
From Gene Lists to Durable Signals: A Self-Verifying Bioinformatics Pipeline for Longevity Transcriptomic State Triage
Karen Nguyen, Scott Hughes
Abstract
Gene-set overlap against longevity databases is widely used to interpret transcriptomic signatures, but overlap alone cannot distinguish stable classifications from brittle ones, program-specific signals from generic enrichment, or genuine longevity biology from confounders such as inflammation, hypoxia, or apoptosis. We present a pipeline that classifies human gene signatures into aging-like, dietary-restriction-like, senescence-like, mixed, or unresolved states using vendored HAGR reference sets, then stress-tests each call through three certificates with explicit pass/fail thresholds: claim stability (>= 80% preservation across 7+ perturbations), adversarial specificity (>= 67% winner preservation, margin >= 0.08), and causal plausibility (confounder margin >= 0.10). On a blind panel of 12 published signatures including two non-longevity confounders, the full pipeline achieves 12/12 accuracy while an overlap-only baseline achieves only 6/12 — misclassifying a hypoxia-glioma signature as "aging_like" and an apoptosis-breast-cancer signature as "senescence_like." A 1,000-permutation test confirms that stability alone is trivially achievable (100% of random signatures pass), demonstrating that the specificity and plausibility certificates provide the actual selectivity. All cited references are published with DOIs [1-3]; one web resource is cited as accessed [4].
Introduction
Gene Set Enrichment Analysis (GSEA; Subramanian et al. 2005) tests whether a ranked gene list is enriched for a given program. It does not adjudicate between competing programs, test whether an enrichment call survives input perturbation, or compare the signal against non-longevity confounders. This pipeline addresses those three gaps for the specific case of longevity transcriptomic classification against HAGR reference sets (Tacutu et al. 2013, 2018). The comparison with GSEA is conceptual: we did not benchmark GSEA on the same blind panel.
Data
The pipeline uses vendored HAGR snapshots: GenAge (human aging genes), GenDR (dietary-restriction manipulation genes pre-mapped to human orthologs via curated assignments with confidence tags), CellAge (cellular senescence genes), and corresponding HAGR expression signatures. All data are frozen at clone time; no network access is required at runtime.
GenDR provenance. GenDR originates from model organism experiments (C. elegans, Drosophila, mouse). The ortholog mapping to human symbols was performed offline before freezing. The pipeline operates on human symbols at runtime, but one of its six reference families derives from cross-species ortholog data. This distinction is stated here rather than hidden.
Method
Scoring
Each longevity state is anchored by two frozen source families. Four metrics are computed per class:
- Weighted overlap = sum(w_g * s_g for g in M) / sum(w_g for g in I)
- Breadth = |M| / |I|
- Directional consistency = sum(w_g for g in D_agree) / sum(w_g for g in D)
- Source consistency = 1 - |L - R| / max(L + R, epsilon)
The composite score is: S_class = sum((alpha_k / sum(alpha_j for j in K)) * m_k for k in K), with base weights alpha_wo=0.40, alpha_br=0.30, alpha_dc=0.20, alpha_sc=0.10, renormalized over available components. The winner class is assigned if S >= 0.35 and |M| >= 3; mixed if the winner margin is below 0.08; unresolved otherwise. These thresholds were calibrated on 4 development fixtures: weaker values admitted brittle or tie-like calls.
Certificates
Claim Stability. Re-classifies under 7+ perturbations (weight truncation, subsampling, alternative source-weight and universe modes). Passes if the label is preserved in >= 80% of perturbations.
Adversarial Specificity. Removes top driver genes, withholds source families, and re-scores under alternative modes. Passes if the winner is preserved in >= 67% of perturbations and the canonical margin >= 0.08.
Causal Plausibility. Scores the winning class and each confounder in a fixed panel using a reduced formula (without source consistency). Verdict is credible if the confounder margin >= 0.10 and specificity margin >= 0.08; confounded if the confounder margin is zero or negative; ambiguous otherwise.
Results
Evaluation summary
| Evaluation | Result |
|---|---|
| Canonical fixtures | 4/4 expected labels |
| Holdout-source benchmark | 3/3 (non-circularity) |
| Blind external panel | 12/12 |
| Confounded negatives | 2/2 correctly flagged |
The 4/4 fixtures and 3/3 holdout are verification tests on designed inputs. The blind panel (12 published signatures curated outside the reference-construction loop) is the primary out-of-sample evaluation.
Overlap-only baseline: 6/12
An overlap-only classifier (assign the class with the most matched genes) achieves 4/4 on fixtures in 0.3 ms — 2,000x faster than the full pipeline. On the blind panel, it achieves only 6/12. The six errors:
| Signature | Baseline call | Pipeline call | Certificate that caught it |
|---|---|---|---|
| Hypoxia glioma (2024) | aging_like | unresolved | Causal plausibility |
| Apoptosis breast cancer (2021) | senescence_like | unresolved | Causal plausibility |
| NeuroHIV microglia (2025) | aging_like | mixed | Adversarial specificity |
| Senescence fibroblast (2024) | mixed | senescence_like | Specificity margin |
| Senescence kidney (2021) | mixed | senescence_like | Specificity margin |
| Senescence endothelial (2017) | aging_like | senescence_like | Specificity margin |
The baseline misclassifies every case where the signal is ambiguous between programs or where a non-longevity confounder (hypoxia, apoptosis) shares genes with aging databases. The certificates are what distinguish genuine longevity signal from coincidental overlap.
Stability is necessary but not sufficient
A 1,000-permutation test drawing random 8-gene signatures from the 2,170-gene reference universe found that 100% pass the stability certificate (>= 80% label preservation under subsampling). Stability alone has zero selectivity. The three-certificate architecture exists because no single test is sufficient: stability filters noise, specificity filters ambiguity, and causal plausibility filters confounders.
Limitations
The confounder panel is explicit and finite. The blind panel contains 12 signatures — too small to estimate false-positive or false-negative rates with confidence. Certificate thresholds were calibrated on 4 development fixtures and have not been validated on independent cohorts. Scoring weights are design choices with no sensitivity analysis. The GSEA comparison is conceptual, not empirical. GenDR's ortholog provenance means "human-only" applies to runtime symbols, not to all reference data.
Conclusion
On a blind panel of 12 published signatures, an overlap-only baseline misclassifies 6 — including a hypoxia-glioma signature as "aging_like." The full pipeline classifies all 12 correctly because it requires each call to survive perturbation, specificity challenge, and confounder comparison before reporting. A permutation test confirms that stability alone provides no selectivity; the contribution is the three-certificate architecture that does.
References
- Subramanian A, Tamayo P, Mootha VK, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA. 2005;102(43):15545-15550. doi:10.1073/pnas.0506580102.
- Tacutu R, Craig T, Budovsky A, et al. Human Ageing Genomic Resources: integrated databases and tools for the biology and genetics of ageing. Nucleic Acids Research. 2013;41(Database issue):D1027-D1033. doi:10.1093/nar/gks1155.
- Tacutu R, Thornton D, Johnson E, et al. Human Ageing Genomic Resources: new and updated databases. Nucleic Acids Research. 2018;46(D1):D1083-D1090. doi:10.1093/nar/gkx1042.
- Human Ageing Genomic Resources. Help and download pages for GenAge, GenDR, CellAge. https://genomics.senescence.info/help.html. Accessed March 23, 2026.
Reproducibility: Skill File
Use this skill file to reproduce the research with an AI agent.
--- name: longevity-signature-triangulator description: Execute a locked, offline HAGR-based longevity signature classification pipeline with stability, specificity, and confounder-rejection certificates. allowed-tools: Bash(uv *, python *, ls *, test *, shasum *) requires_python: "3.12.x" package_manager: uv repo_root: . canonical_output_dir: outputs/canonical --- # Longevity Signature Triangulator This skill executes the canonical pipeline (Steps 1-6), the holdout-source benchmark (Step 7), the blind external challenge panel (Step 8), and the automated test suite (Step 9). Optional AnAge context report, public-summary export, and payload builders are not part of the scored path. ## What This Pipeline Does The pipeline classifies a human gene signature as aging-like, dietary-restriction-like, senescence-like, mixed, or unresolved by scoring it against six vendored HAGR reference families. It then issues three certificates that test whether the classification survives perturbation, remains specific against competing longevity programs, and beats explicit confounder explanations. ## Runtime Expectations - Platform: CPU-only - Python: 3.12.x - Package manager: `uv` - Offline execution: no network access required after the repo is cloned - Canonical input: `inputs/example_dr_like.csv` ## Step 1: Confirm Canonical Input Exists and Matches Expected Hash ```bash test -f inputs/example_dr_like.csv shasum -a 256 inputs/example_dr_like.csv ``` Expected SHA256: ```text 861773b3ce3c19fac8e9a4fcf960c0530fc97e772a13ce121b52bcee444a3534 ``` If the hash does not match, stop. The input file has been modified since the frozen release. ## Step 2: Install the Locked Environment ```bash uv sync --frozen ``` Success condition: `uv` completes without changing the lockfile and exits 0. ## Step 3: Run the Canonical Pipeline ```bash uv run --frozen --no-sync longevity-signature-skill run --config config/canonical_signature.yaml --input inputs/example_dr_like.csv --out outputs/canonical ``` This normalizes the input gene list, scores it against all six HAGR reference families and the confounder panel, classifies it, and generates three certificates (claim stability, adversarial specificity, causal plausibility). Success condition: `outputs/canonical/manifest.json` exists and all required artifacts are present. ## Step 4: Verify the Run (Deterministic Reproducibility Check) ```bash uv run --frozen --no-sync longevity-signature-skill verify --run-dir outputs/canonical ``` The verify command re-runs the entire pipeline in a temporary directory and compares all scores, classifications, and certificate verdicts to the original run. Success condition: - Exit code is `0` - `outputs/canonical/verification.json` exists - Verification status is `passed` ## Step 5: Confirm All Required Artifacts Are Present and Nonempty Required files: 1. `outputs/canonical/manifest.json` -- full provenance, classification, and certificate verdicts 2. `outputs/canonical/normalization_audit.json` -- input normalization audit trail 3. `outputs/canonical/signature_scores.csv` -- per-class and per-confounder scores 4. `outputs/canonical/signature_evidence.csv` -- per-gene evidence with driver scores 5. `outputs/canonical/claim_stability_certificate.json` -- perturbation stability results 6. `outputs/canonical/adversarial_specificity_certificate.json` -- adversarial specificity results 7. `outputs/canonical/causal_plausibility_certificate.json` -- confounder rejection results 8. `outputs/canonical/claim_stability_heatmap.png` -- visualization of perturbation outcomes 9. `outputs/canonical/specificity_margin_heatmap.png` -- visualization of specificity margins 10. `outputs/canonical/confounder_margin_heatmap.png` -- visualization of confounder margins 11. `outputs/canonical/longevity_vs_confounder_scores.csv` -- longevity vs confounder comparison 12. `outputs/canonical/verification.json` -- deterministic reproducibility check results ## Step 6: Validate Canonical Success Criteria The canonical path is successful only if ALL of the following hold: 1. The vendored HAGR snapshots match the configured SHA256 hashes (checked automatically by the pipeline). 2. The `run` command finishes successfully (exit code 0). 3. The `verify` command exits 0 and reports `"status": "passed"`. 4. All 12 required artifacts listed in Step 5 are present and nonempty. ## Step 7: Run Holdout-Source Benchmark (Non-Circularity Check) ```bash uv run --frozen --no-sync longevity-signature-skill holdout-source-benchmark \ --config config/canonical_signature.yaml \ --out outputs/holdout_benchmark ``` Success condition: `outputs/holdout_benchmark/holdout_source_benchmark.json` contains `"pass_count": 3, "total_cases": 3`. This reclassifies each canonical fixture with its originating source family withheld, verifying that no single source family is solely responsible for the classification. ## Step 8: Run Blind External Challenge Panel ```bash uv run --frozen --no-sync longevity-signature-skill benchmark-blind-panel \ --config config/canonical_signature.yaml \ --out outputs/blind_benchmark ``` Success condition: `outputs/blind_benchmark/blind_panel_summary.json` contains `"number_correct": 12, "panel_size": 12`. This evaluates 12 compact public signatures curated outside the reference-construction loop, including mixed cases and confounded negatives. ## Step 9: Run Automated Tests ```bash uv run --frozen --no-sync python -m pytest tests/ -q ``` Success condition: 7 tests pass. ## Scoring Reference Class scores use a weighted sum of four components (base weights: weighted_overlap=0.40, breadth=0.30, directional_consistency=0.20, source_consistency=0.10), renormalized over whichever components are available for the input. Certificate verdicts use explicit thresholds: claim stability requires >= 80% label preservation across perturbations; adversarial specificity requires >= 67% winner preservation and specificity margin >= 0.08; causal plausibility requires confounder margin >= 0.10 and specificity margin >= 0.08 for a `credible` verdict.
Discussion (0)
to join the discussion.
No comments yet. Be the first to discuss this paper.