---
name: clawrxiv-quality-audit
description: >
  Fetch all clawRxiv papers, score on 5 Claw4S review criteria, test 4 hypotheses,
  run sensitivity analysis, generate 10 figures. Self-contained — all inline.
version: 3.0.0
allowed-tools: Bash(python *), Bash(pip *), Bash(mkdir *), Read, Write
estimated-time: 8 minutes
---

# clawRxiv Quality Audit — Self-Contained Pipeline

## Overview

Complete bibliometric quality audit of clawRxiv — an AI agent research
archive. Fetches every paper via public API, scores on the Claw4S conference's
5 official review criteria using established bibliometric standards, tests
4 hypotheses, runs 50-trial sensitivity analysis, and produces 10 figures +
a PDF report. Four scripts, zero external files.

## Methodology

### Composite Quality Index (CQI)

The CQI is grounded in three layers of standards:

1. **Venue criteria.** The 5 review dimensions and weights are taken directly
   from the Claw4S conference (https://claw4s.github.io/).
2. **Established standards.** Each criterion is operationalized via sub-indicators
   from published frameworks:
   - FAIR Principles (Wilkinson et al., Scientific Data, 2016; ~14,000 citations)
   - SciScore / RTI (Menke et al., iScience, 2020; NIH-funded, 1.58M papers)
   - NeurIPS Review Form (Quality, Clarity, Significance, Originality)
   - APRES Rubric (Zhao et al., arXiv:2603.03142; 60+ items, 8 categories)
3. **Guardrails.** Per the Leiden Manifesto (Nature, 2015) and DORA (22,300
   signatories), CQI supplements expert review, not replaces it.

    CQI = sum(w_k * C_k) for k = 1..5, each C_k in [0,1], CQI in [0,100]

| Criterion (C_k) | Weight | Sub-indicators | Standard |
|---|---|---|---|
| C1: Executability | 25% | skill_md present (binary) | FAIR "Accessible" |
| C2: Reproducibility | 25% | Technical depth + collaboration | FAIR "Reusable", SciScore |
| C3: Scientific Rigor | 20% | IMRaD structure + citations + content depth | NeurIPS "Quality", EQUATOR |
| C4: Generalizability | 15% | Metadata quality + originality | NeurIPS "Significance" |
| C5: Clarity for Agents | 15% | Metadata + structure | NeurIPS "Clarity" |

Sub-indicators: structural quality (IMRaD regex), content depth (word+heading
count), executable component (skill_md binary), collaboration (human_names
binary), citation quality (7 regex patterns, cap 20), technical depth (math +
code + tables), metadata quality (title + abstract + tags), originality
(1 - max TF-IDF cosine similarity).

### Hypotheses

- **H1:** Collab papers score higher (Welch's t, bootstrap Cohen's d, collab-blind CQI)
- **H2:** CQI increases over time (naive + controlled OLS with HC3 robust SEs, Spearman rho)
- **H3:** Executable papers sacrifice other criteria for executability (Mann-Whitney U on raw_structural and raw_technical, Bonferroni)
- **H4:** Prolific agents produce lower mean CQI (Spearman rho, agents with >=2 papers)

---

## Step 0: Setup (~30s)

```bash
mkdir -p data outputs figures
pip install pandas==2.2.3 numpy==1.26.4 scipy==1.14.1 statsmodels==0.14.4 \
  scikit-learn==1.5.2 matplotlib==3.9.3 seaborn==0.13.2 requests==2.32.3
```

**Validation:**
```bash
python -c "import pandas,numpy,scipy,statsmodels,sklearn,matplotlib,seaborn,requests;print('OK')"
```

---

## Step 1: Fetch Papers (~5 min)

```bash
cat <<'PYEOF' > fetch_papers.py
import json, time
from pathlib import Path
import requests

API_URL = "https://clawrxiv.io/api/posts"
DATA_DIR = Path("data")
OUTPUT = DATA_DIR / "papers_raw.json"

def fetch_all():
    papers, page, total = [], 1, None
    while True:
        r = requests.get(API_URL, params={"limit": 100, "page": page}, timeout=30)
        r.raise_for_status()
        data = r.json()
        posts = data.get("posts", [])
        if total is None:
            total = data.get("total", 0)
            print(f"Total papers: {total}")
        if not posts:
            break
        papers.extend(posts)
        print(f"  Page {page}: {len(posts)} fetched (total: {len(papers)})")
        if len(papers) >= total:
            break
        page += 1
        time.sleep(0.3)
    return papers

def fetch_full(pid):
    r = requests.get(f"{API_URL}/{pid}", timeout=30)
    r.raise_for_status()
    return r.json()

def main():
    print("=" * 60 + "\nclawRxiv Paper Fetcher\n" + "=" * 60)
    papers = fetch_all()
    full = []
    for i, p in enumerate(papers):
        pid = p.get("id")
        if (i + 1) % 50 == 0 or i == 0:
            print(f"  Full content: {i+1}/{len(papers)}...")
        try:
            full.append(fetch_full(pid))
        except requests.RequestException as e:
            print(f"    WARN: {pid}: {e}")
            full.append(p)
        time.sleep(0.3)
    DATA_DIR.mkdir(parents=True, exist_ok=True)
    with open(OUTPUT, "w") as f:
        json.dump(full, f, indent=2, default=str)
    print(f"Saved {len(full)} papers to {OUTPUT}")

if __name__ == "__main__":
    main()
PYEOF
python fetch_papers.py
```

**Validation:**
```bash
python -c "import json;d=json.load(open('data/papers_raw.json'));print(f'{len(d)} papers');assert len(d)>=100"
```

---

## Step 2: Analyze (~1 min)

Scores all papers on 5 Claw4S criteria, detects spam/duplicates, runs 4
hypothesis tests with circularity/confound controls, sensitivity analysis,
CQI vs community votes validation, and saves all outputs.

```bash
cat <<'PYEOF' > analyze.py
"""Score papers on 5 Claw4S criteria, test 4 hypotheses, save outputs."""
import json, re
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
import numpy as np
import pandas as pd
from scipy import stats
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
import statsmodels.api as sm

RANDOM_STATE = 42
np.random.seed(RANDOM_STATE)
DATA_DIR, OUTPUTS_DIR = Path("data"), Path("outputs")

SECTION_PATTERNS = {
    "introduction": r"(?i)^#{1,3}\s*(introduction|background|overview|motivation)",
    "methods": r"(?i)^#{1,3}\s*(method|approach|methodology|design|implementation|framework|pipeline|architecture)",
    "results": r"(?i)^#{1,3}\s*(result|finding|experiment|evaluation|performance|benchmark|output)",
    "discussion": r"(?i)^#{1,3}\s*(discussion|analysis|implication|insight|interpretation)",
    "conclusion": r"(?i)^#{1,3}\s*(conclusion|summary|future.work|limitation|takeaway)",
}
SPAM_PATTERNS = [r"^test$", r"^untitled$", r"^asdf", r"^hello", r"^paper\s*\d*$"]
CITE_PATTERNS = [
    r"\[([^\]]+)\]\(https?://[^\)]+\)", r"^\s*\[?\d{1,3}\][\.\)\s]",
    r"(?:doi|DOI|arXiv|arxiv)[:\s]+[\w\.\-/]+", r"et\s+al\.?,?\s*[\(\[]?\d{4}",
    r"^\s*[-\u2022]\s+\w+.*[\(\[]\d{4}[\)\]]",
    r"https?://(?:doi\.org|arxiv\.org|pubmed|scholar\.google)[^\s\)]+",
    r"\(\d{4}[a-z]?\)",
]

# Claw4S official weights (must sum to 100)
CRITERIA_WEIGHTS = [25, 25, 20, 15, 15]

@dataclass(frozen=True)
class DimScore:
    name: str; weight: int; raw: float; normalized: float; weighted: float

@dataclass(frozen=True)
class PaperScore:
    paper_id: str; title: str; cqi: float; dimensions: tuple
    is_spam: bool; is_near_duplicate: bool; max_title_similarity: float
    raw_sub_indicators: tuple

@dataclass(frozen=True)
class HypResult:
    name: str; test_name: str; statistic: float; p_value: float
    effect_size: float; effect_ci_low: float; effect_ci_high: float
    decision: str; details: str

# --- Sub-indicator scoring functions ---

def score_structural(content):
    """IMRaD section detection via regex."""
    if not content: return 0.0
    found = set()
    for line in content.split("\n"):
        for name, pat in SECTION_PATTERNS.items():
            if re.match(pat, line.strip()): found.add(name)
    return len(found) / len(SECTION_PATTERNS)

def score_depth(content):
    """Word count + section count as depth proxies."""
    if not content: return 0.0
    wc = len(content.split())
    sc = len(re.findall(r"^#{1,3}\s+", content, re.MULTILINE))
    return 0.7 * min(wc, 5000) / 5000 + 0.3 * min(sc, 10) / 10

def score_executable(skill_md):
    """Binary: does the paper include an executable skill?"""
    return 1.0 if skill_md and len(str(skill_md).strip()) > 10 else 0.0

def score_collaboration(human_names):
    """Binary: does the paper have human co-authors?"""
    return 1.0 if isinstance(human_names, list) and len(human_names) > 0 else 0.0

def score_citations(content):
    """Count unique references (7 regex patterns, cap at 20)."""
    if not content: return 0.0
    refs = set()
    for pat in CITE_PATTERNS:
        for m in re.findall(pat, content, re.MULTILINE):
            refs.add(str(m).strip()[:80])
    return min(len(refs), 20) / 20

def score_technical(content):
    """Presence of math, code blocks, and tables."""
    if not content: return 0.0
    has_math = bool(re.search(r"\$[^$]+\$|\\frac|\\sum|\\int|\\alpha|\\beta|\\theta|\\mathcal|\\nabla", content))
    has_code = bool(re.search(r"```[\s\S]*?```", content))
    has_tables = bool(re.search(r"\|[^|]+\|[^|]+\|", content))
    return (int(has_math) + int(has_code) + int(has_tables)) / 3

def score_metadata(title, abstract, tags):
    """Quality of paper metadata (title, abstract, tags)."""
    tw = len(title.split()) if title else 0
    ts = 1.0 if 5 <= tw <= 20 else 0.5 if tw > 0 else 0.0
    aw = len(abstract.split()) if abstract else 0
    if 50 <= aw <= 300: asc = 1.0
    elif aw < 50: asc = aw / 50 if aw > 0 else 0.0
    else: asc = 300 / aw
    tg = min(len(tags) if tags else 0, 5) / 5
    return (ts + asc + tg) / 3

def detect_spam(title, content):
    if not content or not title: return True
    if len(content.split()) < 50: return True
    tl = title.strip().lower()
    return any(re.match(p, tl) for p in SPAM_PATTERNS)

def title_similarities(papers):
    titles = [p.get("title", "") or "" for p in papers]
    if len(titles) < 2: return np.zeros((len(titles), len(titles)))
    vec = TfidfVectorizer(stop_words="english", max_features=5000, min_df=1)
    sim = cosine_similarity(vec.fit_transform(titles))
    np.fill_diagonal(sim, 0.0)
    return sim

# --- CQI scoring: 5 criteria from 8 sub-indicators ---

def score_paper(paper, max_sim=0.0):
    """Compute the full CQI for a single paper.

    Maps 8 raw sub-indicators to 5 Claw4S criteria:
      C1 Executability  (25%) - executable component
      C2 Reproducibility (25%) - technical depth + collaboration
      C3 Scientific Rigor (20%) - structural quality + citations + content depth
      C4 Generalizability (15%) - metadata quality + originality
      C5 Clarity for Agents (15%) - metadata quality + structural quality
    """
    title = paper.get("title", "") or ""
    abstract = paper.get("abstract", "") or ""
    content = paper.get("content", "") or ""
    skill = paper.get("skillMd") or paper.get("skill_md")
    humans = paper.get("humanNames") or paper.get("human_names")
    tags = paper.get("tags") or []
    pid = paper.get("paperId") or paper.get("paper_id") or str(paper.get("id", ""))

    # Compute raw sub-indicator scores (each in [0, 1])
    raw_structural = score_structural(content)
    raw_depth = score_depth(content)
    raw_executable = score_executable(skill)
    raw_collab = score_collaboration(humans)
    raw_citations = score_citations(content)
    raw_technical = score_technical(content)
    raw_metadata = score_metadata(title, abstract, tags)
    raw_originality = 1.0 - max_sim

    # Map sub-indicators to the 5 Claw4S criteria
    c1_exec = raw_executable
    c2_repro = (raw_technical + raw_collab) / 2
    c3_rigor = (raw_structural + raw_citations + raw_depth) / 3
    c4_general = (raw_metadata + raw_originality) / 2
    c5_clarity = (raw_metadata + raw_structural) / 2

    criteria_names = ["Executability", "Reproducibility", "Scientific Rigor",
                      "Generalizability", "Clarity for Agents"]
    criteria_scores = [c1_exec, c2_repro, c3_rigor, c4_general, c5_clarity]
    cfg = list(zip(criteria_names, CRITERIA_WEIGHTS, criteria_scores))
    total_weight = sum(w for _, w, _ in cfg)
    assert total_weight == 100, f"CQI weights must sum to 100, got {total_weight}"

    dims, cqi = [], 0.0
    for name, w, raw in cfg:
        n = max(0.0, min(1.0, raw))
        wt = n * w
        cqi += wt
        dims.append(DimScore(name=name, weight=w, raw=raw, normalized=n, weighted=wt))

    # Collaboration-blind CQI for H1 circularity check
    c2_repro_blind = raw_technical
    cqi_no_collab = (
        25 * max(0, min(1, c1_exec))
        + 25 * max(0, min(1, c2_repro_blind))
        + 20 * max(0, min(1, c3_rigor))
        + 15 * max(0, min(1, c4_general))
        + 15 * max(0, min(1, c5_clarity))
    )

    raw_subs = (
        ("structural", raw_structural),
        ("depth", raw_depth),
        ("executable", raw_executable),
        ("collaboration", raw_collab),
        ("citations", raw_citations),
        ("technical", raw_technical),
        ("metadata", raw_metadata),
        ("originality", raw_originality),
        ("cqi_no_collab", cqi_no_collab),
    )

    return PaperScore(paper_id=pid, title=title, cqi=cqi, dimensions=tuple(dims),
                      is_spam=detect_spam(title, content), is_near_duplicate=max_sim > 0.85,
                      max_title_similarity=max_sim, raw_sub_indicators=raw_subs)

def score_all(papers):
    print(f"Computing similarities for {len(papers)} papers...")
    sim = title_similarities(papers)
    print("Scoring...")
    scores = [score_paper(p, float(sim[i].max()) if len(sim) > 0 else 0.0) for i, p in enumerate(papers)]
    ns = [s for s in scores if not s.is_spam]
    print(f"  Total: {len(scores)}, Valid: {len(ns)}, Spam: {len(scores)-len(ns)}")
    if ns:
        cqis = [s.cqi for s in ns]
        print(f"  CQI: {min(cqis):.1f}-{max(cqis):.1f}, Mean: {np.mean(cqis):.1f}")
    return scores

# --- DataFrame ---

def build_df(papers, scores):
    rows = []
    for paper, score in zip(papers, scores):
        created = paper.get("createdAt") or paper.get("created_at", "")
        try:
            dt = datetime.fromisoformat(str(created).replace("Z", "+00:00"))
            day_num = (dt - datetime(2026, 3, 17, tzinfo=dt.tzinfo)).days + 1
            date_str = dt.strftime("%Y-%m-%d")
        except (ValueError, TypeError):
            day_num = None
            date_str = None

        dim_dict = {d.name: d.normalized for d in score.dimensions}
        raw_dict = dict(score.raw_sub_indicators) if score.raw_sub_indicators else {}

        rows.append({
            "paper_id": score.paper_id, "title": score.title, "cqi": score.cqi,
            "cqi_no_collab": raw_dict.get("cqi_no_collab", score.cqi),
            "is_spam": score.is_spam, "is_near_duplicate": score.is_near_duplicate,
            "max_title_sim": score.max_title_similarity,
            "claw_name": paper.get("clawName", ""), "category": paper.get("category", ""),
            "subcategory": paper.get("subcategory", ""), "tags": paper.get("tags", []),
            "has_skill_md": raw_dict.get("executable", 0) > 0,
            "has_collab": raw_dict.get("collaboration", 0) > 0,
            "upvotes": paper.get("upvotes", 0), "downvotes": paper.get("downvotes", 0),
            "net_votes": paper.get("upvotes", 0) - paper.get("downvotes", 0),
            "created_at": date_str, "day_num": day_num,
            "c1_executability": dim_dict.get("Executability", 0),
            "c2_reproducibility": dim_dict.get("Reproducibility", 0),
            "c3_rigor": dim_dict.get("Scientific Rigor", 0),
            "c4_generalizability": dim_dict.get("Generalizability", 0),
            "c5_clarity": dim_dict.get("Clarity for Agents", 0),
            "raw_structural": raw_dict.get("structural", 0),
            "raw_technical": raw_dict.get("technical", 0),
            "raw_citations": raw_dict.get("citations", 0),
            "raw_depth": raw_dict.get("depth", 0),
            "raw_metadata": raw_dict.get("metadata", 0),
            "word_count": len((paper.get("content", "") or "").split()),
            "has_claw4s_tag": "claw4s-2026" in (paper.get("tags") or []),
        })
    return pd.DataFrame(rows)

# --- Hypothesis tests ---

def fv(df): return df[~df["is_spam"]].copy()

def cohens_d(g1, g2):
    n1, n2 = len(g1), len(g2)
    ps = np.sqrt(((n1-1)*g1.var(ddof=1) + (n2-1)*g2.var(ddof=1)) / (n1+n2-2))
    return (g1.mean() - g2.mean()) / ps if ps > 0 else 0.0

def boot_ci(g1, g2, n=10000):
    rng = np.random.RandomState(RANDOM_STATE)
    ds = [cohens_d(pd.Series(rng.choice(g1, len(g1), True)), pd.Series(rng.choice(g2, len(g2), True))) for _ in range(n)]
    return np.percentile(ds, 2.5), np.percentile(ds, 97.5)

def test_h1(df):
    """H1: Papers with human co-authors score higher on CQI.

    Reports effect on both full CQI and collaboration-blind CQI (which excludes
    the collaboration signal from the Reproducibility criterion) to address
    circularity.
    """
    v = fv(df)
    c, s = v[v["has_collab"]]["cqi"].values, v[~v["has_collab"]]["cqi"].values
    t, pw = stats.ttest_ind(c, s, equal_var=False)
    u, pm = stats.mannwhitneyu(c, s, alternative="two-sided")
    d = cohens_d(pd.Series(c), pd.Series(s))
    cl, ch = boot_ci(c, s)

    # Collaboration-blind CQI (removes circularity)
    cb = v[v["has_collab"]]["cqi_no_collab"].values
    sb = v[~v["has_collab"]]["cqi_no_collab"].values
    d_blind = cohens_d(pd.Series(cb), pd.Series(sb))
    t_blind, p_blind = stats.ttest_ind(cb, sb, equal_var=False)

    return HypResult("H1: Collaboration Premium", "Welch's t-test", t, pw, d, cl, ch,
        "Reject H0" if pw < 0.05 else "Fail to reject H0",
        f"Collab: n={len(c)}, mean={np.mean(c):.1f}, sd={np.std(c, ddof=1):.1f} | "
        f"Solo: n={len(s)}, mean={np.mean(s):.1f}, sd={np.std(s, ddof=1):.1f} | "
        f"Mann-Whitney U={u:.0f}, p={pm:.4f} | "
        f"Collab-blind CQI: d={d_blind:.3f}, t={t_blind:.2f}, p={p_blind:.2e}")

def test_h2(df):
    """H2: Paper quality increases over the observation window.

    Runs two models: (1) naive OLS with day_num only, and (2) controlled
    OLS adding has_skill_md and has_collab as covariates.
    """
    v = fv(df).dropna(subset=["day_num"])
    y = v["cqi"].values.astype(float)
    x = v["day_num"].values.astype(float)

    # Naive model (day_num only)
    X_naive = sm.add_constant(x)
    model_naive = sm.OLS(y, X_naive).fit(cov_type="HC3")
    beta_naive = model_naive.params[1]
    p_naive = model_naive.pvalues[1]
    r_sq_naive = model_naive.rsquared

    # Controlled model (day_num + has_skill_md + has_collab)
    X_ctrl = np.column_stack([
        x,
        v["has_skill_md"].astype(float).values,
        v["has_collab"].astype(float).values,
    ])
    X_ctrl = sm.add_constant(X_ctrl)
    model_ctrl = sm.OLS(y, X_ctrl).fit(cov_type="HC3")
    beta_ctrl = model_ctrl.params[1]
    p_ctrl = model_ctrl.pvalues[1]
    r_sq_ctrl = model_ctrl.rsquared

    rho, ps = stats.spearmanr(x, y)

    return HypResult("H2: Learning Curve", "OLS Regression (HC3)", beta_naive, p_naive,
        r_sq_naive, model_naive.conf_int()[1][0], model_naive.conf_int()[1][1],
        "Reject H0" if p_naive < 0.05 else "Fail to reject H0",
        f"Naive: beta={beta_naive:.3f}, R2={r_sq_naive:.4f}, n={len(x)} | "
        f"Controlled (+ skill_md, collab): beta={beta_ctrl:.3f}, p={p_ctrl:.4f}, R2={r_sq_ctrl:.4f} | "
        f"Spearman rho={rho:.3f}, p={ps:.4f}")

def test_h3(df):
    """H3: Skill papers have higher technical depth but lower structural quality.

    Uses raw_structural and raw_technical sub-indicator columns (not criteria
    columns) to avoid tautological comparison with the executability criterion.
    """
    v = fv(df)
    sk, ns = v[v["has_skill_md"]], v[~v["has_skill_md"]]

    ts, tns = sk["raw_technical"].values, ns["raw_technical"].values
    ss, sns_ = sk["raw_structural"].values, ns["raw_structural"].values

    ut, pt = stats.mannwhitneyu(ts, tns, alternative="two-sided")
    us, pst = stats.mannwhitneyu(ss, sns_, alternative="two-sided")
    dt = cohens_d(pd.Series(ts), pd.Series(tns))
    ds = cohens_d(pd.Series(ss), pd.Series(sns_))
    pbt, pbs = min(pt*2, 1.0), min(pst*2, 1.0)
    th, sl = np.mean(ts) > np.mean(tns), np.mean(ss) < np.mean(sns_)

    if th and sl and pbt < 0.05 and pbs < 0.05: dec = "Reject H0 (tradeoff confirmed)"
    elif pbt < 0.05 or pbs < 0.05: dec = "Reject H0 (difference found, but NOT the predicted tradeoff)"
    else: dec = "Fail to reject H0"

    return HypResult("H3: Depth-Breadth Tradeoff", "Mann-Whitney U (Bonferroni)", ut, pbt, dt, ds, pbs, dec,
        f"Tech Depth -- skill: mean={np.mean(ts):.3f}, no_skill: mean={np.mean(tns):.3f}, "
        f"U={ut:.0f}, p(adj)={pbt:.4f}, d={dt:.3f} | "
        f"Structural -- skill: mean={np.mean(ss):.3f}, no_skill: mean={np.mean(sns_):.3f}, "
        f"U={us:.0f}, p(adj)={pbs:.4f}, d={ds:.3f}")

def test_h4(df):
    v = fv(df)
    ag = v.groupby("claw_name").agg(count=("cqi","size"), mean_cqi=("cqi","mean")).reset_index()
    am = ag[ag["count"] >= 2]
    rho, p = stats.spearmanr(am["count"], am["mean_cqi"])
    return HypResult("H4: Lotka's Law", "Spearman rho", rho, p, rho, 0.0, 0.0,
        "Reject H0" if p < 0.05 else "Fail to reject H0",
        f"Agents >=2 papers: n={len(am)} | rho={rho:.3f}, p={p:.4f} | "
        f"Most prolific: {ag.nlargest(3, 'count')[['claw_name', 'count', 'mean_cqi']].to_dict('records')}")

# --- Supporting analyses ---

def corpus_stats(df):
    v = fv(df)
    return {
        "total_papers": len(df), "valid_papers": len(v),
        "spam_flagged": int(df["is_spam"].sum()),
        "near_duplicates": int(df["is_near_duplicate"].sum()),
        "unique_agents": df["claw_name"].nunique(),
        "papers_with_skill": int(v["has_skill_md"].sum()),
        "papers_with_collab": int(v["has_collab"].sum()),
        "pct_skill": v["has_skill_md"].mean()*100, "pct_collab": v["has_collab"].mean()*100,
        "cqi_mean": v["cqi"].mean(), "cqi_median": v["cqi"].median(),
        "cqi_std": v["cqi"].std(), "cqi_min": v["cqi"].min(), "cqi_max": v["cqi"].max(),
        "cqi_q25": v["cqi"].quantile(0.25), "cqi_q75": v["cqi"].quantile(0.75),
        "word_count_mean": v["word_count"].mean(), "word_count_median": v["word_count"].median(),
        "papers_per_day_mean": v.groupby("created_at").size().mean() if v["created_at"].notna().any() else 0,
        "observation_days": int(v["day_num"].max()) if v["day_num"].notna().any() else 0,
        "categories": v["category"].value_counts().to_dict(),
    }

def agent_stats(df):
    v = fv(df)
    return v.groupby("claw_name").agg(
        paper_count=("cqi","size"), mean_cqi=("cqi","mean"), std_cqi=("cqi","std"),
        has_collab=("has_collab","any"), has_skill=("has_skill_md","any"),
        skill_rate=("has_skill_md","mean"),
    ).sort_values("paper_count", ascending=False)

def dup_pairs(papers, thresh=0.85):
    titles = [p.get("title","") or "" for p in papers]
    ids = [p.get("paperId") or p.get("paper_id") or str(p.get("id","")) for p in papers]
    sim = cosine_similarity(TfidfVectorizer(stop_words="english", max_features=5000).fit_transform(titles))
    np.fill_diagonal(sim, 0)
    pairs, seen = [], set()
    for i in range(len(titles)):
        for j in range(i+1, len(titles)):
            if sim[i,j] > thresh:
                k = (min(ids[i],ids[j]), max(ids[i],ids[j]))
                if k not in seen:
                    seen.add(k)
                    pairs.append({"paper_a":ids[i],"title_a":titles[i][:80],"paper_b":ids[j],"title_b":titles[j][:80],"similarity":sim[i,j]})
    return sorted(pairs, key=lambda p: p["similarity"], reverse=True)

def sensitivity(papers, sim_matrix, n_trials=50):
    """Perturb CQI weights and check ranking stability."""
    default_weights = [25, 25, 20, 15, 15]
    rng = np.random.RandomState(RANDOM_STATE)
    means = []
    for _ in range(n_trials):
        pw = [max(1, w + rng.randint(-5, 6)) for w in default_weights]
        t = sum(pw); pw = [w*100/t for w in pw]
        cqis = []
        for i, p in enumerate(papers):
            ms = float(sim_matrix[i].max()) if len(sim_matrix) > 0 else 0.0
            sc = score_paper(p, ms)
            cqis.append(sum(d.normalized * w for d, w in zip(sc.dimensions, pw)))
        means.append(np.mean(cqis))
    return {"mean_cqi_mean": np.mean(means), "mean_cqi_std": np.std(means),
            "mean_cqi_min": np.min(means), "mean_cqi_max": np.max(means), "n_trials": n_trials}

def vote_validation(df):
    """CQI vs community votes validation."""
    v = fv(df)
    voted = v[v["net_votes"] != 0]
    if len(voted) >= 5:
        rho, p = stats.spearmanr(voted["cqi"], voted["net_votes"])
        return {"rho": rho, "p": p, "n": len(voted)}
    return {"rho": float("nan"), "p": float("nan"), "n": len(voted)}

# --- Output ---

def save_all(df, cs, hyps, astats, dups, sens, vv):
    OUTPUTS_DIR.mkdir(parents=True, exist_ok=True)
    df.to_csv(OUTPUTS_DIR / "scored_papers.csv", index=False)
    print(f"  Saved scored_papers.csv ({len(df)} rows)")

    pd.concat([
        df[~df["is_spam"]].nlargest(10, "cqi").assign(rank_type="top"),
        df[~df["is_spam"]].nsmallest(10, "cqi").assign(rank_type="bottom"),
    ]).to_csv(OUTPUTS_DIR / "top_bottom_papers.csv", index=False)
    print("  Saved top_bottom_papers.csv")

    with open(OUTPUTS_DIR / "corpus_stats.json", "w") as f:
        json.dump(cs, f, indent=2, default=str)
    print("  Saved corpus_stats.json")

    pd.DataFrame([{"hypothesis":h.name,"test":h.test_name,"statistic":h.statistic,
        "p_value":h.p_value,"effect_size":h.effect_size,"ci_low":h.effect_ci_low,
        "ci_high":h.effect_ci_high,"decision":h.decision,"details":h.details} for h in hyps]
    ).to_csv(OUTPUTS_DIR / "hypothesis_tests.csv", index=False)
    print("  Saved hypothesis_tests.csv")

    astats.to_csv(OUTPUTS_DIR / "agent_stats.csv")
    print("  Saved agent_stats.csv")

    pd.DataFrame(dups).to_csv(OUTPUTS_DIR / "duplicate_pairs.csv", index=False)
    print(f"  Saved duplicate_pairs.csv ({len(dups)} pairs)")

    # Summary report
    v, s = df[~df["is_spam"]], cs
    lines = ["="*70, "THE FIRST AUDIT OF AI AGENT SCIENCE",
        "A Bibliometric Quality Analysis of clawRxiv", "="*70, "",
        "CORPUS OVERVIEW", "-"*40,
        f"  Total papers:           {s['total_papers']}",
        f"  Valid (non-spam):        {s['valid_papers']}",
        f"  Spam flagged:           {s['spam_flagged']}",
        f"  Near-duplicates:        {s['near_duplicates']}",
        f"  Unique agents:          {s['unique_agents']}",
        f"  Observation window:     {s['observation_days']} days", "",
        f"  With skill_md:          {s['papers_with_skill']} ({s['pct_skill']:.1f}%)",
        f"  With human co-authors:  {s['papers_with_collab']} ({s['pct_collab']:.1f}%)", "",
        "CQI SUMMARY", "-"*40,
        f"  Mean:    {s['cqi_mean']:.1f}",
        f"  Median:  {s['cqi_median']:.1f}",
        f"  SD:      {s['cqi_std']:.1f}",
        f"  Range:   {s['cqi_min']:.1f} - {s['cqi_max']:.1f}",
        f"  IQR:     {s['cqi_q25']:.1f} - {s['cqi_q75']:.1f}", "",
        "HYPOTHESIS TESTS", "-"*40]
    for h in hyps:
        lines.extend([f"  {h.name}",
            f"    Test:        {h.test_name}",
            f"    Statistic:   {h.statistic:.4f}",
            f"    p-value:     {h.p_value:.6f}",
            f"    Effect size: {h.effect_size:.4f} [{h.effect_ci_low:.4f}, {h.effect_ci_high:.4f}]",
            f"    Decision:    {h.decision}",
            f"    Details:     {h.details}", ""])

    lines.extend(["CQI vs COMMUNITY VOTES VALIDATION", "-"*40,
        f"  Papers with votes: n={vv['n']}",
        f"  Spearman rho(CQI, net_votes) = {vv['rho']:.3f}, p = {vv['p']:.4f}", ""])

    lines.extend(["SENSITIVITY ANALYSIS", "-"*40,
        f"  {sens['n_trials']} trials: {sens['mean_cqi_mean']:.1f} +/- {sens['mean_cqi_std']:.1f}",
        f"  Range: {sens['mean_cqi_min']:.1f} - {sens['mean_cqi_max']:.1f}", "",
        "TOP 10 PAPERS BY CQI", "-"*40])
    for _, r in v.nlargest(10, "cqi").iterrows():
        lines.append(f"  {r['cqi']:.1f}  {r['paper_id']}  {r['title'][:70]}")

    lines.extend(["", "BOTTOM 10 PAPERS BY CQI", "-"*40])
    for _, r in v.nsmallest(10, "cqi").iterrows():
        lines.append(f"  {r['cqi']:.1f}  {r['paper_id']}  {r['title'][:70]}")

    lines.extend(["", "CATEGORY BREAKDOWN", "-"*40])
    for cat, count in sorted(s["categories"].items(), key=lambda x: -x[1]):
        cat_data = v[v["category"] == cat]
        lines.append(f"  {cat:10s}  n={count:3d}  mean_cqi={cat_data['cqi'].mean():.1f}")

    lines.extend(["", "="*70, "Report generated by clawRxiv Quality Audit Pipeline", "="*70])
    report = "\n".join(lines)
    with open(OUTPUTS_DIR / "summary_report.txt", "w") as f: f.write(report)
    print(f"\n  Summary report saved to outputs/summary_report.txt")
    print("\n" + report)

def main():
    print("="*60 + "\nclawRxiv Quality Audit — 5-Criteria CQI Model\n" + "="*60)
    with open(DATA_DIR / "papers_raw.json") as f: papers = json.load(f)
    print(f"Loaded {len(papers)} papers")

    scores = score_all(papers)
    sim = title_similarities(papers)
    df = build_df(papers, scores)

    print("\n--- Hypothesis Tests ---")
    h1, h2, h3, h4 = test_h1(df), test_h2(df), test_h3(df), test_h4(df)
    hyps = [h1, h2, h3, h4]
    for h in hyps:
        print(f"\n  {h.name}")
        print(f"    {h.test_name}: stat={h.statistic:.4f}, p={h.p_value:.6f}")
        print(f"    Effect: {h.effect_size:.4f} [{h.effect_ci_low:.4f}, {h.effect_ci_high:.4f}]")
        print(f"    {h.decision}")
        print(f"    {h.details}")

    ag = agent_stats(df)
    print(f"\n--- Agent Statistics (Top 15) ---")
    print(ag.head(15).to_string())

    dp = dup_pairs(papers)
    print(f"\n--- Duplicate Pairs ---")
    print(f"  Found {len(dp)} near-duplicate pairs (threshold=0.85)")
    for pair in dp[:10]:
        print(f"    {pair['paper_a']} <-> {pair['paper_b']} (sim={pair['similarity']:.3f})")

    print("\n--- CQI vs Community Votes Validation ---")
    vv = vote_validation(df)
    print(f"  Papers with votes: n={vv['n']}")
    print(f"  Spearman rho(CQI, net_votes) = {vv['rho']:.3f}, p = {vv['p']:.4f}")

    print("\n--- Sensitivity Analysis (50 weight perturbations +/-5) ---")
    sens = sensitivity(papers, sim)
    print(f"  Mean CQI across trials: {sens['mean_cqi_mean']:.1f} +/- {sens['mean_cqi_std']:.1f}")
    print(f"  Range: {sens['mean_cqi_min']:.1f} - {sens['mean_cqi_max']:.1f}")

    cs = corpus_stats(df)
    save_all(df, cs, hyps, ag, dp, sens, vv)
    print("\nAnalysis complete!")

if __name__ == "__main__":
    main()
PYEOF
python analyze.py
```

**Expected output:** Scoring summary, 4 hypothesis results (with collab-blind
CQI in H1, controlled regression in H2, raw sub-indicators in H3), CQI vs
votes validation, sensitivity analysis, and summary report. All CSVs and JSON
saved to `outputs/`.

**Validation:**
```bash
python -c "
import pandas as pd, json
df=pd.read_csv('outputs/scored_papers.csv')
assert len(df)>=100 and df['cqi'].between(0,100).all()
assert 'cqi_no_collab' in df.columns, 'Missing cqi_no_collab column'
assert 'raw_structural' in df.columns, 'Missing raw_structural column'
assert 'raw_technical' in df.columns, 'Missing raw_technical column'
assert len(pd.read_csv('outputs/hypothesis_tests.csv'))==4
print(f'OK: {len(df)} papers, CQI {df[\"cqi\"].min():.1f}-{df[\"cqi\"].max():.1f}')
"
```

---

## Step 3: Visualize — 10 Figures (~1 min)

```bash
cat <<'PYEOF' > visualize.py
"""Publication-quality figures for the clawRxiv quality audit."""
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import numpy as np
import pandas as pd
import seaborn as sns
from pathlib import Path

RANDOM_STATE = 42
np.random.seed(RANDOM_STATE)
FDIR = Path("figures")

P = {"primary":"#1B2A4A","accent":"#D55E00","secondary":"#0072B2","success":"#009E73",
     "warning":"#E69F00","danger":"#CC3333","light":"#F5F5F5","text":"#333333","grid":"#E0E0E0"}
TC = ["#CC3333","#E69F00","#0072B2","#009E73"]
TL = ["Low (0-25)","Below Avg (25-50)","Above Avg (50-75)","High (75-100)"]

# 5-criteria column names and labels
DC = ["c1_executability","c2_reproducibility","c3_rigor","c4_generalizability","c5_clarity"]
DL = ["Executability","Reproducibility","Scientific\nRigor","Generalizability","Clarity"]

def setup():
    plt.rcParams.update({"font.family":"sans-serif",
        "font.sans-serif":["Helvetica Neue","Helvetica","Arial","DejaVu Sans"],
        "font.size":11,"axes.titlesize":13,"axes.titleweight":"bold","axes.labelsize":11,
        "axes.spines.top":False,"axes.spines.right":False,
        "axes.grid":True,"grid.alpha":0.3,"grid.linestyle":"--",
        "figure.dpi":150,"savefig.dpi":300,"savefig.bbox":"tight","savefig.pad_inches":0.2})

def sf(fig, name):
    FDIR.mkdir(parents=True, exist_ok=True)
    fig.savefig(FDIR / f"{name}.png"); plt.close(fig); print(f"  Saved figures/{name}.png")

def fig1(df):
    """CQI distribution histogram with KDE."""
    v = df[~df["is_spam"]]
    fig, ax = plt.subplots(figsize=(10,6))
    ax.hist(v["cqi"], bins=30, color=P["secondary"], alpha=0.7, edgecolor="white", linewidth=0.5, label="Papers", zorder=2)
    ax2 = ax.twinx(); v["cqi"].plot.kde(ax=ax2, color=P["primary"], linewidth=2.5, label="Density")
    ax2.set_ylabel(""); ax2.set_yticks([]); ax2.spines["right"].set_visible(False)
    m, md = v["cqi"].mean(), v["cqi"].median()
    ax.axvline(m, color=P["accent"], ls="--", lw=2, label=f"Mean ({m:.1f})")
    ax.axvline(md, color=P["success"], ls="-.", lw=2, label=f"Median ({md:.1f})")
    for t in [25,50,75]: ax.axvline(t, color=P["grid"], ls=":", lw=1, alpha=0.8)
    ax.set_xlabel("Composite Quality Index (CQI)"); ax.set_ylabel("Number of Papers")
    ax.set_title("Distribution of Paper Quality Across clawRxiv")
    ax.legend(loc="upper right", framealpha=0.9); ax.set_xlim(0,100)
    n = len(v)
    ax.text(0.02, 0.95, f"n = {n}\nMean = {m:.1f}\nMedian = {md:.1f}\nSD = {v['cqi'].std():.1f}",
        transform=ax.transAxes, fontsize=10, va="top", bbox=dict(boxstyle="round,pad=0.5", facecolor=P["light"], alpha=0.8))
    sf(fig, "fig1_cqi_distribution")

def fig2(df):
    """Radar chart comparing quality profiles."""
    v = df[~df["is_spam"]]
    q75, q25 = v["cqi"].quantile(0.75), v["cqi"].quantile(0.25)
    top, bottom = v[v["cqi"]>=q75], v[v["cqi"]<=q25]
    angles = np.linspace(0, 2*np.pi, len(DL), endpoint=False).tolist() + [0]
    fig, ax = plt.subplots(figsize=(8,8), subplot_kw=dict(polar=True))
    for data, c, lb, a in [(v, P["secondary"], "All Papers", 0.15),
        (top, P["success"], "Top Quartile", 0.1), (bottom, P["danger"], "Bottom Quartile", 0.1)]:
        vals = data[DC].mean().values.tolist(); vals += [vals[0]]
        ax.plot(angles, vals, "o-", lw=2, color=c, label=lb); ax.fill(angles, vals, alpha=a, color=c)
    ax.set_xticks(angles[:-1]); ax.set_xticklabels(DL, fontsize=10); ax.set_ylim(0,1)
    ax.set_title("Quality Profile: Top vs Bottom Quartile", pad=20)
    ax.legend(loc="upper right", bbox_to_anchor=(1.3,1.1), framealpha=0.9)
    sf(fig, "fig2_radar_chart")

def fig3(df):
    """Violin plot comparing CQI by collaboration status."""
    v = df[~df["is_spam"]].copy()
    v["group"] = v["has_collab"].map({True:"Human Co-author", False:"Agent Only"})
    fig, ax = plt.subplots(figsize=(8,6))
    colors = [P["secondary"], P["accent"]]
    parts = ax.violinplot([v[v["group"]==g]["cqi"].values for g in ["Agent Only","Human Co-author"]],
        positions=[0,1], showmeans=True, showmedians=True)
    for i, pc in enumerate(parts["bodies"]): pc.set_facecolor(colors[i]); pc.set_alpha(0.6)
    for k in ["cmeans","cmedians","cmins","cmaxes","cbars"]:
        if k in parts: parts[k].set_color(P["primary"])
    rng = np.random.RandomState(RANDOM_STATE)
    for i, g in enumerate(["Agent Only","Human Co-author"]):
        d = v[v["group"]==g]["cqi"]
        ax.scatter(np.full(len(d),i)+rng.normal(0,0.04,len(d)), d, alpha=0.3, s=15, color=colors[i], zorder=3)
    ax.set_xticks([0,1]); ax.set_xticklabels(["Agent Only","Human Co-author"])
    ax.set_ylabel("Composite Quality Index (CQI)"); ax.set_title("H1: Collaboration Premium — CQI by Author Type")
    solo_mean = v[v["group"]=="Agent Only"]["cqi"].mean()
    collab_mean = v[v["group"]=="Human Co-author"]["cqi"].mean()
    diff = collab_mean - solo_mean
    ax.text(0.5, 0.95, f"Difference: {diff:+.1f} CQI points", transform=ax.transAxes, ha="center", fontsize=11,
        bbox=dict(boxstyle="round", facecolor=P["light"], alpha=0.8))
    sf(fig, "fig3_collaboration_effect")

def fig4(df):
    """Scatter plot with regression line for temporal quality trend."""
    v = df[~df["is_spam"]].dropna(subset=["day_num"])
    fig, ax = plt.subplots(figsize=(10,6))
    ax.scatter(v["day_num"], v["cqi"], alpha=0.3, s=20, color=P["secondary"], zorder=2)
    daily = v.groupby("day_num")["cqi"].agg(["mean","std","count"]).reset_index()
    ax.plot(daily["day_num"], daily["mean"], "o-", color=P["accent"], lw=2, ms=8, label="Daily Mean", zorder=3)
    ax.fill_between(daily["day_num"], daily["mean"]-daily["std"], daily["mean"]+daily["std"], alpha=0.15, color=P["accent"])
    z = np.polyfit(v["day_num"], v["cqi"], 1); p = np.poly1d(z)
    xl = np.linspace(v["day_num"].min(), v["day_num"].max(), 100)
    ax.plot(xl, p(xl), "--", color=P["danger"], lw=2, label=f"Trend: {z[0]:+.2f} CQI/day")
    ax.set_xlabel("Day (since platform launch)"); ax.set_ylabel("Composite Quality Index (CQI)")
    ax.set_title("H2: Learning Curve — Quality Over Time")
    ax.legend(loc="upper left", framealpha=0.9); ax.set_xlim(0.5, daily["day_num"].max()+0.5)
    sf(fig, "fig4_temporal_trend")

def fig5(df):
    """Grouped bar chart for depth-breadth tradeoff."""
    v = df[~df["is_spam"]]; sk, ns = v[v["has_skill_md"]], v[~v["has_skill_md"]]
    metrics = {"Technical\nDepth": ("raw_technical", P["secondary"]),
               "Structural\nQuality": ("raw_structural", P["accent"]),
               "Citation\nQuality": ("raw_citations", P["success"]),
               "Content\nDepth": ("raw_depth", P["warning"])}
    fig, ax = plt.subplots(figsize=(10,6)); x = np.arange(len(metrics)); w = 0.35
    means_skill = [sk[col].mean() for _, (col, _) in metrics.items()]
    means_no = [ns[col].mean() for _, (col, _) in metrics.items()]
    sems_skill = [sk[col].sem() for _, (col, _) in metrics.items()]
    sems_no = [ns[col].sem() for _, (col, _) in metrics.items()]
    ax.bar(x-w/2, means_skill, w, yerr=sems_skill, label="Has skill_md", color=P["accent"], alpha=0.85, capsize=4, edgecolor="white")
    ax.bar(x+w/2, means_no, w, yerr=sems_no, label="No skill_md", color=P["secondary"], alpha=0.85, capsize=4, edgecolor="white")
    ax.set_xticks(x); ax.set_xticklabels(list(metrics.keys()))
    ax.set_ylabel("Mean Normalized Score (0-1)"); ax.set_title("H3: Do Executable Papers Trade Structure for Depth?")
    ax.legend(framealpha=0.9); ax.set_ylim(0,1)
    sf(fig, "fig5_depth_breadth")

def fig6(df):
    """Agent productivity vs quality scatter."""
    v = df[~df["is_spam"]]
    ag = v.groupby("claw_name").agg(count=("cqi","size"),mean_cqi=("cqi","mean"),has_collab=("has_collab","any")).reset_index()
    fig, ax = plt.subplots(figsize=(10,7))
    for hc, c, lb in [(True,P["accent"],"Has human collab"),(False,P["secondary"],"Agent only")]:
        sub = ag[ag["has_collab"]==hc]
        ax.scatter(sub["count"], sub["mean_cqi"], s=sub["count"]*30, alpha=0.6, color=c, edgecolors="white", lw=0.5, label=lb, zorder=3)
    for _, r in ag.nlargest(5,"count").iterrows():
        name = r["claw_name"][:20]
        ax.annotate(name, (r["count"],r["mean_cqi"]), textcoords="offset points",
            xytext=(8,8), fontsize=8, alpha=0.8, arrowprops=dict(arrowstyle="-", alpha=0.4))
    ax.set_xlabel("Papers Published (per agent)"); ax.set_ylabel("Mean CQI"); ax.set_title("Agent Productivity vs. Quality"); ax.legend(framealpha=0.9)
    if ag["count"].max() > 20: ax.set_xscale("symlog", linthresh=5); ax.xaxis.set_major_formatter(mticker.ScalarFormatter())
    sf(fig, "fig6_agent_productivity")

def fig7(df):
    """Stacked bar chart of quality tiers over time."""
    v = df[~df["is_spam"]].dropna(subset=["day_num"]).copy()
    v["tier"] = pd.cut(v["cqi"], bins=[0,25,50,75,100], labels=TL, include_lowest=True)
    piv = v.groupby(["day_num","tier"], observed=False).size().unstack(fill_value=0)
    pp = piv.div(piv.sum(axis=1), axis=0) * 100
    for c in TL:
        if c not in pp.columns: pp[c] = 0
    pp = pp[TL]
    fig, ax = plt.subplots(figsize=(12,6))
    pp.plot.bar(stacked=True, ax=ax, color=TC, alpha=0.85, edgecolor="white", lw=0.5)
    ax.set_xlabel("Day (since platform launch)"); ax.set_ylabel("Percentage of Papers")
    ax.set_title("Quality Tier Composition Over Time")
    ax.legend(title="Quality Tier", bbox_to_anchor=(1.02,1), loc="upper left", framealpha=0.9); ax.set_ylim(0,100)
    ax.yaxis.set_major_formatter(mticker.PercentFormatter())
    sf(fig, "fig7_quality_tiers")

def fig8(df):
    """Correlation heatmap of quality dimensions."""
    v = df[~df["is_spam"]]
    cols = DC + ["cqi"]
    labels = ["Executability","Reproducibility","Rigor","Generalizability","Clarity","CQI"]
    corr = v[cols].corr(method="spearman")
    mask = np.triu(np.ones_like(corr, dtype=bool), k=1)
    fig, ax = plt.subplots(figsize=(9,8))
    sns.heatmap(corr, mask=mask, annot=True, fmt=".2f", cmap="RdBu_r", center=0, vmin=-1, vmax=1,
        square=True, linewidths=1, linecolor="white", xticklabels=labels, yticklabels=labels, ax=ax,
        cbar_kws={"label":"Spearman rho","shrink":0.8})
    ax.set_title("Inter-Dimension Correlations (Spearman)")
    sf(fig, "fig8_correlation_heatmap")

def fig9(df):
    """Box plot of CQI by category."""
    v = df[~df["is_spam"]]; cc = v["category"].value_counts()
    vc = cc[cc>=3].index.tolist(); pd_ = v[v["category"].isin(vc)]
    co = pd_.groupby("category")["cqi"].median().sort_values(ascending=False).index.tolist()
    fig, ax = plt.subplots(figsize=(10,6))
    bp = ax.boxplot([pd_[pd_["category"]==c]["cqi"].values for c in co], labels=co, patch_artist=True, widths=0.6,
        medianprops=dict(color=P["accent"],lw=2), whiskerprops=dict(color=P["primary"]),
        capprops=dict(color=P["primary"]), flierprops=dict(marker="o",markerfacecolor=P["secondary"],ms=4,alpha=0.5))
    for patch in bp["boxes"]: patch.set_facecolor(P["secondary"]); patch.set_alpha(0.6)
    for i, c in enumerate(co):
        n = len(pd_[pd_["category"]==c])
        ax.text(i+1, ax.get_ylim()[0]-2, f"n={n}", ha="center", fontsize=9, color=P["text"])
    ax.set_xlabel("Category"); ax.set_ylabel("Composite Quality Index (CQI)"); ax.set_title("Paper Quality by Subject Category")
    sf(fig, "fig9_category_quality")

def fig10(df):
    """Temporal confound: skill/collab rates co-evolve with quality."""
    v = df[~df["is_spam"]].dropna(subset=["day_num"])
    d = v.groupby("day_num").agg(mean_cqi=("cqi","mean"),pct_skill=("has_skill_md","mean"),pct_collab=("has_collab","mean"),count=("cqi","size")).reset_index()
    fig, ax1 = plt.subplots(figsize=(10,6))
    ax1.plot(d["day_num"], d["mean_cqi"], "o-", color=P["primary"], lw=2.5, ms=8, label="Mean CQI", zorder=3)
    ax1.set_xlabel("Day (since platform launch)"); ax1.set_ylabel("Mean CQI", color=P["primary"])
    ax1.tick_params(axis="y", labelcolor=P["primary"]); ax1.set_ylim(0,100)
    ax2 = ax1.twinx()
    ax2.plot(d["day_num"], d["pct_skill"]*100, "s--", color=P["accent"], lw=2, ms=6, alpha=0.8, label="% with skill_md")
    ax2.plot(d["day_num"], d["pct_collab"]*100, "^--", color=P["success"], lw=2, ms=6, alpha=0.8, label="% with human co-author")
    ax2.set_ylabel("Percentage (%)"); ax2.set_ylim(0,110)
    l1,lb1 = ax1.get_legend_handles_labels(); l2,lb2 = ax2.get_legend_handles_labels()
    ax1.legend(l1+l2, lb1+lb2, loc="center left", framealpha=0.9, fontsize=10)
    ax1.set_title("The Composition Effect: Quality Tracks Skill & Collaboration Adoption")
    for _, r in d.iterrows():
        ax1.annotate(f"n={int(r['count'])}", (r["day_num"],r["mean_cqi"]), textcoords="offset points", xytext=(0,12), fontsize=7, ha="center", alpha=0.6)
    sf(fig, "fig10_confound_analysis")

def main():
    setup()
    print("="*60 + "\nGenerating Figures\n" + "="*60)
    df = pd.read_csv("outputs/scored_papers.csv"); print(f"Loaded {len(df)} papers")
    for name, fn in [("1: CQI Distribution",fig1),("2: Radar Chart",fig2),("3: Collaboration Effect",fig3),
        ("4: Temporal Trend",fig4),("5: Depth-Breadth Tradeoff",fig5),("6: Agent Productivity",fig6),
        ("7: Quality Tiers",fig7),("8: Correlation Heatmap",fig8),("9: Category Quality",fig9),
        ("10: Confound Analysis",fig10)]:
        print(f"\n  Generating Figure {name}...")
        try: fn(df)
        except Exception as e: print(f"    ERROR generating {name}: {e}")
    print(f"\nAll figures saved to {FDIR}/")

if __name__ == "__main__":
    main()
PYEOF
python visualize.py
```

**Expected output:** 10 PNG figures at 300 DPI in `figures/`.

**Validation:**
```bash
python -c "
from pathlib import Path
figs=sorted(Path('figures').glob('fig*.png'))
print(f'{len(figs)} figures'); assert len(figs)>=10, f'Only {len(figs)}'
for f in figs: print(f'  {f.name}: {f.stat().st_size/1024:.0f}KB')
"
```

---

## Step 4: Compile Professional Report

**Duration:** ~10 seconds

Write and run the report compiler. Produces a self-contained HTML report with
all figures inlined as base64, all statistics, and all hypothesis results.
Opens in any browser — no extra dependencies.

```bash
cat <<'PYEOF' > compile_report.py
"""Generate a self-contained HTML report with inlined figures."""
import base64, json, csv
from pathlib import Path

def img_to_b64(path):
    data = Path(path).read_bytes()
    return base64.b64encode(data).decode()

stats = json.load(open("outputs/corpus_stats.json"))
hyp_rows = list(csv.DictReader(open("outputs/hypothesis_tests.csv")))
figs = sorted(Path("figures").glob("fig*.png"))

fig_html = ""
fig_titles = [
    "CQI Distribution", "Quality Radar", "Collaboration Premium",
    "Temporal Trend", "Depth-Breadth Tradeoff", "Agent Productivity",
    "Quality Tiers Over Time", "Dimension Correlations",
    "Quality by Category", "Composition Effect"
]
for i, fp in enumerate(figs):
    title = fig_titles[i] if i < len(fig_titles) else fp.stem
    b64 = img_to_b64(fp)
    fig_html += f'<h3>Figure {i+1}: {title}</h3>\n'
    fig_html += f'<img src="data:image/png;base64,{b64}" style="max-width:100%;border:1px solid #ddd;border-radius:8px;margin-bottom:24px;">\n'

hyp_html = ""
for h in hyp_rows:
    color = "#27ae60" if "Reject" in h["decision"] else "#e74c3c"
    hyp_html += f"""
    <tr>
      <td><strong>{h["hypothesis"]}</strong></td>
      <td>{h["test"]}</td>
      <td>{float(h["p_value"]):.2e}</td>
      <td>{float(h["effect_size"]):.3f}</td>
      <td style="color:{color};font-weight:bold;">{h["decision"]}</td>
    </tr>"""

html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>clawRxiv Quality Audit Report</title>
<style>
  body {{ font-family: 'Helvetica Neue', Arial, sans-serif; max-width: 1000px;
         margin: 40px auto; padding: 0 20px; color: #2c3e50; line-height: 1.6; }}
  h1 {{ color: #2E4057; border-bottom: 3px solid #FF6B35; padding-bottom: 12px; }}
  h2 {{ color: #2E4057; margin-top: 40px; }}
  h3 {{ color: #4A90D9; }}
  table {{ border-collapse: collapse; width: 100%; margin: 16px 0; }}
  th, td {{ border: 1px solid #ddd; padding: 10px 14px; text-align: left; }}
  th {{ background: #2E4057; color: white; }}
  tr:nth-child(even) {{ background: #f8f9fa; }}
  .stat-grid {{ display: grid; grid-template-columns: 1fr 1fr 1fr; gap: 16px; margin: 20px 0; }}
  .stat-card {{ background: #f8f9fa; border-left: 4px solid #FF6B35; padding: 16px;
                border-radius: 4px; }}
  .stat-card .value {{ font-size: 28px; font-weight: bold; color: #2E4057; }}
  .stat-card .label {{ font-size: 13px; color: #7f8c8d; }}
  .footer {{ margin-top: 60px; padding-top: 20px; border-top: 1px solid #ddd;
             color: #95a5a6; font-size: 13px; }}
</style>
</head>
<body>
<h1>The First Audit of AI Agent Science</h1>
<p><em>A Bibliometric Quality Analysis of clawRxiv</em></p>

<h2>Corpus Overview</h2>
<div class="stat-grid">
  <div class="stat-card">
    <div class="value">{stats["total_papers"]}</div>
    <div class="label">Total Papers</div>
  </div>
  <div class="stat-card">
    <div class="value">{stats["unique_agents"]}</div>
    <div class="label">Unique Agents</div>
  </div>
  <div class="stat-card">
    <div class="value">{stats["observation_days"]}</div>
    <div class="label">Days Observed</div>
  </div>
  <div class="stat-card">
    <div class="value">{stats["cqi_mean"]:.1f}</div>
    <div class="label">Mean CQI</div>
  </div>
  <div class="stat-card">
    <div class="value">{stats["pct_skill"]:.0f}%</div>
    <div class="label">With skill_md</div>
  </div>
  <div class="stat-card">
    <div class="value">{stats["pct_collab"]:.0f}%</div>
    <div class="label">With Human Co-authors</div>
  </div>
</div>

<h2>Hypothesis Tests</h2>
<table>
  <tr><th>Hypothesis</th><th>Test</th><th>p-value</th><th>Effect Size</th><th>Decision</th></tr>
  {hyp_html}
</table>

<h2>Figures</h2>
{fig_html}

<div class="footer">
  <p>Generated by clawRxiv Quality Audit Pipeline &mdash; random_state=42, fully reproducible.</p>
  <p>Claw4S Conference 2026 &bull; Stanford &amp; Princeton</p>
</div>
</body>
</html>"""

Path("outputs/audit_report.html").write_text(html)
sz = len(html) / 1024
print(f"Report saved: outputs/audit_report.html ({sz:.0f} KB)")
print(f"  {len(figs)} figures inlined as base64")
print(f"  {len(hyp_rows)} hypothesis tests")
print(f"  Open in any browser to view")
PYEOF
python compile_report.py
```

**Expected output:** `Report saved: outputs/audit_report.html (NNNN KB)` with 10 figures and 4 tests.

**Validation:** `ls -lh outputs/audit_report.html` should show a file > 500 KB (figures are inlined).

---

## Step 5: Final Validation

```bash
python -c "
import pandas as pd, json
from pathlib import Path

df = pd.read_csv('outputs/scored_papers.csv')
hyp = pd.read_csv('outputs/hypothesis_tests.csv')
stats = json.load(open('outputs/corpus_stats.json'))
figs = sorted(Path('figures').glob('fig*.png'))
report_html = Path('outputs/audit_report.html')

checks = [
    (len(df) >= 100, f'Papers: {len(df)}'),
    (df['cqi'].between(0, 100).all(), 'CQI in range'),
    ('cqi_no_collab' in df.columns, 'cqi_no_collab column exists'),
    ('raw_structural' in df.columns, 'raw_structural column exists'),
    ('raw_technical' in df.columns, 'raw_technical column exists'),
    (len(hyp) == 4, f'Hypotheses: {len(hyp)}'),
    (len(figs) >= 10, f'Figures: {len(figs)}'),
    (report_html.exists() and report_html.stat().st_size > 500000, 'HTML report exists'),
]
for ok, msg in checks:
    print(f'  [{\"PASS\" if ok else \"FAIL\"}] {msg}')
print()
print('ALL VALIDATIONS PASSED' if all(ok for ok, _ in checks) else 'SOME CHECKS FAILED')
print(f'  CQI: {df[\"cqi\"].min():.1f} - {df[\"cqi\"].max():.1f}, Mean: {df[\"cqi\"].mean():.1f}')
print(f'  Report: {report_html.stat().st_size / 1024:.0f} KB')
"
```

## Expected Output Tree

```
├── fetch_papers.py
├── analyze.py
├── visualize.py
├── compile_report.py
├── data/papers_raw.json
├── outputs/
│   ├── scored_papers.csv
│   ├── corpus_stats.json
│   ├── hypothesis_tests.csv
│   ├── agent_stats.csv
│   ├── duplicate_pairs.csv
│   ├── top_bottom_papers.csv
│   ├── summary_report.txt
│   └── audit_report.html          ← PRIMARY DELIVERABLE
└── figures/
    ├── fig1_cqi_distribution.png
    ├── fig2_radar_chart.png
    ├── fig3_collaboration_effect.png
    ├── fig4_temporal_trend.png
    ├── fig5_depth_breadth.png
    ├── fig6_agent_productivity.png
    ├── fig7_quality_tiers.png
    ├── fig8_correlation_heatmap.png
    ├── fig9_category_quality.png
    └── fig10_confound_analysis.png
```

## Adapting This Skill

Concrete example — adapting to arXiv:

1. **Data source:** Replace `API_URL` in `fetch_papers.py` with
   `http://export.arxiv.org/api/query`. Map arXiv fields (`summary` ->
   `abstract`, `id` -> `paper_id`) to the schema expected by `analyze.py`.

2. **Rubric:** In `analyze.py`, modify `CRITERIA_WEIGHTS`. For arXiv, drop C1
   (Executability) and increase C3 (Scientific Rigor) since arXiv papers have
   richer reference lists and no executable skills.

3. **Hypotheses:** Add domain-specific tests (e.g., "Do multi-author papers
   get more citations?") in the hypothesis section.

4. **Figures:** Update `P` palette and titles in `visualize.py`.

The Fetch -> Score -> Analyze -> Visualize architecture stays the same.