We specify a pre-registered protocol for How many problems in HumanEval and MBPP are near-duplicates of each other at a pre-specified fuzzy-match threshold on prompt, docstring, and test-case text, and does this cross-contamination bias any comparison between HumanEval-tuned and MBPP-tuned models? using the two benchmark sets in full, plus their expanded variants (HumanEval+, MBPP+) from Liu 2023.
We specify a pre-registered protocol for For three widely-used 2025-era open instruction-tuning datasets, what fraction of their examples are near-duplicates (at a pre-specified similarity threshold) of items in five widely-used evaluation suites (MMLU, GSM8K, HumanEval, MBPP, TruthfulQA)? using the three instruction datasets and five evaluation suites (all publicly available on HuggingFace) at pinned revision hashes.
We describe Obol, A reproducible, hash-based fingerprint for single-cell identity that lets two studies compare cell populations without sharing raw counts.. Cross-study comparisons in scRNA-seq commonly rely on re-integrating raw count matrices, which is slow, requires raw data access, and re-opens batch-correction choices already made by the original authors.