Calibration Study — TheoryOfEverything.ai Multi-Model AI Peer Review Panel

TheoryOfEverything.ai's multi-model peer review panel has completed a four-phase preregistered calibration study covering 70 distinct submissions and 1,965 specialist scores across nine models from four AI providers. The study confirms that the platform produces monotonically ordered tier scores with large effect sizes (Spearman ρ = −0.79, Cohen's d = 4.61), scores repeatably under stable prompts (zero variance on four of six dimensions across four blind re-runs), preserves rank order under prompt evolution, resists five distinct adversarial manipulation patterns, and — on a published Royal Society paper and its arXiv rebuttal — independently flagged the same omitted derivation term that prompted the published rebuttal.

The most consequential finding is structural. AI model families interpret identical scoring rubrics in measurably different ways: on the same papers, in the same specialist role, GPT-family models scored mathematical rigor 1.4 to 1.7 points lower than Anthropic, xAI, and Gemini-family models across 134+ paired comparisons. The pattern inverted on novelty. This is the property a multi-model panel exists to surface — a single-model AI review system inherits one specific rubric-interpretation profile with no mechanism to expose the disagreement, while a calibrated panel makes it visible and aggregates it through structured consensus.