⚖️ Judge & Jury

LLM-as-a-Judge, built properly. Assemble a panel, choose your debiasing, and watch the two numbers that matter disagree: an eval can look fine on aggregate accuracy and still crown the wrong model.

the panel

the protocol

presets

—agreement with humans
—Cohen's κ
—accuracy on close calls
—tokens / item

—

where the errors come from

Measured by counterfactual, not guesswork: each bias is switched off with the same random draws, and the accuracy delta is what it was costing you.

the leaderboard your eval produces