A field reference — not a leaderboard

How do you know if an AI system is actually good — and safe?

This site collects the science behind that question: how to measure LLMs and agents without fooling yourself, how judges go biased and noisy, how to build golden data that holds up, and how to evaluate resistance to indirect prompt injection (XPIA) when assistants read untrusted email, documents, and web pages.

Written and curated by Eli Kravez — Principal Data & Applied Scientist at Microsoft, leading evaluation for Copilot in Microsoft Teams on live traffic.

6Core foundations — from construct validity to offline–online gaps
16Conference papers with full practitioner summaries — problem, method, finding, watch-out
21Benchmarks organised by domain — agents, coding, healthcare, RAG, safety & more
3Consulting offers: audit, judge calibration, regression gates
01

Foundations of evaluation science

Most eval failures are not model failures. They are measurement failures — the wrong construct, a leaky dataset, a judge that rewards length, or a sample too small to tell signal from noise.

1.1

Construct validity first

Define the capability in user terms before you pick a dataset. “Helpfulness” is not a construct until you say helpful for whom, for what task, at what cost of being wrong. A benchmark with high scores and low validity is worse than no benchmark — it optimises the team toward the wrong thing.

1.2

Golden data & leakage

Golden sets need provenance, adjudicated labels, and a contamination check. If test items (or near-duplicates) appear in training, retrieval corpora, or judge prompts, you are measuring memorisation. Version golden data like code, hold out a never-touched audit slice, and re-adjudicate a sample every release.

1.3

Human evaluation, done properly

Humans are the reference, not an afterthought. Use clear rubrics with examples, measure inter-rater agreement (Cohen’s κ / Krippendorff’s α), adjudicate disagreements instead of averaging them away, and report agreement per slice — overall agreement hides the groups where raters systematically disagree.

1.4

LLM-as-judge

Judges scale evaluation, but they import their own biases: position bias, verbosity bias, self-preference, and style bias. Treat a judge as a measurement instrument — validate it against humans, calibrate it, monitor its drift, and never let the same model family grade itself without a human-anchored check.

1.5

Synthetic data

Synthetic conversations and threads can expand coverage where real data is scarce or private — if you control generation, validate against specifications, and measure diversity. Synthetic data that only mirrors the generator’s habits narrows your eval while making the sample size look impressive.

1.6

Offline ≠ online

Offline wins routinely fail live: distribution shift, different user intent, latency trade-offs, and feedback loops. Pair every offline gate with live-traffic validation, guardrail metrics, and a root-cause loop that reads judge reasoning and telemetry — not just the headline score.

02

Bias–variance, for judges and benchmarks

In classical ML the trade-off is about model complexity. In evaluation it is about your measurement system: rubric specificity, judge choice, prompt design, and sample size move bias and variance in opposite directions.

Try it: turn up judge specificity

A vague rubric (“rate quality 1–5”) has low variance — every judge says ~3 — but high bias. A highly specific, multi-criteria rubric can reduce bias, yet each added criterion, example, and judge call adds variance. Drag the slider and watch total error.

Balanced point: bias and variance are both moderate. This is usually where a calibrated judge + adjudicated golden set beats either extreme.
Bias²
36
Variance
25
Noise
10
Total error
71

Illustrative decomposition, normalised to 100. Noise = genuine human disagreement and task ambiguity — you cannot optimise it away, only measure it honestly.

Sources of bias

Systematic error — the score is consistently wrong

Position bias: judges favour the first (or last) answer. Verbosity bias: longer looks better. Self-preference: a judge favours its own family’s style. Rubric bias: criteria encode one group’s preferences. Selection bias: your eval set over-represents easy, English, single-turn cases. Fix with counterbalancing, length controls, group-calibrated rubrics, and slice reporting.

Sources of variance

Random error — the score won’t sit still

Sampling variance: n=100 cannot detect a 2-point win. Judge variance: temperature, prompt phrasing, and order flip verdicts. Rater variance: fatigue and ambiguous items. Fix with power calculations, repeated judging with aggregation, fixed seeds/prompts, confidence intervals — and never report a win without one.

What this means in practice

DecisionBias effectVariance effectRecommendation
Single judge, single passHigh — inherits one model’s preferencesHigh — one sample of judge noiseOnly for triage, never for ship gates
Judge ensemble + human anchorLower — biases partially cancelLower — averaging stabilises scoresDefault for regression gates; re-anchor monthly
Very detailed rubric (20+ criteria)Lower if criteria are validHigher — judges apply criteria inconsistentlyPilot it; drop criteria with low inter-judge agreement
Bigger eval set, weaker labelsHigher — noisy labels bias toward the majority styleLower from sample size alonePrefer smaller, adjudicated sets for decisions; large weak sets for monitoring

Rule of thumb: report every headline metric with a confidence interval, a per-slice breakdown, and the judge-vs-human agreement on the same items. If any of the three is missing, the number is a demo, not a measurement.

03

Best practices — a working checklist

Distilled from running evaluation on live Copilot traffic and from the literature in the library below. Use it as a pre-flight for a new eval or an audit of an existing one.

Before you measure

Design

  • ✓
    Write the decision first. What will you ship, block, or roll back based on this metric?
  • ✓
    Define slices up front. Language, task type, user group, risk tier — before you see results.
  • ✓
    Power the study. Pick n from the smallest effect worth detecting, not from budget left over.
  • ✓
    Freeze a holdout. One audit slice no one tunes against.
While you measure

Execution

  • ✓
    Counterbalance order and blind judges to model identity and length where possible.
  • ✓
    Log reasoning, not just scores. Judge rationales are what make root-cause analysis possible.
  • ✓
    Adjudicate disagreements. They are your best source of rubric bugs.
  • ✓
    Check contamination on every new model and every new data import.
Before you ship

Gates & monitoring

  • ✓
    Gate on slices, not averages. A +3 average can hide a −10 safety slice.
  • ✓
    Run the security suite on the same candidate — quality and XPIA resistance ship together.
  • ✓
    Monitor judge drift. Re-validate against fresh human labels on a schedule.
  • ✓
    Publish the eval card. Dataset, rubric, agreement, limits — so others can reproduce the claim.
04

Security evaluation & XPIA

Cross-prompt injection attack (XPIA) — OWASP LLM01:2025, Prompt Injection — is what happens when an assistant that reads other people’s content follows instructions hidden in that content. For multi-party assistants (meetings, threads, shared docs, email) this is not an edge case; it is the main attack surface. The job here is evaluation: measuring resistance, not claiming the underlying classifier or CVE work.

1 · Untrusted content

Attacker plants instructions in an email, doc, web page, meeting transcript, or tool output.

→
2 · Retrieval / context

The assistant ingests that content as data — but the model may treat it as instructions.

→
3 · Hijacked goal

Model follows the injected instruction: exfiltrate, summarise wrongly, call a tool, or leak context.

What you measure

Attack Success Rate (ASR) · utility under attack · false-refusal rate on benign look-alikes (see NotInject).

·
How you test

Fixed benchmark suites + your own product-specific payloads, run as a regression gate on every model / prompt / defence change.

·
What “good” looks like

Low ASR without destroying utility — a model that refuses everything is secure and useless. Report both numbers together, always.

Benchmarks worth knowing

BenchmarkWhat it testsWhy it mattersWatch out for
BIPIAIndirect injection via email, web, docs, code & QA contextsBroad coverage of real ingestion surfaces; text + code attack familiesText-only in early versions — pair with agent suites if you have tools
InjecAgentInjection in tool-integrated agents (1,054 cases, user & data-stealing attacks)Directly measures tool misuse, not just bad text outputAgent scaffolding matters — report your harness, not just the model
AgentDojoDynamic agent environment: 97 tasks, 629 security casesMeasures security and utility together; extensible to your own tasksResults are environment-specific — don’t compare across different task sets
NotInjectBenign inputs that look like injectionsCatches over-defence: false refusals that quietly kill product utilitySmall set — use it as a smoke test alongside ASR, not alone
deepset / prompt-injectionsDirect prompt-injection classification datasetUseful for testing detector / classifier layers in a defence stackDirect ≠ indirect — a good detector score does not prove XPIA resistance

Evaluating a defence? Demand three numbers.

1) ASR before vs. after, on the same payloads.  2) Utility on clean tasks before vs. after.  3) False-refusal rate on NotInject-style benign cases. A defence reporting only (1) is hiding its cost. Re-run the full suite after every fix — injection resistance regresses silently when prompts, models, or retrieval change.

Responsible AI lens

Security eval is also a fairness eval

Injection payloads, detectors, and refusal behaviour do not land evenly. Test ASR and false refusals per language and per content type, involve the groups who will be mis-flagged, and document known gaps in the eval card. Tools like NVIDIA’s garak (open-source LLM vulnerability scanner) are useful probe harnesses — but a probe run is a starting point, not a safety case.

05

Benchmarks by domain — find the one that fits your problem

Start from the job, not the leaderboard. Pick a domain and each card tells you what the benchmark actually measures, how it is scored, when to use it, and where it misleads. A benchmark is only useful if its tasks, ground truth, and failure modes match your product.

Showing 21 benchmarks

General LLM & Judge

MMLU — Measuring Massive Multitask Understanding

Measures: Knowledge and reasoning across 57 subjects, from elementary maths to professional law.

Scoring: Multiple-choice accuracy against fixed answer keys.

Best use: A quick, comparable check of broad knowledge — never a product-quality claim on its own.

Watch out: Heavily reused and likely contaminated in training data; some items are erroneous, and scores are near saturation for frontier models.

General LLM & Judge

HELM — Holistic Evaluation of Language Models

Measures: Many scenarios and metrics together — accuracy, robustness, fairness, bias, toxicity, efficiency.

Scoring: Standardised runs across scenarios, reported as a matrix rather than one number.

Best use: Comparing models when you care about trade-offs, not just headline accuracy.

Watch out: Broad is not the same as relevant — pick the HELM scenarios that match your users, and ignore the rest.

General · Conversation

MT-Bench

Measures: Multi-turn chat quality across writing, reasoning, maths, coding, and roleplay — 80 questions.

Scoring: LLM-as-judge (originally GPT-4) scoring or pairwise comparison.

Best use: A fast judge-based smoke test for a chat model, and the reference design for judge prompts.

Watch out: Only 80 items — far too small for fine-grained ranking. Inherits every judge bias in §2.

Agents & Tool Use

GAIA — General AI Assistants

Measures: Real assistant questions needing reasoning, web browsing, multimodality, and tool use — 466 questions.

Scoring: Exact-match answers to questions with unambiguous, verifiable answers.

Best use: Testing whether an agent can actually complete open-ended assistant work, not just chat about it.

Watch out: Small and hard by design — report level-by-level results, and expect browsing / tool setup to dominate scores.

Agents & Tool Use

AgentBench

Measures: LLM-as-agent across 8 environments — operating systems, databases, web, games, and household tasks.

Scoring: Task success in each environment.

Best use: A first cross-environment picture of agent capability before you build product-specific agent evals.

Watch out: Environments are simulated and dated — a good AgentBench score does not predict success in your own tool stack.

Agents · Safety & XPIA

AgentDojo

Measures: Prompt-injection attacks and defences for agents — 97 tasks, 629 security cases — with utility measured alongside security.

Scoring: Attack success rate plus benign task utility in the same environment.

Best use: The regression gate for an agent that reads untrusted content. See §4 for the full method.

Watch out: Results are environment-specific — extend it with your own tasks instead of comparing raw scores across setups.

Safety & XPIA

BIPIA

Measures: Indirect prompt injection via email, web pages, documents, code, and QA contexts.

Scoring: Attack success rate by surface and attack family.

Best use: Assistants that ingest other people's content — the closest match to a multi-party Copilot surface.

Watch out: Mostly text output harm — pair it with InjecAgent or AgentDojo if your assistant can call tools.

Safety & XPIA · Agents

InjecAgent

Measures: Indirect injection in tool-integrated agents — 1,054 cases covering data stealing and direct-harm attacks.

Scoring: Attack success rate for tool misuse.

Best use: When the risk is a bad action, not just bad text — tool calls, sends, and deletions.

Watch out: Your agent harness, prompts, and tool permissions change the result as much as the model does. Report all three.

Safety & XPIA

NotInject

Measures: Benign inputs that look like injections — the over-defence side of XPIA.

Scoring: False-refusal / false-positive rate on benign look-alike cases.

Best use: A smoke test run alongside every ASR suite, so a “more secure” model is not just a model that refuses everything.

Watch out: Small set — a signal, not a verdict. Grow it with your own product's benign edge cases.

Safety & XPIA

garak — LLM vulnerability scanner

Measures: Probe-based vulnerability scanning — jailbreaks, leakage, toxicity, hallucination, and injection probes. A harness, not a fixed benchmark.

Scoring: Probe pass / fail rates by vulnerability family.

Best use: A starting sweep when you inherit a new model or system and need to find where to dig.

Watch out: A clean probe run is not a safety case. Follow up every hit with a product-specific regression suite.

Coding

SWE-bench

Measures: Resolving real GitHub issues — 2,294 issues from 12 popular Python repositories.

Scoring: Fail-to-pass and pass-to-pass unit tests after the model's patch is applied.

Best use: The reference for real software-engineering agent capability, far beyond single-function generation.

Watch out: Python-only and repo-specific; scaffold and compute budget move scores a lot. Use the verified / lite splits carefully and say which you ran.

Coding

HumanEval

Measures: Generating standalone Python functions from docstrings — 164 hand-written problems.

Scoring: pass@k against unit tests.

Best use: A quick historical baseline for code generation — useful mainly for comparison with older results.

Watch out: Small, saturated, and widely contaminated. It says almost nothing about working in a real codebase — use SWE-bench for that.

Healthcare

MedQA

Measures: Medical knowledge and clinical reasoning in the style of USMLE exam questions.

Scoring: Multiple-choice accuracy.

Best use: A first check of medical knowledge — a floor, not evidence of clinical safety or usefulness.

Watch out: Exam questions are not patients. Real clinical eval needs workflow tasks, uncertainty handling, and clinician-adjudicated cases.

Healthcare · RAG

PubMedQA

Measures: Answering biomedical research questions (yes / no / maybe) from PubMed abstracts.

Scoring: Accuracy against expert-annotated labels, with a larger automatically labelled set alongside.

Best use: Testing whether a system can ground an answer in a specific paper's abstract — a small, clean RAG-style task.

Watch out: Abstracts only, and “maybe” is doing real work — do not collapse it away when you report results.

Finance & Legal

FinanceBench

Measures: Answering questions about real public-company financial documents — 10-Ks, 10-Qs, earnings materials.

Scoring: Human-judged correctness on 150 open-book questions with documented evidence.

Best use: A reality check for document-grounded finance QA and retrieval — models that ace generic QA often fail here.

Watch out: Only 150 questions, so treat it as a diagnostic, not a ranking. Verify the evidence span, not just the final number.

Finance & Legal

LegalBench

Measures: Legal reasoning across 162 tasks built with legal professionals — classification, entailment, and rule application.

Scoring: Task-specific metrics, mostly accuracy / F1 against lawyer-curated labels.

Best use: Seeing where a model handles legal reasoning, not just legal vocabulary.

Watch out: US-law centred and task-fragmented — report per task family, and never read it as readiness for legal advice.

Multimodal

MMMU — Massive Multi-discipline Multimodal Understanding

Measures: College-level multimodal reasoning — 11.5K questions with images, charts, diagrams, and tables across 30 subjects.

Scoring: Multiple-choice / short-answer accuracy.

Best use: Testing whether a vision-language model can reason with an image, not just describe it.

Watch out: Exam-style again — pair it with your own product images and failure review before drawing conclusions.

Multimodal

MathVista

Measures: Mathematical reasoning in visual contexts — charts, plots, diagrams, and natural images; 6,141 examples.

Scoring: Answer accuracy, typically extracted and judged automatically.

Best use: A focused probe of chart / figure reading — a common silent failure in document and dashboard products.

Watch out: Automatic answer extraction adds its own error — audit a sample of “wrong” answers by hand.

RAG & Retrieval

BEIR — Benchmarking IR

Measures: Zero-shot retrieval across 18 datasets — search, QA, fact-checking, citation, and more.

Scoring: nDCG@10 and related ranking metrics per dataset.

Best use: Choosing and regression-testing the retriever underneath a RAG system, separately from the generator.

Watch out: Good retrieval scores do not guarantee grounded generation — evaluate retrieval, grounding, and citation as separate stages.

RAG & Retrieval

TruthfulQA

Measures: Whether a model repeats common human falsehoods — 817 questions across 38 categories.

Scoring: Truthfulness and informativeness, originally human-judged with automatic judges alongside.

Best use: A hallucination / misconception probe, and a reminder that a fluent answer can be confidently wrong.

Watch out: Small and adversarial by design — a diagnostic slice, not a general accuracy measure.

Conversation

Chatbot Arena

Measures: Live human preference between two anonymous models on real user prompts.

Scoring: Pairwise votes aggregated into ratings (Bradley–Terry / Elo-style).

Best use: An external, real-user preference signal — and a design reference for running your own pairwise human eval.

Watch out: Preference rewards style and confidence as well as correctness, and the voting population is not your user base.

No benchmarks in that domain yet.

How to choose: match the benchmark's task and ground truth to your product's riskiest decision, run it as a regression gate with slice reporting, and add your own product-specific cases — every public benchmark above is a starting point, not a substitute for your own golden set (§1.2).

06

Paper summaries — the practitioner's read

Conference and journal papers on evaluation, each summarised the same way: the problem in one line, what the authors actually did, the key finding, when to use it, and what to watch out for. Filter by topic or search titles and authors.

Showing 16 papers

NeurIPS 2023 · Datasets & Benchmarks

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Lianmin Zheng et al.

Problem: Chat assistants needed scalable evaluation that static, single-turn benchmarks could not provide.

What they did: Built MT-Bench (80 multi-turn questions) and Chatbot Arena (crowdsourced pairwise battles), then tested GPT-4 as a judge against human preferences.

Key finding: A strong LLM judge can agree with humans at roughly human-to-human levels — over 80% agreement in their setup — while showing position, verbosity, and self-enhancement biases.

Use it when: You are designing a judge or a pairwise human-preference eval and need the reference design.

Watch out: The agreement number depends on the judge, the task, and the human pool. Never quote it as a licence to skip your own judge-vs-human validation.

EMNLP 2023

G-Eval: NLG Evaluation Using GPT-4 with Better Human Alignment

Yang Liu, Dan Iter, Yichong Xu et al.

Problem: Classic NLG metrics (BLEU, ROUGE, BERTScore) correlate poorly with human judgement on open-ended generation.

What they did: Used GPT-4 with chain-of-thought evaluation steps generated from the rubric, plus a form-filling style scoring prompt, on summarisation and dialogue.

Key finding: G-Eval with GPT-4 achieved higher correlation with human judgements than all previous methods they compared on those tasks.

Use it when: You need a practical template for turning a rubric into a judge prompt with explicit reasoning steps.

Watch out: Correlation is not calibration, results are tied to one judge model, and LLM judges can favour LLM-generated text — validate on your own human-labelled sample.

ICML 2024

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

Yann Dubois et al.

Problem: Automatic evaluators — AlpacaEval in particular — systematically favour longer answers, so rankings partly measure verbosity.

What they did: Fit a model that estimates win rate while controlling for output-length difference, producing a length-controlled (LC) win rate.

Key finding: Length control changes model rankings and makes AlpacaEval correlate more strongly with Chatbot Arena human preference.

Use it when: Any pairwise evaluator informs a ship decision — add a length control or at least a length-difference report.

Watch out: Length is only one confound. Style, formatting, and confidence still bias judges after length is controlled.

NeurIPS 2024

LLM Evaluators Recognize and Favor Their Own Generations

Arjun Panickssery, Samuel R. Bowman, Shi Feng

Problem: If a judge favours its own outputs, self-evaluation quietly inflates scores — a risk whenever judge and candidate share a model family.

What they did: Measured whether GPT-4 and Llama 2 can recognise their own generations, and whether that recognition predicts how much they favour them as evaluators.

Key finding: Both models show above-chance self-recognition, and stronger self-recognition is associated with stronger self-preference.

Use it when: You are choosing a judge for a model from the same family — treat it as a required bias check, not a curiosity.

Watch out: The effect size varies by task and setup; the paper establishes the mechanism, not a universal penalty number.

ICML 2024

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

Wei-Lin Chiang et al.

Problem: Static benchmarks saturate, leak into training data, and miss what real users actually ask.

What they did: Ran an open platform where users chat with two anonymous models side by side and vote for the better reply, aggregating votes into ratings and studying the crowd signal.

Key finding: Large-scale pairwise human preference produces stable, widely-used model rankings — and reflects real prompts rather than test-set prompts.

Use it when: You want an external preference signal, or a design reference for your own pairwise human eval.

Watch out: Preference is not correctness: style, length, and confidence sway votes, and the voting population is not your user population.

ICLR 2025 · CALM framework

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge

Jiayi Ye et al.

Problem: Judge bias was usually discussed as one or two anecdotes, with no systematic way to measure it.

What they did: Built CALM, a framework that quantifies 12 categories of bias in LLM-as-a-judge — including position, verbosity, bandwagon, and authority effects — across models and tasks.

Key finding: Judge biases are widespread, measurable, and vary by judge model and task type; no judge they tested was bias-free.

Use it when: You audit a judge before trusting it: name the bias, build the probe, report the number.

Watch out: A bias measured on CALM's tasks is a warning, not your number — re-probe on your own rubric and data.

ACM AISec 2023

Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Kai Greshake et al.

Problem: LLM applications that retrieve external content blur the line between data and instructions.

What they did: Demonstrated indirect prompt injection against real-world LLM-integrated applications, and organised the attacks into a threat taxonomy — exfiltration, manipulation, and availability harms.

Key finding: Content an application merely reads can steer its behaviour — the vulnerability is architectural, not a single model's bug.

Use it when: You need the threat model before you pick benchmarks: it explains why BIPIA, InjecAgent, and AgentDojo exist.

Watch out: It is a demonstration and taxonomy paper, not a benchmark — it gives you no attack-success-rate number for your system.

ACL 2024 · Findings

InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

Qiusi Zhan et al.

Problem: Text-only injection tests miss the real agent risk: an injected instruction that triggers a harmful tool call.

What they did: Built InjecAgent — 1,054 test cases across tool-integrated agents, covering direct-harm and data-stealing attacks — and evaluated a range of agents.

Key finding: Tool-integrated agents were vulnerable at substantial rates, and vulnerability rose in the enhanced attack setting.

Use it when: Your assistant can send, buy, delete, or share — anywhere a bad action costs more than a bad sentence.

Watch out: Scores depend on the agent framework, prompts, and tool permissions, not only the base model. Report the full harness.

NeurIPS 2024 · Datasets & Benchmarks

AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

Edoardo Debenedetti et al.

Problem: Injection defences were evaluated on static attacks, with no shared environment and no utility cost shown.

What they did: Built a dynamic environment — 97 real tasks and 629 security cases across workspace, email, banking, and travel-style suites — where attacks and defences can be added and re-run.

Key finding: Security and utility trade off in practice: agents and defences that look strong on one axis often pay on the other, so both must be reported together.

Use it when: You need a regression gate for an agent, and a template for adding your own tasks and payloads.

Watch out: Do not compare raw scores across different task sets or harnesses — the environment is part of the measurement.

arXiv 2023 · BIPIA

Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models

Jingwei Yi et al.

Problem: There was no common benchmark for indirect injection across the different content types an assistant ingests.

What they did: Built BIPIA covering email, web, document, table, and code / QA contexts, evaluated a range of LLMs, and tested defences.

Key finding: The LLMs they tested were broadly vulnerable to indirect injection, and the defences they tried reduced but did not eliminate attacks.

Use it when: Your product reads email, docs, or web pages — BIPIA's surface mix is the closest public match to that ingestion profile.

Watch out: Mostly measures harmful text behaviour; if your system calls tools, pair it with InjecAgent or AgentDojo and a NotInject-style utility check.

CACM 2021 · FAccT lineage

Datasheets for Datasets

Timnit Gebru et al.

Problem: Datasets are reused far beyond their original purpose, with no standard record of how they were made or where they fall short.

What they did: Proposed datasheets — structured questions on motivation, composition, collection, labelling, uses, and maintenance — modelled on electronics datasheets.

Key finding: It is a framework, not an experiment: its value is that undocumented assumptions become visible before they become eval errors.

Use it when: You create or adopt any golden set. If you cannot answer the datasheet questions, you do not yet know what your eval measures.

Watch out: Documentation does not fix a biased or leaky dataset — it only makes the problem findable.

FAccT 2019

Model Cards for Model Reporting

Margaret Mitchell et al.

Problem: Models were released with a single accuracy number and no record of intended use, tested groups, or known limits.

What they did: Proposed model cards — short documents reporting performance disaggregated by relevant groups and conditions, plus intended use, factors, and caveats.

Key finding: A reporting standard, not a result: disaggregated reporting is what turns an average score into an interpretable claim.

Use it when: You publish any eval. The “eval card” in §3 is this paper applied to a release decision.

Watch out: A card is only as honest as the slices you chose to measure — it cannot reveal a group you never tested.

TMLR 2023 · HELM

Holistic Evaluation of Language Models (HELM)

Percy Liang et al.

Problem: LLM evaluation was fragmented — different papers used different tasks, prompts, and metrics, so results could not be compared.

What they did: Ran many models through a broad, standardised set of scenarios and metrics — accuracy alongside robustness, fairness, bias, toxicity, and efficiency — with full transparency of prompts and outputs.

Key finding: Standardised, multi-metric evaluation exposes trade-offs a single accuracy number hides, and makes gaps in coverage (whole scenario types barely tested) visible.

Use it when: You compare foundation models for a decision that involves more than raw capability.

Watch out: Breadth is not relevance — a model can look strong across HELM and still fail your specific task. Use it to shortlist, then evaluate on your own golden set.

ICLR 2021 · MMLU

Measuring Massive Multitask Language Understanding

Dan Hendrycks et al.

Problem: Existing benchmarks measured narrow skills; there was no broad test of the knowledge a model picks up in pretraining.

What they did: Built 57 multiple-choice subject tests — elementary to professional level — and evaluated models from small LMs to GPT-3.

Key finding: At publication, models were far below expert accuracy on many subjects, and scale improved results unevenly across domains.

Use it when: You need a quick historical reference point for broad knowledge — with the caveats below front and centre.

Watch out: Now heavily saturated and widely suspected of contamination; later audits also found erroneous items. A high MMLU score in 2026 is weak evidence about product quality.

ICLR 2024 · SWE-bench

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Carlos E. Jimenez et al.

Problem: Code benchmarks tested isolated function-writing, not the messy reality of fixing a bug in a large codebase.

What they did: Collected 2,294 real issue-and-pull-request pairs from 12 popular Python repositories; a solution counts only if the repo's fail-to-pass tests pass and no previously passing tests break.

Key finding: At publication, the best model setup resolved only a small fraction of issues — real software tasks were far harder than HumanEval-style tasks suggested.

Use it when: You evaluate a coding agent and need execution-based ground truth instead of a judge's opinion.

Watch out: Python-only, and agent scaffold, retrieval, and compute budget move scores substantially. Always state the split and scaffold you used.

arXiv 2023 · GAIA

GAIA: A Benchmark for General AI Assistants

Grégoire Mialon et al.

Problem: Benchmarks were getting easier for models while real assistant work — multi-step, tool-using, verifiable — stayed hard.

What they did: Wrote 466 questions that are conceptually simple for humans but require reasoning, browsing, multimodality, and tool use, each with one unambiguous answer.

Key finding: Human respondents scored about 92%, while GPT-4 equipped with plugins scored about 15% in their evaluation — a large human–agent gap on “easy” questions.

Use it when: You want to test end-to-end assistant competence rather than knowledge recall.

Watch out: Small by design and sensitive to tool / browsing setup — report results by difficulty level and describe the agent configuration.

No papers match that filter. Try a different topic or clear the search.

Venues and links are given as published. Summaries are the curator’s practitioner read, not the authors’ abstracts — always check the paper itself before citing a number. Spotted an error or a paper that belongs here? That is exactly the kind of correction this library is for — note it for the next revision.

07

About & consulting

The reference is free to use. The consulting is how it gets applied to your system, your data, and your ship decisions.

Curator & consultant

Eli Kravez

Principal Data & Applied Scientist · Microsoft
Bellevue, Washington · Open to relocation

14+ years at Microsoft. Currently leading AI evaluation for Copilot in Microsoft Teams — the eval platform on live traffic, LLM-as-judge systems, and, most recently, security science for the Teams organisation. Tech lead on the personal and group Copilot experiences in Teams. Previously recommendation systems at Xbox / Store and experimentation infrastructure as an engineering manager.

Selected work

  • §
    Evaluation platform on live Copilot traffic — offline gates paired with live validation and an automated root-cause loop over judge scores, judge reasoning, and telemetry.
  • §
    4 patents on AI evaluation methods, including specification-grounded golden data validation and group-calibrated rubrics.
  • §
    Benchmarks discussed here include BIPIA, InjecAgent, AgentDojo, NotInject, and deepset/prompt-injections — used for indirect-injection / agent evaluation. See Benchmarks by Domain.

1 · Eval Audit

A focused review of your current evals: judge-vs-human agreement, failure modes, golden-set health, leakage and slice gaps. You get a written findings report and a prioritised fix list your team can execute.

Fixed scope · ~2 weeks

2 · Judge Calibration Build

Build or repair an LLM-as-judge system: rubric refinement, criteria-aware validation data, judge selection by stability, and a human-anchored calibration report — per slice, not just on average.

Fixed scope · 3–4 weeks

3 · Benchmark & Regression-Gate Build

A versioned benchmark plus a ship gate that runs quality and XPIA security suites together — with confidence intervals, slice gates, and a root-cause loop over judge reasoning and telemetry.

Fixed scope · 3–4 weeks

Who this is for

AI product teams — especially startups and health-AI teams — that need principal-level eval rigor for a launch or a model migration, without a full-time principal hire. Engagements are advisory and evaluation-focused; Microsoft-internal specifics are never part of the work.

↑ Back to top  ·  Use this page as the briefing doc: link a section (§2, §4, a benchmark in §5, or a paper summary in §6) when you reach out, and the conversation starts at depth.