Moon
Product Benchmarks Research Pricing News Contact

AI is meaningful when you
can measure how it is doing.

Run an eval

Every model, every release,
every prompt you actually run.

Moon runs the evals your model ships against, tracks the regressions before your users find them, and hands your team the recipe for the next release.

CONVERSE 42

Multiturn dialog quality

4,214 prompts17 domains
GROUND 18

Retrieval faithfulness

1,812 promptsCited answers
DRIFT 9

Regression detection

928 promptsRelease over release

Every prompt, every surface, once a day.

Moon runs your evals on every release, every prompt template and every provider you serve. Silent regressions surface before your users file them.

Judged by humans, not just other models.

Every eval gets two human ratings and a model rating. Moon reconciles the three and shows you where the model rater is fooling itself.

A recipe for the next release.

Every regression comes with the prompt, the diff, the failure mode, and the smallest change likely to close the gap. Ship models you can defend.

Support agent, v42
4,214 prompts, three tone tiers
pass 94.2
Sales opener, v18
1,812 prompts, six industries
pass 88.7
Doc summarizer, v9
928 prompts, five document classes
pass 91.4
Onboarding tutor, v3
612 prompts, four personas
pass 82.1
Recovery flow, v12
418 prompts, edge cases only
pass 96.8
1
Prompt fires
Live traffic or bench replay
2
Two annotator pass
Blind ratings, structured rubric
3
Model rating reconciled
Third rater settles ties
4
Recipe attached
Smallest change likely to close the gap
Regressions caught
+18
Before release, this month
Silent failure rate
0.4%
Down from 3.1% at baseline
Human agreement
0.92
Two rater F1, held out set
Evals per day
14K
Across four providers
Recipes shipped this quarter
42 changes, 118 regressions closed, one model swap
Every change with the eval trace behind it

Three things Moon does
that most eval stacks skip.

Real prompts, real traffic

Moon runs your evals on the prompts your users actually send, not just the benchmarks a paper claimed matter.

Human plus model rating

Every eval gets both. Moon surfaces where the model rater is fooling itself and where the humans disagree.

A recipe, not just a score

Every regression comes with the failure mode, the diff, and the smallest change likely to fix it.

Moon is the first eval platform that told us
which regression to fix first.

Tomás RennerApplied research lead, Field Guide Labs

Six lines of code.
An eval on every release.

Drop in the Moon SDK, register your evals, ship every release with the score attached and the recipe queued.

evals/run.py
# Score a release against your evals

from moon import Evals



evals = Evals("support_agent")

     .register("tone", "grounding", "safety")

     .run(release="v42")



print(evals.summary())

# pass 94.2, regressions 2, recipe attached

Ship the models
you can defend.

Free on the first hundred evals. Bring a prompt, we bring the raters, and the first score lands within an hour.