AI Prompts Library

I am being sold on a model based on the benchmark claims below. Stress-test them.

For each claim:
1. Is the benchmark contamination-prone? Public repos, well-known problem sets and anything predating the model's training cutoff all are.
2. Was the number produced under conditions I can reproduce? Note any scaffolding, retries, best-of-N, or custom harness.
3. What does the benchmark actually measure, and is that the thing I care about?
4. What is the sibling benchmark this result should predict, and does it?
5. What is conspicuously absent from the claim set?

Then tell me: what would I have to test myself to know whether this model is better for MY workload, and what is the cheapest version of that test?

Claims:
[PASTE]
My workload:
[DESCRIBE]

Plan a migration from [CURRENT MODEL] to [TARGET MODEL] for a production workload.

Cover:

1. BEHAVIOURAL DELTAS to expect: tokenizer differences and their effect on cost, system prompt interpretation, refusal boundaries, structured output strictness, tool-calling format, streaming behaviour.
2. WHAT WILL SILENTLY BREAK: the changes that pass tests and fail in production. Be specific.
3. TOKEN COUNT IMPACT: different tokenizers mean the same text costs different amounts. Estimate the direction and rough size.
4. SHADOW TEST PLAN: how to run both models on real traffic and compare, including what to compare on and the sample size needed.
5. ROLLOUT: percentage stages, what metric gates each stage, and the rollback trigger.
6. WHAT TO DELETE AFTERWARDS: the workarounds for the old model's quirks that should not survive the migration.

Workload:
[DESCRIBE]