How the tests work
Each test asks one question of the prompt. A test passes only when the prompt itself states what the test asks for. Anything the model would have to guess does not count, however obvious it seems to you.
Not every test matters for every request. A one-line factual question does not need an example to match. When the Grader’s AI review decides a check cannot matter for a request, it marks it n/a, and that does not count against the prompt. A blank such as [AUDIENCE] counts as supplied, because you fill it in before you send the prompt.
- 1TaskDoes it say exactly what to produce or do?
- 2PurposeDoes it say who or what the result is for?
- 3MaterialDoes it supply the facts and context the model cannot guess?
- 4ScopeDoes it set the size: length, count, duration or depth?
- 5FormDoes it name the output format or structure?
- 6LimitsDoes it say what to leave out or avoid?
- 7StandardDoes it say what a good result looks like?
- 8ReferenceDoes it give an example or a reference to match?
- 9DoubtDoes it say what to do when something is unclear or missing?
- 10FitIs it laid out the way the target model reads best?
To make the tests concrete, this guide follows one weak prompt from start to finish. A product manager has a pile of customer feedback after a release and asks a model for help. Each test adds one fix.
1. Task
Does it say exactly what to produce or do?
Can you look at this feedback?
Group the customer feedback below into themes and summarize each theme.
Why it matters. “Look at” could mean summarize, critique, reply to, translate or count. The model will pick one, and it may not be the one you meant. A precise verb (group, summarize, classify, draft, compare, extract) is the cheapest improvement any prompt can get.
2. Purpose
Does it say who or what the result is for?
Summarize each theme.
The summary is for our product team’s planning meeting, where they will choose the two problems to fix next quarter.
Why it matters. Purpose changes what matters in the answer. A planning meeting needs to know how often each problem comes up; a support lead would want the individual customers to follow up with. Once the model knows how the result will be used, it can make those choices for you. Anthropic’s guidance makes the same point: give the reason behind an instruction, such as what the output will be used for.
3. Material
Does it supply the facts and context the model cannot guess?
Summarize the feedback about our new release.
Feedback, one comment per line:
"""
{{feedback}}
"""
Context: These comments arrived in the two weeks after version 4.2 of our invoicing app shipped. That release moved the Export button into the Share menu and added a dark theme.Why it matters. The model knows nothing about your product, your customers or what changed last week. Whatever it is not told, it fills in with something plausible. Here, knowing that the Export button moved lets it connect a cluster of “where is export?” comments to their cause.
4. Scope
Does it set the size: length, count, duration or depth?
List the main themes.
Scope: The five most common themes, most common first, each summarized in at most two sentences.
Why it matters. Without a size, length is the model’s choice, and it changes from run to run. A size is also a priority: “five themes” means pick the five that matter most, not all eleven. And since output is the expensive side of a call, a size is often the easiest way to cut cost.
5. Form
Does it name the output format or structure?
(no format stated)
Format: A table with three columns: theme, number of comments, one representative quote.
Why it matters. The format decides whether the result can be pasted into a slide, read by code or skimmed in a meeting. If software will read the output, name the exact structure, such as JSON keys or column names, and test that every output follows it.
6. Limits
Does it say what to leave out or avoid?
(nothing excluded)
Leave out: comments about pricing, which finance is reviewing separately, and customer names.
Why it matters. Limits keep out what would distract, repeat someone else’s work, or create a problem, such as customer names in a document that will be shared. Give the reason for a limit, as above, so the model can apply it to cases you did not list. Limits work best for content. For format, describe what you want instead of listing what to avoid; Anthropic’s guidance recommends exactly that.
7. Standard
Does it say what a good result looks like?
Make it clear and useful.
A good result: someone who has not read the feedback can rank the problems by how many customers they affect.
Why it matters. “Clear and useful” is a wish. A standard is a test someone could apply. It tells the model what to optimize for, and it gives you, or an AI judge, a criterion for checking the result. The testing guide shows how to turn standards into checks.
8. Reference
Does it give an example or a reference to match?
(no example)
Example row: | Export is hard to find | 14 | "I spent ten minutes looking for the export button." |
Why it matters. An example shows tone, length and format faster than a description can. Models also copy examples closely, so a single narrow example can narrow every answer. Anthropic’s guidance suggests three to five varied examples when you use them; Google’s suggests giving examples an identical structure and format, and pairing them with written instructions.
9. Doubt
Does it say what to do when something is unclear or missing?
(nothing said about gaps)
If unclear: put comments too vague to assign under a theme called "Unclear" instead of guessing. If you find fewer than five themes, list only the ones you find.
Why it matters. A model with no instruction for gaps tends to fill them with a confident guess. Look at the second sentence: the scope asked for five themes, and without this line the model might invent a fifth to fill the table. OpenAI’s guidance asks for the same thing: say what to do when information is missing.
10. Fit
Is it laid out the way the target model reads best?
Here's our feedback: {{feedback}} can you group it into themes for the planning meeting, top five, as a table, and skip pricing stuff.<feedback>
{{feedback}}
</feedback>
<context>
These comments arrived in the two weeks after version 4.2 shipped. …
</context>
<instructions>
Group the feedback above into themes and summarize each theme. …
</instructions>Why it matters. The same words, laid out differently, are read differently. Anthropic, OpenAI and Google each publish guidance on how to structure a prompt, and they do not agree on everything: tags or headings, material first or last. The guide to Claude, GPT and Gemini sets their recommendations side by side.
The Mohs scale
The Grader reports a prompt’s hardness: the number of tests it passes, from 1 to 10, named after the Mohs scale that mineralogists use. On that scale, each mineral can scratch the ones below it. Talc, at 1, crumbles under a fingernail; diamond, at 10, is scratched by nothing natural. A prompt that passes no tests still reads 1, talc.
| Hardness | Mineral | What it means |
|---|---|---|
| 1 | Talc | Crumbles under a fingernail. The model has to guess nearly everything. |
| 2 | Gypsum | A fingernail (2.5) still marks it. |
| 3 | Calcite | A copper coin (3.5) scratches it. |
| 4 | Fluorite | A knife cuts it with little effort. |
| 5 | Apatite | A steel knife (5.5) only just marks it. |
| 6 | Orthoclase | Scratches window glass. A steel file (6.5) marks it. |
| 7 | Quartz | Scratches steel. Good enough for most work. |
| 8 | Topaz | Scratches quartz. |
| 9 | Corundum | Only diamond marks it. |
| 10 | Diamond | Nothing natural scratches it. |
Two things about the scale are worth keeping in mind.
- It is an order, not a measurement. The Mohs scale ranks minerals; it does not say diamond is ten times harder than talc. Likewise, a 9 is not “90% good”. It means nine of the ten questions are answered in the prompt.
- Not every prompt needs diamond. A quick one-off question is fine at talc. A prompt that will run thousands of times, or that other people will rely on, deserves quartz or harder.
The Grader can score a prompt in two ways. The estimate runs in your browser as you type and looks for telltale words with simple rules. It is instant and free, and easy to fool in both directions. The assay uses an examiner model with your own key and reads for meaning. Use the estimate to see what is plainly missing while you write; use the assay before a prompt goes into production. Neither replaces running the prompt on real inputs.
All ten together
Here is the feedback prompt before and after. The final version follows the plain-label layout that suits any chat model: material first, then the task and its conditions, one labeled part per test.
Can you look at this feedback?
Feedback, one comment per line:
"""
{{feedback}}
"""
Context: These comments arrived in the two weeks after version 4.2 of our invoicing app shipped. That release moved the Export button into the Share menu and added a dark theme.
Task: Group the feedback above into themes and summarize each theme. The summary is for our product team's planning meeting, where they will choose the two problems to fix next quarter.
Scope: The five most common themes, most common first, each summarized in at most two sentences.
Format: A table with three columns: theme, number of comments, one representative quote.
Leave out: comments about pricing, which finance is reviewing separately, and customer names.
A good result: someone who has not read the feedback can rank the problems by how many customers they affect.
Example row:
| Export is hard to find | 14 | "I spent ten minutes looking for the export button." |
If unclear: put comments too vague to assign under a theme called "Unclear" instead of guessing. If you find fewer than five themes, list only the ones you find.With Written for set to Any chat model, the Grader’s free estimate scores the first version 1, talc, and the final version 10, diamond. Switch Written for to Claude and the Fit test fails, because Anthropic’s guidance favors XML tags around each kind of content; the estimate drops to 9 until you add them.
Hardness on the Mohs scale. Written for Claude, the Fit test fails until each part is wrapped in XML tags.
Notice what did not change: the request. The final prompt asks for exactly what the first one meant. It just stops making the model guess.
Key takeaways
- Ten questions: task, purpose, material, scope, form, limits, standard, reference, doubt and fit.
- A test passes only when the prompt says it. What the model must guess does not count.
- The cheapest fixes come first: a precise verb, who the result is for, and a size.
- Give reasons for limits, and say what to do when something is missing.
- Hardness is a count on an ordinal scale. Quartz, 7, is enough for most work.
- A well-written prompt still needs testing on real inputs before it ships.