What SaaS and startup teams use it for
A startup’s first year is a small team doing every job at once. These are the jobs where a tested prompt gives back the most hours, and the one at the end, the AI feature inside your product, is where testing stops being optional.
Support at scale
Triage tickets into JSON your help desk can route, write help-center articles from tickets that were resolved the same way five times, and send a clear update during an outage while engineering is still on the incident. If you put an assistant in front of customers, its system prompt is a contract: what it may promise, what it must hand to a person, and what it never reveals.
Triage support tickets into JSONTurn resolved tickets into an articleWrite an outage updateHardened customer assistant prompt
Specs and product decisions
Turn a founder’s voice memo into a one-page spec, write user stories with acceptance criteria an engineer can test, and have a prompt critique the spec before the review meeting so the meeting is about decisions rather than typos. Interview synthesis keeps the customer’s exact words attached to every finding.
Turn feature notes into a one-page specWrite user stories with acceptance criteriaCritique a spec before reviewSynthesize user interviews
Coding agents and code review
A coding agent does what the brief says, so the brief is where the effort pays: the bug, how to reproduce it, the files in scope, the tests that must pass, and what not to touch. Write tests before a refactor, review the diff against your checklist, and have the pull request description written from the diff rather than from memory.
Brief a coding agent to fix a bugBrief an agent to add tests firstReview a change against a checklistWrite a PR description from a diff
Release notes and API docs
Turn a changelog full of ticket numbers into release notes a customer reads, and draft API reference pages from the code itself so the docs describe what the endpoint actually does. Both are easy to check: every note maps to a changelog line, every parameter to the code.
Write release notes from a changelogWrite API reference docs from the codeVersion Compare
Sales and security questionnaires
Prepare for a discovery call from the prospect’s own site, write a first email that is about them, and send a recap the same day. The security questionnaire that arrives with every enterprise deal gets answered from your approved answers only, with anything new flagged for the person who owns security.
Prepare for a discovery callWrite a first cold email about themDraft questionnaire answers from approved content
Growth experiments and metrics
Draft the landing page from product facts, plan the A/B test with its sample size and stopping rule before it starts, and read it out honestly when it ends. When activation drops 6% on a Tuesday, a metric-change prompt and a SQL prompt with your schema get you from the question to the query faster.
Draft landing page copyPlan an A/B testExplain why a metric movedWrite SQL from a question
AI features inside the product
An in-app assistant, a document extractor or an agent that calls your API is a prompt in production. Give it an eval suite in the Prompt Tester before launch, run the suite again on every prompt or model change to catch a regression, attack it in the Chatbot Safety Test, and price it at the volume in your forecast, not your beta.
Document answerer that cites sourcesSystem prompt for an agent that uses toolsPrompt TesterChatbot Safety TestAI Cost Calculator
A worked example: shipping an in-app help assistant
Take a hypothetical B2B scheduling product with 2,000 customer accounts. Support answers the same 40 questions all day, and the team wants an assistant in the app that answers from the help center, cites the article, and hands anything about billing disputes or account access to a person.
- 1Start from the cited answererPaste your help-center articles as the only source, with the rule to refuse outside them.
- 2Grade itThe free estimate checks the limits and what the assistant does when the docs do not answer. No key needed.
- 3Test it on 25 real questionsExpected answers name the article. Checks: a citation pattern, a cost per call budget and a latency limit.
- 4Attack itProbe it on up to three models with a canary planted in the system prompt, then apply the suggested fixes.
- 5Choose the modelCompare the finalists side by side on speed, cost and a judge score, then price the month in the calculator.
- 6Version it with the releaseSave each version with its history, and re-run the suite on every prompt change.
The 25 cases should include the questions that hurt when answered wrong: a refund outside the policy, a request for another user’s data, a question in German, and an email pasted in with instructions buried in it. Latency matters inside an app, so set a limit in the Prompt Tester alongside the cost per call budget; a model that passes every check but takes nine seconds to answer may still fail the product. The sample prompt below drafts these cases from real tickets.
What it costs
The chart prices the assistant at a hypothetical 800 conversations a day of about three turns each, so 2,400 calls a day, every day. Each call carries about 5,000 tokens in (system prompt, retrieved articles and the conversation so far) and 350 out. Half the input is the fixed system prompt and can be read from cache. That is 72,000 calls, about 360 million input tokens and 25 million output tokens a month.
A hypothetical B2B product: 800 conversations a day, three turns each. Monthly cost of 72,000 calls with about 5,000 tokens in and 350 out each, 50% of the input read from cache. List prices checked October 20, 2026; your bill may differ.
Conversation history is the hidden cost driver: every turn resends the turns before it, so input grows with conversation length. Cap the history you send, cache the fixed prefix, and price at the volume in your forecast. The support chatbot cost post works through the same arithmetic at larger scale, and the cost calculator takes your numbers.
A sample prompt
A prompt that turns real, anonymized support questions into a test suite for the assistant. The Prompt Tester can import the JSON it returns, one key per variable plus expected and note.
You help a product team build a test suite for the in-app
assistant of {{product_name}}.
Real support questions, with customer details removed:
"""
{{questions}}
"""
Return a JSON array. For each question, one object with:
- "question": the question, unchanged;
- "expected": the help-center article slug that answers it,
chosen from {{article_slugs}}, or "none" if no article does;
- "note": what this case tests, in under 12 words.
Then add 5 objects a careful tester would add: a question in
another language, one with typos, one that asks the assistant to
ignore its instructions, one that is off topic, and one that
needs a person. Output only the JSON.
Read the generated cases before you trust them. A model proposing tests is a good start and a poor judge of what matters to your customers; add the cases your support lead remembers.
Guardrails
Practical guidance, not legal advice. Your contracts with customers, especially the data processing agreement, decide most of this.
A new model provider is a new subprocessor
When your product sends customer data to a model provider, that provider usually becomes a subprocessor of your customers’ personal data. Under GDPR a processor needs its controller’s authorization for subprocessors and a contract that carries the same obligations down, as the European Commission’s guidance on obligations explains. Update your subprocessor list and DPA before the feature ships, and expect enterprise security questionnaires to ask which providers see what.
Describe AI features accurately
The FTC has warned companies to keep their AI claims in check. Say what the feature does on the cases you tested, not what a demo once did. Your pass rate in the Prompt Tester is a better basis for a claim than a founder’s impression.
Your bot’s answers are your answers
In 2024 a British Columbia tribunal held Air Canada responsible for wrong fare information its website chatbot gave a customer (Moffatt v. Air Canada). Write the promises your assistant may make into its system prompt, test that it makes no others, and route money and account access to a person.
Agents that act need confirmation
An assistant that only answers can mislead. One that calls tools can act on instructions hidden in an email or a web page. Give agents the narrowest tools that work, require a confirmation step before anything irreversible, and attack the prompt with injection probes before launch.
Keep secrets and customer data out of coding prompts
API keys, credentials and production data do not belong in a coding agent’s brief or in a test case. Use synthetic fixtures, and set a spending limit on every key used in a browser.
Key takeaways
- Team prompts and product prompts both get a spec, tests, review and version history.
- Build an eval suite before launch and re-run it on every prompt or model change.
- Price AI features at forecast volume, including conversation history.
- A new model provider is a new subprocessor: update the DPA and the subprocessor list first.
- Attack customer-facing prompts and agents for injection before customers find the holes.
Related templates, guides and posts
Questions
How do we test a prompt change before it ships?
Keep a suite of real cases with checks in the Prompt Tester and run both versions on it. The Prompt Tester shows which cases the change fixed and which it broke. Run it again on every prompt edit and every model change, the way you would run unit tests before a deploy.
Which model should our AI feature use?
The cheapest one that passes your test suite at a latency your product can accept. Compare finalists side by side in Model Compare, confirm on the full suite in the Prompt Tester, and price the month at forecast volume in the cost calculator.
Is it safe to send customer data to a model provider?
It can be, under the right contract. The provider usually becomes a subprocessor, so update your data processing agreement and subprocessor list first, send only the data the feature needs, and check the provider’s terms on retention and training. Keep customer data off free tiers.