thinkhard.SI

Prompt repository Engineering & code

Brief a coding agent to write a data backfill script with a dry run

Existing records need correcting in bulk, and you want a coding agent to write a backfill script that is safe to rerun and starts with a dry run.

Registry no.
ENG-049
Collection
Engineering & code
Laid out for
Coding agent
Score
10 out of 10
Blanks to fill
11
Added

Engineering & codeCode agent

The prompt

Score 10 of 10, Excellent

Use it when Existing records need correcting in bulk, and you want a coding agent to write a backfill script that is safe to rerun and starts with a dry run.

  • {{script_path}}
  • {{backfill_change}}
  • {{selection_rule}}
  • {{schema}}
  • {{pattern_file}}
  • {{database_details}}
  • {{expected_counts}}
  • {{test_command}}
  • {{staging_dry_run}}
  • {{sample_size}}
  • {{off_limits}}
Full prompt and why it works
Prompt
Goal: Write a backfill script at {{script_path}} (for example scripts/backfill_order_totals.py) that {{backfill_change}} for the records that match {{selection_rule}}, with a dry run, so that the data can be corrected in batches while the application keeps running.

Context:
- Tables and columns involved, with their types and indexes:
"""
{{schema}}
"""
- An existing script or management command to copy for database access, configuration and logging: {{pattern_file}}
- Database and approximate row counts: {{database_details}}
- Expected counts, from queries I ran on staging: {{expected_counts}}
- Test command: {{test_command}}
- Command to run the script against staging in dry-run mode: {{staging_dry_run}}

Constraints:
- Dry run is the default. The script writes only when given an explicit flag such as --execute, and the dry run prints the same counts the real run would change, plus before and after values for a sample of {{sample_size}} records.
- Idempotent: select only rows that still need the change (for example WHERE new_column IS NULL), so a second run changes nothing.
- Batched: walk the table by primary key ranges (keyset, not OFFSET), with a configurable batch size and an optional pause between batches. Each batch commits in its own short transaction, and the script can resume with --start-after-id.
- Log each batch: ID range, rows matched, rows changed, time taken. Print a summary at the end.
- Write through a path that does not trigger side effects such as emails, webhooks or cache events from model save hooks, or stop and tell me if there is no such path.
- Read connection settings from the environment the way the pattern file does; never put a connection string or credential in the script.
- No schema changes in this task. Do not touch {{off_limits}}.
- Never run the script with --execute against any database. Production runs are mine.

Tests, in the project's existing test setup:
- A dry run changes nothing.
- An execute run changes exactly the matching rows and no others.
- A second execute run changes zero rows.
- Batch edges: a table with exactly one batch of matching rows, and one with one row more.

Done when:
- {{test_command}} passes.
- {{staging_dry_run}} reports counts that match {{expected_counts}}. If they differ, stop and report the difference with sample IDs; do not adjust the selection to make them match.

Verify: run the tests and the staging dry run, then report in this format: Usage (flags and examples), Dry-run counts next to the expected ones, Sample of before and after values, Time per batch and estimated total run time, Commands run with the last 20 lines of their output.

Variables

{{script_path}}
Where the script goes, for example scripts/backfill_order_totals.py or a management command path.
{{backfill_change}}
What it does, for example "fills the new total_cents column from the order lines".
{{selection_rule}}
Which records, for example "orders created before the new column existed, where total_cents is null".
{{schema}}
The CREATE TABLE statements or model definitions for the tables involved, with indexes.
{{pattern_file}}
An existing script whose style, database access and logging to copy.
{{database_details}}
For example "PostgreSQL 15, orders table about 30 million rows, busiest 9 am to 6 pm".
{{expected_counts}}
What your own count queries on staging returned, for example "412,880 orders need the change".
{{test_command}}
For example pytest tests/scripts.
{{staging_dry_run}}
The exact command, for example "python scripts/backfill_order_totals.py --env staging", or "I will run it myself".
{{sample_size}}
How many records to show before and after, for example 20.
{{off_limits}}
Tables, files or jobs that must not change, for example the nightly reporting job.

Why it works

  • What good looks like. Done means the staging dry run matches the counts you measured, and a mismatch stops the work instead of being tuned away.
  • What to avoid. Dry run is the default, the script is idempotent and batched, and the agent never runs it with --execute, so a mistake costs a rerun rather than damaged data.
  • Background facts. Expected counts from your own staging queries give the agent a number to hit that it did not choose.

How to use it

  1. Press Fill in blanks and type your details into each blank. What you type stays in this browser.
  2. Copy the filled prompt into the AI tool you already use, such as ChatGPT, Claude or Gemini, or open it in the Grader or the Prompt Tester here.
  3. Before you rely on it, try it on three or four real examples. The Prompt Tester runs them all at once and shows which ones go wrong.
Use it from code

Get this prompt as JSON from the free REST API, with your own text in the blanks, or let an AI assistant fetch it through the MCP server. No key.

curl -s https://thinkhard.si/api/v1/prompts/coding-agent-data-backfill

With the MCP server added to Claude, ChatGPT, Cursor or VS Code, call get_prompt with {"prompt": "coding-agent-data-backfill"}, or just ask for it by name.