Complete AI Training
Sign inGet my AI kit

Your job's AI kit

Get your AI kit

Tell us who you are and what you do. We show you your kit right away and email you the link: skills, prompts, AI agents, MCP servers and courses for your job.

500+ jobs ready, and we make a kit for any other job. No payment needed to look.

Share

AI agent for prompt engineers

Model Upgrade Prompt Migration Agent

Produce a tested migration plan that tells which prompts can move to the new model and which need work

Quality controlAutomationIT and development
See how it worksGet access
Model Upgrade Prompt Migration Agent: what goes in, what the agent does and what you get

What it does

When a new model version arrives, the team wants its speed or price, but each prompt may react differently. Testing all of them by hand is slow, and some breakages only show in rare inputs. This agent runs every production prompt's test set on both the current and the new model and compares scores side by side. It sorts prompts into three groups: ready as is, needs small changes, and needs rework. For the middle group it tries targeted fixes, such as restating the format or adding an example, and retests. If a fix does not bring the prompt within the allowed gap, it moves the prompt to the rework list with notes on what failed. The prompt lead approves the migration plan and the order of switching. Edge case: a prompt that improves on average but fails a critical test still counts as not ready.

How it works

Follow the arrows from top to bottom. The orange dashed arrow is the loop: when a check fails, the agent goes back and tries again.

Start and resultWhat it doesA check on its own workWaits for your OKGoes back and retries
Yes, continueYes, continueApprovedNoNo 1 STARTS WHEN New model approved for evaluation 2 USES A TOOL Load all production prompts and their test sets 3 USES A TOOL Run each test set on current and new model 4 DOES Compare scores and critical test results per prompt 5 DOES Sort prompts into ready, small fix and rework 6 USES A TOOL Apply a targeted fix to each small-fix prompt andretest 7 CHECKS THE RESULT Is the fixed prompt within 2 points and passing allcritical tests? If not: try one other fix; if still failing, move it torework with notes. Back to step 6. 8 CHECKS THE RESULT Have all critical tests been run on the new modelfor every prompt? If not: run the missing tests before planning. Back tostep 3. 9 YOU APPROVE Prompt lead approves the migration order and reworklist 10 RESULT Migration plan with per-prompt results
Read the steps as a list
  1. New model approved for evaluation
  2. Load all production prompts and their test sets
  3. Run each test set on current and new model
  4. Compare scores and critical test results per prompt
  5. Sort prompts into ready, small fix and rework
  6. Apply a targeted fix to each small-fix prompt and retest
  7. Is the fixed prompt within 2 points and passing all critical tests?If not: try one other fix; if still failing, move it to rework with notes. Back to step 6.
  8. Have all critical tests been run on the new model for every prompt?If not: run the missing tests before planning. Back to step 3.
  9. Prompt lead approves the migration order and rework listThe agent waits here for your OK.
  10. Migration plan with per-prompt results

How it decides

A prompt is ready when its new-model score is within 2 points of the old one and it passes every critical test; otherwise it gets one round of fixes before going to rework.

  • Any failed critical test makes a prompt not ready, whatever its average
  • Allow two fix attempts per prompt before rework
  • Order switching by risk: lowest-traffic ready prompts first
  • Report cost and latency change per prompt next to quality

Make it yours

Every agent is a starting point. You choose these settings for your own situation.

  • Allowed score gap (default 2 points)
  • Fix attempts per prompt (default 2)
  • Which tests count as critical (default those tagged critical)
  • Rollout batch size (default 5 prompts per week)

What keeps you in control

It always asks you first

  • Migration order and go-live dates
  • Each prompt switch in production

Hard limits

  • Never switches a production prompt to the new model
  • Uses only test data cleared for model testing

It stops when

  • Done: plan approved
  • Stop: new model fails critical tests on most prompts; recommend waiting

Set it up

We guide you through the set-up, step by step

Members get the full set-up guide for this agent. No technical skills needed: you copy, paste and upload.

10 minto set it up in your AI
5 AIsChatGPT, Claude, Copilot, Gemini, Grok
  • One set of instructions to paste into your AI, with the clicks for ChatGPT, Claude, Microsoft 365 Copilot, Gemini and Grok
  • The agent then walks you through connecting your own data, one source at a time
  • A downloadable copy with the flow chart, the rules and the full guide
Get access to this agent

An example run

What happensAcross 34 prompts, 21 were ready, 9 needed small fixes and 4 needed rework. The contract-clause prompt scored 1 point higher but failed the critical test for missing dates, so it stayed not ready. A restated format block fixed 7 of the 9 on the first try. The other 2 went to rework. The lead approved switching 28 prompts over three weeks.

More agents for prompt engineers

Risk management · IT and development

Prompt Injection Red-Team Agent

Find the ways your prompt can be attacked and fix them without hurting normal use

11 steps1 check3 jobs
Quality control · IT and development

Golden Test Set Maintenance Agent

Keep the test set current so real failures become permanent tests

11 steps2 checks5 jobs
Decisions · IT and development

Model Selection Bake-Off Agent

Choose a model with evidence on quality, cost and speed for the real task

11 steps2 checks7 jobs
Data cleanup · IT and development

Prompt Library Cleanup Agent

Leave a smaller library where each purpose has one tested, owned prompt

11 steps2 checks1 job