Complete AI Training
Sign inGet my AI kit

Your job's AI kit

Get your AI kit

Tell us who you are and what you do. We show you your kit right away and email you the link: skills, prompts, AI agents, MCP servers and courses for your job.

500+ jobs ready, and we make a kit for any other job. No payment needed to look.

Share

AI agent for traders

Backtest Sanity Check Agent

A report showing how much of the backtest edge survives realistic costs and out-of-sample testing

Quality controlAnalysisFinance
See how it worksGet access
Backtest Sanity Check Agent: what goes in, what the agent does and what you get

What it does

A backtest with a smooth equity curve can be an illusion. This agent takes a strategy description and the backtest results, then audits them. It checks the number of trades, the number of tunable parameters against the sample, and whether fees and slippage are set to realistic values. It looks for data a trader could not have had at the time, such as using the close to decide an entry at the open. It then tests the strategy on periods the parameters were not tuned on. When a check fails, it reruns with realistic costs or a split sample and reports how much the edge shrinks. The trader decides whether the idea deserves paper trading. Edge case: a strategy that only works on one year of data is reported as a single-regime result.

How it works

Follow the arrows from top to bottom. The orange dashed arrow is the loop: when a check fails, the agent goes back and tries again.

Start and resultWhat it doesA check on its own workWaits for your OKGoes back and retries
Yes, continueYes, continueApprovedNoNo 1 STARTS WHEN Trader submits a finished backtest 2 DOES Count trades and parameters and compare to thesample 3 USES A TOOL Review fee and slippage settings against the brokerschedule 4 DOES Scan the rules for look-ahead and survivorshipproblems 5 CHECKS THE RESULT Are costs and data realistic? If not: rerun the backtest with realistic costs andreport the change in results. Back to step 3. 6 USES A TOOL Split the data and test on out-of-sample periods 7 CHECKS THE RESULT Does the edge hold out of sample? If not: try a walk-forward split and report how much theedge shrinks. Back to step 6. 8 DOES Write a sanity report with an edge before and after 9 YOU APPROVE Trader decides whether to move to paper trading 10 RESULT Sanity report saved with the strategy
Read the steps as a list
  1. Trader submits a finished backtest
  2. Count trades and parameters and compare to the sample
  3. Review fee and slippage settings against the broker schedule
  4. Scan the rules for look-ahead and survivorship problems
  5. Are costs and data realistic?If not: rerun the backtest with realistic costs and report the change in results. Back to step 3.
  6. Split the data and test on out-of-sample periods
  7. Does the edge hold out of sample?If not: try a walk-forward split and report how much the edge shrinks. Back to step 6.
  8. Write a sanity report with an edge before and after
  9. Trader decides whether to move to paper tradingThe agent waits here for your OK.
  10. Sanity report saved with the strategy

How it decides

The result passes when the edge stays positive after costs, holds in the out-of-sample periods and the sample has enough trades for the parameter count.

  • Require at least 30 trades per tunable parameter
  • Reject results where fees are set to zero
  • Fail the test when out-of-sample profit factor is below 1.1
  • Report any single year contributing over half of the profit

Make it yours

Every agent is a starting point. You choose these settings for your own situation.

  • Fee and slippage assumptions
  • Minimum trades per parameter (default 30)
  • Out-of-sample split (default last 30%)
  • Pass threshold for profit factor

What keeps you in control

It always asks you first

  • Trader moves the strategy to paper trading

Hard limits

  • Never changes the strategy rules to improve results
  • Does not recommend live capital

It stops when

  • Done: the report states edge before and after adjustments
  • Stop: trade list or settings are missing

Set it up

We guide you through the set-up, step by step

Members get the full set-up guide for this agent. No technical skills needed: you copy, paste and upload.

10 minto set it up in your AI
5 AIsChatGPT, Claude, Copilot, Gemini, Grok
  • One set of instructions to paste into your AI, with the clicks for ChatGPT, Claude, Microsoft 365 Copilot, Gemini and Grok
  • The agent then walks you through connecting your own data, one source at a time
  • A downloadable copy with the flow chart, the rules and the full guide
Get access to this agent

An example run

What happensA mean reversion test showed 18% a year over 210 trades with $0 fees. The agent set a $2 commission and 1 cent slippage, which cut the return to 9%. Out of sample, the last two years returned 2%, so the check failed and it ran a walk-forward split, giving 4%. It reported the edge as thin and mostly from one year. The trader chose more testing before paper trading.

More agents for traders

Analysis · Automation

Watchlist Setup Screening Agent

A ranked short list of watchlist candidates that meet the trader's rules and pass the safety checks

10 steps2 checks1 job
Monitoring · Risk management

Strategy Drift and Drawdown Monitor Agent

A clear status of whether the live strategy is inside its tested range, with a size or pause proposal when it is not

10 steps2 checks2 jobs
Planning · Creating

Weekly Trading Plan Builder Agent

An approved weekly plan with scenarios, levels, events and total planned risk inside the trader's limits

10 steps2 checks2 jobs
Analysis · Decisions

Execution Quality and Slippage Audit Agent

A monthly report of execution costs by broker, order type and time, with approved changes that cut them

11 steps2 checks2 jobs