Complete AI Training
Sign inGet my AI kit

Your job's AI kit

Get your AI kit

Tell us who you are and what you do. We show you your kit right away and email you the link: skills, prompts, AI agents, MCP servers and courses for your job.

500+ jobs ready, and we make a kit for any other job. No payment needed to look.

Share

AI agent for statisticians

Public Survey Quality Check Agent

A quality report that says how far each survey result can be trusted, with corrected results and ranges

Quality controlGovernment
See how it worksGet access
Public Survey Quality Check Agent: what goes in, what the agent does and what you get

What it does

A survey result goes into a briefing with no check on who answered, and later someone finds it overstates one group. This agent checks sample size, response rates, question wording and weighting for any survey a team plans to use. It compares the sample with census data on age, income, region and other traits to see who is over- or under-represented. It reads the questions for leading or double-barreled wording. When it finds problems, it recomputes the headline results with corrected weights and reports ranges instead of one number. It checks that the corrected weights bring the sample close to the population and that the weighted results do not hinge on a handful of respondents. The analyst approves the summary. Edge case: a result based on only 18 respondents in a subgroup is reported as not reliable.

How it works

Follow the arrows from top to bottom. The orange dashed arrow is the loop: when a check fails, the agent goes back and tries again.

Start and resultWhat it doesA check on its own workWaits for your OKGoes back and retries
Yes, continueYes, continueApprovedNoNo 1 STARTS WHEN Survey dataset received 2 USES A TOOL Load the data, questionnaire and methodology notes 3 DOES Check sample size, response rate and missing data 4 USES A TOOL Compare sample traits with census benchmarks 5 DOES Review question wording for bias 6 DOES Recompute headline results with corrected weights 7 CHECKS THE RESULT Does the weighted sample match census shares withintolerance? If not: adjust the weights or collapse categories andrecompute. Back to step 6. 8 CHECKS THE RESULT Do results hold when the largest weights aretrimmed? If not: cap extreme weights and report a wider range.Back to step 6. 9 DOES Write the summary with ranges and unreliablesubgroups 10 YOU APPROVE Analyst approves the summary 11 RESULT Quality report attached to the dataset
Read the steps as a list
  1. Survey dataset received
  2. Load the data, questionnaire and methodology notes
  3. Check sample size, response rate and missing data
  4. Compare sample traits with census benchmarks
  5. Review question wording for bias
  6. Recompute headline results with corrected weights
  7. Does the weighted sample match census shares within tolerance?If not: adjust the weights or collapse categories and recompute. Back to step 6.
  8. Do results hold when the largest weights are trimmed?If not: cap extreme weights and report a wider range. Back to step 6.
  9. Write the summary with ranges and unreliable subgroups
  10. Analyst approves the summaryThe agent waits here for your OK.
  11. Quality report attached to the dataset

How it decides

A result is reliable when the sample is large enough and the weighted sample is close to census shares. Subgroups with small samples are reported as not reliable.

  • Mark subgroups under 30 respondents as unreliable
  • Require weighted shares within 2 points of census shares
  • Flag leading or double-barreled questions
  • Report a range, not a single number, when weights are trimmed

Make it yours

Every agent is a starting point. You choose these settings for your own situation.

  • Benchmark source
  • Subgroup minimum (default 30)
  • Weighting variables
  • Tolerance (default 2 points)

What keeps you in control

It always asks you first

  • The summary before it is shared

Hard limits

  • Never changes the raw data
  • Does not decide whether to use the survey

It stops when

  • Done: the report and corrected results are approved
  • Stop: data has no way to be weighted

Set it up

We guide you through the set-up, step by step

Members get the full set-up guide for this agent. No technical skills needed: you copy, paste and upload.

10 minto set it up in your AI
5 AIsChatGPT, Claude, Copilot, Gemini, Grok
  • One set of instructions to paste into your AI, with the clicks for ChatGPT, Claude, Microsoft 365 Copilot, Gemini and Grok
  • The agent then walks you through connecting your own data, one source at a time
  • A downloadable copy with the flow chart, the rules and the full guide
Get access to this agent

An example run

What happensA resident survey had 1,240 respondents, but adults over 65 were 31% of the sample against 19% in the census, so the check failed. After weighting, support for a transit levy fell from 64% to 58%. Trimming the largest weights gave a range of 56 to 60%. A renter subgroup of 22 people was marked not reliable. The analyst approved the summary.

More agents for statisticians

Quality control · Product development

Product Experiment Sample Integrity Agent

Experiment samples that are valid before anyone draws conclusions.

8 steps2 checks3 jobs
Quality control · Science and research

Research Figure Reproduction Agent

A reproducible figure bundle whose values and labels match the analysis.

7 steps2 checks3 jobs
Quality control · Decisions

Research Analysis Assumption Agent

Every analysis has its assumptions checked, with violations decided by the investigator.

8 steps1 check11 jobs
Planning · Decisions

Landing Page A/B Test Planning Agent

A single-change test plan that is launched correctly and read at the right time

11 steps2 checks6 jobs