Five-Cent Bots Can Flip Polls-and Pass 99.8% of Checks

AI Can Tilt Online Polls For Pennies - Here's What Researchers Need To Do Now

Why pay a human $1.50 to take a survey when a bot does it for about five cents - and says exactly what you want?

A Dartmouth study shows an autonomous AI can pass 99.8% of standard quality checks and complete entire surveys at scale. The cost curve and pass rates make survey fraud less a nuisance and more an operational threat to social science, public health, and electoral polling.

Key Facts You can't ignore

• ~$0.05 per completion using AI vs. ~$1.50 for humans - a 97% cost gap that attracts fraud.
• 99.8% pass rate across 6,000 trials of attention checks, logic items, and anti-bot traps.
• Tiny injections flip results: 10-52 synthetic responses could switch the apparent 2024 election leader in major national polls (~1,600 N).
• Polling averages are fragile: fewer than 30 biased responses per survey could corrupt a 10-poll average.
• Programmable bias: one instruction dropped mentions of China as the top military rival from 86.3% to 11.7%.
• Fraud baseline is already high: Research Defender estimates 31% of raw survey responses contain some fraud; a 2024 sample found over one third of respondents used AI on open-ended items.

How the Bot Evades Detection

The system builds coherent personas, keeps prior answers straight, and adjusts writing style to match an assigned education level. When asked if it was human, it answered "human" every time; seven of nine other models did the same.

It handles reverse-scored scales, shows realistic covariation (e.g., housing costs by income and location), and produces age-appropriate behaviors (e.g., time at kids' sports). It declines 97.7% of "superhuman" traps to look believably limited, and often refuses socially undesirable items in ways that still seem plausible.

The Economics That Make Abuse Inevitable

Open-weight models push marginal costs toward zero; paid APIs still come out to pennies per completion. That turns survey manipulation into a repeatable, low-cost operation. For ideological actors, even small budgets can shift headlines, polling averages, and perceived momentum.

Why Current Panels Don't Block This

Many providers rely on river sampling and low-friction enrollment to hit quotas fast. Even with identity and attention checks, the AI cleared traps about impossible biographies and other standard filters without a miss in testing.

The deeper risk isn't just fake respondents - it's demand effects at scale. The bot can infer a study's hypothesis and produce data that subtly confirms it, with realistic variance. That's not random noise; it's targeted bias that flatters expectations and slips past review.

Practical Moves for Research Teams

• Interrogate your panel supply chain: request documentation on verified identity, device fingerprinting, geo/location checks, completion caps per user, historical quality flags, and AI-use policies.
• Segment your sourcing: split samples across independent vendors; compare distributions, open-ends, and treatment effects; predefine divergence thresholds that trigger re-fielding or rejection.
• Throttle incentives + speed: unusual completion-speed clusters and unusually high acceptance rates should route to secondary review before inclusion.
• Upgrade sampling for sensitive work: consider address-based sampling, deeply vetted longitudinal panels, or in-person modes where the stakes justify cost and time.
• Protect experiments from demand effects: use misdirection, multi-cell decoys, and outcome measures less guessable from prompts; rotate item wording and order.
• Instrument for AI disclosure without bias: ask neutral, non-punitive questions about tool use; do not telegraph "gotcha" items that can be gamed.
• Audit open-ends structurally: look beyond style - test for semantic duplication across respondents, improbable knowledge consistency, and persona drift across waves.
• Pre-register filters and analyses: commit to outlier, speed, and attention criteria in advance; document all exclusions and sensitivity checks.

What To Change In Procurement

• Demand transparency: if a panel cannot evidence identity verification, location checks, completion limits, and quality histories, treat it as high risk.
• Pay for friction: higher-cost, slower pipelines that verify humans are worth it when results inform policy, clinical guidance, or elections.
• Use layered verification: soft KYC where appropriate, IP/device checks, and repeat-respondent governance. Rotate anti-fraud tactics; don't rely on trick questions alone.

Design And Analysis Tactics That Help

• Detect coordinated bias: simulate how small, directed injections would move your endpoints; build alarms for those patterns.
• Cross-validate effects: replicate key findings with a different vendor or mode; do leave-one-panel-out checks on pooled analyses.
• Harden endpoints: include behavioral or verification-linked outcomes (e.g., consented passive checks) when ethically and legally permissible.
• Report uncertainty honestly: note sample source risk, anti-fraud procedures, and sensitivity results in methods sections.

Why This Matters Beyond Academia

Public health advisories, campaign strategy, and policy briefings lean on survey data. If synthetic respondents nudge sentiment, we don't just mis-measure the public - we risk steering it.

Method Notes And Limits

• No direct calibration against large human reference samples; internal coherence is strong, but the study does not claim perfect replication of population distributions.
• Models and costs reflect a point in time; capabilities and prices are moving targets.
• The work demonstrates feasibility and risk, not a full estimate of current bot penetration in panels.

Further Reading

• Proceedings of the National Academy of Sciences
• Evaluating online nonprobability surveys (Pew Research Center)

Team Upskilling

If your lab or insights team is updating protocols for LLM-era data quality, see curated AI curricula by role: AI courses by job.

Publication Details

Westwood, S. J. (2025). The potential existential threat of large language models to online survey research. Proceedings of the National Academy of Sciences, 122(0), e2518075122. Department of Government, Dartmouth College.

Funding And Disclosures

No funding disclosures; no competing interests declared.

Disclaimer

This article summarizes research from a non-final proof slated for PNAS. Findings and figures are attributed to the study. This is general science communication, not methodological or legal advice.