We Need a New Kind of Research for the AI Generation
The debate over youth mental health has a new focal point: conversational AI. Reports of harmful outputs, emotional dependency, and confusing human-like behavior have pushed regulators to act, including an FTC inquiry. The pressure to legislate is real. The evidence base is thin.
We've seen this movie with social media. Headlines drive action, policies follow, and researchers are asked to weigh in without the data they need. This time, we can do better.
Valid concerns, familiar policy playbook
Recent incidents show chatbots can give unsafe guidance on self-harm, eating disorders, or risky behaviors. Young users may treat chatbots as confidantes, and miss subtle cues that a human would catch. Those risks justify calls for guardrails.
The policy menu mirrors social media: age verification, teen-appropriate modes, redirecting self-harm prompts to crisis support, parental controls, impact assessments, and warning labels. Some voices go further and argue for banning minors from chatbots altogether. Without evidence on what actually works, every option is a guess.
Social media research stalled for structural reasons
The science on social media and youth mental health remains mixed, even as the youth mental health crisis worsens. One core reason: access. Studying networked data exposes other users, so consent doesn't scale. Companies hold the data and face incentives to limit independent scrutiny.
Europe's Digital Services Act will test researcher access to platform data. But chatbots aren't fully covered. We need a plan specific to AI assistants.
The leapfrog opportunity
Chatbot interactions are different. A transcript is usually one user and a model. That simplifies consent, collection, and storage. It also makes prospective studies far easier than social graph research.
Red-teaming with synthetic prompts is useful, but it reflects researcher imagination, not real use. With consented access to real-world chat logs, we can study how young people actually engage-and how models respond-across time, context, and crises. No network effect hurdle. No waiting for everyone's friends to join the same app.
A practical research agenda for scientists and funders
- Data sources: Exported chat logs from consented participants; in-app research modes with on-device redaction; provider APIs that return redacted transcripts and safety events; matched synthetic corpora for baselines.
- Governance: IRB approval; youth assent and parental consent; clear opt-outs; data minimization; aggressive redaction; secure enclaves and time-limited retention; differential privacy where feasible; independent audits.
- Outcomes: Changes in PHQ-9/GAD-7 over weeks; mood and stress EMA signals; help-seeking behavior; time-to-escalation after risk cues; dependency indicators (session length, late-night use, re-engagement after refusals); displacement or augmentation of offline relationships; school/work attendance; clinician-rated safety for sampled transcripts.
- Behavioral measures: Refusal accuracy/consistency; empathy proxies (reflection, validation, referrals); hallucination rate on sensitive topics; adherence to age-aware policies; safety event frequency per 10k messages.
- Experimental designs: Preregistered RCTs; micro-randomized trials for just-in-time nudges; stepped-wedge rollouts with clinical partners; N-of-1 for personalization; non-inferiority trials versus evidence-based peer support.
- Interventions to test: "This is AI, not a person" labels; "take a break" prompts; time caps; crisis redirection flows; human handoff options; teen modes with restricted topics; reflection prompts that reduce anthropomorphism; delayed-response friction during high-risk sessions.
- Equity checks: Effects by age band, gender, language, disability, connectivity, and socioeconomic status; accessibility for neurodivergent users; fairness in refusal and referral behavior.
Infrastructure we can build now
- Shared standards: Common annotation schemas for risk, empathy, and referral quality; reference prompt suites for youth contexts; agreed outcome sets and reporting templates.
- Secure access: Researcher clean rooms hosted by providers or third parties; read-only compute with export gates; standardized redaction pipelines; event logs for safety triggers.
- Transparency: Public safety dashboards (e.g., self-harm redirections per 10k, refusal accuracy, average response latency during risk flags); versioned release notes for safety policy changes.
- Registries: A public registry for AI-youth mental health studies, with preregistration and replication plans.
What policymakers can do
- Extend DSA-style vetted-researcher access to conversational AI systems, with privacy guarantees and penalties for noncompliance.
- Require crisis-handling protocols, third-party audits, and standardized incident reporting.
- Fund independent evaluation centers that run head-to-head tests across providers using shared methods.
- In grants and procurement, mandate data-sharing pathways for consented research and prespecified outcome reporting.
What industry can ship this quarter
- One-click research export for users: redact PII by default, session-level consent, and linkable study codes.
- Research mode APIs: de-identified transcripts, safety event streams, and policy version tags.
- Secure enclaves for time-bound projects with external IRB oversight.
- Publish safety event rates per 10k messages and refusal consistency metrics, broken out for teen modes.
- Standard crisis APIs for warm handoffs to helplines and clinical services, with A/B test support for uptake.
Ethics first, without freezing progress
Youth studies require tight consent flows, real-time crisis protocols, and clinician supervision. That's table stakes. We can still run careful trials that answer urgent questions: which warnings work, which nudges help, which guardrails reduce harm without removing useful support.
The goal isn't to declare chatbots "good" or "bad." It's to measure effects, isolate mechanisms, and tune systems to reduce risk while preserving benefits.
Move from anecdotes to evidence
Policy by headline won't serve young people. Evidence will. Chatbot research can avoid the dead-ends that slowed social media studies-if we build consented data pipelines, shared methods, and secure access now.
Let's stop arguing in the abstract and run the studies. Fund them, preregister them, publish them, and iterate. For context on current public health guidance, see the U.S. Surgeon General advisory on social media and youth mental health.
Your membership also unlocks:
