Responsible AI: Do We Need Humanlike Agents? It Depends
Agentic AI is moving from tool to teammate - systems that pursue goals, make decisions, and act with little oversight. The harder question isn't just how to build them, but when to make them humanlike in tone, behavior, and presence.
Across a panel of 34 AI experts and a global survey of 1,221 executives, nearly 80% agree: responsible AI governance should question the necessity of overly humanlike agentic systems. The short version - just because you can make AI feel human doesn't mean you should.
Why question humanlike design in the first place?
- Overtrust and confusion: People disclose more, defer more, and assume competence where there may be none when an AI "feels" human.
- Blurred accountability: Anthropomorphic cues can obscure who's responsible for decisions and outcomes.
- Persuasion risk: Humanlike framing increases the chance of manipulation, especially in high-stakes or emotionally charged contexts.
- Regulatory and legal exposure: Claims about AI performing like experts are already drawing scrutiny and penalties. See guidance on AI marketing claims from the U.S. Federal Trade Commission here.
Where humanlike AI helps
- Accessibility and education: Natural language and empathetic tone can lower barriers to use and comprehension.
- Caregiving and companionship: Friendly, social interfaces may support engagement for older adults or isolated users.
- Coaching and tutoring: Conversational guidance can improve adherence and learning outcomes when paired with strong safeguards.
Value here comes from usability and comfort - but it's a design choice with asymmetric risks. The burden of proof sits with builders to show why human likeness is essential.
Where to avoid humanlike cues
- High-stakes decision support: Finance, medical, and legal contexts risk false authority and harmful deference.
- Systems prized for speed and clarity: Algorithmic trading and control systems work because they are explicit and machine-like.
- Persuasive or engagement-optimized apps: Humanlike framing can push users into overuse, dependency, or poor choices.
- Appliances and utilities: A dishwasher doesn't need "emotional speech." Neither does most back-office software.
Core risks to manage
- Misplaced trust: Users think they're talking to a person, overvalue outputs, and skip verification.
- Attachment and overreliance: Emotional mimicry can lead to dependency and distress, especially for vulnerable users.
- Manipulation: Humanlike agents can become dangerously persuasive, intentionally or not.
- Compliance gaps: Undisclosed humanlike features can clash with transparency and consumer protection rules.
Five practical governance moves
- 1) Assess the need first: Require a clear, written justification for any anthropomorphic feature. If the use case doesn't benefit, don't add it.
- 2) Educate builders: Make it explicit that AI mimics empathy; it doesn't feel it. Train teams on anthropomorphism risks and mitigation patterns.
- 3) Design deliberately: Default to the minimum necessary social presence. Add stronger safeguards for children, the elderly, or high-risk contexts.
- 4) Test before deploying: Run controlled pilots that measure behavioral effects of humanlike cues (trust calibration, disclosure rates, deference, escalation).
- 5) Continuously monitor: Track signals of harm and intervene. Examples: growing session length, late-night use, self-harm intent, rising first-response compliance without verification.
Transparency that actually helps
- Always-on disclosure: Clear, persistent "AI assistant" identity - in text and voice - plus reminders in long sessions.
- Style and naming: Avoid human names, avatars, and backstories unless justified; keep tone professional and clearly synthetic.
- Friction for high-stakes tasks: Require confirmations, show confidence levels, cite sources, and route to humans when uncertainty is high.
- Auditability: Log prompts, outputs, and key decisions; enable independent review and safe rollback.
Metrics worth watching post-launch
- Engagement drift: Session length, daily minutes, and use during typical sleep hours.
- Trust calibration: Rate of user verification actions (link clicks, second opinions) after AI recommendations.
- Safety signals: Escalations, self-harm indicators, sensitive data disclosure events.
- Fairness and access: Disparate impact across user groups; opt-out rates for human assistance.
A simple decision checklist
- What problem are we solving, and who benefits or could be harmed?
- Why is human likeness necessary for this outcome, specifically?
- What goes wrong if users think it's human?
- Which users are vulnerable, and what off-ramps to humans exist?
- What evidence from pilots shows benefits exceed risks?
- Who is accountable for outcomes and ongoing monitoring?
Standards to anchor your approach
Map your policies to widely recognized principles for trustworthy AI and consumer protection. As a starting point, see the OECD AI Principles here.
Bottom line
Humanlike AI is a tool, not a default. Use it where it clearly improves access, clarity, or care - and only with strong transparency and monitoring. Elsewhere, keep systems machine-like, explicit, and accountable.
Build team capability
If you're formalizing RAI practices or upskilling stakeholders, you can scan structured options by role here: Complete AI Training - Courses by Job.
Your membership also unlocks:
