AI in Health Care: The Ethical Frontier at AMP 2025
At the Association for Molecular Pathology (AMP) Annual Meeting in Boston, Takunda Matose, MBe, PhD, challenged a common belief: more data does not automatically mean better AI. He urged pathologists and developers to rethink data practices, accuracy claims, and oversight if they want AI to improve clinical outcomes.
His core message was simple: AI is probabilistic. It helps remove bad options and sharpen decisions, but it won't guarantee the right answer. That truth should shape how we collect data, validate tools, and communicate results to patients and clinicians.
Rethinking "more data = better AI"
Health care produces an estimated 14% to 30% of global data, yet roughly 90% of it goes unused. Much of what we store wasn't collected for model development, so adding more of it can add noise, not signal.
Think of AI as a system for conditional probabilities. As new information comes in, the odds shift. The classic Monty Hall problem is a helpful analogy: removing a known bad option improves your chances, but it doesn't guarantee success. Here's a quick refresher if you need it.
Bias isn't optional-manage it
Bias is baked into choices about data sources, training sets, validation cohorts, and annotation. Even a well-built system can pick up new biases during deployment.
One risk is the compositional fallacy-assuming population-level results transfer neatly to individuals. Model accuracy depends on data quality, labeling, and how cohesive your datasets are. You can't shortcut that.
Stop treating accuracy as a single truth
Accuracy isn't universal. It shifts with the clinical task, the population, and the context of use. Vendor-reported metrics may look impressive, but they're not guarantees.
Past performance is only probabilistically predictive of future performance. Treat metrics as decision inputs-not as promises.
Human + AI works when roles are clear
"Human in the loop" isn't a cure-all. Human judgment adds value, but it also adds variability and bias. The key is designing workflows that use each strength at the right moment.
Define where AI should narrow options and where clinicians should make the call. Then pressure-test that division in real clinical settings.
Data are social facts-treat them that way
Data gain meaning in aggregate and carry social implications. A single genetic or clinical data point can reveal insights about families and communities. That raises stakes for privacy, security, and consent.
We need policies that match the tech we're using today-on collection, retention, and disclosure. Transparency is non-negotiable.
Architecture trade-offs: integration vs interoperability
Vertical integration concentrates exposure but reduces points of failure. Interoperability spreads control-and responsibility. Both paths come with trade-offs that should be weighed deliberately.
Standards like FAIR help, but they're hard to operationalize at scale. If you're new to FAIR, start here: FAIR Principles.
Practical checkpoints for clinical teams
- Define a precise clinical question and the decision it supports. Avoid scope creep.
- Use the smallest sufficient dataset. Prioritize relevance, quality, and clear labeling over volume.
- Document every annotation rule. Track who labeled what, when, and how disagreements were resolved.
- Validate on the population you intend to serve. Test transportability before wider rollout.
- Pre-specify metrics and acceptable trade-offs (e.g., sensitivity vs specificity) tied to clinical impact.
- Run prospective evaluations. Monitor calibration, drift, and subgroup performance over time.
- Set failure protocols: what happens when the model abstains, conflicts with clinician judgment, or degrades?
- Clarify human roles. Decide when to defer to the model, when to override, and how to document those choices.
- Minimize data collection. Keep audit trails for retention, access, and disclosures that could affect families.
- Communicate limitations clearly to clinicians and patients. Be upfront about uncertainty and failure rates.
What this means for your practice
AI will never be 100% accurate. That's a feature of probability, not a flaw. If we plan for it-ethically and operationally-we can still improve workflows and outcomes.
For many use cases, smaller, well-annotated datasets and clear governance can beat sprawling data pools. That approach also supports broader access, including settings with fewer resources.
The goal isn't blind trust in algorithms or defaulting to human judgment. It's thoughtful oversight that pairs the strengths of both.
Further learning
If your team is building AI skills for clinical settings, here's a curated list of programs by role: AI courses by job.
Your membership also unlocks:
