AI Shows Promise in Identifying Childhood Cancer Survivors Who Need Extra Support
Researchers at St. Jude Children's Research Hospital found that large language models can analyze patient interviews to detect symptoms severe enough to warrant additional care for childhood cancer survivors. The work, published in Communications Medicine, suggests that how researchers prompt AI systems significantly affects their accuracy.
The team tested two AI models-ChatGPT and Llama-on transcripts from 30 interviews with young survivors and their caregivers. Human experts had previously analyzed the same conversations for signs of pain and fatigue, categorizing symptoms by severity and their physical, cognitive, or social impact. The AI systems performed comparably to the experts, but results varied dramatically based on the prompting method used.
Simple prompts failed; complex ones worked
Researchers compared four prompting strategies. The two simpler approaches-zero-shot and few-shot prompting, which provide minimal context-produced unstable and inaccurate results.
The two complex strategies performed significantly better. Chain-of-thought prompting uses step-by-step logical instructions, while generated knowledge prompting asks the model to develop background information before receiving the task. Both methods distinguished physical and cognitive impacts well, though they showed moderate ability detecting social impacts.
Why this matters for cancer care
Physicians spend roughly 40 to 60 percent of clinical encounters listening to patients describe symptoms and health experiences. That conversational data often remains unanalyzed because reviewing transcripts manually is time-consuming.
I-Chan Huang, the study's corresponding author, said the work "provides a proof of concept that large language models could help analyze that underutilized conversational data to detect symptom severity and its functional impact and assist physician decision-making."
Cancer and its treatments can cause long-term effects in survivors treated during critical developmental periods. Identifying which survivors need targeted support has proven difficult for physicians.
Next steps require more testing
The findings represent an early example of how generative AI and large language models might improve survivorship care. Clinical use will require substantially more testing before implementation.
The research team included scientists from St. Jude, the University of Memphis, Hallym University, Wake Forest University School of Medicine, and Stanford University Medical School. The work was funded by the National Cancer Institute, St. Jude's Cancer Center Support grant, and ALSAC, St. Jude's fundraising organization.
Your membership also unlocks:
