Article on The combined generalist-specia...

Researchers at the Hong Kong University of Science and Technology have built a low-cost AI framework that pairs a generalist medical model with lightweight specialist models. The system, called GSCo, outperformed both standalone approaches on disease diagnosis tasks while using far less computing power than fine-tuning a large foundation model, according to findings published in Nature Biomedical Engineering.

How GSCo works

The Generalist-Specialist Collaboration framework coordinates MedDr, an open-source generalist foundation model developed at HKUST, with a set of specialist models trained for specific diagnostic tasks. MedDr was trained on more than two million multimodal medical data points. The specialist models are light enough to train on a single consumer-grade GPU, such as an NVIDIA RTX 4090. The design addresses a known trade-off in medical AI. Generalist models handle a wide range of tasks but can lack diagnostic precision, while specialist models are accurate but narrow, and often require separate models for different use cases or hospitals. Retraining a large generalist model is also expensive and impractical when patient data must stay on-site. "Generalists and specialists in medical AI each possess distinct strengths: one is flexible but imprecise across many diseases, while the other is accurate but narrow, focusing on a single task," said Dr Chen Hao, research lead and assistant professor in HKUST's Department of Computer Science and Engineering. During inference, the specialist models identify and compare historical cases and provide diagnostic predictions. MedDr then combines that evidence with its own knowledge base to produce the final diagnosis. The researchers said the approach could make AI for Healthcare integration more practical in settings with limited computing resources.

What the benchmarks showed

The team validated MedDr and GSCo on 32 public datasets containing about 260,000 medical images, covering diagnosis, visual question answering, and radiology report generation. Findings were published in Nature Biomedical Engineering. MedDr outperformed existing medical generalist models including RadFM, LLaVA-Med, and Med-Flamingo. GSCo ranked highest in both internal and external benchmarks, beating ten existing vision foundation models. In skin lesion diagnosis, GSCo scored 84 out of 100, compared with MedDr's zero-shot score of 75.45 and the top standalone specialist score of 82.92. In a stress test, MedDr correctly identified 67.6% of tumor images despite systematically biased input. For chest X-ray report generation, six of seven board-certified radiologists preferred GSCo's output over that of the specialist model R2GenGPT. The efficiency gains are substantial. Adapting the generalist model required about 2.5 × 10¹⁹ floating-point operations (FLOPs), while training one specialist model under the study's benchmark setting took no more than about 5.2 × 10¹⁶ FLOPs. That translates to up to a hundredfold reduction in model development costs for new clinical tasks. "GSCo makes adaptation more accessible to hospitals and research teams that lack the resources to repeatedly fine-tune a large generalist foundation model," Chen said. He added that specialist models can be trained on task-specific data using consumer-grade hardware, allowing healthcare institutions to address new clinical tasks "at substantially lower computational cost while keeping sensitive data locally."

Why this matters for IT and development teams

GSCo offers a working template for deploying AI in compute-constrained environments: keep a large foundation model fixed, train small specialist models per task, and let the generalist coordinate their outputs. The study measured the cost gap precisely, and the same pattern applies to AI for IT & Development work beyond healthcare. When a foundation model is too expensive to fine-tune repeatedly, a small task-specific model that feeds into it can get the job done with consumer-grade hardware. The team has made both MedDr and GSCo open source. Next steps include expanding MedDr to support 3D imaging modalities like CT and MRI, plus medical video. Researchers also plan to explore test-time computation, where models perform more extensive reasoning during inference to improve accuracy.