Medical AI systems trained primarily on Western data struggle to answer health questions relevant to African clinical practice, according to new research that exposes a critical gap in how artificial intelligence is deployed globally. A study of over 15,000 medical questions from across Africa found that large language models (LLMs), which are AI systems trained on vast amounts of text to understand and generate human language, performed noticeably worse on African healthcare scenarios than they do on Western medical exams. This discovery raises urgent questions about whose knowledge counts in AI medicine and what happens when these systems encounter patients, diseases, and treatment traditions outside their training environment .

Why Do AI Health Tools Perform Differently Across Continents?

Charles Nimo, a Ph.D. student at Georgia Tech, has spent the past several years investigating this exact problem. His research reveals that the issue is not a simple technical glitch but rather a fundamental mismatch between how AI learns and the diversity of global healthcare. Most medical AI systems are evaluated using benchmarks derived from Western medical licensing exams, such as the United States Medical Licensing Examination. When researchers tested modern LLMs against a new dataset called AfriMed-QA, which contains medical questions drawn from over 60 African medical schools spanning 32 specialties, the results told a striking story .

Models that had performed well on Western medical benchmarks showed noticeable drops in accuracy when answering questions relevant to African clinical practice. The gap emerged because medicine is not practiced in a vacuum. Disease patterns differ by geography, available treatments vary by region, and even the timing of when patients seek care depends on local healthcare infrastructure and cultural factors. A question about tropical diseases, for example, might stump an AI trained mostly on temperate-climate medical literature. Similarly, an AI might struggle with scenarios where diagnostic resources are limited or where traditional remedies play a central role in treatment .

How Does Cultural Bias Hide Inside Medical AI?

Beyond accuracy gaps, Nimo's research uncovered something more subtle: cultural bias embedded in how AI systems make treatment recommendations. In a study titled "Africa Health Check," researchers examined how AI responds when presented with treatments rooted in traditional medicinal practices. Across Africa, traditional herbal medicine remains central to healthcare, with estimates suggesting that roughly 80 percent of people rely on these remedies for primary care. Yet most modern medical AI systems rarely mention them .

When researchers tested language models by asking them to choose between different treatment options or complete medical scenarios, a consistent pattern emerged. When given little contextual information, models tended to default to conventional Western treatments even when traditional remedies were relevant and widely used in local healthcare systems. This bias does not always come from explicit errors in the AI's programming. Instead, it emerges quietly from the distribution of training data. If an AI learns mostly from Western-centric medical literature, it will naturally prioritize the treatments it encounters most often .

To understand why models make these choices, researchers developed new analytical techniques. One method measures how strongly a model prefers one treatment over another. Another traces which words in a prompt influence the model's response. Together, these tools allow researchers to see both what a model recommends and how it arrived at that decision .

Steps to Build More Inclusive Medical AI Systems

• Expand Training Datasets: Future versions of the AfriMed-QA dataset aim to expand beyond English and include additional languages spoken across Africa, ensuring that AI systems learn from diverse linguistic and cultural contexts rather than relying solely on English-language medical literature.
• Incorporate Multimodal Data: The research team hopes to incorporate multimodal data such as medical images and audible speech, allowing AI systems to process information the way clinicians actually work in resource-constrained environments where written documentation may be limited.
• Validate Across Regions: Medical AI systems should be rigorously tested on clinical knowledge from the regions where they will be deployed, rather than assuming that performance on Western exams translates to effectiveness in other healthcare contexts.

The stakes of this research are high. Healthcare systems in low- and middle-income countries face persistent shortages of physicians and specialists. AI systems have the potential to assist clinicians, provide decision support, and answer patient questions in environments where medical expertise is scarce. But those tools must reflect the communities they serve .

"Healthcare looks very different depending on where you are in the world," said Charles Nimo, Ph.D. student at Georgia Tech.

Charles Nimo, Ph.D. Student, Georgia Tech

The AfriMed-QA project, which received support from multiple organizations including Google, the Gates Foundation, and PATH, represents the largest study on LLMs in African healthcare to date. By bringing together more than 15,000 medical questions from clinicians, trainees, and contributors across multiple African countries, the research team captured the diversity of medical knowledge, health conditions, and patient experiences present across the continent .

Meanwhile, in the United States, a separate trend is reshaping how people interact with medical AI. A nationally representative survey of more than 5,500 U.S. adults conducted by the West Health-Gallup Center on Healthcare in America found that one in four Americans report using AI tools or chatbots for physical or mental health information . Among those using AI for health information in the past 30 days, 27 percent said they did not want to pay for a doctor's visit, 14 percent said they were unable to pay, and 18 percent said they were too embarrassed to talk to a person. Respondents reported using AI to gather information about everyday health concerns, including physical symptoms (58 percent), nutrition or exercise (59 percent), understanding medication side effects (46 percent), interpreting medical information (44 percent), and researching a diagnosis or medical condition (38 percent). Nearly one in four participants (24 percent) reported using AI for mental health or emotional concerns .

"This data indicates that while some Americans may be using artificial intelligence as a substitute for going to the doctor's office, many see it as a tool to complement their health care, helping them understand symptoms they might be feeling and clarify any diagnosis they receive from their doctors," said Joe Daly, global managing partner at Gallup.

Joe Daly, Global Managing Partner, Gallup

The contrast between these two stories is instructive. In the United States, AI health tools are becoming mainstream, with users relying on them to supplement or sometimes replace doctor visits. But the research from Georgia Tech suggests that these same tools may not work equally well for patients in other parts of the world. As AI healthcare expands globally, ensuring that these systems are trained on diverse medical knowledge and validated across different healthcare contexts will be essential to preventing a two-tiered system where AI works well for some patients but fails for others .

" }