Microsoft AI beats doctors in complex cases
A new diagnostic system outperformed generalist physicians in simulations, marking a key step toward Microsoft's goal of medical superintelligence.
By Layla Fontaine ·
Microsoft’s new diagnostic AI has demonstrated a significant advantage over human specialists in handling difficult medical scenarios. According to a study published this week in Science, the system correctly identified complex conditions in simulations at a rate of 83%. This performance far exceeded the 36% accuracy rate achieved by primary care doctors and general internists working individually with reference materials. Mustafa Suleyman, the CEO of Microsoft AI, described the findings as a critical move toward achieving what the company calls medical superintelligence.
The technology, known as MAI-DxO, operates by coordinating multiple AI agents that function like a panel of physicians. These agents ask questions, order tests, and debate potential diagnoses before reaching a conclusion. The study used complex cases from the New England Journal of Medicine for its simulations. It is important to note that such complex cases are typically managed by specialists or teams in real-world settings, who were not included in the direct comparison group.
Dominic King, vice president of Health at Microsoft AI, emphasized the practical implications of the results. He noted that the current healthcare system faces a growing mismatch between the demand for medical expertise and its supply. By publishing these results, Microsoft joins rivals like Google and OpenAI in claiming that its AI can match or surpass human doctors. However, the data suggests that much of the improvement stems from more powerful underlying models, with the orchestration system providing additional gains.
Cost and accuracy were both measured during the testing phase. When using OpenAI’s GPT-5 model alone with a basic prompt, the system achieved 78.9% accuracy at a cost of approximately $8,400 per case. The DxO system reduced that cost by about $3,000 while raising accuracy by four points. Results varied with newer models like GPT-5.5, where DxO improved accuracy from 72% to nearly 87% on complex cases. For emergency department cases, however, the improvements in both accuracy and cost were minimal.
Microsoft is now preparing for real-world validation through a partnership with Beth Israel Deaconess Medical Center in Boston. The collaboration will evaluate the clinical reasoning capabilities of the system across various hospital settings. Patients will receive standard care, while those who consent will also interact with the AI. The company plans to compare the AI's recommendations against those of human doctors. Suleyman expressed confidence that 2027 will bring a breakthrough in processing medical records, allowing for highly accurate predictions of patient outcomes.
This study coincides with a broader manifesto from Microsoft on building medical superintelligence. The goal is to move beyond benchmarking individual tasks and integrate the AI into clinician workflows for better overall outcomes. Suleyman believes this focus will differentiate Microsoft from other frontier labs. The company already sells dictation and radiology tools and views this new system as a major expansion of its healthcare portfolio. He stated that the company aims to have the top state-of-the-art clinical decision support products in the next few years.
