83%: The Diagnostic Accuracy Gap That Could Reshape Medical Care

In a simulation involving complex medical cases, a new artificial intelligence system diagnosed correctly 83% of the time, a figure that is more than double the 36% accuracy rate achieved by generalist physicians working individually. This stark contrast, reported in a study published in Science this week, highlights a potential shift in how medical expertise is delivered. Mustafa Suleyman, CEO of Microsoft AI, described the result as a significant step toward achieving what the company calls medical superintelligence.

The system, known as MAI-DxO, functions by coordinating multiple AI agents that mimic a panel of doctors. These agents interrogate patient data, order tests, and debate potential diagnoses. The study utilized difficult cases from the New England Journal of Medicine. It is important to note that such complex cases are typically handled by specialists or teams in real-world settings, and those groups were not included in the comparison. Nevertheless, the performance gap suggests that current AI orchestration can provide substantial benefits in diagnostic reasoning.

The core of the improvement appears to come from the underlying power of large language models rather than the orchestration layer alone. Microsoft tested the system with models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest scores were achieved using OpenAI’s GPT-5 and GPT-5.5. When GPT-5 was used with a basic prompt, it scored 78.9% on complex cases at a cost of approximately $8,400 per case. The DxO orchestration reduced this cost by about $3,000 while adding roughly four points to the accuracy score. However, on emergency department cases, the system showed little improvement in either accuracy or cost efficiency compared to the base model.

The benefits of the orchestration system are most pronounced when using less capable models. For example, Gemini 2.5 Flash scored 46.4% on its own but rose to nearly 69% with the assistance of DxO. Despite these gains, the gap between weaker and stronger models remained larger than the boost provided by the orchestration. Microsoft researchers noted that accuracy gains naturally diminish as models approach their theoretical limits. The system aims to optimize both cost and accuracy, but its value is currently highest in complex scenarios where base models struggle.

Microsoft is moving beyond simulations to real-world testing. The company has partnered with the Mayo Clinic to train a foundation model for healthcare, with a release planned for next year. Suleyman believes this model will enable significant leaps in capability. He also announced a new partnership with Beth Israel Deaconess Medical Center in Boston to evaluate the clinical reasoning capabilities of DxO across different hospital settings. In this study, patients will receive standard treatment while consenting to parallel engagement with the AI system. Researchers will then compare the AI’s recommendations against those of human doctors.

This effort coincides with a broader industry push to define medical superintelligence as the capacity to surpass contemporary human-led health systems in reasoning and coordination. Microsoft aims to move beyond benchmarking individual tasks to integrating with clinicians for better outcomes. Dominic King, vice president of Health at Microsoft AI, stated that the technology is ready for real-world evaluations. He emphasized that the complexity of issues in settings like a Boston emergency department provides a rigorous test for these systems.

Suleyman, who previously co-founded DeepMind, sees this as a continuation of his career-long focus on using AI to solve medical issues. He expects healthcare models to improve by an order of magnitude in capability within the next 24 to 36 months. He argues that progress accelerates once a flywheel effect is established, where models in production are iterated upon using real-world usage data to identify and patch gaps. This process has already driven rapid progress in coding and chat interfaces, and he anticipates a similar trajectory for clinical decision support starting next year.

The company views this as a strategic differentiator against other frontier labs. Microsoft already sells dictation and radiology tools to clinicians and sees the new diagnostic system as an expansion of those efforts. Suleyman stated that the goal is to have the number one state-of-the-art medical clinical decision support product in the coming years. He believes that by 2027, the company will be on the verge of a breakthrough in coding medical records, allowing for highly accurate predictions about how a medical record might unfold. This foresight, he argued, is invaluable for clinicians managing patient care.