Microsoft says its MAI-DxO diagnostic AI was more accurate than generalist doctors in a study published in Science

Microsoft AI said in a post dated 22 October 2026 that its work on the Microsoft AI Diagnostic Orchestrator (MAI-DxO) had been published that day in Science.^[1] Crossref’s record of the paper, “Sequential diagnosis with language models”, lists it in Science volume 394, issue 6822, pages 423–428, dated 22 October 2026.^[2] The paper’s abstract says the system achieved 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases and 80.3% on MIMIC-IV-ED emergency department admissions, against 36.1% and 46.8% for generalist physicians.^[2] The abstract adds that “these findings were obtained in a controlled benchmark setting”.^[2] The Verge, TechCrunch and Gizmodo reported the study on 22 October.^[3][4][5]

Microsoft said MAI-DxO works like a panel of virtual physicians that propose diagnoses, pick questions or tests and challenge the emerging view over several rounds, and that its Sequential Diagnosis Benchmark (SDBench) covers 304 complex NEJM cases and a set of more typical emergency-department cases from MIMIC-IV-ED.^[1] Microsoft said physicians scored 22% on the NEJM cases without reference resources and 36% with them, making MAI-DxO about 3.7 and 2.3 times more accurate, and that it reached diagnoses at half or even one third of the cost.^[1] Gizmodo reported that those savings are based on estimated costs in simulated cases rather than actual medical expenses.^[5] Microsoft said it is open-sourcing SDBench for select NEJM cases so researchers can reproduce the evaluation and challenge the findings.^[1]

TechCrunch reported that Microsoft tested MAI-DxO with models from OpenAI, Google, Anthropic, Meta and DeepSeek, and that its best results came with OpenAI’s GPT-5 and GPT-5.5.^[4] TechCrunch reported that GPT-5 on its own with a basic prompt scored 78.9% on the complex cases at about $8,400 per case, and that the orchestrator cut that cost by about $3,000 while raising accuracy by only about four points.^[4] TechCrunch reported that with GPT-5.5 the orchestrator raised accuracy on complex cases from 72% to nearly 87%, while on emergency department cases both set-ups scored about 85%, and wrote that the study’s own data suggests much of the improvement comes from more powerful underlying models.^[4] Microsoft said the gains persisted and improved with newer models including GPT-5.5, and The Verge reported that the paper does not include newer models such as GPT-6 Astra, Gemini 4 Argon or Opus 5.5.^[1][3]

TechCrunch noted that cases this complex are usually handled by specialists or physician teams, who were not part of the comparison, and Gizmodo reported that the researchers said they “would expect the performance gap to narrow” against specialists.^[4][5] The Verge reported that the system has only been tested on past medical cases, and quoted Mustafa Suleyman, chief executive of Microsoft AI, as saying “It needs clinical validation” and “That is a leap, but this is a major milestone along that journey.”^[3] Microsoft first posted the work as an arXiv preprint on 27 June 2025, which reported 85.5% accuracy for MAI-DxO configured for maximum accuracy and 80% when paired with OpenAI’s o3, against a 20% average for generalist physicians.^[7][5] Microsoft’s June 2025 post said the 21 physicians in that study worked without access to colleagues, textbooks or generative AI.^[6] At the time, TIME quoted Eric Topol of the Scripps Research Translational Institute, whom Microsoft’s 2025 post thanks for his insights, as saying “It's impressive what they did,” and that “it doesn't change medical practice until they take it out on the real medical highway.”^[8][6] TIME also quoted Keith Dreyer, chief data science officer at Massachusetts General Hospital and Brigham and Women’s Hospital Center for Clinical Data Science, as saying “In my mind, they are not testing any individual model that is optimized for health care,” and reported his view that the results do not necessarily bring such systems closer to regulatory approval.^[8] Gizmodo wrote that widespread adoption would raise legal questions about accountability when an AI-assisted diagnosis goes wrong.^[5]

Microsoft said a new partnership with Beth Israel Deaconess Medical Center in Boston will include a prospective study in which patients interact with the system directly, with performance measured against physicians’ independent assessments.^[1] The Verge reported that Dr Dominic King, Microsoft AI’s health lead and a co-author of the study, said testing will likely “get underway in the next couple of months”, and that neither King nor Suleyman would commit to a firm timeline.^[3] TechCrunch reported that Suleyman said Microsoft is working with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year, and that Microsoft already sells dictation and radiology tools to clinicians and sees the orchestrator as an expansion of those efforts.^[4] Microsoft said it also published an article on its idea of “medical superintelligence” in BMJ Digital Health and AI this week, and The Verge quoted Suleyman as predicting “something that is indistinguishable from human performance on medical superintelligence in the next couple of years”.^[1][3]