Back to Featured
Stanford HAI News2026-10-19 00:0029 days ago

Stanford HAI released MedHELM, PhysicianBench, and HealthAdminBench, finding that medical AI agents' real-world completion rate is below half

Measuring What Medical AI Can Actually Do

AI Briefing

The Stanford team summarized three years of work building medical AI benchmarks. On PhysicianBench, the best model, GPT-5.5, completed only 46% of tasks in a single attempt; on HealthAdminBench, Claude Opus 4.6 completed 36.3%. MedHELM covers 121 clinical tasks across 5 major categories. Evaluations showed the lowest scores in clinical decision support and administrative workflows; failures mainly resulted from lost context when transferring information across documents and systems. Both benchmarks are open with synthetic data, public code, and leaderboards.

Source: Stanford HAI News · hai.stanford.edu

Finished reading the news? Let’s talk about your business.

Leave your contact details, and I’ll get in touch with you personally.

WECHAT

Add me on WeChat

Press and hold or select the WeChat ID below to copy it.