Artificial intelligence has transformed how doctors diagnose disease and predict illness, but a fundamental problem remains unsolved: the invisible bureaucratic machinery that governs who gets care, when, and how. Administrative work in U.S. healthcare costs more than $1 trillion annually, nearly 25 cents of every dollar spent on medical care. Yet despite the excitement around AI breakthroughs, this administrative layer has remained largely untouched by automation.

Why Is Healthcare Administration So Hard for AI to Automate?

A research team at Stanford University, led by PhD students Suhana Bedi and Ryan Welch alongside senior author Nigam Shah, set out to measure whether AI could actually handle administrative workflows. They built four realistic software environments modeled on tools healthcare workers use daily: an electronic health record (EHR), two insurer portals, and a fax system. Inside these environments, they created 135 expert-designed tasks drawn from common administrative workflows, including prior authorization requests, appeals and denials management, and durable medical equipment ordering.

The results were sobering. The best-performing AI agent completed only 36.3 percent of full tasks successfully. Another system achieved the strongest subtask success rate, correctly completing 82.8 percent of individual subtasks, yet still fell far short on end-to-end task completion. This massive gap reveals a critical insight: AI systems can handle individual steps but struggle when tasks require moving between multiple systems, gathering information in one place, and using it later in another.

"That surprised me. That is a huge gap between high subtask performance and end-to-end task completion. Almost a 50% drop," noted Nigam Shah, senior author of the study.

Nigam Shah, MBBS, PhD, Stanford Medicine

Consider a real task from the benchmark: submitting a prior authorization request. To complete it, an AI must move step-by-step through multiple systems, first opening a patient's chart, extracting diagnosis and procedure codes, downloading required clinical documents, then navigating to an insurer's portal to enter patient details, attach those documents, and submit the request. Finally, it must return to the medical record and log the authorization confirmation. Each of these steps is explicitly required. Missing even one means failure.

What Are the Biggest Obstacles AI Faces in Healthcare Administration?

The Stanford research identified several critical failure points that reveal why current AI systems are not ready for real-world deployment:

• Document Handling Across Systems: Downloading a file from one system and attaching it correctly to another emerged as a major source of failure across all models tested, even though this is routine work for human staff.
• Long-Term Memory and Information Tracking: AI agents struggled with remembering hidden long-term dependencies, lost track of important information over time, and often failed to use the scratchpad memory space researchers provided for storing key facts across steps.
• Recognizing When to Stop: Many AI systems failed to recognize when a workflow should halt. In a durable medical equipment task, when an AI found a required evaluation document dated more than six months old, it continued the process instead of stopping and documenting why the order could not be completed.
• Execution Over Abstract Reasoning: Failure often had less to do with abstract reasoning than with execution. Agents frequently avoided file operations such as downloads and uploads, even when those actions were essential to completing the task.

The study reveals a broader truth about AI in the real world: intelligence is not just knowing what to do. It is being able to do it reliably in messy environments where interfaces are clunky, tasks unfold over time, and small omissions can invalidate the whole effort. Healthcare happens to be full of exactly those conditions.

How Can Healthcare Systems Prepare for AI-Assisted Administration?

The Stanford findings are not entirely discouraging. In a separate fine-tuning experiment, an open-source model trained on just 100 tasks improved held-out task success by 23 percentage points and outperformed the strongest frontier model in that evaluation. This suggests that with focused training on real healthcare workflows, AI systems can improve significantly.

Experts emphasize that the path forward requires building realistic testing environments and measuring progress rigorously. Ryan Welch, one of the lead researchers, stated that the most tangible impact of their work is providing the field with a realistic testing bed for healthcare agents. By building environments that reflect actual workflows, researchers can test agents in a way that is much closer to deployment reality.

"We think the most tangible impact is that HealthAdminBench gives the field a realistic testing bed for healthcare agents, along with a rigorous way to measure progress. By building environments that reflect those workflows, we can start to test agents in a way that is much closer to deployment reality," explained Ryan Welch, PhD student at Stanford Medicine.

Ryan Welch, PhD Student, Stanford Medicine

The administrative burden in healthcare shapes whether patients get tests approved, whether claims are denied, and how much time clinicians spend on bureaucracy instead of care. It is also one of the engines of clinician burnout. A system that could reliably handle prior authorizations or equipment orders might reduce delays, save money, and free up staff for higher-value work. But HealthAdminBench suggests that current systems are not ready to be trusted on their own.

Shah offered a measured outlook on the timeline for deployment-ready systems. "Within reach," he said, noting that models only get better and there is now a focus on capturing workflow traces, which can then be used to teach models to become far more capable.

While AI may be transforming medicine through diagnosis and prediction, the bureaucratic machinery that quietly governs healthcare delivery remains largely untouched. Solving this $1 trillion problem requires not just smarter AI, but AI systems that can reliably execute complex, multi-step workflows in the chaotic software ecosystem of real healthcare settings. The Stanford research provides a roadmap for measuring progress toward that goal.