Every day, clinicians make life-critical decisions while navigating vast amounts of patient information under significant time pressure. As healthcare becomes increasingly data-rich, identifying the right information quickly and acting on it with confidence has the potential to save lives.
Artificial intelligence is beginning to help, but the next generation of clinical AI must do more than recognize patterns or answer questions. Emerging systems, known as agentic AI, can retrieve patient information, reason across complex clinical data, generate recommendations, and verify their own outputs before supporting medical decisions.
For these systems to be adopted in hospitals, they must be accurate, transparent, and trustworthy. Clinicians need decision support tools that explain their reasoning, fit seamlessly into clinical workflows, and place patient safety at the center of every recommendation.
At NYU Abu Dhabi, researchers in the Clinical AI Lab, led by Assistant Professor Farah Shamout, are developing the foundations for safe agentic AI in healthcare.
Building on AI models that help identify hospitalized and intensive care unit (ICU) patients at risk of deterioration, the team is developing an agentic clinical decision support system. Rather than acting as a standalone prediction tool, the system retrieves relevant information from electronic health records, reasons across multiple sources of clinical data, generates patient-specific risk assessments, and verifies its outputs before presenting recommendations to clinicians.
When clinicians must review years of medical history in minutes, identifying the information that matters most is critical. Instead of processing entire medical records, the team, in a project led by PhD candidate Saeed Shurrab, has developed methods that enable AI to retrieve the most clinically relevant information from a patient's longitudinal health record. Their research also explores how multiple AI agents can collaborate on complex clinical reasoning tasks while maintaining reliability and transparency.
The team's research has also revealed an important insight through the AgentRx benchmark study, led by PhD candidate Baraa Al Jorf: simply combining multiple AI agents does not necessarily improve clinical performance. Understanding how collaborative AI systems should be designed is critical to building decision support tools that clinicians can trust.
To reduce the risk of AI errors influencing clinical decisions, the lab, in a study led by Postdoctoral Associate Dr. Congbo Ma, developed MedGuards, safety frameworks that automatically identify, verify, and correct clinically significant errors in AI-generated medical summaries and recommendations before they reach healthcare professionals. In a project led by PhD candidate L. Julian Lechuga Lopez in collaboration with Dr. Tim Rudner, the team has also developed an uncertainty quantification framework that flags model predictions with low model confidence as a safety guardrail: MedCertAIn.
While much of today's healthcare AI focuses on improving prediction accuracy, NYUAD is addressing the broader challenge of making AI reliable enough for real-world clinical practice. The team's research brings together trustworthy information retrieval, multimodal reasoning, clinical risk prediction, and automated error verification within a unified agentic framework.
The project is being developed in close collaboration with colleagues and clinicians at NYUAD, NYU Langone Health and medical partners in the UAE, ensuring the technology reflects real clinical workflows and healthcare priorities. Together, the partners are translating this research into an agentic clinical decision support platform that helps identify patients at high risk of deterioration in hospital and ICU settings, enabling earlier intervention while maintaining physician oversight and patient safety.
The goal is not to replace clinicians, but to help them make faster, better-informed decisions by delivering reliable, transparent decision support that strengthens clinical expertise. By helping identify patients at risk earlier, and reducing the cognitive burden of interpreting increasingly complex medical information, this research is laying the foundations for AI that can be safely integrated into patient care, improving patient outcomes while keeping clinicians firmly in control.