AI in Regulated Industries: FDA Compliance, QMS, and Human Oversight | Propulsion 2026

AI is reshaping quality, regulatory, and manufacturing operations faster than the rulebooks can keep up. For product companies operating under FDA oversight, the question is no longer whether to adopt AI, but how to do it without creating the next warning letter.

At Propulsion 2026, Propel CRO Zack Jha, KPMG's Life Sciences Quality and Regulatory Practice Lead Rajesh Misra, and industry veteran Zak Macht sat down for a frank conversation with moderator Dario Ambrosini about what it actually takes to deploy AI responsibly in regulated environments. From a real FDA warning letter triggered by an AI agent that failed to flag a missing process validation, to the future of real-time batch release and AI-powered post-market surveillance, this session covers the full landscape.

Rajesh issues a challenge to stop chasing incremental wins and pursue the kind of bold, business-transforming AI use cases that can deliver 20-30% bottom-line impact. Zack grounds that ambition with a hard-won caution from the cloud era: know your costs before your CFO gets surprised. And across the board, the panel agrees that the companies building AI into the fabric of their QMS, not bolting it on as a side project, are the ones that will come out ahead.

Key Takeaways

Human Oversight Is a Compliance Requirement, Not a Philosophy

An FDA warning letter cited a manufacturer whose AI agent failed to flag a missing process validation, and the team relied on the agent as their backstop. Today, human review of AI outputs is a baseline regulatory expectation, full stop.

Build AI Into Your QMS, Not Around It

Companies running AI pilots outside their quality management systems are creating a traceability gap that auditors will find. Any AI output should be logged, tracked, and auditable to the same standard as a manually written SOP.

Stop Optimizing for Small Wins, Go After Transformation

Rajesh Misra pushes back on the POC-and-park trap that kills more than 80-90% of AI programs before they scale. The companies seeing real impact are targeting outcomes like real-time batch release (currently a 7-10 day industry average) and AI-accelerated product recalls, not document summarization.

The FDA Is Using AI to Audit You

Regulators are no longer waiting for you to hand over information. FDA inspectors are coming prepared with AI-assisted pre-audit analysis, pointed questions, and increasingly compressed timelines, including a new one-day surveillance audit format. Assume they already know what they're looking for.

Post-Market Surveillance Is the Underrated AI Opportunity

From complaint triage and CAPA triggering to proactive field service routing and real-time product recall tracing, the post-market loop is rich with high-value, AI-ready use cases. Zack Jha highlights how AI-powered signal detection could compress the time from market feedback to design review from months to weeks.

Cybersecurity Is Now a Baseline Regulatory Expectation

For any connected medical device, cyber is no longer optional. FDA's 510(k) submissions now require a cybersecurity plan, and inspectors are actively reviewing it. Zack Jha and Rajesh Misra both flag that AI systems introduce new exposure, and companies need their cybersecurity posture in order before they scale AI adoption.

From AI to ROI – the insights are waiting. Watch all Propulsion 2026 sessions.

Learn More

Want to see how AI built for regulated industries shows up in the product?

Explore Propel One to learn how Propel's agentic AI brings the same "govern it, trace it, scale it" mindset to PLM and QMS, so your quality and regulatory teams spend less time chasing compliance and more time driving the decisions that move products forward.

About Propulsion 2026

Propulsion brings together everything product and manufacturing leaders need to know in one place. Bringing you Propel’s boldest platform reveals, critical AI and market insights from top industry analysts, and real customer results. These sessions are designed to cut through the noise so you can put what matters most to work immediately.