A couple of things make a real and lasting change of this sort difficult.
First of all, the agency would have to commit up front to accepting certain kinds of data as evidence of efficacy and safety, which it has historically been extremely reluctant to do. The answer to questions in the general vein of "will you accept this" has always seemed to be "bring us the data and we'll see", which of course means that after you've spent hundreds of millions of dollars collecting that data, you do absolutely everything to make it bulletproof (and that risk aversion is exactly the problem you've been talking about, Adam). This is a serious cultural/structural problem.
Second is that you need long-term stable leadership at the FDA. The political chaos happening right now, with rumors of the head being fired and experienced leaders leaving, is... not that. We can hope that this changes with the next presidential administration, but that's a few years away, and more aspirational than rational.
The Sean Khozin quote really lands "the second sponsor, and the tenth, and the hundredth." That's where every promising trial innovation goes quiet. The pilot is the easy part; it's a controlled environment with a willing vendor and cooperative sites. The messy reality of a multinational phase III trial pulling from five EHR systems across three countries is a completely different problem.
The point about sponsor risk-aversion is underappreciated too. A lot of the cumbersome infrastructure we're now trying to automate away wasn't mandated. It accumulated because no one wanted to be the first company to do less of something FDA might later decide it actually wanted. That cultural inertia is arguably harder to fix than the technical plumbing.
Coming from a food science background, I find the parallel to food safety data systems surprisingly direct same fragmentation, same reluctance to share real-time data upstream to regulators, same pattern of pilots that look great in controlled settings and stall at scale. The bottleneck is almost never the technology.
Your three recommendations are exactly right, especially the third. Without FDA being specific about what it will actually accept, sponsors will keep doing SDV the old way just to be safe. Guidance that's too general has essentially no effect on a risk-averse industry, we've seen that pattern repeat.
Really useful breakdown of something that got a lot of fanfare without a lot of explanation.
Always valuable to have an insider perspective here. Curious to know the author's thoughts on potential upside in terms of time and budget saved (assuming a best case scenario of widespread adoption through P3 trials).
I don't have a precise estimate, but there are a couple of ways to look at the upside: statically and dynamically.
The static upside: we shave a few weeks off between each development phase and a couple of months off the FDA review time (the kind of benefit we see in FDA's real-time oncology review program), since sponsors need to spend less time cleaning their data and prepping it for submission and can get FDA input sooner.
The dynamic upside: Real-time trials get sponsors their data more quickly, which makes it easier to run innovative trial designs (especially Bayesian and adaptive methods), seamless trials, and other approaches. That could shave years off drug development.
On top of that, the tech used here makes the trials simpler to run, which could open up the possibility of more trial sites, more patients, and faster recruitment. The impact is much harder to quantify but the difference over time could be dramatic; not only speeding up existing trials but enabling new trials that were once prohibitively expensive or logistically difficult to pull off.
It's still going, and I'm optimistic that it will outlast Makary and remain an important initiative at the agency. FDA actually just held an industry Q&A on Friday on the program (three days after Makary's departure), which I take as a sign that they're committed to keep moving forward on this.
A couple of things make a real and lasting change of this sort difficult.
First of all, the agency would have to commit up front to accepting certain kinds of data as evidence of efficacy and safety, which it has historically been extremely reluctant to do. The answer to questions in the general vein of "will you accept this" has always seemed to be "bring us the data and we'll see", which of course means that after you've spent hundreds of millions of dollars collecting that data, you do absolutely everything to make it bulletproof (and that risk aversion is exactly the problem you've been talking about, Adam). This is a serious cultural/structural problem.
Second is that you need long-term stable leadership at the FDA. The political chaos happening right now, with rumors of the head being fired and experienced leaders leaving, is... not that. We can hope that this changes with the next presidential administration, but that's a few years away, and more aspirational than rational.
The Sean Khozin quote really lands "the second sponsor, and the tenth, and the hundredth." That's where every promising trial innovation goes quiet. The pilot is the easy part; it's a controlled environment with a willing vendor and cooperative sites. The messy reality of a multinational phase III trial pulling from five EHR systems across three countries is a completely different problem.
The point about sponsor risk-aversion is underappreciated too. A lot of the cumbersome infrastructure we're now trying to automate away wasn't mandated. It accumulated because no one wanted to be the first company to do less of something FDA might later decide it actually wanted. That cultural inertia is arguably harder to fix than the technical plumbing.
Coming from a food science background, I find the parallel to food safety data systems surprisingly direct same fragmentation, same reluctance to share real-time data upstream to regulators, same pattern of pilots that look great in controlled settings and stall at scale. The bottleneck is almost never the technology.
Your three recommendations are exactly right, especially the third. Without FDA being specific about what it will actually accept, sponsors will keep doing SDV the old way just to be safe. Guidance that's too general has essentially no effect on a risk-averse industry, we've seen that pattern repeat.
Really useful breakdown of something that got a lot of fanfare without a lot of explanation.
Always valuable to have an insider perspective here. Curious to know the author's thoughts on potential upside in terms of time and budget saved (assuming a best case scenario of widespread adoption through P3 trials).
I don't have a precise estimate, but there are a couple of ways to look at the upside: statically and dynamically.
The static upside: we shave a few weeks off between each development phase and a couple of months off the FDA review time (the kind of benefit we see in FDA's real-time oncology review program), since sponsors need to spend less time cleaning their data and prepping it for submission and can get FDA input sooner.
The dynamic upside: Real-time trials get sponsors their data more quickly, which makes it easier to run innovative trial designs (especially Bayesian and adaptive methods), seamless trials, and other approaches. That could shave years off drug development.
On top of that, the tech used here makes the trials simpler to run, which could open up the possibility of more trial sites, more patients, and faster recruitment. The impact is much harder to quantify but the difference over time could be dramatic; not only speeding up existing trials but enabling new trials that were once prohibitively expensive or logistically difficult to pull off.
Great overview. But is this dead in the water now after Makary's ousting?
It's still going, and I'm optimistic that it will outlast Makary and remain an important initiative at the agency. FDA actually just held an industry Q&A on Friday on the program (three days after Makary's departure), which I take as a sign that they're committed to keep moving forward on this.