Who is this for?
If you landed on this page, you're probably interested in applying large language models to medical devices. You may have a medical device or may not know if your application is even considered a medical device at all. Or you may be looking at different regulatory pathways and wondering which one is the right one for your device. As of writing of this article, the regulatory pathway for LLMs are unclear. And there's many gray zones.
I could not find suitable answers from FDA or other sources so I decided to forge a new pathway. I took my previous experiences with AI applied to convolutional neural networks and tried to generalize them to this new age of large language and large multimodal models. I believe large language models should be used responsibly and to find low-risk usages of them first. Incorrect or hallucinated data is worse than no data at all. And therefore, the use case of LLMs should be carefully monitored and made part of the risk assessment of your device. A key risk assessment should look at the possibility of LLM hallucination and mitigate it appropriately either by design or by risk control measures and almost never by labeling because no one really reads those, just being completely honest.
Key Takeaways from the LLM Presubmission
- FDA stated that the topic of incorporating LLM is a novel idea and more internal conversation needs to be had about these types of devices, which might affect future feedback.
- Sponsor asked the Agency about using synthetic data. Sponsor proposed having a board-certified radiologist review it. FDA stated that given their current understanding there would a tremendous amount of information needed on the model. Thus, it might be easier to provide real data.
- FDA stated that depending on prevalence, consecutive collection is usually good method to follow. However, the Agency emphasized that for LLM, FDA does not have clear criteria on what they are looking for, but welcomes Sponsor justifications.
- Sponsor stated that it is now possible to run LLM locally, so have reasonable understanding that it would be safe to use in production. FDA stated that using a “frozen” model and providing a clear understanding of how it is being deployed would be recommended in future submission.
- Sponsor asked the Agency if there are any other general concerns for the use of LLM. FDA stated they haven’t seen a lot of projects that use LLMs. FDA stated that a frozen application would be desirable, but if Sponsor is not able to do that, then reason to not do that then there would be a concern of testing device in a “black box” environment would be much more difficult.
What are the steps to a presub anyway?
Step 1: Send your Presub to the FDA. Wait 60 days for a response.
This attachment contains the original pre-submission request that we had sent to the FDA. When you're creating a pre-submission request, you essentially need to do some design input work up front. You need to come up with your intended use statement and identify high-risk regulatory items that it is worth asking the FDA about this early on. It is a bit of an art and a science to figure out what questions to ask FDA. Questions that are too generic likely have answers already in the FDA guidance documents. Questions that are too specific are not likely to result in a definitive answer from FDA. For example, sample size determination is almost always a question that the FDA will not respond to in a pre-submission meeting, or at least not definitively.
Step 2: FDA responds 1 week before the agreed upon meeting date
One week before the FDA pre-submission meeting, the FDA will submit written feedback about the questions that you have asked them. You then have the option to skip the meeting if all of your questions are addressed. However, I would recommend having the meeting anyway. Because email interactions will only get you so far, and nothing beats an interactive Zoom call, or even a face-to-face call with FDA.
Step 3: Send your presubmission presentation to the FDA ahead of time at least 48 hours.
So, after you get FDA's initial feedback, you have about five days to compile a presentation for FDA. You may not want to discuss all agenda items because the FDA may have already given you clear instructions that you're satisfied with in the previous interaction. Although the FDA does not appreciate giving too much additional information at this point because they will not have time to review, after all that's what the 60 days was for, you can provide a couple of illustrative graphics and key information points that will make the FDA's responses that much more productive.
Step 4: Send the FDA your meeting minutes
So you just had your FDA pre-submission meeting, now what? You will have to compile all of the notes taken during the meeting into a set of meeting minutes. You'll have to summarize what the FDA's responses were and your takeaways from the meeting. Interestingly, there is an art to meeting minutes that can help extract the most actionable information from FDA, even at this stage. For example, if you had a bit of confusion about years of experience for readers, you can state your question as an answer something like "FDA suggests 5 years of experience". The FDA will either agree or disagree and provide their own interpretation and guidance in the next step.
Step 5: FDA sends back their revised minutes and it becomes part of the official record.
This is the final step, FDA submits their last clarifying comments to the minutes. Where do you go from here? Oftentimes, a pre-submission meeting may just give you more questions than answers. However, it gives the regulatory team key insights on what the FDA is concerned about so that you can focus your validation efforts and risk analysis efforts onto these key high risk areas.
Where to from here?
This is unprecedented regulatory territory. If you need a thought leader or team of thought leaders on your side, please don't hesitate to reach out. We are happy to forge a pathway for your large language model, large vision model, large action model, or large whatever model, including radiology foundation models. No one has done it before, and that's why you need to hire thought leaders that can generalize the existing regulations and guidances into brand new territory. Contact us today.
Why stop at a presub?
Do you want access to a fully un-redacted 510(k) submission package for a cleared software as a medical device with a machine learning component? This would have honestly saved me a year on my first FDA application.