An AI-enabled device gets authorized with human oversight written into its conditions of use. Whether that oversight exists is decided months later, by somebody making a staffing decision.
We filed a comment with the FDA this week on the generative AI medical device docket. Xillium does not build devices, and we took no position on classification or market pathway. We wrote about one thing that runs through both the risk framework and the postmarket monitoring section. Oversight is treated as a property of the device. In deployment it is a property of the organization doing the deploying.
Three points.
Effective autonomy is set at deployment, not at authorization. The same cleared device gets real review in one organization and near-automatic acceptance in another.
Oversight degrades under caseload without producing an error signal. Nothing reports that review got thinner.
Escalation authority matters more than review volume. A reviewer who cannot stop the work is not oversight.
What we recommended is that postmarket monitoring capture oversight as a stated condition of use, through four deployment fields, rather than assuming it.
I had not filed a federal comment before this one. It turns out to be straightforward. The docket is FDA-2026-N-7874, Considerations for the Regulation of Generative AI-Enabled Medical Devices, and it is open until October 19.
If you run the operations where these systems actually land, your experience is the thing that record is short on.
Ours is filed and awaiting posting, so I cannot link to it yet.
Xillium helps define the human oversight AI-supported workflows require before deployment. What that looks like in practice is on our Solutions page.
