A nurses' union in northern Michigan just won something worth paying attention to: a contractual seat at the table for how AI gets rolled out in their hospital.
Their ask wasn't "no AI." It was "don't let it replace nursing judgment."
One nurse on the bargaining committee gave a simple example. If a blood pressure cuff isn't positioned correctly, the reading gets pulled into the system anyway, no questions asked. A nurse would catch that. The AI doesn't know to ask.
That's not a hypothetical concern. A recent Stanford and Harvard study found that even top-performing AI models produced 12 to 15 severely harmful errors per 100 cases. The worst-performing models got it wrong 40 times out of 100.
This is the part that should give every healthcare leader pause. The errors aren't rare edge cases happening at the margins. They're happening often enough that frontline staff are organizing around it.
Nurses aren't resisting technology. They're resisting being cut out of the loop on outputs that directly affect patient safety.
That instinct is correct. The organizations that figure out how to keep human judgment embedded in these workflows, not bolted on as an afterthought, are the ones who'll actually be able to scale AI safely.
The ones that don't will find out the hard way, one grievance or one bad outcome at a time.
Xillium keeps frontline judgment in the loop when AI-supported systems may be wrong. What that looks like in practice is on our Solutions page.
