A Confidence Score Does Not Move the Obligation

Your vendor’s confidence score is a statement about their product. Your compliance obligation is unaffected by it.

Those are two different things, and the gap between them is where practices get hurt.

That number is not fixed. Vendors retrain models and versions change, and unless your contract pins the model version, nobody has to tell you when it shifts.

A 94 in March and a 94 in September may not be measuring the same thing. Most contracts don’t pin it. Most practices have never asked. Which means the figure your workflow is built around can move underneath you, on the vendor’s schedule, for reasons you never see.

Second, the score describes how familiar the input looked. Not whether the output was right. Not whether it was appropriate for this patient under this policy.

So a high score tells you nothing was unusual. That is precisely the condition under which nobody looks closely. The items most likely to slip through are frequently the ones that scored well.

Third, and this is the one that surfaces when something goes wrong, an auditor will not ask what your vendor’s system reported. They will ask who reviewed it, what they saw, and what standard they applied.

There is no field on a vendor dashboard that answers that question. There was never going to be.

The vendor sells you a tool and takes responsibility for the tool. You are responsible for the decision. That allocation does not change because the tool became more sophisticated, and no number the vendor gives you moves any part of it onto them.