A clinical research lead once asked us, only half joking, whether there was a cloud following her around collecting everything she did.
Fair question. The honest answer was yes — and the uncomfortable part wasn’t the collecting. It was that nobody had architected what the system was allowed to do with what it saw.
That’s the state of most AI rollouts in regulated environments right now. The model is fine. The governance around the model is vibes.
If you’re putting agents anywhere near clinical or regulated data, three architecture moves do more for your validation story than any model upgrade:
-
Pin the model version. If behavior can silently change under you, you cannot claim a validated state. Upgrades become controlled changes, not surprises.
-
Make the agent propose, not decide. Drafts, flags, and recommendations that a qualified human commits. The signature stays where the accountability lives.
-
Keep a golden test set. A fixed batch of known inputs with known correct outputs, run against every change. It’s the cheapest audit evidence you will ever produce.
None of this limits what AI can do for you. It’s what lets you say yes to it in an environment where “the vendor updated something” is not an acceptable root cause.
The cloud is following everyone around now. Architect what it’s allowed to do.