A skill works when nobody’s there to catch it.
Most don’t get tested that way. They get tested by their builder, on a curated set of examples, in the same week the skill got built. The whole time, the builder is doing half the skill’s invisible job — filtering edge cases, catching weird outputs, quietly fixing things that don’t look right before anybody downstream sees them.
The day the skill gets handed off, all of that disappears.
The right test isn’t “does it produce the right answer when I run it?” It’s “does it produce the right answer when somebody else runs it, on data I haven’t seen, with nobody around to catch what slips?”
Most skills fail that test, and the failures look very similar.
Inputs come in missing or contradictory, and instead of flagging it, the skill produces something plausible. The number of skills I’ve watched silently invent data — because nobody told them refusing was allowed — is the most common failure mode I’ve seen.
Even when the data is fine, the output is often the wrong shape. If someone’s using the output to approve a contract, it has to surface what drives approval, in the order it gets checked. If someone’s using it to write a report, the output is structured around the report’s sections. Summaries are not decisions, and skills that produce them make people feel busy without actually moving the work forward.
And even when the inputs are clean and the output is right, the skill often only works in its builder’s hands. You hand it off and it falls over. What you built was a clever prompt, not a portable skill. A skill that requires its author to operate hasn’t finished being designed.
Most skills fail at least one of these. That’s fine, as long as it’s caught now. Failing them quietly in production, with three teams downstream taking the output as truth, is a much more expensive kind of fine.
What gets skills shipped isn’t passing the demo. It’s surviving without you in the room.