To test whether a skill helps, give the same fixed task to the same model at least three times without the skill and three times with it, then compare the results against checks you wrote down in advance, along with tokens and time. One run is not enough: in 7 of our 15 timed proofs, a single pair of runs could have pointed the wrong way on output length.
Every proof on this site uses the same side-by-side setup, and Anthropic recommends the same starting point: measure a baseline without the skill first, then compare. The sections below show what our own runs reveal about why repeats matter, then the steps to run the test yourself.
What do our proofs show about testing a skill?
Use This Skill proves skills side by side: one brief, three runs without the skill and three with it, on claude-opus-5-5, with all six outputs published. Fifteen proofs, run on October 24 and November 1, 2026, also record the input tokens, output tokens and seconds of every run in data/proofs.json. We read those numbers on November 16, 2026.
| Skill | Proven | Median output tokens, without → with | Median seconds, without → with | Single-run pairs that point the other way (of 9) |
|---|---|---|---|---|
| tdd | Oct 24 | 27,884 → 16,704 | 230 → 138 | 0 |
| improve-codebase-architecture | Oct 24 | 32,503 → 28,181 | 265 → 232 | 0 |
| mattpocock-prototype | Oct 24 | 25,221 → 16,994 | 219 → 140 | 0 |
| triage | Oct 24 | 17,079 → 18,159 | 146 → 155 | 2 |
| grilling | Oct 24 | 27,322 → 23,086 | 242 → 216 | 2 |
| design-taste-frontend | Oct 24 | 20,057 → 27,543 | 173 → 266 | 2 |
| ui-styling | Oct 24 | 26,788 → 29,305 | 224 → 249 | 2 |
| slides | Nov 1 | 20,452 → 30,623 | 174 → 261 | 0 |
| banner-design | Nov 1 | 17,486 → 23,720 | 153 → 222 | 1 |
| design-system | Nov 1 | 24,402 → 37,762 | 193 → 300 | 0 |
| brand | Nov 1 | 22,508 → 23,056 | 196 → 193 | 4 |
| receiving-code-review | Nov 1 | 13,088 → 15,494 | 108 → 134 | 0 |
| differential-review | Nov 1 | 19,158 → 21,928 | 157 → 183 | 0 |
| crypto-protocol-diagram | Nov 1 | 22,144 → 19,433 | 200 → 167 | 2 |
| vercel-react-native-skills | Nov 1 | 19,436 → 25,811 | 159 → 217 | 0 |
The last column pairs each of the three runs without the skill with each of the three runs with it, nine pairs in all, and counts the pairs where the skill's effect on output length points the opposite way from the medians. Three findings stand out:
- Runs of the same brief differ a lot. On the same side of the same proof, output length varied by a median of 20% from the shortest run to the longest, and by up to 66%: banner-design's three runs without the skill wrote 13,929, 17,486 and 23,078 tokens.
- One run a side can mislead. In 7 of the 15 proofs, at least one single pair of runs shows the opposite result from the medians. For brand, 4 of the 9 pairs do, so a one-run test would have been close to a coin toss.
- A skill is not automatically cheaper or faster. The skill shortened the median output in 5 of 15 proofs and the median time in 6. Time moved in the same direction as output length in 14 of the 15, so a skill mostly saves time when it makes the model write less.
Loading the skill's text added between 683 and 48,544 input tokens per run, with a median of 8,296. For what that means for a whole session, see how many skills Claude Code can load.
Why can't tokens and time alone tell me if a skill helps?
Because shorter is not the same as better. A design skill that writes 13,000 more tokens may be drawing the sections the brief asked for; a testing skill that writes 11,000 fewer may be skipping work, or cutting waste. The numbers tell you what the skill costs. Whether it helps depends on checks about the output itself.
Our test-writing comparison shows the difference. On the same cron-parser brief, tdd cut the median output from 27,884 to 16,704 tokens. The more useful finding came from reading all six outputs: with tdd or test-driven-development, 0 of 6 runs tested an internal parser function, against 5 of 6 without a skill. That is a check you can write before the runs and count after them.
How do I run the test myself?
Anthropic's best-practices page calls this evaluation-driven development: run representative tasks without the skill, note where Claude falls short, build at least three scenarios that test those gaps, measure the baseline, and then write only enough skill to close them. The Agent Skills specification site lays out the same loop in more detail. In practice:
- Write the task down and freeze it. Use a realistic prompt, the kind a person would type, with any input files. Do not adjust it between runs.
- Write the checks before you look. Two to five things a good result must contain, each one observable: "the test file imports only the public function", "the page has a pricing section". The specification site advises against vague checks such as "the output is good" and brittle ones that demand exact wording.
- Run the baseline in clean sessions. Run the task at least three times with the skill removed. Each run should start fresh, with no leftover context from earlier runs.
- Run the same task with the skill. Same model, same settings, same number of runs. Record tokens and time for every run, not just the total.
- Grade, then read. Count how many runs on each side pass each check. Then read the outputs yourself, because checks only cover what you thought to write down.
- Compare cost with benefit. The specification site puts it plainly: a skill that adds a few seconds but raises the pass rate a lot is probably worth it, and one that doubles tokens for a small gain may not be.
If the results are close, run more. A difference that disappears when you swap one run for another is not yet a result.
Can a tool run this for me?
Anthropic's skill-creator can. According to Anthropic's post of March 3, 2026, it writes evals, runs them in parallel with independent agents in clean contexts, and has a benchmark mode that tracks pass rate, elapsed time and token usage. Comparator agents judge a skill against no skill, or two versions against each other, without knowing which output is which. Anthropic says the update is available in Claude.ai and Cowork, and as a plugin for Claude Code. skill-creator itself is in the directory, with its own proof.
The best-practices page also notes that there is not yet a built-in way to run evaluations of the structure it shows, so you can build your own. The six steps above need nothing more than a folder per run and a note of each run's tokens and time.
What should I test besides the output?
Two other questions decide whether a skill helps you in practice.
| Question | How to test it | Where to read more |
|---|---|---|
| Does the agent pick the skill when it should? | Try prompts that should trigger it and prompts that should not, and note which load it | Why a skill does not trigger, writing a description |
| Does it work on the models you use? | Repeat the runs on each model; Anthropic suggests Haiku, Sonnet and Opus | Anthropic's best-practices page |
| Is it safe to install? | Check the folder before the first run | The live skill check |
Our proofs answer none of these three. They measure what a skill changes once it is loaded, on one model. Each proof sends the model the skill's full text and asks it to follow the skill, with no tools, so it does not measure triggering or a long agent session. The methodology page gives the rules, and every proof, with all six outputs, is on the proven page.
How we got these numbers
Tokens and seconds come from data/proofs.json, written by scripts/prove.mjs. Each run is one call to the model; input tokens include cached tokens, and seconds are wall-clock time for the call. The summary for each side is the median of its three runs. "Pairs that point the other way" compares every run without the skill with every run with it, nine pairs per proof, against the direction of the medians on output tokens. The 20% and 66% spreads are (longest run minus shortest run) divided by the shortest, on each side of each proof. The 27 earlier proofs did not record tokens or time and are not counted here.