Testing vs Automation · both proven

skill-creator vs triage

Both skills were run through Claude twice: once on a fixed brief without the skill, once with it. The briefs differ because each is written for the job the skill claims to do, so compare each skill against its own baseline, then decide which difference matters for your work. They are in different categories, so this is a choice of job, not a head-to-head on the same job.

skill-creator

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Category: Testing
Source: anthropics/skills
License: Not stated
Proof model: claude-opus-5-5
Brief: “Write a complete SKILL.md for a skill called release-notes that turns a list of merged pull request titles into customer-facing release notes grouped by feature, fix and breaking change, in the tone of a small developer-tools company. Present the finished SKILL.md as a single HTML page, with the frontmatter and body shown in a code block and a short note under it on how you decided what to include.”
Skill commit: dbd4588

Without the skillWith skill-creator

⇆

Without, runWith skill-creator, run3 independent runs per side, none selected by hand.

Download the with-skill page

triage

Move issues and external PRs through a state machine of triage roles, categorise, verify, grill if needed, and write agent-ready briefs.

Category: Automation
Source: mattpocock/skills
License: MIT
Proof model: claude-opus-5-5
Brief: “You are triaging three new issues on an open-source Markdown editor. #412 "App crashes" by @kmorr: "It crashes when I paste stuff. Please fix asap." #413 "Export to PDF loses images" by @dlee: "On v2.3.1 (macOS 15), exporting a note with two PNG images to PDF produces a PDF with blank boxes where the images were. Steps: create note, drag in two PNGs, File > Export > PDF. Expected images in the PDF. Worked in v2.2.0." #414 "Add vim keybindings" by @tpz: "Would love vim mode. Lots of editors have it." Present the triage as a single HTML page: for each issue, the category, the state you would move it to and why, the comment you would post, and for any issue ready for an agent to implement, the agent brief.”
Skill commit: 5beb35b

Without the skillWith triage

⇆

Without, runWith triage, run3 independent runs per side, none selected by hand.

Download the with-skill page

We don’t pick a winner for you: the pages are the evidence. Both proofs can be repeated with the brief, model and commit shown above. See alternatives to skill-creator or alternatives to triage.