Compare commits

...
Author SHA1 Message Date
Matt PocockandClaude Opus 5 bda79a3c3c fix: trim the Redact section to prose
Drop the curl exemplar and the enumerated secret and artifact lists —
the model does not need to be told what a secret looks like. Three
sentences carry the same rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:17:31 +01:00
Matt PocockandClaude Opus 5 efce423018 fix: make diagnosing-bugs redact secrets
A Snyk audit (W007, HIGH) flagged the skill for insecure credential
handling: it tells the agent to "paste the invocation and its output",
builds curl loops, and collects artifacts — three paths by which a live
token can end up reproduced in the agent's response.

Add a Redact section making redaction the first move on each, and point
the two call sites at it. Warn in the HITL template that `capture`
prints its value back to the terminal, where the agent reads it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:15:46 +01:00
3 changed files with 20 additions and 2 deletions
@@ -0,0 +1,9 @@
---
"mattpocock-skills": patch
---
Make `diagnosing-bugs` redact secrets.
- Add a **Redact** section to `SKILL.md`. The skill has the agent show commands, outputs and captured artifacts; the section makes redaction the first move on each — write `<REDACTED>`, build loops against env vars so the credential stays in the environment, and quote only the signal-carrying lines of a captured artifact.
- The Phase 1 completion criterion said "paste the invocation and its output". It now says show it redacted, and Phase 1 asks the user for a **redacted** captured artifact.
- Note in `scripts/hitl-loop.template.sh` that `capture` prints its value back to the terminal, so it takes observations while signing in stays a `step`.
+8 -2
View File
@@ -9,6 +9,12 @@ A discipline for hard bugs. Skip phases only when explicitly justified.
When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching. When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching.
## Redact
This skill has you show commands, outputs and captured artifacts. **Redact every secret first** — write `<REDACTED>` in its place. Build loops against env vars, so the credential stays in the environment rather than in what you show. Captured artifacts carry auth headers: quote only the lines that carry the signal.
If the redacted output is not enough to diagnose the bug, say so and ask the user.
## Phase 1 — Build a feedback loop ## Phase 1 — Build a feedback loop
**This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you. **This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you.
@@ -46,11 +52,11 @@ The goal is not a clean repro but a **higher reproduction rate**. Loop the trigg
### When you genuinely cannot build a loop ### When you genuinely cannot build a loop
Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop. Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop.
### Completion criterion — a tight loop that goes red ### Completion criterion — a tight loop that goes red
Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (paste the invocation and its output), and that is: Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (show the invocation and its output, redacted), and that is:
- [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_. - [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_.
- [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above). - [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above).
@@ -11,6 +11,9 @@
# capture VAR "<question>" → show question, read response into VAR # capture VAR "<question>" → show question, read response into VAR
# #
# At the end, captured values are printed as KEY=VALUE for the agent to parse. # At the end, captured values are printed as KEY=VALUE for the agent to parse.
#
# `capture` prints its value back to the terminal, where the agent reads it — so
# capture observations, and leave signing in to the user as a `step`.
set -euo pipefail set -euo pipefail