fix: make diagnosing-bugs redact secrets

A Snyk audit (W007, HIGH) flagged the skill for insecure credential
handling: it tells the agent to "paste the invocation and its output",
builds curl loops, and collects artifacts — three paths by which a live
token can end up reproduced in the agent's response.

Add a Redact section making redaction the first move on each, and point
the two call sites at it. Warn in the HITL template that `capture`
prints its value back to the terminal, where the agent reads it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt Pocock
2026-08-06 09:15:46 +01:00
co-authored by Claude Opus 5
parent 8b36d4fb26
commit efce423018
3 changed files with 24 additions and 2 deletions
@@ -0,0 +1,9 @@
---
"mattpocock-skills": patch
---
Make `diagnosing-bugs` redact secrets.
- Add a **Redact** section to `SKILL.md`. The skill has the agent show commands, outputs and captured artifacts; the section makes redaction the first move on each — write `<REDACTED>`, show a command's shape (`Bearer $API_TOKEN`) rather than its credential, build loops against env vars, and redact HAR files, log dumps and payloads before quoting them.
- The Phase 1 completion criterion said "paste the invocation and its output". It now says show it redacted, and Phase 1 asks the user for a **redacted** captured artifact.
- Note in `scripts/hitl-loop.template.sh` that `capture` prints its value back to the terminal, so it takes observations while signing in stays a `step`.
+12 -2
View File
@@ -9,6 +9,16 @@ A discipline for hard bugs. Skip phases only when explicitly justified.
When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching.
## Redact
This skill has you show commands, outputs and captured artifacts. **Redact every secret first** — API keys, tokens, passwords, cookies, session IDs, connection strings, signed URLs. Write `<REDACTED>` in its place.
- **Show a command's shape, not its credential**: `curl -H "Authorization: Bearer $API_TOKEN" …`. The env var reference is the redacted form, and it still runs.
- **Build loops against env vars**, so the credential stays in the environment rather than in the file you write or the output you quote.
- **Redact captured artifacts** — HAR files, log dumps and request payloads carry auth headers. Quote only the lines that carry the signal.
If the redacted output is not enough to diagnose the bug, say so and ask the user.
## Phase 1 — Build a feedback loop
**This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you.
@@ -46,11 +56,11 @@ The goal is not a clean repro but a **higher reproduction rate**. Loop the trigg
### When you genuinely cannot build a loop
Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop.
Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop.
### Completion criterion — a tight loop that goes red
Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (paste the invocation and its output), and that is:
Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (show the invocation and its output, redacted), and that is:
- [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_.
- [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above).
@@ -11,6 +11,9 @@
# capture VAR "<question>" → show question, read response into VAR
#
# At the end, captured values are printed as KEY=VALUE for the agent to parse.
#
# `capture` prints its value back to the terminal, where the agent reads it — so
# capture observations, and leave signing in to the user as a `step`.
set -euo pipefail