From efce423018fc6468a3239621f1c1bcaacc723801 Mon Sep 17 00:00:00 2001 From: Matt Pocock Date: Thu, 6 Aug 2026 09:15:46 +0100 Subject: [PATCH] fix: make diagnosing-bugs redact secrets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A Snyk audit (W007, HIGH) flagged the skill for insecure credential handling: it tells the agent to "paste the invocation and its output", builds curl loops, and collects artifacts — three paths by which a live token can end up reproduced in the agent's response. Add a Redact section making redaction the first move on each, and point the two call sites at it. Warn in the HITL template that `capture` prints its value back to the terminal, where the agent reads it. Co-Authored-By: Claude Opus 5 (1M context) --- .changeset/diagnosing-bugs-redact-secrets.md | 9 +++++++++ skills/engineering/diagnosing-bugs/SKILL.md | 14 ++++++++++++-- .../diagnosing-bugs/scripts/hitl-loop.template.sh | 3 +++ 3 files changed, 24 insertions(+), 2 deletions(-) create mode 100644 .changeset/diagnosing-bugs-redact-secrets.md diff --git a/.changeset/diagnosing-bugs-redact-secrets.md b/.changeset/diagnosing-bugs-redact-secrets.md new file mode 100644 index 0000000..9a19519 --- /dev/null +++ b/.changeset/diagnosing-bugs-redact-secrets.md @@ -0,0 +1,9 @@ +--- +"mattpocock-skills": patch +--- + +Make `diagnosing-bugs` redact secrets. + +- Add a **Redact** section to `SKILL.md`. The skill has the agent show commands, outputs and captured artifacts; the section makes redaction the first move on each — write ``, show a command's shape (`Bearer $API_TOKEN`) rather than its credential, build loops against env vars, and redact HAR files, log dumps and payloads before quoting them. +- The Phase 1 completion criterion said "paste the invocation and its output". It now says show it redacted, and Phase 1 asks the user for a **redacted** captured artifact. +- Note in `scripts/hitl-loop.template.sh` that `capture` prints its value back to the terminal, so it takes observations while signing in stays a `step`. diff --git a/skills/engineering/diagnosing-bugs/SKILL.md b/skills/engineering/diagnosing-bugs/SKILL.md index f400de7..ff140b5 100644 --- a/skills/engineering/diagnosing-bugs/SKILL.md +++ b/skills/engineering/diagnosing-bugs/SKILL.md @@ -9,6 +9,16 @@ A discipline for hard bugs. Skip phases only when explicitly justified. When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching. +## Redact + +This skill has you show commands, outputs and captured artifacts. **Redact every secret first** — API keys, tokens, passwords, cookies, session IDs, connection strings, signed URLs. Write `` in its place. + +- **Show a command's shape, not its credential**: `curl -H "Authorization: Bearer $API_TOKEN" …`. The env var reference is the redacted form, and it still runs. +- **Build loops against env vars**, so the credential stays in the environment rather than in the file you write or the output you quote. +- **Redact captured artifacts** — HAR files, log dumps and request payloads carry auth headers. Quote only the lines that carry the signal. + +If the redacted output is not enough to diagnose the bug, say so and ask the user. + ## Phase 1 — Build a feedback loop **This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you. @@ -46,11 +56,11 @@ The goal is not a clean repro but a **higher reproduction rate**. Loop the trigg ### When you genuinely cannot build a loop -Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop. +Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop. ### Completion criterion — a tight loop that goes red -Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (paste the invocation and its output), and that is: +Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (show the invocation and its output, redacted), and that is: - [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_. - [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above). diff --git a/skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh b/skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh index 40afc46..43daedd 100644 --- a/skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh +++ b/skills/engineering/diagnosing-bugs/scripts/hitl-loop.template.sh @@ -11,6 +11,9 @@ # capture VAR "" → show question, read response into VAR # # At the end, captured values are printed as KEY=VALUE for the agent to parse. +# +# `capture` prints its value back to the terminal, where the agent reads it — so +# capture observations, and leave signing in to the user as a `step`. set -euo pipefail