SOX Testing

Plans a control area's testing for a period: builds the control matrix, sizes and draws samples, scaffolds the workpaper, and coordinates evidence annotation and deterministic procedures. Calls /sox-python for any deterministic step, /sox-annotate-xlsx when evidence is xlsx with embedded screenshots, and /sox-from-video when evidence is a recorded auditor walkthrough. Dispatches the sox-workpaper-grader agent at the end for an independent verdict.

For any procedure expressible as code — random sample draw, three-way match, reconciliation tie-out, threshold check, exception roll-up. Generates a Python script, runs it, captures stdout/stderr, and appends a per-sample-per-test detail tab with the full source code, output, runtime, and exit code.

Takes an xlsx whose sheets contain embedded evidence screenshots. For each requested attribute, identifies the pixel bounding box and extracts the on-screen observed value (typed by a closed attribute_kind enum — checkbox / toggle / text / currency / dropdown / date), then writes back red-outline rectangle Excel shapes plus bold red gutter labels (Approver: Requester's Manager) anchored over the original images. A parallel context scan flags ambient signals that modify the value's meaning (grayed-out rows, status badges, error banners, lock icons), and a reviewer agent double-checks both. The orchestrator merges all three signals to auto-judge pass / fail / needs_human per attribute and writes the comparison into the Summary Reasoning cell. Pixels are never burned in.

Takes an mp4 walkthrough plus a transcript. Parses the transcript for moments where the auditor reviews each in-scope attribute, extracts the corresponding video frames, identifies bounding boxes and observed values via vision, runs the same context + review passes, and writes per-sample-per-test detail tabs with annotated frames. Self-contained — vendors its own opencv-based frame extractor.

For when the evidence lives behind a login the tester already has. Drives Chrome through the Claude for Chrome extension to collect screenshots on demand, capturing chain-of-custody metadata (source URL, viewport, timestamp) and feeding the manifest into the same boxer / context / reviewer / writer pipeline /sox-annotate-xlsx uses. Read-only is enforced: every non-navigation click is filtered through a deny-list of mutating verbs (Approve / Submit / Save / Delete / Pay), and the run pauses on SSO / MFA pages for the tester to authenticate. Cowork session + Chrome extension required.

Pre-processor that takes a firm-supplied xlsx template (typically with one sample completed as a guide), auto-detects its shape (per-sample tabs, master matrix, per-test tabs, or single tab), profiles every placeholder cell to a semantic field, and scaffolds copies of the exemplar for the rest of the population. Emits a template-profile.json that the evidence helpers consume via --template-profile — bypassing the canonical layout when the firm template is the right fit.

Reads a completed workpaper and builds a portable .skill ZIP that replays the same test next period. Copies the deterministic scripts verbatim (SHA-verified for tamper-evidence), captures the locked attribute lists for evidence tests, and bakes methodology and the test plan into a generated SKILL.md. The builder writes no new Python — the deterministic core is preserved exactly as it ran the first time.

Auto-loaded reference skill (user-invocable: false). Holds sample-size buckets by risk level, the four sample-selection methods, evidence sufficiency standards, deficiency classification with indicators, deficiency aggregation rules, and the full control-type taxonomy — ITGC, manual, automated, IT-dependent manual, and entity-level. Other skills consult this one rather than duplicating methodology.

Dispatched by /sox-annotate-xlsx, /sox-from-video, and /sox-from-web once per image. Views a single screenshot or video frame, identifies pixel bounding boxes for each requested attribute, and extracts the on-screen observed value (checkbox state, displayed text, dropdown selection, currency amount) typed by a closed attribute_kind enum. Returns a boxes JSON plus an observed-values map so the orchestrator can auto-judge pass/fail from text alone. The orchestrator never sees the image bytes. Run in parallel: a single message with multiple Agent calls.

Dispatched in parallel with the boxer (default-on; --no-context skips) by every evidence-collecting skill. Independently locates each requested field by its label and surfaces ambient signals that modify how the observed value should be interpreted — disabled-indicator (grayed-out rows, locked icons), status-badge (active / inactive / draft / expired), error-banner, unsaved-indicator, permission-indicator, nearby-state. Plus a top-level screen_state that halts the pipeline if the screen is a modal, an error page, a permission-denied view, or still loading. Catches the "configured but disabled" case the boxer alone can't see.

Dispatched once per image after the boxer + context pair (default-on; --no-review skips). Views a Pillow-rendered overlay (the original image with each proposed box drawn on it and labeled by field) plus the original image, and emits two independent verdicts per box — position (ok / shift / wrong-field / miss-extra) and value (ok / wrong-value / unreadable). Drift gets routed back to the boxer for revision (capped at 2 retries) on the appropriate axis only, so a value re-read doesn't move a correct box and a box move doesn't regress a correct value.

Dispatched by /sox-from-video once per recording. Reads a transcript (.vtt, .srt, .txt, .docx), identifies the moments where the auditor reviewed each in-scope attribute, and emits a timestamps.json. Transcript text never returns to the orchestrator — only a small summary with timestamp count, samples covered, tests covered, and any gaps.

Dispatched by /sox-testing after the workpaper is fully assembled. Reads the workpaper rubric, introspects the xlsx via openpyxl, scores each criterion (Required + Recommended), and writes a verdict JSON with overall: pass | fail plus any blocking failures. Runs in a fresh context window with no exposure to the orchestrator's reasoning — that isolation is what makes the verdict meaningful.

Download the plugin below and install it in Claude Code — the .claude-plugin/plugin.json manifest sits at the root of the zip. The plugin registers seven user-facing slash-command skills, an auto-loaded methodology reference, and five subagents the orchestrator dispatches automatically.

Open any directory you want to use as a SOX testing workspace and run /sox-testing procure-to-pay 2024-Q4 (or your own control area and period). The skill walks through control identification, sample sizing, and sample selection — delegating to /sox-python so the random seed and selection logic are captured in a tamper-evident detail tab.

If your firm has a bespoke xlsx layout, run /sox-from-template <template.xlsx> first. The skill profiles the template, scaffolds copies of the exemplar tab for the remaining samples, and emits a profile JSON the evidence helpers consume to write into your layout instead of the canonical one.

For screenshot evidence, run /sox-annotate-xlsx <evidence.xlsx> "Approver, Amount Threshold, Self-Approval Checkbox". For walkthrough recordings, run /sox-from-video <walkthrough.mp4> <transcript>. For live collection through Chrome, run /sox-from-web <collection-plan> (Cowork session with the Claude for Chrome extension required). All three fan out to the boxer + context + reviewer agents in parallel and write annotated detail tabs back to the workpaper, with the orchestrator auto-judging pass/fail per attribute from the structured signals.

Once the workpaper is signed off, run /sox-replay-build <workpaper.xlsx> to package the deterministic scripts, locked attribute lists, methodology, and test plan into a portable .skill. Drop the .skill into next period's workspace and replay the same test without rewriting code.