Automated code investigation

Someone has to read the code at 3am.

Legwork opens your repository at the exact commit that broke, reads until it finds the cause, and hands you the evidence — every claim cited to a line, every dead end still on the record.

CASE 0427 Closed

High · Payment API

Concurrent cache eviction interrupts active archive reads

The clear path removes the content-addressed store without coordinating with active readers.

Time to answer
4m 12s
Files read
14
Hypotheses
2

The part nobody wants to do at 3am

Every investigation is recorded as it happens, including the paths that went nowhere. This is the whole of case 0427.

  1. 00:00

    Resolved payments-api to the deploy that preceded the first alert.

    0fab160216e8

  2. 00:11

    Sandbox created. No DNS, no HTTP, no private network, no metadata endpoint.

  3. 00:26

    Read 14 files across internal/cache and internal/archive.

  4. 01:38
    Ruled out

    Hypothesis: the archive reader leaks file handles under load. Handle count is flat across the incident window.

    internal/archive/reader.go:210–244

  5. 02:52

    Hypothesis: cache eviction removes the store while readers still hold open paths.

  6. 03:47
    Confirmed

    The evict path calls os.RemoveAll without coordinating with active readers.

    internal/cache/layers.go:94–107

  7. 04:12

    Sandbox destroyed. Working copy gone.

CASE 0427 · FINDING

Concurrent cache eviction interrupts active archive reads

E-01 · internal/cache/layers.go:94–107

if err := os.RemoveAll(dir); err != nil {

Not verified

Production eviction rate was out of scope. The frequency of this collision is inferred from the code path, not measured.

The answer arrives as a record

A diagnosis you cannot check is a guess with better grammar. Every claim Legwork makes points at source it actually read, at a commit that cannot move underneath you.

And when it could not check something, it says so in the finding rather than rounding up to confidence. The limits are part of the answer.

Three things to wire up

  1. 1

    Approve the repositories

    Install the read-only GitHub App and pick what Legwork may read. It never gets write access, and it never mints a token outside the worker.

  2. 2

    Post the incident

    Send it when your alerting fires. One endpoint, idempotent, per-service.

  3. 3

    Take the callback

    A signed callback arrives when the case closes — root cause, evidence, and the limits of what was checked.

POST an incident

curl -X POST https://legwork.dev/api/v1/services/payments-api/incidents \
  -H "Authorization: Bearer $LEGWORK_API_KEY" \
  -d '{"title":"5xx spike on /checkout","severity":"high"}'

Evidence in. Nothing out.

  • Read-only GitHub App
  • No network egress
  • Sandbox destroyed after every run
  • Content purged at 30 days