structured-log-jq-cookbook

Structured-Log jq Query Cookbook

Item #5 of 2026-07-03-follow-up-learning-plan.md. Working counter-example to “transcripts aren’t observability”: any agent emitting JSONL events (one JSON object per line, at minimum an action/msg field, a success/level field, and a timestamp) can reuse these recipes instead of re-deriving jq syntax each time. All recipes below were run and verified against ~/internal/harness/logs/fixtures/sample-stress-test.jsonl (action-oriented, fields: action, success, timestamp, taskId) and ~/internal/harness/logs/fixtures/sample-cli.jsonl (pino-style, fields: level, time, msg).

Adjust field names (.action, .success, .msg, .time) to match your own schema - the query shapes transfer even when the field names don’t.

1. Success rate per action (group-by)

jq -s -r 'group_by(.action)
  | map({action: .[0].action, total: length, success: (map(select(.success)) | length)})
  | .[] | "\(.action): \(.success)/\(.total)"' events.jsonl

decision:decompose: 1/2
decision:peek: 2/3
task_complete: 3/3

Use this to spot an action type that’s quietly failing more than others - per-action success rate, not just a global pass/fail count.

2. Chronological event/success timeline

jq -s -r 'sort_by(.timestamp) | .[] | "\(.timestamp) \(.action) success=\(.success)"' events.jsonl

Sorting explicitly (rather than trusting file order) matters once logs are merged from multiple sources or rotated.

3. Overall failure rate

jq -s -r '(map(select(.success==false)) | length) as $f
  | length as $t
  | "\($f)/\($t) = \(($f/$t*100)|floor)%"' events.jsonl

2/8 = 25%

One-line health-check number, cheap enough to run in a pre-flight or CI gate.

4. First failure timestamp/task per action

jq -s -r 'group_by(.action)
  | map(select(any(.success==false)))
  | .[] | .[0].action + " first failure taskId=" + (map(select(.success==false)) | .[0].taskId)' \
  events.jsonl

Answers “when did this action type start failing” without eyeballing the whole file - useful for bisecting a regression against a deploy/config change timestamp.

5. Filter by log level / message pattern (pino-style logs)

jq -r 'select(.msg | test("ERROR")) | "\(.time) \(.msg)"' cli.jsonl

2026-03-04T06:40:31.000Z ERROR Transport error for "dokuwiki":
2026-03-04T06:40:34.000Z ERROR controller disposed

Works for numeric pino level fields too (select(.level >= 50) for error-and-above), not just string-matched msg.

Notes

• jq -s (slurp) is required whenever an aggregation needs to see all records at once (group-by, counts, sort). Per-line filters (#5) don’t need it and stream more efficiently on large files.
• These are read-only aggregations over static/rotated fixture files. For a live-tailed log, pair with tail -F file.jsonl | jq --unbuffered ... instead of jq -s.
• Source data: ~/internal/harness/logs/fixtures/sample-stress-test.jsonl, ~/internal/harness/logs/fixtures/sample-cli.jsonl.