Stratum & Docket — an agentic data lake with a human in the loop
Stratum fetches documents from permissively-licensed public sources — Wikipedia, via its API's documented etiquette: sequential requests, a descriptive User-Agent, batched titles — deduplicates exact matches, bounds content size and applies rule-based categorisation before writing a JSONL manifest for review. Docket reads that manifest and walks a reviewer through each entry — approve, relabel, reject, skip — writing one labelled decision per entry to a resumable, append-only output.
Architecture
- 1Fetch — permissively-licensed sources only, per their published etiquette
- 2Normalise — deduplicate exact matches, bound size, categorise by rule
- 3Manifest — JSONL, one entry per document
- 4Review — Docket walks a human through each entry
- 5Record — append-only labelled decisions, resumable
The problem
Data for training and evaluating agents needs provenance you can defend and labels a person actually agreed to. Most pipelines get one of those. The failure modes are quiet: silently truncated content, a scraper that ignored a robots policy, a review tool that let a decision be overwritten without a trace.
Approach & decisions
- Oversized content is a rejected ContentTooLarge error, never silently truncated — a manifest entry is either whole or absent.
- Project Gutenberg is deliberately unimplemented, not worked around: its robot-access policy forbids automated access to the main site and IP-blocks violators. The policy citation lives in the source.
- Docket's output is append-only. A --redo flag forces a re-review without ever rewriting a previous decision, and unparseable input re-prompts rather than silently advancing.
- Two repositories rather than one crate with two binaries, so each gets its own CI, connected only by a documented JSONL contract — the same cross-repo coupling trade-off ADLC makes by declining to depend on Grit.
- Both test suites run fully offline against trait-injected fakes — a fake HTTP transport for Stratum, a scripted prompter for Docket — matching the no-network-in-tests convention across the portfolio.
Results
- Both repositories green in CI with secret and dependency scanning on every push.
- Fully offline test suites — no network, no real stdin.
- Honest scope: Wikipedia is the only source implemented; Docket is a terminal tool, not a web UI, and not a preference-pair constructor.