Sherlog

It's 3 AM and production is down. The on-call engineer spends the first hour just finding the cause. Sherlog is an autonomous on-call SRE that does that hour in about a minute, with evidence, and then fixes it safely. Sherlog watches a system's metrics, logs, traces and change events (deploys, config, flags). It detects incidents without an LLM, using learned baselines, log templates and exception fingerprints. It then condenses thousands of log lines into a small evidence pack and hands it to Meta's models. Muse Spark Open triages each suspect service. Muse Spark investigates with ten read-only tools (metrics, traces, logs, the code as deployed, the deploy diff, dependencies, past incidents) and returns a structured diagnosis: root cause, confidence, evidence, ruled-out hypotheses and a recommended action. Sherlog then takes a safe, reversible action (rollback, config revert, flag off, restart or scale) with autonomy levels and human approval by default. It verifies that the system actually recovered against its baselines. If the system didn't recover, Sherlog reverts the action and re-investigates. Finally it writes the postmortem and remembers the incident ("seen before: #12, fixed by rollback"). When the cause is a bad deploy, Muse Code writes the code fix and a regression test. Sherlog's fixer proves the test fails on the deployed code and passes with the fix, then opens a pull request for a human to review. To test it honestly we built a real target: a four-service shop (gateway, orders, inventory, payments on Postgres and Redis) with constant traffic and full OpenTelemetry (Prometheus, Loki, Tempo). It also has a simulated deploy platform and a chaos controller with 15 fault scenarios: bad deploys, hangs, slow queries, pool exhaustion, network latency and partitions, retry storms, memory leaks, CPU burn, Redis down, a bad flag and red herrings. Sherlog can't cheat: it sees only what a human SRE would. Results: - Detection: 15/15 scenarios, each as exactly one incident; 0 false positives in 10 fault-free minutes; median 20 s to detect. - Diagnosis with live Meta models on the acceptance scenarios: 4/4 correct on service, category and action. Median 78 s from incident to diagnosis, $0.39 for all four. - Muse Code fix for a KeyError deploy: written and independently verified in 34 s, for about $0.09. Sherlog is built to run for real: - Cost follows errors, not log volume: Loki filters and counts, so at 20,000 lines/s Sherlog processes about 19. - Restarts are warm. - A monthly model budget with a circuit breaker, plus secret and PII redaction before any prompt. - Roles, sessions, CSRF and an audit log. - Action guardrails that don't trust the model: telemetry that looks like a prompt injection forces human approval. A web console shows the live service map, the investigation streaming in, approve/reject, verification check by check, the Chaos Lab and postmortems.