Abdo

Built for "Build for Your World": mcp-trust-check is a command-line gate for developers who install third-party MCP servers. It is powered by Muse Spark Open, Meta's open-weights model, served through Baseten partner inference. THE PROBLEM You approve an MCP server once, and your AI agent then treats every tool description as trusted instructions. A server can hide an instruction inside a description (tool poisoning), or change a description after you approved it (rug pull). Local stdio servers also run with your own privileges. WHAT IT DOES - approve: spawns the server over stdio, fetches every tool definition, validates it deterministically (hidden Unicode, HTML comments, base64 blobs, length), screens it with the LLM reviewer, and saves a canonical-JSON + SHA-256 baseline. A hash only proves what was approved, not that it is safe, so the first approval is screened too. - check: read-only, for CI. Re-fetches the definitions, diffs them against the baseline, and sends only added or changed tools to the reviewer. Exit code 1 when a tool is flagged or the diff is invalid. HOW IT USES META'S MODEL Muse Spark Open (meta-models/muse-spark-open-1026) reads each tool definition and returns a structured JSON verdict {suspicious, reason}. The untrusted text is wrapped in a random per-call delimiter, the output is validated, and calls are retried on failure. The verdict is advisory; a human decides. EVALUATION 128 hand-written tool definitions, 3 runs each, majority vote; 14 arguable cases are excluded from the headline numbers. On the remaining 114: 0 missed attacks, 4 false positives (4/56 = 7.1% on hard benign cases); in matched malicious/benign pairs both were correct in 10 of 14. It misses persuasion-style attacks that contain no instruction. Raw results are in eval-results/. BUILT DURING THE HACKATHON Everything in the repository: the MCP client (modern 2026-07-28 lifecycle with legacy fallback), validator, baseline store, diff, reviewer, CLI, tests and eval harness. LIMITS Author-written set, not an independent benchmark. The reviewer sees only the tool definition, so malicious runtime behavior behind a clean description is out of scope (no sandbox). Definitions are sent to Baseten. RUN IT See the README Quick start: npm ci, copy .env.example to .env and set BASETEN_API_KEY, then npm run demo:rug-pull:approve and demo:rug-pull:check.