Open source · MCP preflight + runtime wrap
Catch it before the agent does.
Status: early, pre-alpha. Two tools in this repo, at different levels of maturity:
tools/call path between an MCP host and server, and rewrites what the model reads when a tool result looks like a silent failure. This one is real code with a real proof, but it currently does nothing on any server you haven't hand-built a baseline for. See "What doesn't work yet" below before you rely on it.
Fourgate does not claim to prevent silent failures. Today it classifies exactly one kind (silent_empty), only when a baseline exists, and passes everything else through unchanged.MCP tools can return a technically-valid, schema-valid "success" that is actually empty, fake, or stale — an expired auth token that still returns 200, a search that silently returns {results: []} instead of an error, a payload so large the agent truncates its own reasoning around it. The JSON-RPC layer sees no error. The agent sees "done" and acts on it. Fourgate's premise is that this class of failure has to be caught on the path the model actually reads, not in a log a human checks later.
Spawns your MCP server as a real subprocess and speaks JSON-RPC to it directly — not through a client library that would just crash on the same bug it's supposed to catch.
git clone https://github.com/hifzabuildsai/fourgate.git
cd fourgate
pip install -r requirements.txt
python3 checker/preflight.py fixtures/broken_server.py # catches a real bug
python3 checker/preflight.py fixtures/clean_server.py # same server, fixed -> ALL CLEAR
# or side by side:
bash demo.sh
Fourgate Preflight Report — fixtures/broken_server.py
============================================================
ℹ [tools_list] Discovered 3 tool(s): ['add_numbers', 'get_greeting', 'divide']
✘ [stdout_cleanliness] Non-JSON output on stdout broke the JSON-RPC stream: 'DEBUG: adding 1 + 1'
✔ [tool_call:add_numbers:valid] Call completed normally.
...
------------------------------------------------------------
8 passed, 2 failed, 0 warnings
RESULT: NOT SAFE TO SHIP — fix the failures above first.
Fix the line (print(...) → print(..., file=sys.stderr)) and re-run: 8 passed, 0 failed, 0 warnings — ALL CLEAR.
A host-side stdio proxy that sits on the live tools/call path between an MCP host and server, and rewrites the response the model reads into a structured verdict: {kind, tool, evidence, recovery}.
What's proven, today:
silent_empty classifier, gated on a per-tool baseline. It only flags a result when it has a known-good shape to compare against; without a baseline it passes the result through untouched. Three deliberate false-positive cases (e.g. a legitimate "no results" text response) are tested and pass.--selfcheck) that fails loudly if the wrap isn't actually on the call path — so a silent-failure detector can't itself silently no-op.Confirm before relying on this — path/flags below are drafted from description, not yet verified against the pushed entrypoint.
python3 wrap/fourgate_wrap.py -- <your-mcp-server-launch-command>
--selfcheck — verifies the wrap is genuinely intercepting tools/call, not bypassed. Fails if it can't prove it's on-path.
--observe PATH — shape-only JSONL logging: counts, booleans, byte sizes. No argument or result values are written. Error responses are logged too, not just successes.
fake_success, auth_expiry, and token_bloat are named in the design but not implemented. Only silent_empty classifies anything today..dxt extension builds reject this install path. DXT packages launch the MCP server directly as a self-contained bundle; there's currently no supported way to insert the wrap into that launch command. This only works today for hosts where you control the server's launch invocation directly.--observe; there's no push/webhook path yet.24 real tools/call invocations across 4 live servers (filesystem, fetch, duckduckgo, heventure), logged and committed at specs/field-observations.md.
Result: zero silent_empty hits. Every result carried non-empty content; errors came back as errors. One zero-result search returned "No matches found" as a text string — the false-positive gate correctly let it pass rather than flagging it.
What actually failed, four times, all silently, all before any tools/call happened: a server unreachable, an SDK import crash, a dependency install that broke a different server, and a config key in the wrong path so a server never loaded. In each case the agent reported the tool as unavailable and used something else instead. None of these are in scope for the current silent_empty classifier, which only runs once a call is made — this is a real gap the field data surfaced, not a hypothetical one.
The open question driving the baseline work: the one verified external report is from Prathmesh Patel (CEO @mcpjams, ex-Asana API/MCP lead) — an Atlassian MCP server returned {issues: [], isLast: true} for a nonexistent project, and the agent reported no issues and offered to create some. As written today, Fourgate would not flag this: it's a named empty collection, and the current classifier only acts where a baseline exists. This is the concrete case the unbuilt baseline-generation work is meant to close.
your MCP server (subprocess, stdio)
│
▼
Fourgate (raw JSON-RPC client)
│ reads every line raw, before treating it as a message
▼
initialize → tools/list → tools/call (valid / malformed / edge case)
│
▼
pass/fail report (human-readable or --json)
fake_success, auth_expiry, token_bloat classifiers--observe.dxt buildsMIT — see LICENSE.
No reviews yet. Be the first to rate this tool.
Sign in to leave a review.