MCP Gateway (AI Agent Tool Bridge)
A bridge exposing internal services (GPU inference, RAG memory, Docker, browser, search, host status) as standardized MCP tools behind one authenticated endpoint.
Evidence tier: Verified | code (private). The gateway runs and serves the tools listed below. The implementation lives in a private repository; the public repository carries the architecture, the tool reference, the design decisions and the measurements. Source access is by request.
View the public repository on GitHub
The public repository contains the full architecture documentation and tool reference. For source code access, contact me.
What this proves
This project proves I can design, deploy and measure an AI gateway with real constraints rather than describe one: one authenticated endpoint over 53 tools, per-principal authorisation that filters the advertised surface, and published numbers for latency, failure rate and guardrail behaviour. The evaluation, the rejected alternatives and the known limitations are in the same repository, which is the part most portfolios leave out.
What exists
- 53 tool definitions, 42 advertised to the default principal. The gap is the ACL, not a bug.
- Services behind it: GPU inference (chat, TTS, ASR, embeddings), RAG memory (PostgreSQL plus Qdrant), read-only Docker access through a socket proxy, host status and generated-data retention status, web search and research, a headless browser and an anti-detection browser profile, and the LangGraph agent.
- Network model: every backing service is LAN-only. The gateway is the only externally reachable component, so the attack surface is one authenticated door instead of nine.
- Identity and limits: bearer token mapped to a named principal, per-principal tool allowlist with default deny, a rate ceiling on the RAG surface, and admission control ahead of external providers.
- Measured behaviour, 2026-10-25, single host, single-flight:
- local tool: p50 24.9 ms, p95 43.3 ms, 0 errors in 20 calls
- one RAG hop: p50 88.0 ms, p95 332.6 ms, one 3.4 s tail, 0 errors in 15 calls
- one GPU hop (embedding): p50 358.4 ms, p95 426.9 ms, 0 errors in 10 calls
- guardrail evaluation: 20 of 20 cases pass against the contract as deployed
Where to read the detail
| Document | What it contains |
|---|---|
| Architecture diagram | Mermaid flow from agent client to LAN-only service, with the ACL and the socket proxy |
| Tool reference | All 53 tools by category, generated from the deployed source |
| EVALUATION.md | Latency by hop, failure rate, the 20-case guardrail eval, the cost model, and what was not measured |
| DECISIONS.md | Why streamable-http, why per-principal ACLs, why only search is cached, and the cost of each choice |
| Model card | The models behind the GPU tools, their intended use and their limits |
| Data card | What the memory tools store, what they never store, and how deletion actually works |
What Broke / Challenges
A working call reported zero results. Agents read MCP results through an adapter with a fixed
details ceiling. When a tool returned more than that ceiling, the transport replaced the payload with a
shape-compatible stub, so sc.results || [] evaluated to an empty array. The call had succeeded, the data
existed, and the agent reported “nothing found”. It was measured three times before it was believed, and
the belief that survived longest was the wrong one: that the memory service was broken.
The fix was a version floor on the adapter, which returns an explicit intermediate_result_too_large
error instead of a stub, plus a caller rule: check the omission flag before trusting the shape, and
validate a surprising empty with one direct call before blaming a service. The lesson generalised: a
truncated result must never be silent, because an empty result produced by truncation is worse than an
error, since the agent acts on it.
A pinned dependency that broke every browser tool. camoufox was pinned at 0.5.4 because a rebuild
resolved 0.5.5, which fetched browser build 152.0.4-beta.29 while the durable fingerprint profile still
pointed at beta.28. Every anti-detect browser tool then failed to launch. The lesson was not “pin
versions” in the abstract: the browser artefact is chosen by camoufox’s own CLI, so the pin has to cover
that too, and the combination has to be proven in a throwaway container before the image is rebuilt.
A 524 at the edge did not mean the work stopped. A long transcription returned 524 at the fixed edge timeout while the request stayed parked upstream, holding threads and gigabytes of resident memory on the host that also serves the chat model, so the queue behind it stopped draining and even one-second requests began to fail. The fix was operational: probe the queue before posting, keep one request in flight, size each request to the deadline, and on a 524 stop the batch rather than retry into a queue already holding the first copy.
The evaluation harness was wrong three times. It first scored 7 of 10 and then 5 of 20 before it
scored 20 of 20. Every correction changed the harness, not the gateway: classifying a refusal by
searching text for “error” missed “is not permitted”; treating only isError as a refusal missed the
JSON error payloads this gateway actually returns; and two probes expected tools the principal is not
granted. That is written into EVALUATION.md, because a green score from a harness nobody checked is not
evidence.
What is not claimed
- Not a hosted or multi-tenant service. One host, one operator, LAN-only backing services, no replica. A host restart is a full outage.
- No accuracy claim. The gateway routes calls; it does not answer questions. Model quality is a different layer and is not measured here.
- No throughput claim. All lanes ran single-flight. Concurrency was not measured.
- No currency figure for cost. Local calls cost GPU time rather than tokens, and host electricity and depreciation are not instrumented. The cost shape is published; a price would be invented.
- Not independently audited. The security model is described, not penetration-tested by a third party.
- The public repository does not contain the source. It documents the architecture, the tool contract, the decisions and the measurements, and says so in its first block.