Advanced Patterns

Advanced MCP design focuses on boundaries and failure behavior at scale.

Patterns here prioritize safety, operability, and controlled flexibility.

The current repository makes these trade-offs visible because the router is hard-coded, tracing is print-based, memory is in-process, and src/security/auth.py is still a placeholder.

Pattern overview

flowchart LR
    subgraph Control Plane
      G[Governance Rules]
      E[Evaluation Pipeline]
    end

    subgraph Runtime Plane
      O[Orchestrator]
      K[Tool Gateway]
      X[Execution Sandbox]
    end

    G --> O
    E --> O
    O --> K --> X

    style G fill:#1976d2,color:#fff
    style E fill:#1976d2,color:#fff
    style O fill:#1976d2,color:#fff
    style K fill:#ff9800,color:#fff
    style X fill:#ff9800,color:#fff

Pattern catalog

Pattern	Problem solved	Implementation direction
Policy-driven routing	Inconsistent tool behavior	Apply allowlists and confidence thresholds
Tool gateway	Heterogeneous tool contracts	Normalize auth, schema validation, and retries
Execution sandbox	Risky side effects	Isolate tool runs with strict resource limits
Continuous evaluation	Silent quality regressions	Run benchmark prompts on every release

Current code limitations and the next maturity step

Limitation	File	Risk	Recommended next step
Keyword routing only	src/llm/router.py	Misses intent variations and can overfit prompts	Add schema-driven intent classification and eval sets
In-memory memory store	src/memory/store.py	Lost history on restart	Add persistent storage and retention controls
Print-based tracing	src/observability/tracer.py	Hard to search and aggregate	Emit structured JSON logs or OpenTelemetry spans
Placeholder auth	src/security/auth.py	No access control boundary	Implement API key or JWT enforcement

Recommended mature architecture

flowchart TB
    U[User Request] --> API[API Gateway]
    API --> P[Policy Check]
    P --> ORCH[Orchestrator]
    ORCH --> ROUTE[Router Service]
    ROUTE --> GW[Tool Gateway]
    GW --> SAFE[Sandboxed Tool Execution]
    SAFE --> MEM[Persistent Memory]
    SAFE --> OBS[Structured Observability]

    style API fill:#1976d2,color:#fff
    style P fill:#1976d2,color:#fff
    style ORCH fill:#1976d2,color:#fff
    style ROUTE fill:#ff9800,color:#fff
    style GW fill:#ff9800,color:#fff
    style SAFE fill:#ff9800,color:#fff
    style MEM fill:#ff9800,color:#fff
    style OBS fill:#ff9800,color:#fff

Risk matrix

Risk	Likelihood	Impact	Mitigation
Prompt injection into tools	Medium	High	Input sanitization and tool-level policy checks
Latency cascades	Medium	Medium	Timeout budgets and circuit breakers
Data leakage in traces	Low	High	Redaction and role-based access for telemetry

When does policy-driven routing become necessary?

It becomes necessary when business risk is non-trivial, such as financial or sensitive data actions. Purely model-led decisions are rarely sufficient in regulated workflows.

How do we balance flexibility and control?

Keep flexible intent understanding, but enforce strict execution policy at the gateway. This separation allows adaptation without compromising safety.

What is the practical upgrade path from the current code?

First make routing schema-based, then persist memory, then replace print traces with structured telemetry. After that, add auth and a sandbox boundary for any tool that can mutate data.

--8<-- "_abbreviations.md"