diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..3bacc6b --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,336 @@ +# MCP Gateway Architecture + +## Before vs After + +### BEFORE (Claude Only) +``` +┌─────────────────┐ +│ Claude.ai │ +└────────┬────────┘ + │ MCP Protocol + ▼ +┌─────────────────────────────────────┐ +│ MCP Gateway (Port 4444) │ +│ - OAuth 2.1 provider │ +│ - Tool aggregation │ +│ - Session management │ +└─────────────────────────────────────┘ + │ + ┌────┴────┬────────┬──────────┐ + ▼ ▼ ▼ ▼ + ERPNext Wave TrueNAS Home Assistant + ERP Finance Storage Automation +``` + +### AFTER (Claude + Open-UI) +``` +┌─────────────────┐ ┌─────────────────┐ +│ Claude.ai │ │ Open-UI │ +└────────┬────────┘ └────────┬────────┘ + │ MCP Protocol │ OpenAI API + │ │ + └───────────────┬───────────┘ + ▼ +┌─────────────────────────────────────────────┐ +│ MCP Gateway (Port 4444) │ +│ ┌─────────────────────────────────────┐ │ +│ │ OAuth 2.1 (for Claude) │ │ +│ │ /oauth/* endpoints │ │ +│ │ /mcp protocol │ │ +│ └─────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────┐ │ +│ │ OpenAI API (for Open-UI) [NEW] │ │ +│ │ /v1/models │ │ +│ │ /v1/chat/completions │ │ +│ │ Bearer token auth │ │ +│ └─────────────────────────────────────┘ │ +└─────────────────────────────────────────────┘ + │ + ┌────┴────┬────────┬──────────┐ + ▼ ▼ ▼ ▼ + ERPNext Wave TrueNAS Home Assistant + ERP Finance Storage Automation +``` + +## New Components Added + +### 1. OpenAI Routes Module +**File**: `gateway-proxy/openai_routes.py` + +```python +async def list_models() # GET /v1/models +async def chat_completions() # POST /v1/chat/completions +async def _get_mcp_tools() # Fetch available tools +async def _stream_response() # Stream responses to client +``` + +### 2. OpenAI Adapter (Optional) +**File**: `openai_adapter.py` + +High-level abstraction for tool conversion and MCP calls: +```python +class OpenAIAdapter: + async def get_mcp_tools() # Get tools from MCP + async def call_mcp_tool() # Execute a tool + def _convert_mcp_tool_to_openai() # Convert schemas + async def chat_completions() # Handle chat requests +``` + +### 3. Gateway Proxy Enhancement +**File**: `gateway-proxy/gateway_proxy.py` (modified) + +```python +# Imports +from openai_routes import chat_completions, list_models + +# Routes +Route("/v1/models", openai_models, methods=["GET"]), +Route("/v1/chat/completions", openai_completions, methods=["POST"]), +``` + +## Request Flow + +### Claude → MCP Gateway → Tools + +``` +1. Claude sends MCP request + ↓ +2. OAuth 2.1 validation + ↓ +3. Handle /mcp endpoint + ↓ +4. Parse MCP protocol + ↓ +5. Route to appropriate MCP backend + ↓ +6. Execute tool + ↓ +7. Return MCP response + ↓ +8. Claude processes result +``` + +### Open-UI → OpenAI API → Tools [NEW] + +``` +1. Open-UI sends OpenAI request (POST /v1/chat/completions) + ├─ Headers: Authorization: Bearer TOKEN + ├─ Body: { model, messages, stream } + ↓ +2. Bearer token validation + ├─ Extract token from Authorization header + ├─ Hash and check against ACCESS_TOKENS + ↓ +3. Parse OpenAI request + ├─ Extract messages + ├─ Determine if stream needed + ↓ +4. Fetch available MCP tools + ├─ Call MCP gateway: tools/list + ├─ Convert to OpenAI format + ↓ +5. Build OpenAI response + ├─ List available tools + ├─ Format as OpenAI compatible + ↓ +6. Return response (or stream) + ├─ If stream=true: SSE format + ├─ If stream=false: JSON response + ↓ +7. Open-UI receives and displays +``` + +## API Compatibility + +### Endpoints +| Endpoint | Client | Protocol | Auth | +|----------|--------|----------|------| +| `/mcp` | Claude | MCP 2.0 | OAuth 2.1 | +| `/v1/models` | Open-UI | OpenAI | Bearer Token | +| `/v1/chat/completions` | Open-UI | OpenAI | Bearer Token | +| `/health` | Any | JSON | None | +| `/status` | Any | JSON | None | + +### Authentication + +**Claude (MCP):** +- Uses OAuth 2.1 +- Token stored in `ACCESS_TOKENS` dictionary +- Hash-based validation + +**Open-UI (OpenAI):** +- Uses Bearer tokens +- Token in `Authorization: Bearer` header +- Same hash-based validation + +### Response Formats + +**Claude (MCP):** +```json +{ + "jsonrpc": "2.0", + "result": { ... }, + "id": 1 +} +``` + +**Open-UI (OpenAI):** +```json +{ + "id": "chatcmpl-...", + "object": "chat.completion", + "created": 1234567890, + "model": "mcp-gateway", + "choices": [{ "message": {...} }], + "usage": {...} +} +``` + +## Streaming Support + +### Open-UI Streaming Format +Server-Sent Events (SSE): +``` +data: {"id":"...", "object":"chat.completion.chunk", "choices":[...]} + +data: [DONE] +``` + +Implemented in `_stream_response()` function. + +## Tool Execution Flow (Future Enhancement) + +Current: **Tools are listed only** +``` +Open-UI asks: "What tools are available?" +Gateway responds: "Here are the tools..." +``` + +Future: **Tools will be executable** +``` +Open-UI asks: "Create invoice for $100" +Gateway: + 1. Intercepts request + 2. Calls tool with parameters + 3. Returns result + 4. Open-UI shows outcome +``` + +To enable, modify `chat_completions()` to: +1. Parse `tool_use` in messages +2. Call `_call_mcp_tool()` +3. Return structured tool responses + +## Security Architecture + +### Token Management +``` +┌─────────────────┐ +│ Bearer Token │ (64-char urlsafe) +└────────┬────────┘ + │ SHA256 hash + ▼ +┌─────────────────┐ +│ Token Hash │ (stored in ACCESS_TOKENS) +└────────┬────────┘ + │ Compared on each request + ▼ +┌─────────────────┐ +│ Request Valid? │ ✅ / ❌ +└─────────────────┘ +``` + +### Multi-Tenant Ready +Each Open-UI instance can have its own token: +```python +ACCESS_TOKENS = { + "hash_of_token_1": {"client": "open-ui-1"}, + "hash_of_token_2": {"client": "open-ui-2"}, + "hash_of_claude": {"client": "claude.ai"} +} +``` + +## Deployment Architecture + +### Single Container +``` +Docker Container +├─ Python runtime +├─ Starlette web framework +├─ gateway_proxy.py +│ ├─ OAuth 2.1 endpoints +│ ├─ MCP routing +│ └─ OpenAI routes (NEW) +├─ openai_routes.py (NEW) +└─ openai_adapter.py (NEW) +``` + +### Multi-Backend +``` +Gateway connects to: +├─ ERPNext (REST API) +├─ Wave Finance (GraphQL) +├─ TrueNAS (REST API) +└─ Home Assistant (REST API) +``` + +Each as separate MCP servers. + +## Performance Considerations + +### Current Limitations +- No response caching +- No rate limiting +- Sequential tool discovery +- Full tool list returned each request + +### Future Optimizations +- Cache tool definitions (5 min TTL) +- Rate limit by token +- Parallel backend queries +- Filter tools by capability +- Batch tool calls + +## Monitoring & Observability + +### Log Lines Added +``` +INFO - OpenAI-compatible endpoints registered at /v1/* +INFO - Bearer token validated +INFO - MCP tools fetched: N tools available +ERROR - Unauthorized access attempt +ERROR - Tool call failed: X +``` + +### Health Checks +```bash +# Gateway health +GET /health + +# Full status +GET /status +``` + +## Future Enhancements + +### Phase 2: Tool Execution +- [ ] Parse tool_use in messages +- [ ] Execute MCP tools +- [ ] Return structured results + +### Phase 3: Advanced Features +- [ ] Function parameter validation +- [ ] Error recovery +- [ ] Async tool execution +- [ ] Caching layer + +### Phase 4: Enterprise Ready +- [ ] Rate limiting +- [ ] Audit logging +- [ ] Metrics/Prometheus +- [ ] Multi-tenancy +- [ ] Custom LLM routing + +--- + +**Summary**: Your MCP Gateway now runs a dual-protocol architecture supporting both Claude's MCP protocol and Open-UI's OpenAI API, all from a single gateway service.