Codex Responses Lite clients routed to a chat-native OpenAI-compatible
provider lost tool use in three places: non-streaming Chat responses
leaked the raw chat.completion envelope instead of Responses output
items, internal reasoning continuity fields leaked into the outbound
Chat body causing some upstreams to reject the request, and the
Responses to Chat request translator ignored additional_tools,
custom_tool_call, and custom_tool_call_output items entirely.
Also fixes apiType (chat vs responses) for openai-compatible nodes
being resolved from the immutable provider ID instead of the stored
node config, so editing a node's API Type had no runtime effect.
Normalize every provider to one cache-inclusive convention via
canonicalizeUsage() before persist, and price cached + cache_creation as
subsets of prompt_tokens in calculateCostFromTokens() to stop
double-counting. usageRepo now delegates cost math to a single source.
Surface Cached tokens/cost across dashboard (overview, tokens, cost,
details). Merge Claude message_start cache with message_delta output so
cache counts survive. Compatible LLM nodes now allow multiple API-key
connections (key pool).
Co-authored-by: Cursor <cursoragent@cursor.com>
Gemini response translation wrote OpenAI-shaped bookkeeping into the
shared state.toolCalls map, which the downstream openai-to-claude
translator uses for Claude block metadata. That pre-population skipped
blockIndex creation, so Anthropic input_json_delta events lost index.
Track Gemini function calls via state.geminiToolCallCount instead,
leaving state.toolCalls clean for the Claude translator.
Closes#2218
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds long-lived API-key (ksk_) authentication for Kiro/AWS CodeWhisperer
and a direct claude:kiro / kiro:claude translation route that avoids the
lossy OpenAI two-hop pivot.
- translator: claude-to-kiro request + kiro-to-claude response translators,
registered on the exact source:target pair (direct route ahead of the
OpenAI pivot in index.js). claude-to-kiro uses shared schema constants
(ROLE/CLAUDE_BLOCK/DEFAULT_IMAGE_MIME) per app convention.
- auth: POST /api/oauth/kiro/api-key imports + validates a key via
ListAvailableProfiles, persists authMethod="api_key" (no refresh token).
- executor: send tokentype: API_KEY header and try *.amazonaws.com hosts
first for api-key creds; OAuth keeps kiro.dev first.
- fix: never inject the default placeholder profileArn for api-key auth
(CodeWhisperer 403s an ARN not owned by the key's account).
- ui: API Key method in the Kiro connect modal; surface api-key accounts
on the Quota Tracker and provider count.
- stream: env-overridable TTFT vs stall timeouts + Kiro keepalive frame.
- tests: claude-kiro-direct + kiro-profile-arn (11 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Add helpers/usageHelper.js for conditional prompt/completion token details.
Apply to gemini/codex/claude response translators; keep each provider's token
math intact. No behavior change; golden + gate: no regression.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add helpers/chunkBuilder.js; apply to claude/gemini/kiro/ollama/commandcode/
openai-responses response translators. Caller supplies id/created/model so each
keeps exact id-generation + usage semantics. Extend golden response stream to
openai-responses (codex). No behavior change; gate: no regression.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: sanitize Read tool args to prevent retry loops from non-Anthropic models
* fix: sanitize invalid Read pages from tool args
Non-Anthropic models sometimes emit optional Read args like pages: "" for
non-PDF files, which Claude Code rejects before the tool runs. Drop invalid
pages values, keep valid PDF page ranges, and coerce numeric string bounds
before clamping limit/offset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Some OpenAI-compatible providers emit optional string tool parameters
as empty strings (e.g. pages: "") instead of omitting them. Claude
Code rejects pages: "" as invalid, breaking the Read tool for
non-PDF files routed through 9router.
Add sanitizeToolArguments() that parses tool-call arguments and
removes known optional empty-string fields before emitting
input_json_delta back to Claude format. Currently handles the
Read tool pages field specifically.
Includes regression test.
Fixes#1278
Co-authored-by: JoJo <noreply@github.com>
Replace empty reasoning_content with explicit </think> closing tag when exiting thinking block to properly signal end of reasoning section in streaming responses.
- Encode thoughtSignature into tool_call.id using _TSIG_ delimiter and base64url
- Decode _TSIG_ on request to restore thoughtSignature for Gemini multi-turn thinking
- Track pendingThoughtSignature across parts for deferred signature attachment
- Add LocalMutex (2-layer locking) to prevent ELOCKED on concurrent DB access
- Increase lockfile retries from 5 to 15 for multi-process robustness
- Restore db.json seed on first run to prevent ENOENT on lockfile.lock
- Use process.env.BASE_URL fallback in models test route
- Remove gemini-3-flash-lite-preview from provider models
Co-authored-by: kwanLeeFrmVi <quanle96@outlook.com>
Closes#450
Made-with: Cursor
Apply fix from PR #354 by @tannk4w to properly signal tool_calls finish_reason
when model emits tool calls, allowing OpenAI-compatible clients to continue with
tool result processing instead of stopping prematurely.
Refactored finish_reason logic into computeFinishReason() helper to eliminate
duplication and improve maintainability across flush and completion paths.
Co-authored-by: tannk4w <tannk@tmi-soft.vn>
Thanks to @tannk4w, @trungtq2799, @quanhavn, and @East-rayyy for the thorough
review and improvement suggestions on the original PR.
Made-with: Cursor
Some upstream providers (e.g. Antigravity) return non-standard finish_reason
values like 'other' instead of the OpenAI-standard 'tool_calls' when the
model invokes tools. This causes downstream consumers (e.g. OpenClaw) to
fail to execute tool calls, breaking agentic sub-agent workflows.
Changes:
- nonStreamingHandler: post-translation guard that normalizes finish_reason
to 'tool_calls' when message.tool_calls is present
- sseToJsonHandler: accumulate tool_calls from streaming deltas in
parseSSEToOpenAIResponse; extract function_call items from Responses API
output in handleForcedSSEToJson
- openai-responses translator: use toolCallIndex to choose between
'tool_calls' and 'stop' in flush and response.completed events
Tested: 7 scenarios (non-stream text, single/multiple tool calls, stream
text/tool calls, multi-turn tool conversation, tools present but unused)
- Added handling for HTTP_STATUS.NOT_ACCEPTABLE in error types and messages.
- Enhanced the `prepareClaudeRequest` function to filter built-in tools for non-Anthropic providers and clean up empty tool arrays.
- Updated the `openaiToClaudeRequest` function to handle built-in tools more effectively and ensure proper tool conversion.
- Improved the `claudeToOpenAIResponse` function to skip processing for built-in server tool blocks.
- Refined error message handling in the `parseUpstreamError` function to ensure meaningful output.
- Adjusted command checks for tool installations across various settings routes to use `command -v` for better compatibility.
* feat: add AI request details feature with latency tracking
Add comprehensive request history and debugging capability to the Usage dashboard:
**Storage Layer** (usageDb.js):
- Add saveRequestDetail() for storing full request/response details
- Implement FIFO queue with 1000-record limit in request-details.json
- Auto-sanitize sensitive headers (authorization, api-key, cookie, token)
- Add getRequestDetails() with pagination and filtering support
- Add getRequestDetailById() for single record lookup
**Pipeline Integration** (chatCore.js):
- Track request start time and calculate total latency
- Record TTFT (Time To First Token) and total latency for all requests
- Capture full request details (messages, model, parameters)
- Save response content for non-streaming, mark streaming responses
- Handle error cases with detailed error information
- Async non-blocking saves to avoid impacting request performance
**API Layer** (/api/usage/request-details):
- GET endpoint with pagination (page, pageSize: 1-100)
- Filter by provider, model, connectionId, status, date range
- Returns { details: [...], pagination: {...} } format
**UI Components**:
- Drawer.js: Right slide-out panel with backdrop blur and ESC close
- Pagination.js: Full pagination with page size selector (10/20/50)
- RequestDetailsTab.js: Complete table view with filters and detail drawer
**Dashboard Integration**:
- Add "Details" tab to Usage page (4th tab after Overview/Logger/Limits)
- Table columns: Timestamp, Model, Provider, Input Tokens, Output Tokens, Latency (TTFT/Total), Action
- Provider filter dropdown (9 providers supported)
- Date range filters (start/end datetime)
- Click "Detail" button to view full request/response JSON in slide-out drawer
**Features**:
- Real-time latency monitoring (TTFT & Total)
- Complete request/response inspection for debugging
- Filterable and searchable request history
- Responsive design with mobile-friendly filters
- Data security with automatic header sanitization
- Performance: async saves don't block request pipeline
**Files Created/Modified**:
- src/lib/usageDb.js (modified)
- open-sse/handlers/chatCore.js (modified)
- src/app/api/usage/request-details/route.js (new)
- src/shared/components/Drawer.js (new)
- src/shared/components/Pagination.js (new)
- src/app/(dashboard)/dashboard/usage/components/RequestDetailsTab.js (new)
- src/app/(dashboard)/dashboard/usage/page.js (modified)
Closes: AI Observability Dashboard feature
* feat: enhance request details with full config and streaming content capture
Improve Request Details feature to capture comprehensive request parameters
and actual streaming response content:
**Request Configuration Enhancement** (chatCore.js):
- Add extractRequestConfig() helper function to capture all request parameters
- Include temperature controls: temperature, top_p, top_k
- Include token limits: max_tokens, max_completion_tokens
- Include thinking/reasoning modes: thinking, reasoning, enable_thinking
- Include OpenAI parameters: presence_penalty, frequency_penalty, seed, stop,
tools, tool_choice, response_format, n, logprobs, top_logprobs, logit_bias,
user, parallel_tool_calls, prediction, store, metadata
- Apply to all request types: non-streaming, streaming, and error cases
**Streaming Content Capture** (chatCore.js & stream.js):
- Add onStreamComplete callback mechanism to stream processors
- Accumulate content from all formats: OpenAI, Claude, Gemini
- Track content from delta.content, delta.reasoning_content, delta.text,
delta.thinking, and Gemini content.parts
- Save initial record with "[Streaming in progress...]" marker
- Update record with actual content when stream completes
- Include usage tokens when available from stream
**Files Modified**:
- open-sse/handlers/chatCore.js - extractRequestConfig() + streaming capture
- open-sse/utils/stream.js - onStreamComplete callback + content accumulation
**Benefits**:
- View complete request configuration in Request Details (thinking mode, etc.)
- See actual streaming response content instead of placeholder
- Better debugging and observability for AI requests
Refs: #request-details-enhancement
* feat: separate thinking/reasoning content from response content
Improve Request Details to display thinking process separately from final response:
**Backend Changes**:
- stream.js: Capture content and thinking separately in streaming mode
- Add accumulatedThinking variable alongside accumulatedContent
- Route delta.content to content, delta.reasoning_content to thinking
- Support OpenAI (reasoning_content), Claude (thinking), Gemini (part.thought)
- Update onStreamComplete callback to return { content, thinking } object
- chatCore.js: Update response structure to include thinking field
- Non-streaming: Extract thinking from reasoning_content field
- Streaming: Receive { content, thinking } from stream callback
- Error responses: Include thinking: null
- Initial streaming save: Include thinking: null
**Frontend Changes**:
- RequestDetailsTab.js: Display thinking and content in separate sections
- Add amber/yellow themed "Thinking Process" section with psychology icon
- Show "Final Response" label when thinking is present
- Use distinct visual styling for thinking (amber bg) vs content (gray bg)
- Only show thinking section when thinking content exists
**Benefits**:
- Users can clearly see model's reasoning process vs final answer
- Better debugging for models with thinking capabilities (Claude, o1, etc.)
- Visual distinction makes it easy to identify thinking vs response
Refs: #thinking-content-separation
* fix: map Claude thinking to reasoning_content field
Fix Claude thinking content to be properly captured as reasoning_content
instead of regular content, enabling separate display in Request Details:
**Changes**:
- claude-to-openai.js: Use reasoning_content field for thinking blocks
- thinking start: send { reasoning_content: "" } instead of { content: "```\n```" }
- thinking delta: map to reasoning_content instead of content
- thinking stop: send { reasoning_content: "" } instead of { content: "```\n```" }
**Why This Matters**:
- Previously Claude thinking was sent as `content` field, mixed with actual response
- Now thinking uses `reasoning_content` field, matching OpenAI's o1 format
- stream.js can now properly route thinking to accumulatedThinking variable
- Request Details UI will show Claude thinking in separate "Thinking Process" section
**Supported Thinking Formats**:
- OpenAI: delta.reasoning_content → thinking
- Claude: delta.thinking → reasoning_content (now fixed)
- Gemini: part.thought === true → thinking
Refs: #claude-thinking-fix
* feat(observability): capture and display full 4-layer request chain
Capture complete request/response chain in AI Request Details:
- Add providerRequest field (translated request sent to provider)
- Add providerResponse field (raw provider response, streaming indicator)
- Update chatCore.js at all 5 saveRequestDetail() call sites
- Reorganize UI into 4 collapsible sections with Material icons
- Preserve backward compatibility for old records
- Add distinct styling for streaming indicator
* fix(observability): resolve React duplicate key warning in request details table
- Use composite key (detail.id + index) to ensure unique keys
- Prevents React warnings when database contains duplicate IDs from old ID generation
* fix(observability): display actual content in streaming request details
Change providerResponse field for streaming requests from placeholder
"[Streaming - raw response not captured]" to actual final content.
This improves debugging experience by showing the real AI response
in the "Provider Response (Raw)" section instead of a confusing
placeholder message.
Files changed:
- open-sse/handlers/chatCore.js: Save contentObj.content to providerResponse
- src/app/.../RequestDetailsTab.js: Remove special handling for placeholder
* refactor(observability): migrate request details to SQLite for improved concurrency
- Replace LowDB JSON storage with better-sqlite3
- Enable WAL mode for true concurrent read/write support
- Add 5 indexes to accelerate queries (timestamp, provider, model, connection_id, status)
- Perform pagination at the database level to reduce memory footprint
- Maintain 1000 record limit with automatic cleanup of old data
- Ensure API compatibility via re-exports, requiring no caller changes
Performance improvements:
- Concurrent Writes: Lock-free WAL mode prevents data contention
- Query Efficiency: Index-based searches replace full dataset loading
- Data Integrity: Atomic operations prevent file corruption
* fix(observability): resolve pagination statistics display issues
- Fix issue where totalItems=0 showed 'Showing 1 to 0 of 0 results'
- Hide pagination controls when totalItems=0 or totalPages<=1
- Standardize API response fields: pagination.total -> pagination.totalItems
Before: Incorrect stats shown for empty data, and pager visible even for single-page results
After: Stats hidden for empty data, pager hidden when navigation is unnecessary
* feat(observability): display friendly provider names in request details
- Add /api/usage/providers endpoint to dynamically fetch provider list with names
- Replace hardcoded provider options with dynamic loading from database
- Display friendly provider names instead of IDs in both table and detail drawer
- Support custom provider nodes (e.g., OpenAI-compatible) with user-defined names
- Add provider name caching to optimize performance
* fix(observability): use INSERT OR REPLACE for request details to handle streaming updates
* fix(observability): resolve zero-token display issue by ensuring streaming usage capture and fixing key mismatch
* fix(observability): separate TTFT and total latency calculation for streaming requests
* feat(observability): implement SQLite write queue and JSON size limits
- Added in-memory buffer and batch writing for SQLite to prevent lock contention
- Implemented with configurable 1MB limit to prevent DB bloat
- Added dashboard UI for observability performance and data management settings
- Integrated graceful shutdown handlers to prevent data loss
* fix(observability): resolve ReferenceError by declaring dbInstance
- Added new observability settings in the dashboard for max records, batch size, flush interval, and max JSON size.
- Introduced `extractRequestConfig` function to capture full request configurations.
- Enhanced error handling by saving detailed request information on failures.
- Updated usage tracking to include new token metrics.
- Modified streaming functions to support detailed content and reasoning tracking.
- Support multiple OpenAI-compatible providers with custom prefix/baseUrl
- Add provider nodes CRUD (create/read/update/delete)
- URL building: baseUrl + /chat/completions or /responses
- Model import from /models endpoint
- API key validation via /models
- Usage type safety across all translators
- OAuth token auto-refresh for expired tokens
