The default trait implementation returned a single error chunk that the
SSE mapper silently converted to `data: [DONE]`, producing empty
streaming responses from the OpenAI-compatible endpoint. Mirror the
non-streaming chat_with_history pattern: extract system + last user
message and delegate to stream_chat_with_system, which all providers
already implement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add underscores to long numeric literals (1234567890 → 1_234_567_890)
- Allow cast_possible_truncation for rough token estimates
- Replace loop/match with while-let for event stream parsing
- Merge identical match arms for event types
- Add #[allow(clippy::cast_possible_truncation)] on test helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement the streaming provider trait methods for Bedrock, enabling
real-time token-by-token responses via the ConverseStream endpoint.
Key implementation details:
- Uses /model/{id}/converse-stream endpoint with SigV4 signing
- Parses AWS binary event-stream format (application/vnd.amazon.eventstream)
with a minimal parser (~60 lines) — no new crate dependencies needed
- Handles contentBlockDelta events for text extraction, plus error and
exception events
- Uses mpsc channel + stream::unfold pattern (matching compatible.rs)
- Clones credentials for async task ownership
The binary event-stream parser extracts frame lengths, header sections
(looking for :event-type), and payload bytes. CRC validation is skipped
since TLS already provides integrity guarantees.
Includes 10 new tests for URL formatting, binary parsing, and
deserialization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Anthropic provider had no Image variant in NativeContentOut, so
[IMAGE:data:image/jpeg;base64,...] markers produced by the multimodal
pipeline were sent to the API as plain text. The API counted every
base64 character as a token, reliably exceeding the 200k token limit
for any real image (a typical Telegram-compressed photo produced
~130k tokens of base64 text alone).
Fix:
- Add ImageSource struct and Image variant to NativeContentOut that
serializes to the Anthropic Messages API image content block format
- Add parse_inline_image() to decode data URI markers into Image blocks
- Add build_user_content_blocks() to split user message content into
Text and Image blocks using the existing parse_image_markers helper
- Update convert_messages() user arm to use build_user_content_blocks()
- Handle Image in the apply_cache_to_last_message no-op arm
Fixes#1626
Set vision: true so image inputs are accepted by the capability gate.
Set native_tool_calling: true to align capabilities() with the existing
supports_native_tools() which always returned true, eliminating the
silent inconsistency between the two.
Adds a unit test that fails if either capability regresses.
- security: honor explicit command paths in allowed_commands list
- security: respect workspace_only=false in resolved path checks
- config: enforce 0600 permissions on every config save (unix)
- config: reject temp-directory paths in active workspace marker
- provider: preserve reasoning_content in tool-call conversation history
- provider: add allow_user_image_parts parameter for minimax compatibility
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The supports_native_tools() method was hardcoded to return true,
but it should return the value of self.native_tool_calling to
properly disable native tool calling for providers like MiniMax.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(channels,providers): remap Docker /workspace paths and enable vision for custom provider
Two fixes:
1. Telegram channel: when a Docker-containerised runtime writes a file to
/workspace/<path>, the host-side sender couldn't find it because the
container mount point differs from the host workspace dir. Remap
/workspace/<rel> → <host_workspace_dir>/<rel> in send_attachment before
the path-exists check so generated media is delivered correctly.
2. Provider factory: custom: provider was created with vision disabled,
causing all image messages to be rejected with a capability error even
though the underlying OpenAI-compatible endpoint supports vision. Switch
to new_with_vision(..., true) so image inputs are forwarded correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): restore Qdrant vector database backend
Re-adds the Qdrant memory backend that was removed from main in a
recent upstream merge. Restores:
- src/memory/qdrant.rs — full QdrantMemory implementation with lazy
init, HTTP REST client, embeddings, and Memory trait
- src/memory/backend.rs — Qdrant variant in MemoryBackendKind, profile,
classify and profile dispatch
- src/memory/mod.rs — module export, factory routing with build_qdrant_memory
- src/config/schema.rs — QdrantConfig struct and qdrant field on MemoryConfig
- src/config/mod.rs — re-export QdrantConfig
- src/onboard/wizard.rs — qdrant field in MemoryConfig initializer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Scheduled jobs created via channel conversations (Discord, Telegram, etc.)
never delivered output back to the channel because:
1. The agent had no channel context (channel name + reply_target) in its
system prompt, so it could not populate the delivery config.
2. The schedule tool only creates shell jobs with no delivery support,
and the cron_add tool's delivery schema was opaque.
3. OpenAiCompatibleProvider was missing the native_tool_calling field,
causing a compile error.
Changes:
- Inject channel context (channel name + reply_target) into the system
prompt so the agent knows how to address delivery when scheduling.
- Improve cron_add tool description and delivery parameter schema to
guide the agent toward correct delivery config.
- Update schedule tool description to warn that output is only logged
and redirect to cron_add for channel delivery.
- Fix missing native_tool_calling field in OpenAiCompatibleProvider.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Register Novita AI in provider factory with NOVITA_API_KEY env var
- Add to integrations registry with active/available status detection
- Configure onboarding wizard with default model and API endpoint
- Add to PR labeler provider keyword hints
- Update providers reference documentation
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* ci(homebrew): prefer HOMEBREW_UPSTREAM_PR_TOKEN with fallback
* ci(homebrew): handle existing upstream remote and main base
* fix: always emit toolResult blocks for tool_use responses
The Bedrock Converse API requires that every toolUse block in an
assistant message has a corresponding toolResult block in the
subsequent user message. Two bugs caused violations of this contract:
1. When parse_tool_result_message failed (e.g. malformed JSON or
missing tool_call_id), the fallback emitted a plain text user
message instead of a toolResult block, causing Bedrock to reject
the request with "Expected toolResult blocks at messages.N.content
for the following Ids: ..."
2. When the assistant made multiple tool calls in a single turn, each
tool result was pushed as a separate ConverseMessage with role
"user". Bedrock expects all toolResult blocks for a turn to appear
in a single user message.
Fix (1) by making the fallback construct a toolResult with status
"error" containing the raw content, and attempting to extract the
tool_use_id from the previous assistant message if JSON parsing fails.
Fix (2) by merging consecutive tool-result user messages into a single
ConverseMessage during convert_messages.
Also accept alternate field names (tool_use_id, toolUseId) in addition
to tool_call_id when parsing tool result messages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Will Sarg <12886992+willsarg@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
MiniMax API does not support OpenAI-style native tool definitions
(`tools` parameter in chat completions). Sending them causes a 500
Internal Server Error with "unknown error (1000)" on every request.
Add a `native_tool_calling` field to `OpenAiCompatibleProvider` so each
constructor can declare its tool-calling capability independently.
MiniMax (via `new_merge_system_into_user`) now sets this to `false`,
causing the agent loop to inject tool instructions into the system
prompt as text instead of sending native JSON tool definitions.
Closes#1387
(cherry picked from commit 2b92a774fb)
(cherry picked from commit 1816e8a829)
Co-authored-by: keiten arch <tang.zhengliang@ivis-sh.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Gemini CLI oauth_creds.json can omit client_id/client_secret, causing refresh requests to fail with HTTP 400 invalid_request (could not determine client ID).
Parse id_token claims (aud/azp) as a client_id fallback, preserve env/file overrides, and keep refresh form logic explicit. Also add camelCase deserialization aliases and regression tests for refresh-form and id_token parsing edge cases.
Refs #1424
Fix OpenAI Codex vision support by converting file paths to data URIs
before sending requests to the API.
## Problem
OpenAI Codex API was rejecting vision requests with 400 error:
"Invalid 'input[0].content[1].image_url'. Expected a valid URL,
but got a value with an invalid format."
Root cause: provider was sending raw file paths (e.g. `/tmp/test.png`)
instead of data URIs (e.g. `data:image/png;base64,...`).
## Solution
Add image normalization in both `chat_with_system` and `chat_with_history`:
- Call `multimodal::prepare_messages_for_provider()` before building request
- Converts file paths to base64 data URIs
- Validates image size and MIME type
- Works with both local files and remote URLs
## Changes
- `src/providers/openai_codex.rs`:
- Normalize images in `chat_with_system()`
- Normalize images in `chat_with_history()`
- Simplify `ResponsesInputContent.image_url` from nested object to String
- Fix unit test assertion for flat image_url structure
- `tests/openai_codex_vision_e2e.rs`:
- Add E2E test for second profile vision support
- Validates capabilities, request success, and response content
## Verification
✅ Unit tests pass: `cargo test --lib openai_codex`
✅ E2E test passes: `cargo test openai_codex_second_vision -- --ignored`
✅ Second profile accepts vision requests (200 OK)
✅ Returns correct image descriptions
## Impact
- Enables vision support for all OpenAI Codex profiles
- Second profile works without rate limits
- Fallback chain: default → second → gemini
- No breaking changes to existing non-vision flows
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add vision capability declaration (vision: true)
- Extend ResponsesInputContent to support image_url field
- Update build_responses_input() to parse [IMAGE:...] markers
- Add ImageUrlContent structure for data URI images
- Maintain backward compatibility with text-only messages
- Add comprehensive unit tests for image handling
Enables multimodal input for gpt-5.3-codex and similar models.
Image markers are parsed and sent as separate input_image content items.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Добавлен автоматический refresh протухших OAuth токенов Gemini при вызове warmup().
## Проблема
При использовании Gemini как fallback провайдера, OAuth токены могут протухнуть пока daemon работает. Это приводит к ошибкам при попытке переключения с OpenAI Codex на Gemini.
Сценарий:
1. Daemon работает, но не делает запросов к Gemini
2. OAuth токены Gemini истекают (TTL = 1 час)
3. Происходит ошибка на OpenAI Codex → fallback на Gemini
4. Gemini провайдер использует протухшие токены → запрос падает
## Решение
### Изменения в `GeminiProvider::warmup()`
Добавлена проверка и обновление токенов для `ManagedOAuth`:
- Вызывается `AuthService::get_valid_gemini_access_token()` который автоматически обновляет токены если нужно
- Для `OAuthToken` (CLI): пропускается (существующее поведение)
- Для API key: проверяется через публичный API (существующее поведение)
### Тесты
**Unit тесты** (`src/providers/gemini.rs`):
- `warmup_managed_oauth_requires_auth_service()` — проверка что ManagedOAuth требует auth_service
- `warmup_cli_oauth_skips_validation()` — проверка что CLI OAuth пропускает валидацию
**E2E тест** (`tests/gemini_fallback_oauth_refresh.rs`):
- `gemini_warmup_refreshes_expired_oauth_token()` — live тест с expired токеном и реальным refresh
- `gemini_warmup_with_valid_credentials()` — простой тест что warmup работает с валидными credentials
### Зависимости
Добавлена dev-зависимость `scopeguard = "1.2"` для безопасного восстановления файлов в тестах.
## Верификация
Проверено на live daemon с Telegram ботом:
- OpenAI Codex упал с 429 rate limit
- Fallback на Gemini сработал успешно
- Бот ответил через Gemini без ошибок
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
NVIDIA's NIM API (integrate.api.nvidia.com) does not support the
OpenAI Responses API endpoint. When chat completions returns a
non-success status, the fallback to /v1/responses also fails with
404, producing a confusing double-failure error.
Use `new_no_responses_fallback()` for the NVIDIA provider, matching
the approach already used for GLM and other chat-completions-only
providers.
Fixes#1282
Thinking/reasoning models (Kimi K2.5, GLM-4.7, DeepSeek-R1) return a
reasoning_content field in assistant messages containing tool calls.
ZeroClaw was silently dropping this field when constructing conversation
history, causing provider APIs to reject follow-up requests with 400
errors: "thinking is enabled but reasoning_content is missing in
assistant tool call message".
Add reasoning_content: Option<String> as an opaque pass-through at every
layer of the pipeline: ChatResponse, ConversationMessage, NativeMessage
structs, parse/convert/build functions, and dispatcher. The field is
skip_serializing_if = None so it is invisible for non-thinking models.
Closes#1327
Fixes two related issues with Gemini OAuth:
1. CLI command `zeroclaw auth refresh --provider gemini` was hardcoded to
only support OpenAI Codex, making manual token refresh impossible for
Gemini profiles. Extended the CLI handler to support both providers.
2. GeminiProvider.build_generate_content_request() was missing bearer token
for ManagedOAuth auth type. The method applied OAuth bearer token only
for CLI OAuth (GeminiAuth::OAuthToken), but not for managed profiles
(GeminiAuth::ManagedOAuth), causing 401 Unauthorized errors even after
successful token refresh.
Changes:
- src/main.rs: AuthCommands::Refresh now handles both openai-codex and
gemini providers via pattern match
- src/providers/gemini.rs: Extended OAuth bearer token handling to include
GeminiAuth::ManagedOAuth case (line 837)
Verification:
- Manual test: zeroclaw auth refresh --provider gemini --profile second
- E2E test: echo "hello" | zeroclaw agent --provider gemini --model gemini-2.5-pro
- Unit tests: cargo test providers::gemini (38 passed)
Risk: Low (isolated auth flow changes, no API contract changes)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Chummy
Shadman Hossain
guitaripod
Allen Huang
argenis de la rosa
Dominik Horváth
NB😈
Alex
Bojan Zivic
Amit Kotlovski
Aleksandr Prilipko
cee ray
Vernon Stinebaker
EC2 Default User
reidliu41