Ports remaining changes from feat/unify-web-fetch-providers that were
not yet integrated into dev:
- config/schema.rs: add `user_agent` field (default "ZeroClaw/1.0") to
HttpRequestConfig, WebFetchConfig, and WebSearchConfig, with a shared
default_user_agent() helper. Field is serde-default so existing configs
remain backward compatible.
- tools/http_request.rs: accept user_agent in constructor; pass it to
reqwest::Client via .user_agent() replacing the implicit default.
- tools/web_fetch.rs: accept user_agent in constructor; replace hardcoded
"ZeroClaw/0.1 (web_fetch)" in build_http_client with the configured value.
- tools/web_search_tool.rs: accept user_agent in constructor; replace
hardcoded Chrome UA string in search_duckduckgo and add .user_agent()
to the Brave and Firecrawl client builders.
- tools/mod.rs: wire user_agent from each config struct into the
corresponding tool constructor (HttpRequestTool, WebFetchTool,
WebSearchTool).
- onboard/wizard.rs: add setup_web_tools() as wizard Step 6 "Web &
Internet Tools" (total steps bumped from 9 to 10). Configures
WebSearchConfig, WebFetchConfig, and HttpRequestConfig interactively
with provider selection and optional API key/URL prompts. Step 5
setup_tool_mode() http_request and web_search outputs are now discarded
(_, _) since step 6 owns that configuration. Uses dev's generic
api_key/api_url schema fields unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit fb83da8db021903cf5844852bdb67b9b259941d7)
`supports_vision` is currently hardcoded per-provider. The same Ollama instance can run `llava` (vision) or
`codellama` (no vision), but the code fixes vision support at the provider level with no user override.
This adds a top-level `model_support_vision: Option<bool>` config key — tri-state:
- **Unset (default):** provider's built-in value, zero behavior change
- **`true`:** force vision on (e.g. Ollama + llava)
- **`false`:** force vision off
Follows the exact same pattern as `reasoning_enabled`. Override is applied at the wrapper layer (`ReliableProvider` /
`RouterProvider`) — no concrete provider code is touched.
## Changes
**Config surface:**
- Top-level `model_support_vision` field in `Config` struct with `#[serde(default)]`
- Env override: `ZEROCLAW_MODEL_SUPPORT_VISION` / `MODEL_SUPPORT_VISION`
**Provider wrappers (core logic):**
- `ReliableProvider`: `vision_override` field + `with_vision_override()` builder + `supports_vision()` override
- `RouterProvider`: same pattern
**Wiring (1-line each):**
- `ProviderRuntimeOptions` struct + factory functions
- 5 construction sites: `loop_.rs`, `channels/mod.rs`, `gateway/mod.rs`, `tools/mod.rs`, `onboard/wizard.rs`
**Docs (i18n parity):**
- `config-reference.md` — Core Keys table
- `providers-reference.md` — new "Ollama Vision Override" section
- Vietnamese sync: `docs/i18n/vi/` + `docs/vi/` (4 files)
## Non-goals
- Does not change any concrete provider implementation
- Does not auto-detect model vision capability
## Test plan
- [x] `cargo fmt --all -- --check`
- [x] `cargo clippy --all-targets -- -D warnings` (no new errors)
- [x] 5 new tests passing:
- `model_support_vision_deserializes` — TOML parse + default None
- `env_override_model_support_vision` — env var override + invalid value ignored
- `vision_override_forces_true` — ReliableProvider override
- `vision_override_forces_false` — ReliableProvider override
- `vision_override_none_defers_to_provider` — passthrough behavior
## Risk and Rollback
- **Risk:** Low. `None` default = zero behavior change for existing users.
- **Rollback:** Revert commit. Field is `#[serde(default)]` so old configs without it will deserialize fine.
(cherry picked from commit a1b8dee785)
Add 5 LLM-callable IPC tools (agents_list, agents_send, agents_inbox,
state_get, state_set) backed by a shared SQLite database, enabling
independent ZeroClaw processes on the same host to discover and
communicate with each other. Gated behind [agents_ipc] enabled = true.
Related #88 (item 3: Sessions / Sub-Agent Orchestration)
Related #1518 (design spec)
* fix(channels,providers): remap Docker /workspace paths and enable vision for custom provider
Two fixes:
1. Telegram channel: when a Docker-containerised runtime writes a file to
/workspace/<path>, the host-side sender couldn't find it because the
container mount point differs from the host workspace dir. Remap
/workspace/<rel> → <host_workspace_dir>/<rel> in send_attachment before
the path-exists check so generated media is delivered correctly.
2. Provider factory: custom: provider was created with vision disabled,
causing all image messages to be rejected with a capability error even
though the underlying OpenAI-compatible endpoint supports vision. Switch
to new_with_vision(..., true) so image inputs are forwarded correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(memory): restore Qdrant vector database backend
Re-adds the Qdrant memory backend that was removed from main in a
recent upstream merge. Restores:
- src/memory/qdrant.rs — full QdrantMemory implementation with lazy
init, HTTP REST client, embeddings, and Memory trait
- src/memory/backend.rs — Qdrant variant in MemoryBackendKind, profile,
classify and profile dispatch
- src/memory/mod.rs — module export, factory routing with build_qdrant_memory
- src/config/schema.rs — QdrantConfig struct and qdrant field on MemoryConfig
- src/config/mod.rs — re-export QdrantConfig
- src/onboard/wizard.rs — qdrant field in MemoryConfig initializer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
copilot is the only provider that performs a device-code flow automatically on
first run. openai-codex and gemini (when OAuth-backed) require an explicit
`zeroclaw auth login --provider <name>` step. Split the device-flow next-steps
block to reflect this distinction.
Addresses Copilot review comment on PR #1509.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace hardcoded OPENROUTER_API_KEY hint with provider-aware logic:
- keyless local providers (ollama, llamacpp, etc.) show chat/gateway/status hints
- device-flow providers (copilot, gemini, openai-codex) show OAuth/first-run hint
- all other providers show the correct provider-specific env var via provider_env_var()
Also adds canonical alias "github-copilot" -> "copilot" in canonical_provider_name(),
and a new provider_supports_device_flow() helper with accompanying test.
Additionally fixes pre-existing compile blockers that prevented CI from running:
- fix(security): correct raw string literals in leak_detector.rs that terminated
early due to unescaped " inside r"..." (use r#"..."# instead)
- fix(gateway): add missing wati: None in two test AppState initializations
- fix(gateway): use serde::Deserialize path on WatiVerifyQuery struct
- fix(security): add #[allow(unused_imports)] on new pub use re-exports in mod.rs
- fix(security): remove unused serde::{Deserialize, Serialize} import
- chore: apply cargo fmt to files that had pending formatting diffs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Register Novita AI in provider factory with NOVITA_API_KEY env var
- Add to integrations registry with active/available status detection
- Configure onboarding wizard with default model and API endpoint
- Add to PR labeler provider keyword hints
- Update providers reference documentation
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Problem: The existing http_request tool returns raw HTML/JSON, which is nearly unusable for LLMs to extract
meaningful content from web pages.
- Why it matters: All mainstream AI agents (Claude Code, Gemini CLI, Aider) have dedicated web content extraction
tools. ZeroClaw lacks this capability, limiting its ability to research and gather information from the web.
- What changed: Added a new web_fetch tool that fetches web pages and converts HTML to clean plain text using
nanohtml2text. Includes domain allowlist/blocklist, SSRF protection, redirect following, and content-type aware
processing.
- What did not change (scope boundary): http_request tool is untouched. No shared code extracted between http_request
and web_fetch (DRY rule-of-three: only 2 callers). No changes to existing tool behavior or defaults.
Label Snapshot (required)
- Risk label: risk: medium
- Size label: size: M
- Scope labels: tool, config
- Module labels: tool: web_fetch
- If any auto-label is incorrect, note requested correction: N/A
Change Metadata
- Change type: feature
- Primary scope: tool
Linked Issue
- Closes #
- Related #
- Depends on #
- Supersedes #
Supersede Attribution (required when Supersedes # is used)
N/A
Validation Evidence (required)
cargo fmt --all -- --check # pass
cargo clippy --all-targets -- -D warnings # no new warnings (pre-existing warnings only)
cargo test --lib -- web_fetch # 26/26 passed
cargo test --lib -- tools::tests # 12/12 passed
cargo test --lib -- config::schema::tests # 134/134 passed
- Evidence provided: unit test results (26 new tests), manual end-to-end test with Ollama + qwen2.5:72b
- If any command is intentionally skipped, explain why: Full cargo clippy --all-targets has 43 pre-existing errors
unrelated to this PR (e.g. await_holding_lock, format! appended to String). Zero errors from web_fetch code.
Security Impact (required)
- New permissions/capabilities? Yes — new web_fetch tool can make outbound HTTP GET requests
- New external network calls? Yes — fetches web pages from allowed domains
- Secrets/tokens handling changed? No
- File system access scope changed? No
- If any Yes, describe risk and mitigation:
- Deny-by-default: enabled = false by default; tool is not registered unless explicitly enabled
- Domain filtering: allowed_domains (default ["*"] = all public hosts) + blocked_domains (takes priority).
Blocklist always wins over allowlist.
- SSRF protection: Blocks localhost, private IPs (RFC 1918), link-local, multicast, reserved ranges, IPv4-mapped
IPv6, .local TLD — identical coverage to http_request
- Rate limiting: can_act() + record_action() enforce autonomy level and rate limits
- Read-only mode: Blocked when autonomy is ReadOnly
- Response size cap: 500KB default truncation prevents context window exhaustion
- Proxy support: Honors [proxy] config via tool.web_fetch service key
Privacy and Data Hygiene (required)
- Data-hygiene status: pass
- Redaction/anonymization notes: No personal data in code, tests, or fixtures
- Neutral wording confirmation: All test identifiers use neutral project-scoped labels
Compatibility / Migration
- Backward compatible? Yes — new tool, no existing behavior changed
- Config/env changes? Yes — new [web_fetch] section in config.toml (all fields have defaults)
- Migration needed? No — #[serde(default)] on all fields; existing configs without [web_fetch] section work unchanged
i18n Follow-Through (required when docs or user-facing wording changes)
- i18n follow-through triggered? No — no docs or user-facing wording changes
Human Verification (required)
- Verified scenarios:
- End-to-end test: zeroclaw agent with Ollama qwen2.5:72b successfully called web_fetch to fetch
https://github.com/zeroclaw-labs/zeroclaw, returned clean plain text with project description, features, star count
- Tool registration: tool_count increased from 22 to 23 when enabled = true
- Config: enabled = false (default) → tool not registered; enabled = true → tool available
- Edge cases checked:
- Missing [web_fetch] section in existing config.toml → works (serde defaults)
- Blocklist priority over allowlist
- SSRF with localhost, private IPs, IPv6
- What was not verified:
- Proxy routing (no proxy configured in test environment)
- Very large page truncation with real-world content
Side Effects / Blast Radius (required)
- Affected subsystems/workflows: all_tools_with_runtime() signature gained one parameter (web_fetch_config); all 5
call sites updated
- Potential unintended effects: None — new tool only, existing tools unchanged
- Guardrails/monitoring for early detection: enabled = false default; tool_count in debug logs
Agent Collaboration Notes (recommended)
- Agent tools used: Claude Code (Opus 4.6)
- Workflow/plan summary: Plan mode → approval → implementation → validation
- Verification focus: Security (SSRF, domain filtering, rate limiting), config compatibility, tool registration
- Confirmation: naming + architecture boundaries followed (CLAUDE.md + CONTRIBUTING.md): Yes — trait implementation +
factory registration pattern, independent security helpers (DRY rule-of-three), deny-by-default config
Rollback Plan (required)
- Fast rollback command/path: git revert <commit>
- Feature flags or config toggles: [web_fetch] enabled = false (default) disables completely
- Observable failure symptoms: tool_count in debug logs drops by 1; LLM cannot call web_fetch
Risks and Mitigations
- Risk: SSRF bypass via DNS rebinding (attacker-controlled domain resolving to private IP)
- Mitigation: Pre-request host validation blocks known private/local patterns. Same defense level as existing
http_request tool. Full DNS-level protection would require async DNS resolution before connect, which is out of scope
for this PR.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
(cherry picked from commit 04597352cc)
