Commit Graph

138 Commits

Author SHA1 Message Date
argenis de la rosa
1ebaa673b5 feat(gateway): add streaming mode for webhook responses 2026-02-27 11:28:48 -05:00
Samy
860e8b2442 fix: address remaining CodeRabbit review items
- Fix convert_channels() return type: dict → tuple[dict, list[str]]
- Remove unnecessary f-string prefixes on plain string literals
- Add blank lines after markdown headings (MD022 compliance)
- Handle triple-quote edge case in TOML multiline string output
- Add handler-level validation tests: missing message rejection,
  empty messages detection, whitespace-only user message, and
  context extraction correctness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 04:43:16 +00:00
sudomove
0aa2994423 fix: address PR review issues — auth guard, doc refs, TOML escaping
- Add non-loopback auth guard to /v1/chat/completions (matching /api/chat)
- Fix migration guide references to non-existent files (api_chat.rs,
  openai_compat_shim.rs, mod_patch.rs) — endpoints live in openclaw_compat.rs
- Remove phantom `provider` field from /api/chat response docs
- Add TOML string escaping to config converter to handle special chars
- Add proper JSON parse error handling in config converter
- Update deployment checklist and troubleshooting to match actual file layout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 04:43:16 +00:00
Samy
dde8b82ea0 feat(gateway): add OpenClaw migration compat layer with /api/chat and tools-enabled /v1/chat/completions
Add a complete OpenClaw → ZeroClaw migration toolkit:

- POST /api/chat: ZeroClaw-native endpoint with full agent loop (tools, memory,
  context enrichment). Supports session_id scoping and context[] injection for
  conversation history. Same code path as Linq/WhatsApp/Nextcloud handlers.

- POST /v1/chat/completions: OpenAI-compatible shim that routes through
  run_gateway_chat_with_tools instead of the simple provider.chat_with_history
  path. Extracts last user message + up to 10 messages of conversation context
  from the messages[] array. Supports streaming (simulated SSE). Drop-in
  replacement for OpenClaw callers with zero code changes.

Both endpoints include full observability instrumentation (AgentStart, LlmRequest,
LlmResponse, RequestLatency, AgentEnd), auth (pairing + webhook secret), rate
limiting, auto-save to memory, and response sanitization.

Also adds:
- scripts/convert-openclaw-config.py: Converts openclaw.json → config.toml with
  provider mapping, channel detection, and migration notes
- docs/migration/openclaw-migration-guide.md: Full migration walkthrough with
  endpoint reference, config mapping, callsite examples, and deployment checklist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 04:43:16 +00:00
argenis de la rosa
e3e4878ade fix(gateway): add explicit webhook usage and empty-message errors 2026-02-26 23:24:35 -05:00
argenis de la rosa
bfe3e4295d feat(security): add opt-in perplexity adversarial suffix filter 2026-02-26 22:55:23 -05:00
argenis de la rosa
21e13c8ae5 fix(qq): add sandbox mode and passive msg id fallback 2026-02-26 22:53:06 -05:00
argenis de la rosa
8180e7dc82 feat(skills): add WASM skill engine with secure registry install 2026-02-26 22:09:24 -05:00
argenis de la rosa
4196fd32a4 fix(gateway): align webchat system prompt with tool protocol 2026-02-26 21:43:43 -05:00
argenis de la rosa
b27b44829a chore: promote dev snapshot to main (resolve #1978/#1970) 2026-02-26 21:09:33 -05:00
Chummy
c54a30f68c supersede: file-replay changes from #1897
Automated conflict recovery via changed-file replay on latest main.
2026-02-26 16:37:17 +00:00
Reid
f836291200
fix(gateway): acknowledge WebSocket subprotocol to unblock agent chat (#1954) 2026-02-26 09:25:35 -05:00
Argenis
f220973192 fix(web/gateway): prevent empty dashboard replies after tool calls (#1930)
* fix(gateway): prevent empty websocket tool-call responses

* fix(web): render fallback for empty done messages
2026-02-26 04:51:05 -05:00
Chum Yin
9b0e70b2f2
supersede: file-replay changes from #1895 (#1926)
Automated conflict recovery via changed-file replay on latest main.
2026-02-26 04:15:47 -05:00
argenis de la rosa
ae0159bad6 fix(linq): support current v3 webhook payload shape 2026-02-25 17:25:08 -05:00
Chummy
f0774d75f7 fix(ci): align feishu gateway test fixtures with schema defaults 2026-02-26 02:19:14 +08:00
Chummy
fd86e67d67 fix: restore config reexports after dev rebase 2026-02-26 02:19:14 +08:00
Chummy
1fcf2df28b feat: harden non-CLI approval governance and runtime policy sync 2026-02-26 02:19:14 +08:00
Chummy
efdd40787c feat(config): add deprecated runtime reasoning_level compatibility alias 2026-02-25 21:00:59 +08:00
Chummy
1f257d7bf8 Sanitize WebSocket chat done responses to prevent tool artifact leaks 2026-02-25 19:54:09 +08:00
Chummy
3b6786d0d7 Fix tool-call artifact leaks across channel and gateway replies 2026-02-25 19:54:09 +08:00
Chummy
d5cd65bc4f hardening: tighten gateway auth and secret lifecycle handling 2026-02-25 18:33:28 +08:00
Chummy
2ecfa0d269 hardening: enforce channel tool boundaries and websocket auth 2026-02-25 18:33:28 +08:00
argenis de la rosa
aac87ca437 feat(provider): add reasoning level override
(cherry picked from commit 8d46469c40)
2026-02-25 17:51:00 +08:00
FlashFamily
931cf40636 fix: resolve all clippy warnings across codebase
Fix all clippy errors reported by `cargo clippy --all-targets -- -D warnings`
on Rust 1.93, covering both the original codebase and upstream dev changes.

Changes by category:
- format!() appended to String → write!/writeln! (telegram, discord)
- Redundant field names, unnecessary boolean not (agent/loop_)
- Long numeric literals (wati, nextcloud, telegram, gemini)
- Wildcard match on single variant (security/leak_detector)
- Derivable Default impls (config/schema)
- &Option<T> → Option<&T> or allow (config/schema, config/mod, gateway/api)
- Identical match arms merged (gateway/ws, observability, providers, main, onboard)
- Cast truncation allowed with rationale (discord, lark)
- Unnecessary borrows/returns removed (multiple files)
- Unused imports removed (channels/mod, peripherals/mod, tests)
- MSRV-gated APIs allowed locally (memory/hygiene, tools/shell, tools/screenshot)
- Unnecessary .get().is_none() → !contains_key() (gemini)
- Explicit iteration → reference loop (gateway/api)
- Test-only: useless vec!, field_reassign_with_default, doc indentation

Validated: cargo fmt, cargo clippy --all-targets -- -D warnings, cargo test
Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit 49e90cf3e4)
2026-02-25 17:50:56 +08:00
Chummy
cd4d816a83 fix(providers): keep runtime options backward compatible 2026-02-25 10:56:31 +08:00
reidliu41
3a38c80c05 feat(config): add model_support_vision override for per-model vision control
`supports_vision` is currently hardcoded per-provider. The same Ollama instance can run `llava` (vision) or
  `codellama` (no vision), but the code fixes vision support at the provider level with no user override.

  This adds a top-level `model_support_vision: Option<bool>` config key — tri-state:
  - **Unset (default):** provider's built-in value, zero behavior change
  - **`true`:** force vision on (e.g. Ollama + llava)
  - **`false`:** force vision off

  Follows the exact same pattern as `reasoning_enabled`. Override is applied at the wrapper layer (`ReliableProvider` /
   `RouterProvider`) — no concrete provider code is touched.

  ## Changes

  **Config surface:**
  - Top-level `model_support_vision` field in `Config` struct with `#[serde(default)]`
  - Env override: `ZEROCLAW_MODEL_SUPPORT_VISION` / `MODEL_SUPPORT_VISION`

  **Provider wrappers (core logic):**
  - `ReliableProvider`: `vision_override` field + `with_vision_override()` builder + `supports_vision()` override
  - `RouterProvider`: same pattern

  **Wiring (1-line each):**
  - `ProviderRuntimeOptions` struct + factory functions
  - 5 construction sites: `loop_.rs`, `channels/mod.rs`, `gateway/mod.rs`, `tools/mod.rs`, `onboard/wizard.rs`

  **Docs (i18n parity):**
  - `config-reference.md` — Core Keys table
  - `providers-reference.md` — new "Ollama Vision Override" section
  - Vietnamese sync: `docs/i18n/vi/` + `docs/vi/` (4 files)

  ## Non-goals

  - Does not change any concrete provider implementation
  - Does not auto-detect model vision capability

  ## Test plan

  - [x] `cargo fmt --all -- --check`
  - [x] `cargo clippy --all-targets -- -D warnings` (no new errors)
  - [x] 5 new tests passing:
    - `model_support_vision_deserializes` — TOML parse + default None
    - `env_override_model_support_vision` — env var override + invalid value ignored
    - `vision_override_forces_true` — ReliableProvider override
    - `vision_override_forces_false` — ReliableProvider override
    - `vision_override_none_defers_to_provider` — passthrough behavior

  ## Risk and Rollback

  - **Risk:** Low. `None` default = zero behavior change for existing users.
  - **Rollback:** Revert commit. Field is `#[serde(default)]` so old configs without it will deserialize fine.

(cherry picked from commit a1b8dee785)
2026-02-25 10:56:31 +08:00
Chummy
b4df1dc30d feat(tools): add web_fetch provider dispatch and shared URL validation 2026-02-25 03:30:45 +08:00
Shadman Hossain
d6824afd21 style: fix clippy warnings and cargo fmt in new code
- Add underscores to long numeric literals (1234567890 → 1_234_567_890)
- Allow cast_possible_truncation for rough token estimates
- Replace loop/match with while-let for event stream parsing
- Merge identical match arms for event types
- Add #[allow(clippy::cast_possible_truncation)] on test helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 22:22:16 +08:00
Shadman Hossain
18780b27fe feat: add OpenAI-compatible /v1/chat/completions and /v1/models endpoints
Add an OpenAI-compatible API surface to the gateway so that standard
OpenAI client libraries can interact with ZeroClaw directly.

Endpoints:
- POST /v1/chat/completions — supports both streaming (SSE) and
  non-streaming responses, bearer token auth, rate limiting
- GET /v1/models — returns the gateway's configured model

The chat completions endpoint accepts the standard OpenAI request format
(model, messages, temperature, stream) and returns responses in the
OpenAI envelope format. Streaming uses SSE with delta chunks and a
[DONE] sentinel. A 512KB body limit is applied (vs 64KB default) since
chat histories can be large.

When the underlying provider doesn't support native streaming, the
handler falls back to wrapping the non-streaming response in a single
SSE chunk for transparent compatibility.

Includes 8 unit tests for request/response serialization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 22:22:16 +08:00
Chummy
d6ca79a52e fix(gateway): fill qq fields in node control test AppState 2026-02-24 22:03:53 +08:00
Chummy
c876a03819 feat(gateway): add experimental node-control scaffold API 2026-02-24 22:03:53 +08:00
Chummy
54dd7a4a9b feat(qq): add webhook receive mode with challenge validation 2026-02-24 19:30:36 +08:00
Chummy
0083aece57 fix(gateway): normalize masked reliability api_keys in config PUT 2026-02-24 19:03:50 +08:00
Chummy
b3b5055080 feat: replay custom provider api mode, route max_tokens, and lark image support 2026-02-24 15:59:49 +08:00
Chummy
04e5950020 fix(gateway): remove unused websocket sink import 2026-02-24 14:21:34 +08:00
Chummy
68f1ba1617 chore(fmt): normalize gateway import order for webchat fix 2026-02-24 14:21:34 +08:00
Preventnetworkhacking
35a5815513 fix(gateway): enable tool execution in web chat agent
Web chat was calling provider.chat_with_history() directly, bypassing
the agent loop. Tool calls were rendered as raw XML instead of executing.

Changes:
- Add tools_registry_exec to AppState for executable tools
- Replace chat_with_history with run_tool_call_loop in ws.rs
- Maintain conversation history per WebSocket session
- Add multimodal and max_tool_iterations config to AppState

Closes #1524
2026-02-24 14:21:34 +08:00
Chummy
ace493b32f chore(fmt): format gateway api after dashboard-save fix 2026-02-24 13:30:43 +08:00
argenis de la rosa
9751433803 fix(gateway): preserve masked config values on dashboard save
Replace line-based TOML masking with structured config masking so secret fields keep their original types (including reliability.api_keys arrays).\nHydrate dashboard PUT payloads with runtime config_path/workspace_dir and restore masked secret placeholders from current config before validation/save.\nAlso allow GET on /api/doctor for dashboard/client compatibility to avoid 405 responses.
2026-02-24 13:22:07 +08:00
Chummy
1290b73faa fix: align codex provider runtime options with current interfaces 2026-02-24 12:24:51 +08:00
Mike Johnson-Maxted
2f29ec75ef fix(onboard): use provider-aware env var hint in quick setup next steps
Replace hardcoded OPENROUTER_API_KEY hint with provider-aware logic:
- keyless local providers (ollama, llamacpp, etc.) show chat/gateway/status hints
- device-flow providers (copilot, gemini, openai-codex) show OAuth/first-run hint
- all other providers show the correct provider-specific env var via provider_env_var()

Also adds canonical alias "github-copilot" -> "copilot" in canonical_provider_name(),
and a new provider_supports_device_flow() helper with accompanying test.

Additionally fixes pre-existing compile blockers that prevented CI from running:
- fix(security): correct raw string literals in leak_detector.rs that terminated
  early due to unescaped " inside r"..." (use r#"..."# instead)
- fix(gateway): add missing wati: None in two test AppState initializations
- fix(gateway): use serde::Deserialize path on WatiVerifyQuery struct
- fix(security): add #[allow(unused_imports)] on new pub use re-exports in mod.rs
- fix(security): remove unused serde::{Deserialize, Serialize} import
- chore: apply cargo fmt to files that had pending formatting diffs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 11:46:49 +08:00
Adam Singer
388e168158 [bug] Regex build failure 2026-02-24 11:34:12 +08:00
argenis de la rosa
5c63ec380a Merge branch 'main' into dev — consolidate all upstream releases 2026-02-23 14:03:17 -05:00
Ken Yeung
ecc8865cb7
feat: add WATI WhatsApp Business API channel (#1472)
Add a new WATI channel for WhatsApp Business API integration via the
WATI managed platform. WATI simplifies WhatsApp integration with its
own REST API and webhook system.

- New WatiChannel implementation (webhook mode, REST send)
- WatiConfig with api_token, api_url, tenant_id, allowed_numbers
- Gateway routes: GET/POST /wati for webhook verification and messages
- Flexible webhook parsing handles WATI's variable field names
- 15 unit tests covering parsing, allowlist, timestamps, phone normalization
2026-02-23 08:02:00 -05:00
reidliu41
d3f0a79fe9 Summary
- Problem: The existing http_request tool returns raw HTML/JSON, which is nearly unusable for LLMs to extract
  meaningful content from web pages.
- Why it matters: All mainstream AI agents (Claude Code, Gemini CLI, Aider) have dedicated web content extraction
  tools. ZeroClaw lacks this capability, limiting its ability to research and gather information from the web.
- What changed: Added a new web_fetch tool that fetches web pages and converts HTML to clean plain text using
  nanohtml2text. Includes domain allowlist/blocklist, SSRF protection, redirect following, and content-type aware
  processing.
- What did not change (scope boundary): http_request tool is untouched. No shared code extracted between http_request
   and web_fetch (DRY rule-of-three: only 2 callers). No changes to existing tool behavior or defaults.

Label Snapshot (required)

  - Risk label: risk: medium
  - Size label: size: M
  - Scope labels: tool, config
  - Module labels: tool: web_fetch
  - If any auto-label is incorrect, note requested correction: N/A

  Change Metadata

  - Change type: feature
  - Primary scope: tool

  Linked Issue

  - Closes #
  - Related #
  - Depends on #
  - Supersedes #

  Supersede Attribution (required when Supersedes # is used)

  N/A

  Validation Evidence (required)

  cargo fmt --all -- --check   # pass
  cargo clippy --all-targets -- -D warnings  # no new warnings (pre-existing warnings only)
  cargo test --lib -- web_fetch  # 26/26 passed
  cargo test --lib -- tools::tests  # 12/12 passed
  cargo test --lib -- config::schema::tests  # 134/134 passed

  - Evidence provided: unit test results (26 new tests), manual end-to-end test with Ollama + qwen2.5:72b
  - If any command is intentionally skipped, explain why: Full cargo clippy --all-targets has 43 pre-existing errors
  unrelated to this PR (e.g. await_holding_lock, format! appended to String). Zero errors from web_fetch code.

  Security Impact (required)

  - New permissions/capabilities? Yes — new web_fetch tool can make outbound HTTP GET requests
  - New external network calls? Yes — fetches web pages from allowed domains
  - Secrets/tokens handling changed? No
  - File system access scope changed? No
  - If any Yes, describe risk and mitigation:
    - Deny-by-default: enabled = false by default; tool is not registered unless explicitly enabled
    - Domain filtering: allowed_domains (default ["*"] = all public hosts) + blocked_domains (takes priority).
  Blocklist always wins over allowlist.
    - SSRF protection: Blocks localhost, private IPs (RFC 1918), link-local, multicast, reserved ranges, IPv4-mapped
  IPv6, .local TLD — identical coverage to http_request
    - Rate limiting: can_act() + record_action() enforce autonomy level and rate limits
    - Read-only mode: Blocked when autonomy is ReadOnly
    - Response size cap: 500KB default truncation prevents context window exhaustion
    - Proxy support: Honors [proxy] config via tool.web_fetch service key

  Privacy and Data Hygiene (required)

  - Data-hygiene status: pass
  - Redaction/anonymization notes: No personal data in code, tests, or fixtures
  - Neutral wording confirmation: All test identifiers use neutral project-scoped labels

  Compatibility / Migration

  - Backward compatible? Yes — new tool, no existing behavior changed
  - Config/env changes? Yes — new [web_fetch] section in config.toml (all fields have defaults)
  - Migration needed? No — #[serde(default)] on all fields; existing configs without [web_fetch] section work unchanged

  i18n Follow-Through (required when docs or user-facing wording changes)

  - i18n follow-through triggered? No — no docs or user-facing wording changes

  Human Verification (required)

  - Verified scenarios:
    - End-to-end test: zeroclaw agent with Ollama qwen2.5:72b successfully called web_fetch to fetch
  https://github.com/zeroclaw-labs/zeroclaw, returned clean plain text with project description, features, star count
    - Tool registration: tool_count increased from 22 to 23 when enabled = true
    - Config: enabled = false (default) → tool not registered; enabled = true → tool available
  - Edge cases checked:
    - Missing [web_fetch] section in existing config.toml → works (serde defaults)
    - Blocklist priority over allowlist
    - SSRF with localhost, private IPs, IPv6
  - What was not verified:
    - Proxy routing (no proxy configured in test environment)
    - Very large page truncation with real-world content

  Side Effects / Blast Radius (required)

  - Affected subsystems/workflows: all_tools_with_runtime() signature gained one parameter (web_fetch_config); all 5
  call sites updated
  - Potential unintended effects: None — new tool only, existing tools unchanged
  - Guardrails/monitoring for early detection: enabled = false default; tool_count in debug logs

  Agent Collaboration Notes (recommended)

  - Agent tools used: Claude Code (Opus 4.6)
  - Workflow/plan summary: Plan mode → approval → implementation → validation
  - Verification focus: Security (SSRF, domain filtering, rate limiting), config compatibility, tool registration
  - Confirmation: naming + architecture boundaries followed (CLAUDE.md + CONTRIBUTING.md): Yes — trait implementation +
   factory registration pattern, independent security helpers (DRY rule-of-three), deny-by-default config

  Rollback Plan (required)

  - Fast rollback command/path: git revert <commit>
  - Feature flags or config toggles: [web_fetch] enabled = false (default) disables completely
  - Observable failure symptoms: tool_count in debug logs drops by 1; LLM cannot call web_fetch

  Risks and Mitigations

  - Risk: SSRF bypass via DNS rebinding (attacker-controlled domain resolving to private IP)
    - Mitigation: Pre-request host validation blocks known private/local patterns. Same defense level as existing
  http_request tool. Full DNS-level protection would require async DNS resolution before connect, which is out of scope
   for this PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
(cherry picked from commit 04597352cc)
2026-02-23 20:30:21 +08:00
reidliu41
a606e004e5 fix(gateway): skip pairing dialog in web UI when require_pairing is false 2026-02-23 15:01:46 +08:00
InuDial
9f844173b8 Use channels(&self) and channels_except_webhook(&self)
to reduce repeat
2026-02-21 19:38:19 +08:00
chumyin0912@gmail.com
179e7949c2 fix(gateway): align dashboard API client and embed built web assets 2026-02-21 16:14:01 +08:00
Zeki Kocabıyık
79337c76e8 feat(gateway): add embedded web dashboard with React frontend
Add a complete web management panel for ZeroClaw, served directly from
the binary via rust-embed. The dashboard provides real-time monitoring,
agent chat, configuration editing, and system diagnostics — all
accessible at http://localhost:5555/ after pairing.

Backend (Rust):
- Add 15+ REST API endpoints under /api/* with bearer token auth
- Add WebSocket agent chat at /ws/chat with query param auth
- Add SSE event stream at /api/events via BroadcastObserver
- Add rust-embed static file serving at /_app/* with SPA fallback
- Extend AppState with tools_registry, cost_tracker, event_tx
- Extract doctor::diagnose() for structured diagnostic results
- Add Serialize derives to IntegrationStatus, CliCategory, DiscoveredCli

Frontend (React + Vite + Tailwind CSS):
- 10 dashboard pages: Dashboard, AgentChat, Tools, Cron, Integrations,
  Memory, Config, Cost, Logs, Doctor
- WebSocket client with auto-reconnect for agent chat
- SSE client (fetch-based, supports auth headers) for live events
- Full EN/TR internationalization (~190 translation keys)
- Dark theme with responsive layouts
- Auth flow via 6-digit pairing code, token stored in localStorage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 16:14:01 +08:00