Compare commits

...

105 Commits

Author SHA1 Message Date
Argenis
feaca20582
feat: desktop companion app + device-aware installer + CI/CD (#4500)
* feat: add desktop companion app integration and CI/CD pipeline

- Add `zeroclaw desktop` CLI command to launch/install companion app
- Add device-aware installer (desktop/server/mobile/embedded/container)
- Replace from-source Tauri build with pre-built .dmg download flow
- Add `build-desktop` job to beta and stable release workflows
- Build universal macOS .dmg via Tauri on macos-14 runners
- Include .dmg in GitHub Release assets alongside CLI binaries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: bump version to 0.6.1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update Cargo.lock for 0.6.1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 03:59:11 -04:00
Nim G
40af505e90
feat(security): wire LeakDetector into outbound message path (#4457)
* feat(security): wire LeakDetector into outbound message path

Wire LeakDetector.scan() into channel response sanitization (via
sanitize_channel_response) and cron job delivery (via
deliver_announcement) to prevent credential leakage to external
channels.

Changes:
- src/channels/mod.rs: Add leak detection to sanitize_channel_response()
  with tracing::warn! on detection. Follows prior art from commit fb3b7b8e.
- src/cron/scheduler.rs: Add leak detection to deliver_announcement()
  before sending output to any channel (Telegram, Discord, Slack,
  Mattermost, Signal, Matrix, WhatsApp, QQ).
- Added unit tests in both modules to verify redaction behavior.

All outbound channel messages and cron job delivery are now scanned for
API keys, AWS credentials, JWTs, database URLs, private keys, and
high-entropy tokens. Detected leaks are redacted before delivery and
logged via tracing::warn! with pattern details.

Ref: SPEC-1.3-leak-detector-outbound.md (gap closure §1.3)
Prior art: commit fb3b7b8e (zeroclaw_homecomming branch)

* fix(security): enforce leak redaction with RedactedOutput newtype

Replace raw String return from scan_and_redact_output with a
RedactedOutput newtype. All channel dispatch in deliver_announcement
must go through this type, so the compiler rejects any path that
skips leak detection. Replaces tests that called LeakDetector in
isolation with tests that exercise scan_and_redact_output through
the RedactedOutput boundary.
2026-03-24 01:43:49 -04:00
Nim G
f812dbcb85
feat(channels): add message redaction API to Channel trait (#4458)
* feat(channels): add message redaction API to Channel trait

Add redact_message() method to Channel trait with default no-op implementation.
Implement for MatrixChannel using existing room.redact() SDK method.
Add comprehensive test coverage for trait default and Matrix implementation.

Changes:
- src/channels/traits.rs: Add redact_message() default method and test
- src/channels/matrix.rs: Implement redact_message() for MatrixChannel
- tests/integration/channel_matrix.rs: Add RedactMessage event, test channel impl, and lifecycle test

Tests added:
- default_redact_message_returns_success (trait default)
- redact_message_lifecycle (MatrixTestChannel)
- Updated minimal_channel_all_defaults_succeed

All tests pass. No breaking changes.

* test(channels): clarify redact_message test coverage

* fix(channels): use OwnedRoomId for matrix redact_message room lookup

RoomId is unsized (wraps str), so inlining .parse() in
get_room(&target_room.parse()?) fails to compile with the
channel-matrix feature. Use an intermediate OwnedRoomId binding,
matching the pattern used in send() and listen().
2026-03-24 01:43:41 -04:00
Nim G
59225d97b3
feat(memory): add bulk memory deletion by namespace and session (#4459)
Add purge_namespace() and purge_session() methods to Memory trait with
default bail! implementations. Implement both in SQLite backend using
spawn_blocking pattern matching forget(). Expose via new memory_purge
tool with security enforcement.

Changes:
- src/memory/traits.rs: Add purge_namespace/purge_session trait methods
- src/memory/sqlite.rs: Implement purge methods + 8 backend tests
- src/tools/memory_purge.rs: New tool following memory_forget pattern
- src/tools/mod.rs: Register memory_purge module and export

Tests:
- SQLite backend: 8 tests (namespace/session purge, preservation, count, noop)
- Tool: 8 tests (name/schema, purge success, noop, missing param, readonly/rate-limit)

All tests pass. Clippy clean. Formatted with cargo fmt.
2026-03-24 01:43:39 -04:00
Nim G
483f773e1d
feat(ci): add per-component path labels and labeler workflow (#4461)
* feat(ci): add per-component labels for channels, providers, and tools

labeler.yml only had base scope labels (channel, provider, tool) matching
entire directories. The PR workflow and ci-map specs already describe
fine-grained module labels (channel:telegram, provider:kimi, tool:shell)
but the labeler config never implemented them.

Adds 27 channel, 13 provider, and 12 tool group entries. Base labels are
preserved for the compaction rule in pr-labeler.yml.

* feat(ci): add pr-path-labeler workflow to invoke actions/labeler

labeler.yml has path-label config but no workflow was invoking it.
Adds pr-path-labeler.yml on pull_request_target to apply path and
per-component labels automatically. Updates actions source policy
doc to reflect the new workflow and action.

* docs(contributing): add label registry as single reference for all PR labels

Label definitions were scattered across labeler.yml, label-policy.json,
pr-workflow.md, and ci-map.md with no consolidated view. Adds a single
registry documenting all 99 labels by category, their definitions, how
each is applied, and which automation exists vs. is specced but missing.

* docs(contributing): correct label-registry automation status

The CI was simplified from 20+ workflows to 4. Size, risk,
contributor tier, and triage label automation (pr-labeler.yml,
pr-auto-response.yml, pr-check-stale.yml) were removed during
that simplification. Update the registry to reflect that these
labels are now applied manually, not "not yet implemented."

See PR #2931 for the upstream docs cleanup.
2026-03-24 01:43:35 -04:00
Argenis
b4bbe820a2
feat(memory): add pgvector support and Postgres knowledge graph (#4028) (#4488)
* feat(memory): add pgvector support and Postgres knowledge graph (#4028)

Enhance PostgresMemory with optional pgvector extension for hybrid
vector+keyword recall. Add namespace and importance columns. Create
PgKnowledgeGraph with recursive CTEs for graph traversal (no AGE
dependency). Extend ConsolidationResult with facts/trend fields for
richer extraction. All behind memory-postgres feature flag.

New file: knowledge_graph_pg.rs (5 unit tests)
Modified: postgres.rs (pgvector init, namespace/importance columns),
consolidation.rs (facts/trend fields), StorageProviderConfig

* fix: update PostgresMemory::new call in CLI with pgvector params
2026-03-24 01:04:01 -04:00
Argenis
1702bb2747
fix: route WebSocket connections through configured proxy (#4408)
* fix: route WebSocket connections through configured proxy (#4408)

tokio_tungstenite::connect_async does not honour proxy settings. Add
proxy-aware WebSocket connect helpers (HTTP CONNECT and SOCKS5) in
config::schema and update all six channel WebSocket connections
(discord, discord_history, slack, dingtalk, lark, qq) to use
ws_connect_with_proxy instead of connect_async.

* fix: update Cargo.lock with tokio-socks dependency
2026-03-24 01:02:46 -04:00
Argenis
b6f661c3c5
feat(agent): add complexity-based eval and auto-classify routing (#3928) (#4486)
Add heuristic complexity estimation (Simple/Standard/Complex) and
post-response quality evaluation to enable cheapest-capable model
routing. When rule-based classification misses, auto_classify falls
back to complexity-tier hints for model selection. Eval gate checks
response quality and suggests retry with higher-tier model if below
threshold.

New file: eval.rs (14 unit tests)
Modified: AgentConfig (eval, auto_classify fields), classify_model()
2026-03-24 00:51:18 -04:00
Argenis
ac543cff20
fix(tools): allow git_operations to run in workspace subdirectories (#4483)
Fixes #4409. The git_operations tool now accepts an optional 'path'
parameter to specify a subdirectory within the workspace. Path
traversal outside the workspace is rejected for security.
2026-03-24 00:44:41 -04:00
Argenis
c88affa020
fix(cron): treat empty allowed_tools as None and sanitize tool names (#4482)
Fixes #4442. Empty allowed_tools arrays caused all tools to be filtered
out, making cron jobs fail. Also adds tool name validation in the
OpenRouter provider to skip tools with names that violate the OpenAI
naming regex.
2026-03-24 00:44:39 -04:00
Argenis
67998ad702
fix(docker): add [autonomy] section with auto_approve to Docker configs (#4481)
Fixes #4445. Docker/podman containers running in daemon mode had no
[autonomy] section in the baked config.toml, causing file_write and
file_edit tools to be auto-denied in non-interactive mode.
2026-03-24 00:44:32 -04:00
Argenis
c104b23ddb
feat(agent): add token efficiency analyzer layer (#3892) (#4484)
Add history pruning and context-aware tool filtering to reduce token
usage per API call. History pruner collapses old tool call/result pairs
and drops oldest messages when over budget. Context analyzer suggests
relevant tools per iteration based on keyword matching.

New files: history_pruner.rs, context_analyzer.rs
Modified: AgentConfig (new fields), ToolFilterGroup (filter_builtins),
agent loop (pruning integration)
2026-03-24 00:38:15 -04:00
Argenis
698adca707
fix: add channel-lark to default features (#4472)
Resolves #3540. Lark/Feishu channel was not included in default features, causing builds via cargo install, brew, and other standard install methods to produce binaries without Lark support. Also fixes pre-existing lark audio test failures that were previously hidden.
2026-03-24 00:23:43 -04:00
Argenis
50a877b4c1
fix(ci): use shared concurrency group for master push events (#4479)
The concurrency group used github.sha for push events, giving each
merge its own group. Back-to-back merges queued multiple full CI runs
that never cancelled each other, exhausting runner capacity.

Use a fixed 'push-master' group so only the latest push to master
runs CI — older runs are cancelled automatically.
2026-03-24 00:15:26 -04:00
Argenis
fa7b615508
feat(providers): support Bearer token API keys for Amazon Bedrock (#4473)
Add BedrockAuth enum supporting both SigV4 and BearerToken authentication.
BEDROCK_API_KEY env var takes precedence over AWS AKSK credentials.
When a Bearer token is provided, requests use Authorization: Bearer header
instead of SigV4 signing. Existing SigV4 auth remains fully backward
compatible.

Closes #3742
2026-03-24 00:08:13 -04:00
Argenis
ea9eccfe8b
feat: make shell tool timeout configurable via config.toml (#4468)
Add `[shell_tool]` section to config schema with `timeout_secs` field
(default 60s). The ShellTool struct now carries the timeout as an
instance field instead of relying on the hardcoded constant, and the
full tool registry reads the value from Config at construction time.

Closes #4331
2026-03-24 00:07:46 -04:00
Argenis
eb036b4d95
feat(skills): add TEST.sh validation/testing framework (#4476)
Add a `zeroclaw skills test [name] [--verbose]` command that discovers
and executes TEST.sh files inside skill directories. Each line in a
TEST.sh follows the format `command | expected_exit_code | pattern` and
is validated against actual exit code and output (substring or regex).

- Add `Test` variant to `SkillCommands` enum in lib.rs
- Create `src/skills/testing.rs` with test runner, parser, and
  pretty-printed results using the console crate
- Wire the new subcommand into `handle_command()` in skills/mod.rs
- Add example TEST.sh for the browser skill
- Include 14 unit tests covering parsing, pattern matching, execution,
  aggregation, and edge cases

Closes #3697
2026-03-23 23:58:52 -04:00
Argenis
1c07d5b411
feat(tools): add built-in ask_user tool for interactive prompts (#4474)
Implement a new ask_user tool that sends a question to a messaging
channel and waits for the user's response with configurable timeout.
Supports optional structured choices and works across all channels
(Telegram, Discord, Slack, CLI, Web) via the late-bound channel map
pattern used by PollTool and ReactionTool.

Closes #3698
2026-03-23 23:58:50 -04:00
Argenis
0fe3834349
feat(channels): migrate inline transcription to TranscriptionManager (#4475)
Migrate Telegram, Discord, Slack, WhatsApp Web, and Matrix channels
from inline TranscriptionConfig/transcribe_audio() calls to the
centralized TranscriptionManager pattern for unified provider selection,
fallback, and global cap enforcement.

Each channel now:
- Stores a transcription_manager: Option<Arc<TranscriptionManager>> field
- Builds the manager in with_transcription() from config
- Calls manager.transcribe() instead of the legacy transcribe_audio() shim
- Matrix replaces the whisper-cpp shell-out with the manager API

Closes #4311
2026-03-23 23:58:05 -04:00
Argenis
33f9d66b54
feat(channel): add transcribe_non_ptt_audio config for WhatsApp STT (#4470)
Allow transcription of forwarded/regular audio messages on WhatsApp,
not just push-to-talk voice notes. Controlled by the new
`transcribe_non_ptt_audio` option in `[transcription]` (default: false).

Closes #4343
2026-03-23 23:43:28 -04:00
Argenis
368f39829f
fix: strip [Used tools: xxx] logs from channel responses (#4400)
The tool-context summary (e.g. `[Used tools: browser_open]`) was
prepended to assistant history entries for all non-Telegram channels so
the LLM could retain awareness of prior tool usage.  This caused the
model to learn and reproduce the bracket format in its own output,
delivering raw log lines to end-users instead of meaningful tool results.

Three changes:

1. Stop prepending `[Used tools: …]` to stored history for all channels.
   The LLM already receives tool context through the tool-call/result
   messages built by `run_tool_call_loop`, making the summary redundant.

2. Strip any `[Used tools: …]` prefix from cached history on reload, so
   existing sessions are cleaned up retroactively.

3. Strip the prefix in `sanitize_channel_response` as a safety net, in
   case the model still echoes it from older context.
2026-03-23 23:34:29 -04:00
Argenis
9376c26018
feat(tools): add Claude Code task runner with Slack progress and SSH handoff (#4466)
Implement ClaudeCodeRunner that spawns Claude Code in a tmux session with
HTTP hooks to POST tool execution events back to ZeroClaw's gateway,
updating a Slack message in-place with progress, plus an SSH handoff link.

Components:
- src/tools/claude_code_runner.rs: session lifecycle (tmux spawn, hook
  config generation, event handling, TTL cleanup)
- src/gateway/api.rs: POST /hooks/claude-code endpoint for hook events
- src/channels/slack.rs: chat.update + chat.postMessage with ts return
- src/config/schema.rs: ClaudeCodeRunnerConfig (enabled, ssh_host,
  tmux_prefix, session_ttl)
- src/tools/mod.rs: register runner tool

Closes #4287
2026-03-23 23:24:58 -04:00
Argenis
08e131d7c6
feat(slack): add /config command with Block Kit UI for model switching (#4464)
Add interactive /config command for Slack that renders Block Kit
static_select dropdowns for provider and model selection. Interactive
block_actions payloads from Socket Mode are translated into synthetic
/models and /model commands so existing runtime command handling
applies the selection to the per-sender RouteSelectionMap.

- Enable runtime model switching for the slack channel
- Add ShowConfig variant to ChannelRuntimeCommand
- Parse /config in parse_runtime_command (gated by supports_runtime_model_switch)
- Build Block Kit JSON with provider/model dropdowns and current selection
- Detect __ZEROCLAW_BLOCK_KIT__ prefix in SlackChannel::send() to post raw blocks
- Handle interactive envelope type in Socket Mode for block_actions events
- Non-Slack channels get a plain-text /config fallback

Closes #4286
2026-03-23 23:24:56 -04:00
Argenis
36db977b35
feat: adopt AGENTS.md as the primary agent instruction format (#4462)
Replace the AGENTS.md symlink with a real file containing all cross-tool
agent instructions. Reduce CLAUDE.md to Claude Code-specific directives
only, with a reference back to AGENTS.md for shared conventions.

Closes #4289
2026-03-23 23:24:45 -04:00
Argenis
92b0ebb61a
fix: include channel-lark in Docker build defaults (#4463)
Resolves #3538. Docker builds only included memory-postgres in ZEROCLAW_CARGO_FEATURES, which meant Lark/Feishu long-connection (WebSocket) support was not compiled into container images.
2026-03-23 23:22:36 -04:00
Argenis
9c312180a2
chore: bump version to 0.6.0 (#4455)
Includes macOS desktop menu bar app, media pipeline channels,
SOP engine improvements, and CLI tool integrations.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 21:16:46 -04:00
Argenis
a433c37c53
feat: add macOS desktop menu bar app (Tauri) (#4454)
* feat(tauri): add desktop app scaffolding with Tauri commands

Add Tauri v2 workspace member under apps/tauri with gateway client,
shared state, and Tauri command handlers for status, health, channels,
pairing, and agent messaging. The desktop app communicates with the
ZeroClaw gateway over HTTP and is buildable independently via
`cargo check -p zeroclaw-desktop`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(tauri): add system tray icon, menu, and events

Add tray module with menu construction (Show Dashboard, Status, Open
Gateway, Quit) and event handlers. Wire tray setup into the Tauri
builder's setup hook. Add icons/ placeholder directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tauri): add mobile entry, platform capabilities, icons, and tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(tauri): add auto-pairing, gradient icons, and WebView integration

Desktop app now auto-pairs with the gateway on startup using localhost
admin endpoints, injects the token into the WebView localStorage, and
loads the dashboard from the gateway URL. Adds blue gradient Z tray
icons (idle/working/error/disconnected states), proper .icns app icon,
health polling with tray updates, and Tauri-aware API/WS/SSE paths in
the web frontend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(tauri): add dock icon, keep-alive for tray mode, and install preflight

- Set macOS dock icon programmatically via NSApplication API so it shows
  the blue gradient Z even in dev builds without a .app bundle
- Add RunEvent::ExitRequested handler to keep app alive as a menu bar app
  when all windows are closed
- Add desktop app preflight checks to install.sh (Rust, Xcode CLT,
  cargo-tauri, Node.js) with automatic build on macOS

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 20:55:22 -04:00
Argenis
af8e805016
fix(tools): correct CLI args for codex and gemini harness tools (#4401)
* feat(tools): add Codex CLI and Gemini CLI harness tools

Add two new ACP harness tools following the same pattern as ClaudeCodeTool:

- CodexCliTool: delegates to `codex -q` for OpenAI Codex CLI
- GeminiCliTool: delegates to `gemini -p` for Google Gemini CLI

Both tools share the same security model (rate limiting, act-policy
enforcement, workspace containment, env sanitisation, kill-on-drop
timeout) and are config-gated via `[codex_cli]` / `[gemini_cli]`
sections with `enabled = true`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): use 'codex exec' instead of invalid '-q' flag

* feat(tools): add OpenCode CLI harness tool

Add opencode_cli tool following the same pattern as codex_cli and
gemini_cli. Uses `opencode run "<prompt>"` for non-interactive
delegation. Includes config struct, factory registration, env
sanitization, timeout, and tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): correct CLI invocation args for codex and gemini tools

- Fix codex_cli to use `codex -q` (quiet mode) instead of incorrect
  `codex exec` subcommand, matching the documented behavior and the
  actual @openai/codex CLI interface.
- Fix gemini_cli install instruction to reference `@google/gemini-cli`
  instead of the incorrect `@anthropic-ai/gemini-cli` package name.

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: rareba <rareba@users.noreply.github.com>
2026-03-23 17:42:07 -04:00
Argenis
9f127f896d
fix(tools): validate task_id to prevent path traversal in delegate tool (#4405)
* feat(tools): add background and parallel execution to delegate tool

Add three new execution modes to the delegate tool:

- background: when true, spawns the sub-agent in a background tokio task
  and returns a task_id immediately. Results are persisted to
  workspace/delegate_results/{task_id}.json.

- parallel: accepts an array of agent names, runs them all concurrently
  with the same prompt, and returns all results when complete.

- action parameter with check_result/list_results/cancel_task support
  for managing background task lifecycle.

Cascade control: background sub-agents use child CancellationToken
derived from the parent, enabling cancel_all_background_tasks() to
abort all running background agents when the parent session ends.

Existing synchronous delegation flow is fully preserved (opt-in only).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tools): validate task_id as UUID to prevent path traversal in delegate tool

The check_result and cancel_task actions accept a user-provided task_id
that is used directly in filesystem path construction. A malicious
task_id like "../../etc/passwd" could read or overwrite arbitrary files.

Since task_ids are always generated as UUIDs internally, this adds UUID
format validation before any filesystem operations, rejecting invalid
task_id values with a clear error message.

Also updates existing tests to use valid UUID-format task_ids and adds
dedicated path traversal rejection tests.

* fix: add missing attachments field to wati ChannelMessage after media pipeline merge

* fix(channels): add missing attachments field to voice_wake and lark

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:57:24 -04:00
Argenis
b98971c635
fix(sop): fix state file leak and add deterministic execution tests (#4404)
* feat(sop): add deterministic execution mode with typed steps and approval checkpoints

Add opt-in deterministic execution to the SOP workflow engine, inspired
by OpenClaw's Lobster engine. Deterministic mode bypasses LLM round-trips
for step transitions, executing steps sequentially with piped outputs.

Key additions:
- SopExecutionMode::Deterministic variant and `deterministic: true` SOP.toml flag
- SopStepKind enum (Execute/Checkpoint) for marking approval pause points
- StepSchema for typed input/output validation (JSON Schema fragments)
- DeterministicRunState for persisting/resuming interrupted workflows
- DeterministicSavings for tracking LLM calls saved
- SopRunAction::DeterministicStep and CheckpointWait action variants
- SopRunStatus::PausedCheckpoint status
- Engine methods: start_deterministic_run, advance_deterministic_step,
  resume_deterministic_run, persist/load_deterministic_state
- SopConfig in config/schema.rs with sops_dir, default_execution_mode,
  max_concurrent_total, approval_timeout_secs, max_finished_runs
- Wire `pub mod sop` in lib.rs (previously dead/uncompiled module)
- Fix pre-existing test issues: TempDir import, async test annotations

All 86 SOP core tests pass (engine: 42, mod: 17, dispatch: 13, types: 14).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(sop): resolve clippy warnings and fix metrics test failures

- Add `ampersona-gates = []` feature declaration to Cargo.toml to fix
  clippy `unexpected cfg condition value` errors in sop/mod.rs,
  sop/audit.rs, and sop/metrics.rs.
- Use dynamic Utc::now() timestamps in metrics test helper `make_run()`
  instead of hardcoded 2026-02-19 dates, which had drifted outside the
  7-day/30-day windowed metric windows causing 7 test failures.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(sop): remove non-functional ampersona-gates feature flag

The ampersona-gates feature flag referenced ampersona_core and
ampersona_engine crates that do not exist, causing cargo check
--all-features to fail. Remove the feature flag and all gated code:

- Remove ampersona-gates from Cargo.toml [features]
- Delete src/sop/gates.rs (entire module behind cfg gate)
- Remove gated methods from audit.rs (log_gate_decision, log_phase_state)
- Remove gated MetricsProvider impl and tests from metrics.rs
- Simplify sop_status.rs gate_eval field and append_gate_status
- Update observability docs (EN + zh-CN)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(sop): fix formatting in metrics.rs

* fix(sop): wire SOP tools into tool registry

The five SOP tools (sop_list, sop_advance, sop_execute, sop_approve,
sop_status) existed as source files but were never registered in
all_tools_with_runtime. They are now conditionally registered when
sop.sops_dir is configured.

Also fixes:
- Add mod sop + SopCommands re-export to main.rs (binary crate)
- Handle new DeterministicStep/CheckpointWait variants in match arms
- Add missing struct fields (deterministic, kind, schema, llm_calls_saved)
  to test constructors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(sop): fix state file leak and add deterministic execution tests

- Fix persist_deterministic_state fallback: use system temp dir instead
  of current working directory when SOP location is unset, preventing
  state files from polluting the working directory.
- Add comprehensive test coverage for deterministic execution path:
  start, advance, checkpoint pause, completion with savings tracking,
  and rejection of non-deterministic SOPs.
- Add tests for deterministic flag in TOML manifest loading and
  checkpoint kind parsing from SOP.md.
- Add serde roundtrip tests for DeterministicRunState, SopStepKind,
  SopExecutionMode::Deterministic, and SopRunStatus::PausedCheckpoint.

* ci: retrigger CI

* fix: add missing attachments field to wati ChannelMessage after media pipeline merge

* fix(channels): add missing attachments field to voice_wake and lark

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: rareba <rareba@users.noreply.github.com>
2026-03-23 16:40:26 -04:00
Argenis
2ee0229740
fix(gateway): improve WebSocket chat error handling and diagnostics (#4407)
- Add error codes to WS error messages (AGENT_INIT_FAILED, AUTH_ERROR, PROVIDER_ERROR, INVALID_JSON, UNKNOWN_MESSAGE_TYPE, EMPTY_CONTENT)
- Send Close frame with code 1011 on agent init failure
- Add tracing logs for agent init and turn errors
- Increase MAX_API_ERROR_CHARS from 200 to 500
- Frontend: handle abnormal close codes and show config error hints

Fixes #3681
Supersedes #4366
Original work by mark-linyb
2026-03-23 15:30:18 -04:00
Argenis
0d2b57ee2e
fix(channels): ensure newline between narration and draft status lines (#4394)
Narration text from native tool-call providers that doesn't end with a newline now gets one appended before dispatch to the draft updater. Prevents garbled output in Telegram drafts.

Closes #4348
2026-03-23 15:30:15 -04:00
Argenis
b85a445955
fix(channels): prevent draft streaming hang after tool loop completion (#4393)
Drop delta_tx before awaiting draft_updater so the mpsc channel closes and the updater task can terminate. Without this, draft-capable channels (e.g. Telegram) hang indefinitely after the tool loop completes.

Closes #4300
2026-03-23 15:30:12 -04:00
Argenis
dbd8c77519
feat(channels): add automatic media understanding pipeline (#4402)
* feat(channels): add automatic media understanding pipeline for inbound messages

Add MediaPipeline that pre-processes inbound channel message attachments
before the agent sees them:

- Audio: transcribed via existing transcription infrastructure, annotated
  as [Audio transcription: ...]
- Images: annotated with [Image: <file> attached] (vision-aware)
- Video: annotated with [Video: <file> attached] (placeholder for future API)

The pipeline is opt-in via [media_pipeline] config section (default: disabled).
Individual media types can be toggled independently.

Changes:
- New src/channels/media_pipeline.rs with MediaPipeline struct and tests
- New MediaPipelineConfig in config/schema.rs
- Added attachments field to ChannelMessage for media pass-through
- Wired pipeline into process_channel_message after hooks, before agent

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(channels): add attachments field to integration test fixtures

Add missing `attachments: vec![]` to all ChannelMessage struct literals
in channel_matrix.rs and channel_routing.rs after the new attachments
field was added to the struct in traits.rs.

Also fix schema.rs test compilation: make TempDir import unconditional
and add explicit type annotations on tokio::fs calls to resolve type
inference errors in the bootstrap file tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(channels): add missing attachments field to gmail_push and discord_history constructors

These channels were added to master after the media pipeline PR was
originally branched. The ChannelMessage struct now requires an
attachments field, so initialise it to an empty Vec for channels
that do not yet extract attachments.

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 15:14:11 -04:00
Argenis
34db67428f
fix(gateway): stream ws agent chat responses (#4390)
Add Agent::turn_streamed() that forwards TurnEvent (Chunk, ToolCall,
ToolResult) through an mpsc channel during execution. The WebSocket
gateway uses tokio::join! to relay these events to the client in real
time instead of waiting for the full turn to complete.

Introduce chunk_reset message type so the frontend clears its draft
buffer before the authoritative done message arrives.  Update the React
AgentChat page to render streamed text live in the typing indicator
area, replacing the bounce-dot animation when content is available.

Backward-compatible: the done message still carries the full_response
field unchanged, and providers that do not support streaming fall back
to the non-streaming chat path transparently.

Closes #4372
2026-03-23 15:01:04 -04:00
Argenis
01d0c6b23a
fix(config): add cost tracking configuration to template and example files (#4387)
Add the [cost] section to dev/config.template.toml and
examples/config.example.toml documenting all CostConfig options
(enabled, daily_limit_usd, monthly_limit_usd, warn_at_percent,
allow_override) with their defaults and commented-out model pricing
examples.

Fixes #4373
2026-03-23 15:01:01 -04:00
Argenis
79f0a5ae30
fix(matrix): handle unused Result from backups().disable() (#4386)
Add `let _ =` prefix to explicitly discard the Result from
`client.encryption().backups().disable().await`, suppressing the
`unused_must_use` compiler warning.

Closes #4374
2026-03-23 15:00:58 -04:00
Argenis
5bdeeba213
feat(wati): add audio and voice message transcription (#4391)
* fix(transcription): remove deployment-specific WireGuard references from doc comments

LocalWhisperProvider and LocalWhisperConfig doc comments referenced
WireGuard and specific internal infrastructure. Deployment topology is
operator choice — replace with neutral examples.

* feat(wati): add audio and voice message transcription

* fix(wati): SSRF host validation, early fromMe check, download size cap

Validate that media_url host matches the configured api_url host before
fetching, preventing SSRF with credential leakage.  Move the fromMe
check before the HTTP download to avoid wasting bandwidth on outgoing
messages.  Add MAX_WATI_AUDIO_BYTES (25 MiB) Content-Length pre-check.
Skip empty transcripts in parse_audio_as_message.  Short-circuit
with_transcription() when config.enabled is false.

* fix(wati): remove dead field, enforce streaming size cap, log errors

- Remove unused transcription config field (clippy dead code)
- Use streaming download to enforce size cap without Content-Length
- Log network errors instead of silently swallowing with .ok()?

---------

Co-authored-by: Nim G <theredspoon@users.noreply.github.com>
2026-03-23 14:58:21 -04:00
Argenis
b5447175ff
feat(mattermost): add audio transcription via TranscriptionManager (#4389)
* fix(transcription): remove deployment-specific WireGuard references from doc comments

LocalWhisperProvider and LocalWhisperConfig doc comments referenced
WireGuard and specific internal infrastructure. Deployment topology is
operator choice — replace with neutral examples.

* feat(mattermost): add audio transcription via TranscriptionManager

Add transcription support for Mattermost audio attachments. Routes
audio through TranscriptionManager when configured, with duration
limit enforcement and wiremock-based integration tests.

* fix(mattermost): add download size cap, HTTP status check, warn logging

Replace chained .ok()? calls in try_transcribe_audio_attachment with
explicit error handling that logs warnings on HTTP failure, non-success
status codes, and oversized files.  Add MAX_MATTERMOST_AUDIO_BYTES
(25 MiB) Content-Length pre-check.  Remove mp4 and webm from the
extension-only fallback in is_audio_file().  Short-circuit
with_transcription() when config.enabled is false.

* fix(mattermost): add [Voice] prefix, filter empty, fix config init

- Add [Voice] prefix to transcribed audio matching Slack/Discord
- Filter empty transcription results
- Only store transcription config on successful manager init
- Update test expectation for [Voice] prefix

---------

Co-authored-by: Nim G <theredspoon@users.noreply.github.com>
2026-03-23 14:58:18 -04:00
Argenis
0dc05771ba
feat(lark): add audio message transcription (#4388)
* fix(transcription): remove deployment-specific WireGuard references from doc comments

LocalWhisperProvider and LocalWhisperConfig doc comments referenced
WireGuard and specific internal infrastructure. Deployment topology is
operator choice — replace with neutral examples.

* feat(lark): add audio message transcription

* test(channels): add three missing lark audio transcription tests

- lark_audio_skips_when_manager_none: parse_event_payload_async returns
  empty when transcription_manager is None
- lark_audio_routes_through_transcription_manager: wiremock end-to-end
  test proving file_key → download → whisper → ChannelMessage chain
- lark_audio_token_refresh_on_invalid_token_response: wiremock test
  verifying 401 triggers token invalidation and retry

Adds a #[cfg(test)] api_base_override field to LarkChannel so wiremock
can intercept requests that normally go to hardcoded Lark/Feishu API
base URLs.

* fix(lark): add audio download size cap, event_type guard, skip disabled transcription

Add MAX_LARK_AUDIO_BYTES (25 MiB) Content-Length pre-check before
reading audio response bodies on both the normal and token-refresh
paths.  Add the missing im.message.receive_v1 event_type guard in
parse_event_payload_async so non-message callbacks are rejected before
the audio branch.  Short-circuit with_transcription() when
config.enabled is false.

* fix(lark): address review feedback for audio transcription

- Wire parse_event_payload_async in webhook handler (was dead code)
- Use streaming download with enforced size cap (no Content-Length bypass)
- Fix test: lark_manager_some_when_valid_config asserted is_some with enabled=false
- Fix test: add missing header to lark_audio_skips_when_manager_none payload
- Remove redundant group-mention check in audio arm (shared check covers it)

---------

Co-authored-by: Nim G <theredspoon@users.noreply.github.com>
2026-03-23 14:58:11 -04:00
Argenis
10f9ea3454
fix(security): update blocked_commands_basic test after #4338 (#4399)
* fix(security): update blocked_commands_basic test after allowing python/node

After adding python3 and node to default_allowed_commands in #4338,
the test asserting they are blocked is now wrong. Replace with ruby
and perl which remain correctly blocked.

* fix(security): also fix python3 reference in pipe validation test

The command_with_pipes_validates_all_segments test also referenced
python3 in a blocked assertion. Replace with ruby.
2026-03-23 14:16:42 -04:00
linyibin
3ec532bc29
fix(config): add cost tracking configuration section to template (#4368)
Add [cost] section to dev/config.template.toml with all configuration options and commented per-model pricing examples. Fixes #3679.
2026-03-23 14:14:59 -04:00
Nisit Sirimarnkit
f1688c5910
fix(agent): use Gregorian datetime and prioritize date context in prompts (#4369)
Force explicit Gregorian year/month/day via Datelike/Timelike traits instead of chrono format() which inherits system locale (e.g. Buddhist calendar on Thai systems). Prepend datetime before memory context in user messages. Rename DateTimeSection header to CRITICAL CONTEXT.
2026-03-23 14:14:56 -04:00
Argenis
fd9f140268
feat(tools): add cross-channel poll creation tool (#4396)
* feat(tools): add cross-channel poll creation tool

Adds a poll tool that enables cross-channel poll creation with voting
support. Changes all_tools_with_runtime return type from 3-tuple to
4-tuple to accommodate the new reaction handle.

Original PR #4243 by rareba.

* ci: retrigger CI
2026-03-23 13:58:01 -04:00
Argenis
b2dccf86eb
feat(tools): add Firecrawl fallback for JS-heavy sites (#4395)
* feat(tools): add Firecrawl fallback for JS-heavy and bot-blocked sites

Adds optional Firecrawl API integration as a fallback when standard web
fetching fails due to JavaScript-heavy pages or bot protection. Includes
configurable API key, timeout, and domain allowlist/blocklist.

Original PR #4244 by rareba.

* ci: retrigger CI
2026-03-23 13:57:56 -04:00
Argenis
f0c106a938
fix(update): diagnose arch mismatch in validate_binary before execution (#4379)
When a downloaded binary has the wrong architecture, the previous code
attempted to execute it and surfaced a raw "Exec format error (os error 8)".
Now validate_binary reads the ELF/Mach-O header first, compares the binary
architecture against the host, and reports a clear diagnostic like:
"architecture mismatch: downloaded binary is aarch64 but this host is x86_64".

Closes #4291
2026-03-23 13:36:44 -04:00
Keith_void
e276e66c05
fix(config): resolve macOS test failures from path canonicalization (#4362)
- Prefer HOME env var in default_config_dir() before falling back to UserDirs::new()
- Pre-create temp_default_dir in test so is_temp_directory() can canonicalize /var -> /private/var symlink on macOS
2026-03-23 13:33:19 -04:00
Giulio V
2300f21315
feat(channels): extend /new session reset to all channels (#4361)
Move supports_runtime_model_switch() guard from parse_runtime_command() entry to /model and /models match arms only, so /new is available on every channel.

Closes #4236
2026-03-23 13:33:06 -04:00
Giulio V
2575edb1d2
feat(tools): add LLM task tool for structured JSON-only sub-calls (#4241)
* feat(tools): add LLM task tool for structured JSON-only sub-calls

Add LlmTaskTool — a lightweight tool that runs a single prompt through
an LLM provider with no tool access and optionally validates the
response against a caller-supplied JSON Schema. Ideal for structured
data extraction, classification, and transformation in workflows.

- Parameters: prompt (required), schema (optional JSON Schema),
  model (optional override), temperature (optional override)
- Uses configured default provider/model from root config
- Validates response JSON against schema (required fields, type checks)
- Strips markdown code fences from LLM responses before validation
- Gated by ToolOperation::Act security policy
- Registered in all_tools_with_runtime (always available)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use non-constant value instead of approximate PI in tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: rareba <rareba@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 12:50:22 -04:00
Giulio V
d31f2c2d97
feat(agent): add loop detection guardrail for repetitive tool calls (#4240)
* feat(agent): add loop detection guardrail for repetitive tool calls

Introduces a LoopDetector that monitors a sliding window of recent tool
calls and detects three repetitive patterns:

1. Exact repeat — same tool+args called consecutively (default 3+)
2. Ping-pong — two tools alternating for 4+ cycles
3. No progress — same tool with different args but identical results (5+)

Each pattern escalates through Warning -> Block -> CircuitBreaker.
Configurable via [pacing] section: loop_detection_enabled (default true),
loop_detection_window_size (default 20), loop_detection_max_repeats
(default 3).

Wired into run_tool_call_loop alongside the existing time-gated
identical-output detector. Respects loop_ignore_tools exclusion list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(agent): fix channel test interaction with loop detector

The max_tool_iterations channel tests use an IterativeToolProvider that
intentionally repeats identical tool calls. The loop detector (enabled by
default) fires its circuit breaker before max_tool_iterations is reached,
causing the test to fail. Disable loop detection in these two tests so
they exercise only the max_tool_iterations boundary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(agent): address PR #4240 review — loop detector correctness and test precision

Critical fixes:
- Fix result_index/tool_calls misalignment: use enumerate() before
  filter_map() so the index stays aligned with tool_calls even when
  ordered_results contains None entries from skipped tool calls.
- Fix hash_value JSON key-order sensitivity: canonicalise() recursively
  sorts object keys before serialisation so {"a":1,"b":2} and
  {"b":2,"a":1} hash identically.

Tightened test assertions:
- ping_pong_escalates_with_more_cycles: assert Block with 5 cycles
  (was loose Warning|Block|Break match).
- no_progress_escalates_to_block_and_break: assert Break at 7 calls
  (was loose Block|Break match).
- no_progress_not_triggered_when_all_args_identical: assert Warning
  specifically (was accepting Ok as alternative).

New tests:
- ping_pong_detects_alternation_with_varying_args (item 3)
- window_eviction_prevents_stale_pattern_detection (item 4)
- hash_value_is_key_order_independent + nested variant (item 2)
- pacing_config_serde_defaults_match_manual_default (item 5)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: rareba <rareba@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 12:49:47 -04:00
Argenis
9d7f6c5aaf
fix(skills): surface actionable warning when skills are skipped due to script policy (#4383)
When a skill directory contains script files and `skills.allow_scripts`
is false (the default), the skill was silently skipped with only a
tracing::warn that most users never see. The LLM then reports "tool is
not available in this environment" with no guidance on how to fix it.

Now the loader emits a user-visible stderr warning that names the
skipped skill and suggests setting `skills.allow_scripts = true`.

Closes #4292
2026-03-23 12:47:32 -04:00
Argenis
f9081fcfa7
fix(update): use exact target triples in find_asset_url to prevent wrong binary selection (#4377)
The previous substring match (e.g. "aarch64-unknown-linux") could match
both gnu and android release assets. Switch to full triples like
"aarch64-unknown-linux-gnu" so find_asset_url only selects the correct
platform binary.

Closes #4293
2026-03-23 12:41:17 -04:00
Argenis
a4d95dec0e
fix(cost): enable cost tracking by default (#3679) (#4382)
Cost tracking was disabled by default (enabled: false), causing the
/api/cost endpoint to always return zeros unless users explicitly
opted in. Change the default to enabled: true so cost tracking works
out of the box for all models and channels.
2026-03-23 12:38:35 -04:00
Argenis
31508b8ec7
fix(build): disable prometheus on 32-bit ARM targets in installer (#3677) (#4384)
The prometheus crate requires AtomicU64 which is unavailable on 32-bit
ARM (armv7l/armv6l). Detect 32-bit ARM in install.sh and build with
--no-default-features, re-enabling only channel-nostr and
skill-creation (excluding observability-prometheus).
2026-03-23 12:35:09 -04:00
Argenis
3d9069552c
fix(gateway): send error responses for unrecognized WebSocket message types (#3681) (#4381)
Previously, the /ws/chat handler silently ignored messages with
unrecognized types, leaving clients waiting for a response that never
comes. Now sends explicit error messages describing the expected format.
2026-03-23 12:33:23 -04:00
Argenis
3b074041bf
fix(ollama): strip /api/chat suffix from user-provided base URL (#4376)
When users configure the Ollama api_url with the full endpoint path
(e.g. http://host:11434/api/chat), normalize_base_url only stripped
/api but not /api/chat, causing the final request URL to become
/api/chat/api/chat which fails on remote servers.

Closes #4342
2026-03-23 12:19:26 -04:00
Argenis
9a95318b85
fix(security): add python, python3, pip, node to default allowed commands (#4375)
The default_allowed_commands() list was missing common scripting
runtimes. After #3691 added serde defaults to AutonomyConfig, users
who omitted allowed_commands from their config silently fell back to
the restrictive default list, breaking shell tool access to Python.

Closes #4338
2026-03-23 12:19:23 -04:00
Argenis
ccd572b827
fix(matrix): handle unused Result from backups().disable() (#4374)
Add let _ = prefix to suppress unused Result warning on
client.encryption().backups().disable().await call.

Closes #4339
2026-03-23 12:19:10 -04:00
Argenis
41dd23175f
chore(ci): unify release pipeline for full auto-sync (#4283)
- Add tag-push trigger to Release Stable workflow so `git push origin v0.5.9`
  auto-triggers the full pipeline (builds, Docker, crates.io, website,
  Scoop, AUR, Homebrew, tweet) in one shot
- Add Homebrew Core as downstream job (was manual-only, never auto-triggered)
- Add `workflow_call` to pub-homebrew-core.yml so it can be called from
  the stable release workflow
- Skip beta releases on version bump commits (prevents beta/stable race)
- Skip auto crates.io publish when stable tag exists (prevents double-publish)
- Auto-create git tag on manual dispatch so tag always exists for downstream
- Fix cut_release_tag.sh to reference correct workflow name
2026-03-22 20:56:47 -04:00
Argenis
864d754b56
chore: bump version to 0.5.9 (#4282)
Release 0.5.9 includes:
- feat: enable internet access by default (#4270)
- feat: add browser automation skill and VNC setup scripts (#4281)
- feat: add voice message transcription support (#4278)
- feat: add image and file support for Feishu/Lark channel (#4280)
- feat: declarative cron job configuration (#4045)
- feat: add SearXNG search provider support (#4272)
- feat: register skill tools as callable tool specs (#4040)
- feat: named sessions with reconnect and validation (#4275)
- feat: restore time-decay scoring for memory (#4274)
- fix: prevent thinking level prefix leak across turns (#4277)
- fix: link enricher title extraction byte offset bug (#4271)
- fix: WhatsApp Web delivery channel with backend validation (#4273)
2026-03-22 20:23:55 -04:00
Argenis
ccd52f3394
feat: add browser automation skill and VNC setup scripts (#4281)
* feat: add browser automation skill and VNC setup scripts

- Add browser skill template for agent-browser CLI integration
- Add VNC setup scripts for GUI browser access (Xvfb, x11vnc, noVNC)
- Add comprehensive browser setup documentation
- Enables headless browser automation for AI agents

Tested on: Ubuntu 24.04, ZeroClaw 0.5.7, agent-browser 0.21.4

Co-authored-by: OpenClaw Assistant

* fix(docs): fix markdown lint errors and update browser config docs

- SKILL.md: add blank lines around headings (MD022)
- browser-setup.md: wrap bare URLs in angle brackets (MD034)
- browser-setup.md: rename duplicate "Access" heading (MD024)
- Update config examples to reflect browser enabled by default
- Add examples for restricting/disabling browser via config

---------

Co-authored-by: Argenis <theonlyhennygod@users.noreply.github.com>
2026-03-22 19:35:20 -04:00
Argenis
eb01aa451d
feat: add voice message transcription support (#4278)
Closes #4231. Adds voice memo detection and transcription for Slack and Discord channels. Audio files are downloaded, transcribed via the existing transcription module, and passed as text to the LLM.
2026-03-22 19:18:07 -04:00
Argenis
c785b45f2d
feat: add image and file support for Feishu/Lark channel (#4280)
Closes #4235. Adds image and file message type handling in the Lark channel - downloads images/files via Lark API, detects MIME types, and passes content to the model for analysis.
2026-03-22 19:14:43 -04:00
Argenis
ffb8b81f90
fix(agent): prevent thinking level prefix from leaking across turns (#4277)
* feat(agent): add thinking/reasoning level control per message

Users can set reasoning depth via /think:high etc. with resolution
hierarchy (inline > session > config > default). 6 levels from Off
to Max. Adjusts temperature and system prompt.

* fix(agent): prevent thinking level prefix from leaking across interactive turns

system_prompt was mutated in place for the first message's thinking
directive, then used as the "baseline" for restoration after each
interactive turn. This caused the first turn's thinking prefix to
persist across all subsequent turns.

Fix: save the original system_prompt before any thinking modifications
and restore from that saved copy between turns.
2026-03-22 19:09:12 -04:00
Argenis
65f856d710
fix(channels): link enricher title extraction byte offset bug (#4271)
* feat(channels): add automatic link understanding for inbound messages

Auto-detects URLs in inbound messages, fetches content, extracts
title+summary, and enriches the message before the agent sees it.
Includes SSRF protection (rejects private IPs).

* fix(channels): use lowercased string for title extraction to prevent byte offset mismatch

extract_title used byte offsets from the lowercased HTML to index into
the original HTML. Multi-byte characters whose lowercase form has a
different byte length (e.g. İ → i̇) would produce wrong slices or panics.
Fix: extract from the lowercased string directly. Add multibyte test.
2026-03-22 19:09:09 -04:00
Argenis
1682620377
feat(tools): enable internet access by default (#4270)
* feat(tools): enable internet access by default

Enable web_fetch, web_search, http_request, and browser tools by
default so ZeroClaw has internet access out of the box. Security
remains fully toggleable via config (set enabled = false to disable).

- web_fetch: enabled with allowed_domains = ["*"]
- web_search: enabled with DuckDuckGo (free, no API key)
- http_request: enabled with allowed_domains = ["*"]
- browser: enabled with allowed_domains = ["*"], agent_browser backend
- text_browser: remains opt-in (requires external binary)

* fix(tests): update component test for browser enabled by default

Update config_nested_optional_sections_default_when_absent to expect
browser.enabled = true, matching the new default.
2026-03-22 19:07:12 -04:00
Argenis
aa455ae89b
feat: declarative cron job configuration (#4045)
Add support for defining cron jobs directly in the TOML config file via
`[[cron.jobs]]` array entries. Declarative jobs are synced to the SQLite
database at scheduler startup with upsert semantics:

- New declarative jobs are inserted
- Existing declarative jobs are updated to match config
- Stale declarative jobs (removed from config) are deleted
- Imperative jobs (created via CLI/API) are never modified

Each declarative job requires a stable `id` for merge tracking. A new
`source` column (`"imperative"` or `"declarative"`) distinguishes the
two creation paths. Shell jobs require `command`, agent jobs require
`prompt`, validated before any DB writes.
2026-03-22 19:03:00 -04:00
Argenis
a9ffd38912
feat(memory): restore time-decay scoring lost in main→master migration (#4274)
Apply exponential time decay (2^(-age/half_life), 7-day half-life) to
memory entry scores post-recall. Core memories are exempt (evergreen).
Consolidate duplicate half-life constants into a single public constant
in the decay module.

Based on PR #4266 by 5queezer with constant consolidation fix.
2026-03-22 19:01:40 -04:00
Argenis
86a0584513
feat: add SearXNG search provider support (#4272)
Closes #4152. Adds SearXNG as a search provider option with JSON output support, configurable instance URL, env override support, and 7 new tests.
2026-03-22 19:01:35 -04:00
Argenis
abef4c5719
fix(cron): add WhatsApp Web delivery channel with backend validation (#4273)
Apply PR #4258 changes to add whatsapp/whatsapp-web/whatsapp_web match
arm in deliver_announcement, feature-gated behind whatsapp-web.

Added is_web_config() guard to bail early when the WhatsApp config is
for Cloud API mode (no session_path), preventing a confusing runtime
failure with an empty session path.
2026-03-22 18:58:26 -04:00
Argenis
483b2336c4
feat(gateway): add named sessions with reconnect and validation fixes (#4275)
* fix(cron): add WhatsApp Web delivery channel with backend validation

Apply PR #4258 changes to add whatsapp/whatsapp-web/whatsapp_web match
arm in deliver_announcement, feature-gated behind whatsapp-web.

Added is_web_config() guard to bail early when the WhatsApp config is
for Cloud API mode (no session_path), preventing a confusing runtime
failure with an empty session path.

* feat(gateway): add named sessions with human-readable labels

Apply PR #4267 changes with bug fixes:
- Add get_session_name trait method so WS session_start includes the
  stored name on reconnect (not just when ?name= query param is present)
- Rename API now returns 404 for non-existent sessions instead of
  silently succeeding
- Empty ?name= query param on WS connect no longer clears existing name
2026-03-22 18:58:15 -04:00
Argenis
14cda3bc9a
feat: register skill tools as callable tool specs (#4040)
Skill tools defined in [[tools]] sections are now registered as first-class
callable tool specs via the Tool trait, rather than only appearing as XML
in the system prompt. This enables the LLM to invoke skill tools through
native function calling.

- Add SkillShellTool for shell/script kind skill tools
- Add SkillHttpTool for http kind skill tools
- Add skills_to_tools() conversion and register_skill_tools() wiring
- Wire registration into both CLI and process_message agent paths
- Update prompt rendering to mark registered tools as callable
- Update affected tests across skills, agent/prompt, and channels
2026-03-22 18:51:24 -04:00
Argenis
6e8f0fa43c
docs: add ADR for tool shared state ownership contract (#4057)
Define the contract for long-lived shared state in multi-client tool
execution, covering ownership (handle pattern), identity assignment
(daemon-provided ClientId), lifecycle (validation at registration),
isolation (per-client for security state), and reload semantics
(config hash invalidation).
2026-03-22 18:40:34 -04:00
argenis de la rosa
a965b129f8 chore: bump version to 0.5.8
Release trigger bump after recent fixes.
2026-03-22 16:29:45 -04:00
Argenis
c135de41b7
feat(matrix): add allowed_rooms config for room-level gating (#4230) (#4260)
Add an `allowed_rooms` field to MatrixConfig that controls which rooms
the bot will accept messages from and join invites for. When the list
is non-empty, messages from unlisted rooms are silently dropped and
room invites are auto-rejected. When empty (default), all rooms are
allowed, preserving backward compatibility.

- Config: add `allowed_rooms: Vec<String>` with `#[serde(default)]`
- Message handler: replace disabled room_id filter with allowlist check
- Invite handler: auto-accept allowed rooms, auto-reject others
- Support both canonical room IDs and aliases, case-insensitive
2026-03-22 14:41:43 -04:00
Argenis
2d2c2ac9e6
feat(telegram): support forwarded messages with attribution (#4265)
Parse forward_from, forward_from_chat, and forward_sender_name fields
from Telegram message updates. Prepend forwarding attribution to message
content so the LLM has context about the original sender.

Closes #4118
2026-03-22 14:36:31 -04:00
Argenis
5e774bbd70
feat(multimodal): route image messages to dedicated vision provider (#4264)
When vision_provider is configured in [multimodal] config, messages
containing [IMAGE:] markers are automatically routed to the specified
vision-capable provider instead of failing on the default text provider.

Closes #4119
2026-03-22 14:36:29 -04:00
Argenis
33015067eb
feat(tts): add local Piper TTS provider (#4263)
Add a piper TTS provider that communicates with a local Piper/Coqui TTS
server via an OpenAI-compatible HTTP endpoint. This enables fully offline
voice pipelines: Whisper (STT) → LLM → Piper (TTS).

Closes #4116
2026-03-22 14:36:26 -04:00
Argenis
6b10c0b891
fix(approval): merge default auto_approve entries with user config (#4262)
When a user provides a custom `auto_approve` list in their TOML
config (e.g. to add an MCP tool), serde replaces the built-in
defaults instead of merging. This causes default safe tools like
`weather`, `calculator`, and `file_read` to lose auto-approve
status and get silently denied in non-interactive channel runs.

Add `ensure_default_auto_approve()` which merges built-in entries
into the user's list after deserialization, preserving user
additions while guaranteeing defaults are always present. Users
who want to require approval for a default tool can use
`always_ask`, which takes precedence.

Closes #4247
2026-03-22 14:28:09 -04:00
Argenis
bf817e30d2
fix(provider): prevent async runtime panic during model refresh (#4261)
Wrap `fetch_live_models_for_provider` calls in
`tokio::task::spawn_blocking` so the `reqwest::blocking::Client`
is created and dropped on a dedicated thread pool instead of
inside the async Tokio context. This prevents the
"Cannot drop a runtime in a context where blocking is not allowed"
panic when running `models refresh --provider openai`.

Closes #4253
2026-03-22 14:22:47 -04:00
Alix-007
0051a0c296
fix(matrix): enforce configured room scope on inbound events (#4251)
Co-authored-by: Alix-007 <267018309+Alix-007@users.noreply.github.com>
2026-03-22 14:08:13 -04:00
Canberk Özkan
d753de91f1
fix(skills): prevent panic by ensuring UTF-8 char boundary during truncation (#4252)
Fixed issue #4139.

Previously, slicing a string at exactly 64 bytes could land in the middle of a multi-byte UTF-8 character (e.g., Chinese characters), causing a runtime panic.

Changes:
- Replaced direct byte slicing with a safe boundary lookup using .char_indices().
- Ensures truncation always occurs at a valid character boundary at or before the 64-byte limit.
- Maintained existing hyphen-trimming logic.

Co-authored-by: loriscience <loriscience@gmail.com>
2026-03-22 14:08:01 -04:00
Argenis
f6b2f61a01
fix(matrix): disable automatic key backup when no backup key is configured (#4259)
Explicitly call `client.encryption().backups().disable()` when backups
are not enabled, preventing the matrix_sdk_crypto crate from attempting
room key backups on every sync cycle and spamming the logs with
"Trying to backup room keys but no backup key was found" warnings.

Closes #4227
2026-03-22 13:55:45 -04:00
Argenis
70e7910cb9
fix(web): remove unused import blocking release pipeline (#4234)
fix(web): remove unused import blocking release pipeline
2026-03-22 01:35:26 -04:00
argenis de la rosa
a8868768e8 fix(web): remove unused ChevronsUpDown import blocking release pipeline
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:20:51 -04:00
Argenis
67293c50df
chore: bump version to 0.5.7 (#4232)
chore: bump version to 0.5.7
2026-03-22 01:14:08 -04:00
argenis de la rosa
1646079d25 chore: bump version to 0.5.7
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:49:41 -04:00
Argenis
25b639435f
fix: merge voice-wake feature (PR #4162) with conflict resolution (#4225)
* feat(channels): add voice wake word detection channel

Add VoiceWakeChannel behind the `voice-wake` feature flag that:
- Captures audio from the default microphone via cpal
- Uses energy-based VAD to detect speech activity
- Transcribes speech via the existing transcription API (Whisper)
- Checks for a configurable wake word in the transcription
- On detection, captures the following utterance and dispatches it
  as a ChannelMessage

State machine: Listening -> Triggered -> Capturing -> Processing -> Listening

Config keys (under [channels_config.voice_wake]):
- wake_word (default: "hey zeroclaw")
- silence_timeout_ms (default: 2000)
- energy_threshold (default: 0.01)
- max_capture_secs (default: 30)

Includes tests for config parsing, state machine, RMS energy
computation, and WAV encoding.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(config): fix pre-existing test compilation errors in schema.rs

- Remove #[cfg(unix)] gate on `use tempfile::TempDir` import since
  TempDir is used unconditionally in bootstrap file tests
- Add explicit type annotations on tokio::fs::* calls to resolve
  type inference failures (create_dir_all, write, read_to_string)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(channels): exclude voice-wake from all-features CI check

Add a `ci-all` meta-feature in Cargo.toml that includes every feature
except `voice-wake`, which requires `libasound2-dev` (ALSA) not present
on CI runners. Update the check-all-features CI job to use
`--features ci-all` instead of `--all-features`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 00:49:12 -04:00
Argenis
77779844e5
feat(memory): layered architecture upgrade + remove mem0 backend (#4226)
feat(memory): layered architecture upgrade + remove mem0 backend
2026-03-22 00:47:42 -04:00
Argenis
f658d5806a
fix: honor [autonomy] config section in daemon/channel mode
Fixes #4171
2026-03-22 00:47:32 -04:00
Argenis
7134fe0824
Merge pull request #4223 from zeroclaw-labs/fix/4214-heartbeat-utf8-safety
fix(heartbeat): prevent UTF-8 panic, add memory bounds and path validation
2026-03-22 00:41:47 -04:00
Argenis
263802b3df
Merge pull request #4224 from zeroclaw-labs/fix/4215-thai-i18n-cleanup
fix(i18n): remove extra keys and translate notion in th.toml
2026-03-22 00:41:21 -04:00
Argenis
3c25fddb2a
fix: merge Gmail Pub/Sub push PR #4164 (already integrated via #4200) (#4222)
* feat(channels): add Gmail Pub/Sub push notifications for real-time email

Add GmailPushChannel that replaces IMAP polling with Google's Pub/Sub
push notification system for real-time email-driven automation.

- New channel at src/channels/gmail_push.rs implementing the Channel trait
- Registers Gmail watch subscription (POST /gmail/v1/users/me/watch)
  with automatic renewal before the 7-day expiry
- Handles incoming Pub/Sub notifications at POST /webhook/gmail
- Fetches new messages via Gmail History API (startHistoryId-based)
- Dispatches email messages to the agent with full metadata
- Sends replies via Gmail messages.send API
- Config: gmail_push.enabled, topic, label_filter, oauth_token,
  allowed_senders, webhook_url
- OAuth token encrypted at rest via existing secret store
- Webhook endpoint added to gateway router
- 30+ unit tests covering notification parsing, header extraction,
  body decoding, sender allowlist, and config serialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(config): fix pre-existing test compilation errors in schema.rs

- Remove #[cfg(unix)] gate on `use tempfile::TempDir` import since
  TempDir is used unconditionally in bootstrap file tests
- Add explicit type annotations on tokio::fs::* calls to resolve
  type inference failures (create_dir_all, write, read_to_string)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(channels): fix extract_body_text_plain test

Gmail API sends base64url without padding. The decode_body function
converted URL-safe chars back to standard base64 but did not restore
the padding, causing STANDARD decoder to fail and falling back to
snippet. Add padding restoration before decoding.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Giulio V <vannini.gv@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 00:40:42 -04:00
Argenis
a6a46bdd25
fix: add weather tool to default auto_approve list
Fixes #4170
2026-03-22 00:21:33 -04:00
Argenis
235d4d2f1c
fix: replace ILIKE substring matching with full-text search in postgres memory recall()
Fixes #4204
2026-03-22 00:20:11 -04:00
argenis de la rosa
bd1e8c8e1a merge: resolve conflicts with master + remove memory-mem0 from ci-all
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:18:09 -04:00
Argenis
f81807bff6
fix: serialize env-dependent codex tests to prevent race (#4210) (#4218)
Add a process-scoped Mutex that all env-var-mutating tests in
openai_codex::tests must hold.  This prevents std::env::set_var /
remove_var calls from racing when Rust's test harness runs them on
parallel threads.

Affected tests:
- resolve_responses_url_prefers_explicit_endpoint_env
- resolve_responses_url_uses_provider_api_url_override
- resolve_reasoning_effort_prefers_configured_override
- resolve_reasoning_effort_uses_legacy_env_when_unconfigured
2026-03-22 00:14:01 -04:00
argenis de la rosa
bb7006313c feat(memory): layered architecture upgrade + remove mem0 backend
Implement 6-phase memory system improvement:
- Multi-stage retrieval pipeline (cache → FTS → vector)
- Namespace isolation with strict filtering
- Importance scoring (category + keyword heuristics)
- Conflict resolution via Jaccard similarity + superseded_by
- Audit trail decorator (AuditedMemory<M>)
- Policy engine (quotas, read-only namespaces, retention rules)
- Deterministic sort tiebreaker on equal scores

Remove mem0 (OpenMemory) backend — all capabilities now covered
natively with better performance (local SQLite vs external REST API).

46 battle tests + 262 existing tests pass. Backward-compatible:
existing databases auto-migrate, existing configs work unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:09:43 -04:00
Argenis
9a49626376
fix: use POSIX-compatible sh -c instead of dash-specific -lc (#4209) (#4217)
* fix: build web dashboard during install.sh (#4207)

* fix: use POSIX-compatible sh -c instead of dash-specific -lc in cron scheduler (#4209)
2026-03-22 00:07:37 -04:00
Argenis
8b978a721f
fix: build web dashboard during install.sh (#4207) (#4216) 2026-03-22 00:02:54 -04:00
argenis de la rosa
75b4c1d4a4 fix(heartbeat): prevent UTF-8 panic, add memory bounds and path validation in session context
- Use char_indices for safe UTF-8 truncation instead of byte slicing
- Replace unbounded Vec with VecDeque rolling window in load_jsonl_messages
- Add path separator validation for channel/to to prevent directory traversal
2026-03-22 00:01:44 -04:00
argenis de la rosa
b2087e6065 fix(i18n): remove extra keys and translate untranslated notion entry in th.toml 2026-03-21 23:59:46 -04:00
Nisit Sirimarnkit
ad8f81ad76
Merge branch 'master' into i18n/thai-tool-descriptions 2026-03-22 10:28:47 +07:00
ninenox
c58e1c1fb3 i18n: add Thai tool descriptions 2026-03-22 10:09:03 +07:00
Martin Minkus
cb0779d761 feat(heartbeat): add load_session_context to inject conversation history
When `load_session_context = true` in `[heartbeat]`, the daemon loads the
last 20 messages from the target user's JSONL session file and prepends them
to the heartbeat task prompt before calling the LLM.

This gives the companion context — who the user is, what was last discussed —
so outreach messages feel like a natural continuation rather than a blank-slate
ping. Defaults to `false` (opt-in, no change to existing behaviour).

Key behaviours:
- Session context is re-read on every heartbeat tick (not cached at startup)
- Skips context injection if only assistant messages are present (prevents
  heartbeat outputs feeding back in a loop)
- Scans sessions directory for matching JSONL files using flexible filename
  matching: {channel}_{to}.jsonl, {channel}_*_{to}.jsonl, or
  {channel}_{to}_*.jsonl — handles varying session key formats
- Injects file mtime as "last message ~Xh ago" so the LLM knows how long
  the user has been silent

Config example:
  [heartbeat]
  enabled = true
  interval_minutes = 120
  load_session_context = true
  target = "telegram"
  to = "your_username"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 02:44:19 +00:00
212 changed files with 28307 additions and 3572 deletions

View File

@ -1,97 +0,0 @@
# Mem0 Integration: Dual-Scope Recall + Per-Turn Memory
## Context
Mem0 auto-save works but the integration is missing key features from mem0 best practices: per-turn recall, multi-level scoping, and proper context injection. This causes the bot to "forget" on follow-up turns and not differentiate users.
## What's Missing (vs mem0 docs)
1. **Per-turn recall** — only first turn gets memory context, follow-ups get nothing
2. **Dual-scope** — no sender vs group distinction. All memories use single hardcoded `user_id`
3. **System prompt injection** — memory prepended to user message (pollutes session history)
4. **`agent_id` scoping** — mem0 supports agent-level patterns, not used
## Changes
### 1. `src/memory/mem0.rs` — Use session_id for multi-level scoping
Map zeroclaw's `session_id` param to mem0's `user_id`. This enables per-user and per-group memory namespaces without changing the `Memory` trait.
```rust
// Add helper:
fn effective_user_id(&self, session_id: Option<&str>) -> &str {
session_id.filter(|s| !s.is_empty()).unwrap_or(&self.user_id)
}
// In store(): use effective_user_id(session_id) as mem0 user_id
// In recall(): use effective_user_id(session_id) as mem0 user_id
// In list(): use effective_user_id(session_id) as mem0 user_id
```
### 2. `src/channels/mod.rs` ~line 2229 — Per-turn dual-scope recall
Remove `if !had_prior_history` gate. Always recall from both sender scope and group scope (for group chats).
```rust
// Detect group chat
let is_group = msg.reply_target.contains("@g.us")
|| msg.reply_target.starts_with("group:");
// Sender-scope recall (always)
let sender_context = build_memory_context(
ctx.memory.as_ref(), &msg.content, ctx.min_relevance_score,
Some(&msg.sender),
).await;
// Group-scope recall (groups only)
let group_context = if is_group {
build_memory_context(
ctx.memory.as_ref(), &msg.content, ctx.min_relevance_score,
Some(&history_key),
).await
} else {
String::new()
};
// Merge (deduplicate by checking substring overlap)
let memory_context = merge_memory_contexts(&sender_context, &group_context);
```
### 3. `src/channels/mod.rs` ~line 2244 — Inject into system prompt
Move memory context from user message to system prompt. Re-fetched each turn, doesn't pollute session.
```rust
let mut system_prompt = build_channel_system_prompt(...);
if !memory_context.is_empty() {
system_prompt.push_str(&format!("\n\n{memory_context}"));
}
let mut history = vec![ChatMessage::system(system_prompt)];
```
### 4. `src/channels/mod.rs` — Dual-scope auto-save
Find existing auto-save call. For group messages, store twice:
- `store(key, content, category, Some(&msg.sender))` — personal facts
- `store(key, content, category, Some(&history_key))` — group context
Both async, non-blocking. DMs only store to sender scope.
### 5. `src/memory/mem0.rs` — Add `agent_id` support (optional)
Pass `self.app_name` as `agent_id` param to mem0 API for agent behavior tracking.
## Files to Modify
1. `src/memory/mem0.rs` — session_id → user_id mapping
2. `src/channels/mod.rs` — per-turn recall, dual-scope, system prompt injection, dual-scope save
## Verification
1. `cargo check --features whatsapp-web,memory-mem0`
2. `cargo test --features whatsapp-web,memory-mem0`
3. Deploy to Synology
4. Test DM: "我鍾意食壽司" → next turn "我鍾意食咩" → should recall
5. Test group: Joe says "我鍾意食壽司" → someone else asks "Joe 鍾意食咩" → should recall from group scope
6. Check mem0 server logs: GET with `user_id=sender` AND `user_id=group_key`
7. Check mem0 server logs: POST with both user_ids for group messages

View File

@ -118,3 +118,7 @@ PROVIDER=openrouter
# Optional: Brave Search (requires API key from https://brave.com/search/api)
# WEB_SEARCH_PROVIDER=brave
# BRAVE_API_KEY=your-brave-search-api-key
#
# Optional: SearXNG (self-hosted, requires instance URL)
# WEB_SEARCH_PROVIDER=searxng
# SEARXNG_INSTANCE_URL=https://searx.example.com

301
.github/labeler.yml vendored
View File

@ -36,6 +36,145 @@
- any-glob-to-any-file:
- "src/channels/**"
"channel:bluesky":
- changed-files:
- any-glob-to-any-file:
- "src/channels/bluesky.rs"
"channel:clawdtalk":
- changed-files:
- any-glob-to-any-file:
- "src/channels/clawdtalk.rs"
"channel:cli":
- changed-files:
- any-glob-to-any-file:
- "src/channels/cli.rs"
"channel:dingtalk":
- changed-files:
- any-glob-to-any-file:
- "src/channels/dingtalk.rs"
"channel:discord":
- changed-files:
- any-glob-to-any-file:
- "src/channels/discord.rs"
- "src/channels/discord_history.rs"
"channel:email":
- changed-files:
- any-glob-to-any-file:
- "src/channels/email_channel.rs"
- "src/channels/gmail_push.rs"
"channel:imessage":
- changed-files:
- any-glob-to-any-file:
- "src/channels/imessage.rs"
"channel:irc":
- changed-files:
- any-glob-to-any-file:
- "src/channels/irc.rs"
"channel:lark":
- changed-files:
- any-glob-to-any-file:
- "src/channels/lark.rs"
"channel:linq":
- changed-files:
- any-glob-to-any-file:
- "src/channels/linq.rs"
"channel:matrix":
- changed-files:
- any-glob-to-any-file:
- "src/channels/matrix.rs"
"channel:mattermost":
- changed-files:
- any-glob-to-any-file:
- "src/channels/mattermost.rs"
"channel:mochat":
- changed-files:
- any-glob-to-any-file:
- "src/channels/mochat.rs"
"channel:mqtt":
- changed-files:
- any-glob-to-any-file:
- "src/channels/mqtt.rs"
"channel:nextcloud-talk":
- changed-files:
- any-glob-to-any-file:
- "src/channels/nextcloud_talk.rs"
"channel:nostr":
- changed-files:
- any-glob-to-any-file:
- "src/channels/nostr.rs"
"channel:notion":
- changed-files:
- any-glob-to-any-file:
- "src/channels/notion.rs"
"channel:qq":
- changed-files:
- any-glob-to-any-file:
- "src/channels/qq.rs"
"channel:reddit":
- changed-files:
- any-glob-to-any-file:
- "src/channels/reddit.rs"
"channel:signal":
- changed-files:
- any-glob-to-any-file:
- "src/channels/signal.rs"
"channel:slack":
- changed-files:
- any-glob-to-any-file:
- "src/channels/slack.rs"
"channel:telegram":
- changed-files:
- any-glob-to-any-file:
- "src/channels/telegram.rs"
"channel:twitter":
- changed-files:
- any-glob-to-any-file:
- "src/channels/twitter.rs"
"channel:wati":
- changed-files:
- any-glob-to-any-file:
- "src/channels/wati.rs"
"channel:webhook":
- changed-files:
- any-glob-to-any-file:
- "src/channels/webhook.rs"
"channel:wecom":
- changed-files:
- any-glob-to-any-file:
- "src/channels/wecom.rs"
"channel:whatsapp":
- changed-files:
- any-glob-to-any-file:
- "src/channels/whatsapp.rs"
- "src/channels/whatsapp_storage.rs"
- "src/channels/whatsapp_web.rs"
"gateway":
- changed-files:
- any-glob-to-any-file:
@ -101,6 +240,73 @@
- any-glob-to-any-file:
- "src/providers/**"
"provider:anthropic":
- changed-files:
- any-glob-to-any-file:
- "src/providers/anthropic.rs"
"provider:azure-openai":
- changed-files:
- any-glob-to-any-file:
- "src/providers/azure_openai.rs"
"provider:bedrock":
- changed-files:
- any-glob-to-any-file:
- "src/providers/bedrock.rs"
"provider:claude-code":
- changed-files:
- any-glob-to-any-file:
- "src/providers/claude_code.rs"
"provider:compatible":
- changed-files:
- any-glob-to-any-file:
- "src/providers/compatible.rs"
"provider:copilot":
- changed-files:
- any-glob-to-any-file:
- "src/providers/copilot.rs"
"provider:gemini":
- changed-files:
- any-glob-to-any-file:
- "src/providers/gemini.rs"
- "src/providers/gemini_cli.rs"
"provider:glm":
- changed-files:
- any-glob-to-any-file:
- "src/providers/glm.rs"
"provider:kilocli":
- changed-files:
- any-glob-to-any-file:
- "src/providers/kilocli.rs"
"provider:ollama":
- changed-files:
- any-glob-to-any-file:
- "src/providers/ollama.rs"
"provider:openai":
- changed-files:
- any-glob-to-any-file:
- "src/providers/openai.rs"
- "src/providers/openai_codex.rs"
"provider:openrouter":
- changed-files:
- any-glob-to-any-file:
- "src/providers/openrouter.rs"
"provider:telnyx":
- changed-files:
- any-glob-to-any-file:
- "src/providers/telnyx.rs"
"service":
- changed-files:
- any-glob-to-any-file:
@ -121,6 +327,101 @@
- any-glob-to-any-file:
- "src/tools/**"
"tool:browser":
- changed-files:
- any-glob-to-any-file:
- "src/tools/browser.rs"
- "src/tools/browser_delegate.rs"
- "src/tools/browser_open.rs"
- "src/tools/text_browser.rs"
- "src/tools/screenshot.rs"
"tool:composio":
- changed-files:
- any-glob-to-any-file:
- "src/tools/composio.rs"
"tool:cron":
- changed-files:
- any-glob-to-any-file:
- "src/tools/cron_add.rs"
- "src/tools/cron_list.rs"
- "src/tools/cron_remove.rs"
- "src/tools/cron_run.rs"
- "src/tools/cron_runs.rs"
- "src/tools/cron_update.rs"
"tool:file":
- changed-files:
- any-glob-to-any-file:
- "src/tools/file_edit.rs"
- "src/tools/file_read.rs"
- "src/tools/file_write.rs"
- "src/tools/glob_search.rs"
- "src/tools/content_search.rs"
"tool:google-workspace":
- changed-files:
- any-glob-to-any-file:
- "src/tools/google_workspace.rs"
"tool:mcp":
- changed-files:
- any-glob-to-any-file:
- "src/tools/mcp_client.rs"
- "src/tools/mcp_deferred.rs"
- "src/tools/mcp_protocol.rs"
- "src/tools/mcp_tool.rs"
- "src/tools/mcp_transport.rs"
"tool:memory":
- changed-files:
- any-glob-to-any-file:
- "src/tools/memory_forget.rs"
- "src/tools/memory_recall.rs"
- "src/tools/memory_store.rs"
"tool:microsoft365":
- changed-files:
- any-glob-to-any-file:
- "src/tools/microsoft365/**"
"tool:shell":
- changed-files:
- any-glob-to-any-file:
- "src/tools/shell.rs"
- "src/tools/node_tool.rs"
- "src/tools/cli_discovery.rs"
"tool:sop":
- changed-files:
- any-glob-to-any-file:
- "src/tools/sop_advance.rs"
- "src/tools/sop_approve.rs"
- "src/tools/sop_execute.rs"
- "src/tools/sop_list.rs"
- "src/tools/sop_status.rs"
"tool:web":
- changed-files:
- any-glob-to-any-file:
- "src/tools/web_fetch.rs"
- "src/tools/web_search_tool.rs"
- "src/tools/web_search_provider_routing.rs"
- "src/tools/http_request.rs"
"tool:security":
- changed-files:
- any-glob-to-any-file:
- "src/tools/security_ops.rs"
- "src/tools/verifiable_intent.rs"
"tool:cloud":
- changed-files:
- any-glob-to-any-file:
- "src/tools/cloud_ops.rs"
- "src/tools/cloud_patterns.rs"
"tunnel":
- changed-files:
- any-glob-to-any-file:

View File

@ -7,7 +7,7 @@ on:
branches: [master]
concurrency:
group: ci-${{ github.event.pull_request.number || github.sha }}
group: ci-${{ github.event.pull_request.number || 'push-master' }}
cancel-in-progress: true
permissions:

19
.github/workflows/pr-path-labeler.yml vendored Normal file
View File

@ -0,0 +1,19 @@
name: PR Path Labeler
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
label:
name: Apply path labels
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/labeler@8558fd74291d67161a8a78ce36a881fa63b766a9 # v5
with:
sync-labels: true

View File

@ -1,6 +1,22 @@
name: Pub Homebrew Core
on:
workflow_call:
inputs:
release_tag:
description: "Existing release tag to publish (vX.Y.Z)"
required: true
type: string
dry_run:
description: "Patch formula only (no push/PR)"
required: false
default: false
type: boolean
secrets:
HOMEBREW_UPSTREAM_PR_TOKEN:
required: false
HOMEBREW_CORE_BOT_TOKEN:
required: false
workflow_dispatch:
inputs:
release_tag:

View File

@ -41,6 +41,14 @@ jobs:
echo "Current version: ${current}"
echo "Previous version: ${previous}"
# Skip if stable release workflow will handle this version
# (indicated by an existing or imminent stable tag)
if git ls-remote --exit-code --tags origin "refs/tags/v${current}" >/dev/null 2>&1; then
echo "Stable tag v${current} exists — stable release workflow handles crates.io"
echo "changed=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if [[ "$current" != "$previous" && -n "$current" ]]; then
echo "changed=true" >> "$GITHUB_OUTPUT"
echo "version=${current}" >> "$GITHUB_OUTPUT"

View File

@ -26,22 +26,43 @@ jobs:
outputs:
version: ${{ steps.ver.outputs.version }}
tag: ${{ steps.ver.outputs.tag }}
skip: ${{ steps.ver.outputs.skip }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 2
- name: Compute beta version
id: ver
shell: bash
run: |
set -euo pipefail
base_version=$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -1)
# Skip beta if this is a version bump commit (stable release handles it)
commit_msg=$(git log -1 --pretty=format:"%s")
if [[ "$commit_msg" =~ ^chore:\ bump\ version ]]; then
echo "Version bump commit detected — skipping beta release"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Skip beta if a stable tag already exists for this version
if git ls-remote --exit-code --tags origin "refs/tags/v${base_version}" >/dev/null 2>&1; then
echo "Stable tag v${base_version} exists — skipping beta release"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
beta_tag="v${base_version}-beta.${GITHUB_RUN_NUMBER}"
echo "version=${base_version}" >> "$GITHUB_OUTPUT"
echo "tag=${beta_tag}" >> "$GITHUB_OUTPUT"
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "Beta release: ${beta_tag}"
release-notes:
name: Generate Release Notes
if: github.repository == 'zeroclaw-labs/zeroclaw'
needs: [version]
if: github.repository == 'zeroclaw-labs/zeroclaw' && needs.version.outputs.skip != 'true'
runs-on: ubuntu-latest
outputs:
notes: ${{ steps.notes.outputs.body }}
@ -132,7 +153,8 @@ jobs:
web:
name: Build Web Dashboard
if: github.repository == 'zeroclaw-labs/zeroclaw'
needs: [version]
if: github.repository == 'zeroclaw-labs/zeroclaw' && needs.version.outputs.skip != 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
@ -244,9 +266,65 @@ jobs:
path: zeroclaw-${{ matrix.target }}.${{ matrix.ext }}
retention-days: 7
build-desktop:
name: Build Desktop App (macOS Universal)
needs: [version]
if: needs.version.outputs.skip != 'true'
runs-on: macos-14
timeout-minutes: 40
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
with:
toolchain: 1.92.0
targets: aarch64-apple-darwin,x86_64-apple-darwin
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
prefix-key: macos-tauri
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Tauri CLI
run: cargo install tauri-cli --locked
- name: Sync Tauri version with Cargo.toml
shell: bash
run: |
VERSION=$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -1)
cd apps/tauri
if command -v jq >/dev/null 2>&1; then
jq --arg v "$VERSION" '.version = $v' tauri.conf.json > tmp.json && mv tmp.json tauri.conf.json
else
sed -i '' "s/\"version\": \"[^\"]*\"/\"version\": \"$VERSION\"/" tauri.conf.json
fi
echo "Tauri version set to: $VERSION"
- name: Build Tauri app (universal binary)
working-directory: apps/tauri
run: cargo tauri build --target universal-apple-darwin
- name: Prepare desktop release assets
run: |
mkdir -p desktop-assets
find target -name '*.dmg' -exec cp {} desktop-assets/ZeroClaw.dmg \; 2>/dev/null || true
find target -name '*.app.tar.gz' -exec cp {} desktop-assets/ZeroClaw-macos.app.tar.gz \; 2>/dev/null || true
find target -name '*.app.tar.gz.sig' -exec cp {} desktop-assets/ZeroClaw-macos.app.tar.gz.sig \; 2>/dev/null || true
echo "--- Desktop assets ---"
ls -lh desktop-assets/
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: desktop-macos
path: desktop-assets/*
retention-days: 7
publish:
name: Publish Beta Release
needs: [version, release-notes, build]
needs: [version, release-notes, build, build-desktop]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@ -256,16 +334,21 @@ jobs:
pattern: zeroclaw-*
path: artifacts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: desktop-macos
path: artifacts/desktop-macos
- name: Generate checksums
run: |
cd artifacts
find . -type f \( -name '*.tar.gz' -o -name '*.zip' \) -exec sha256sum {} + | sed 's| \./[^/]*/| |' > SHA256SUMS
find . -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.dmg' \) -exec sha256sum {} + | sed 's| \./[^/]*/| |' > SHA256SUMS
cat SHA256SUMS
- name: Collect release assets
run: |
mkdir -p release-assets
find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name 'SHA256SUMS' \) -exec cp {} release-assets/ \;
find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.dmg' -o -name 'SHA256SUMS' \) -exec cp {} release-assets/ \;
cp install.sh release-assets/
echo "--- Assets ---"
ls -lh release-assets/

View File

@ -1,6 +1,9 @@
name: Release Stable
on:
push:
tags:
- "v[0-9]+.[0-9]+.[0-9]+" # stable tags only (no -beta suffix)
workflow_dispatch:
inputs:
version:
@ -33,11 +36,22 @@ jobs:
- name: Validate semver and Cargo.toml match
id: check
shell: bash
env:
INPUT_VERSION: ${{ inputs.version || '' }}
REF_NAME: ${{ github.ref_name }}
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
input_version="${{ inputs.version }}"
cargo_version=$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -1)
# Resolve version from tag push or manual input
if [[ "$EVENT_NAME" == "push" ]]; then
# Tag push: extract version from tag name (v0.5.9 -> 0.5.9)
input_version="${REF_NAME#v}"
else
input_version="$INPUT_VERSION"
fi
if [[ ! "$input_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Version must be semver (X.Y.Z). Got: ${input_version}"
exit 1
@ -49,9 +63,13 @@ jobs:
fi
tag="v${input_version}"
if git ls-remote --exit-code --tags origin "refs/tags/${tag}" >/dev/null 2>&1; then
echo "::error::Tag ${tag} already exists."
exit 1
# Only check tag existence for manual dispatch (tag push means it already exists)
if [[ "$EVENT_NAME" != "push" ]]; then
if git ls-remote --exit-code --tags origin "refs/tags/${tag}" >/dev/null 2>&1; then
echo "::error::Tag ${tag} already exists."
exit 1
fi
fi
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
@ -255,9 +273,64 @@ jobs:
path: zeroclaw-${{ matrix.target }}.${{ matrix.ext }}
retention-days: 14
build-desktop:
name: Build Desktop App (macOS Universal)
needs: [validate]
runs-on: macos-14
timeout-minutes: 40
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
with:
toolchain: 1.92.0
targets: aarch64-apple-darwin,x86_64-apple-darwin
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
prefix-key: macos-tauri
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Tauri CLI
run: cargo install tauri-cli --locked
- name: Sync Tauri version with Cargo.toml
shell: bash
run: |
VERSION=$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -1)
cd apps/tauri
if command -v jq >/dev/null 2>&1; then
jq --arg v "$VERSION" '.version = $v' tauri.conf.json > tmp.json && mv tmp.json tauri.conf.json
else
sed -i '' "s/\"version\": \"[^\"]*\"/\"version\": \"$VERSION\"/" tauri.conf.json
fi
echo "Tauri version set to: $VERSION"
- name: Build Tauri app (universal binary)
working-directory: apps/tauri
run: cargo tauri build --target universal-apple-darwin
- name: Prepare desktop release assets
run: |
mkdir -p desktop-assets
find target -name '*.dmg' -exec cp {} desktop-assets/ZeroClaw.dmg \; 2>/dev/null || true
find target -name '*.app.tar.gz' -exec cp {} desktop-assets/ZeroClaw-macos.app.tar.gz \; 2>/dev/null || true
find target -name '*.app.tar.gz.sig' -exec cp {} desktop-assets/ZeroClaw-macos.app.tar.gz.sig \; 2>/dev/null || true
echo "--- Desktop assets ---"
ls -lh desktop-assets/
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: desktop-macos
path: desktop-assets/*
retention-days: 14
publish:
name: Publish Stable Release
needs: [validate, release-notes, build]
needs: [validate, release-notes, build, build-desktop]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@ -267,16 +340,21 @@ jobs:
pattern: zeroclaw-*
path: artifacts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: desktop-macos
path: artifacts/desktop-macos
- name: Generate checksums
run: |
cd artifacts
find . -type f \( -name '*.tar.gz' -o -name '*.zip' \) -exec sha256sum {} + | sed 's| \./[^/]*/| |' > SHA256SUMS
find . -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.dmg' \) -exec sha256sum {} + | sed 's| \./[^/]*/| |' > SHA256SUMS
cat SHA256SUMS
- name: Collect release assets
run: |
mkdir -p release-assets
find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name 'SHA256SUMS' \) -exec cp {} release-assets/ \;
find artifacts -type f \( -name '*.tar.gz' -o -name '*.zip' -o -name '*.dmg' -o -name 'SHA256SUMS' \) -exec cp {} release-assets/ \;
cp install.sh release-assets/
echo "--- Assets ---"
ls -lh release-assets/
@ -286,6 +364,14 @@ jobs:
NOTES: ${{ needs.release-notes.outputs.notes }}
run: printf '%s\n' "$NOTES" > release-notes.md
- name: Create tag if manual dispatch
if: github.event_name == 'workflow_dispatch'
env:
TAG: ${{ needs.validate.outputs.tag }}
run: |
git tag -a "$TAG" -m "zeroclaw $TAG"
git push origin "$TAG"
- name: Create GitHub Release
env:
GH_TOKEN: ${{ secrets.RELEASE_TOKEN }}
@ -461,6 +547,16 @@ jobs:
dry_run: false
secrets: inherit
homebrew:
name: Update Homebrew Core
needs: [validate, publish]
if: ${{ !cancelled() && needs.publish.result == 'success' }}
uses: ./.github/workflows/pub-homebrew-core.yml
with:
release_tag: ${{ needs.validate.outputs.tag }}
dry_run: false
secrets: inherit
# ── Post-publish: tweet after release + website are live ──────────────
# Docker push can be slow; don't let it block the tweet.
tweet:

View File

@ -1 +0,0 @@
CLAUDE.md

92
AGENTS.md Normal file
View File

@ -0,0 +1,92 @@
# AGENTS.md — ZeroClaw
Cross-tool agent instructions for any AI coding assistant working on this repository.
## Commands
```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test
```
Full pre-PR validation (recommended):
```bash
./dev/ci.sh all
```
Docs-only changes: run markdown lint and link-integrity checks. If touching bootstrap scripts: `bash -n install.sh`.
## Project Snapshot
ZeroClaw is a Rust-first autonomous agent runtime optimized for performance, efficiency, stability, extensibility, sustainability, and security.
Core architecture is trait-driven and modular. Extend by implementing traits and registering in factory modules.
Key extension points:
- `src/providers/traits.rs` (`Provider`)
- `src/channels/traits.rs` (`Channel`)
- `src/tools/traits.rs` (`Tool`)
- `src/memory/traits.rs` (`Memory`)
- `src/observability/traits.rs` (`Observer`)
- `src/runtime/traits.rs` (`RuntimeAdapter`)
- `src/peripherals/traits.rs` (`Peripheral`) — hardware boards (STM32, RPi GPIO)
## Repository Map
- `src/main.rs` — CLI entrypoint and command routing
- `src/lib.rs` — module exports and shared command enums
- `src/config/` — schema + config loading/merging
- `src/agent/` — orchestration loop
- `src/gateway/` — webhook/gateway server
- `src/security/` — policy, pairing, secret store
- `src/memory/` — markdown/sqlite memory backends + embeddings/vector merge
- `src/providers/` — model providers and resilient wrapper
- `src/channels/` — Telegram/Discord/Slack/etc channels
- `src/tools/` — tool execution surface (shell, file, memory, browser)
- `src/peripherals/` — hardware peripherals (STM32, RPi GPIO)
- `src/runtime/` — runtime adapters (currently native)
- `docs/` — topic-based documentation (setup-guides, reference, ops, security, hardware, contributing, maintainers)
- `.github/` — CI, templates, automation workflows
## Risk Tiers
- **Low risk**: docs/chore/tests-only changes
- **Medium risk**: most `src/**` behavior changes without boundary/security impact
- **High risk**: `src/security/**`, `src/runtime/**`, `src/gateway/**`, `src/tools/**`, `.github/workflows/**`, access-control boundaries
When uncertain, classify as higher risk.
## Workflow
1. **Read before write** — inspect existing module, factory wiring, and adjacent tests before editing.
2. **One concern per PR** — avoid mixed feature+refactor+infra patches.
3. **Implement minimal patch** — no speculative abstractions, no config keys without a concrete use case.
4. **Validate by risk tier** — docs-only: lightweight checks. Code changes: full relevant checks.
5. **Document impact** — update PR notes for behavior, risk, side effects, and rollback.
6. **Queue hygiene** — stacked PR: declare `Depends on #...`. Replacing old PR: declare `Supersedes #...`.
Branch/commit/PR rules:
- Work from a non-`master` branch. Open a PR to `master`; do not push directly.
- Use conventional commit titles. Prefer small PRs (`size: XS/S/M`).
- Follow `.github/pull_request_template.md` fully.
- Never commit secrets, personal data, or real identity information (see `@docs/contributing/pr-discipline.md`).
## Anti-Patterns
- Do not add heavy dependencies for minor convenience.
- Do not silently weaken security policy or access constraints.
- Do not add speculative config/feature flags "just in case".
- Do not mix massive formatting-only changes with functional changes.
- Do not modify unrelated modules "while here".
- Do not bypass failing checks without explicit explanation.
- Do not hide behavior-changing side effects in refactor commits.
- Do not include personal identity or sensitive information in test data, examples, docs, or commits.
## Linked References
- `@docs/contributing/change-playbooks.md` — adding providers, channels, tools, peripherals; security/gateway changes; architecture boundaries
- `@docs/contributing/pr-discipline.md` — privacy rules, superseded-PR attribution/templates, handoff template
- `@docs/contributing/docs-contract.md` — docs system contract, i18n rules, locale parity

View File

@ -1,90 +1,16 @@
# CLAUDE.md — ZeroClaw
# CLAUDE.md — ZeroClaw (Claude Code)
## Commands
> **Shared instructions live in [`AGENTS.md`](./AGENTS.md).**
> This file contains only Claude Code-specific directives.
```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test
```
## Claude Code Settings
Full pre-PR validation (recommended):
Claude Code should read and follow all instructions in `AGENTS.md` at the repository root for project conventions, commands, risk tiers, workflow rules, and anti-patterns.
```bash
./dev/ci.sh all
```
## Hooks
Docs-only changes: run markdown lint and link-integrity checks. If touching bootstrap scripts: `bash -n install.sh`.
_No custom hooks defined yet._
## Project Snapshot
## Slash Commands
ZeroClaw is a Rust-first autonomous agent runtime optimized for performance, efficiency, stability, extensibility, sustainability, and security.
Core architecture is trait-driven and modular. Extend by implementing traits and registering in factory modules.
Key extension points:
- `src/providers/traits.rs` (`Provider`)
- `src/channels/traits.rs` (`Channel`)
- `src/tools/traits.rs` (`Tool`)
- `src/memory/traits.rs` (`Memory`)
- `src/observability/traits.rs` (`Observer`)
- `src/runtime/traits.rs` (`RuntimeAdapter`)
- `src/peripherals/traits.rs` (`Peripheral`) — hardware boards (STM32, RPi GPIO)
## Repository Map
- `src/main.rs` — CLI entrypoint and command routing
- `src/lib.rs` — module exports and shared command enums
- `src/config/` — schema + config loading/merging
- `src/agent/` — orchestration loop
- `src/gateway/` — webhook/gateway server
- `src/security/` — policy, pairing, secret store
- `src/memory/` — markdown/sqlite memory backends + embeddings/vector merge
- `src/providers/` — model providers and resilient wrapper
- `src/channels/` — Telegram/Discord/Slack/etc channels
- `src/tools/` — tool execution surface (shell, file, memory, browser)
- `src/peripherals/` — hardware peripherals (STM32, RPi GPIO)
- `src/runtime/` — runtime adapters (currently native)
- `docs/` — topic-based documentation (setup-guides, reference, ops, security, hardware, contributing, maintainers)
- `.github/` — CI, templates, automation workflows
## Risk Tiers
- **Low risk**: docs/chore/tests-only changes
- **Medium risk**: most `src/**` behavior changes without boundary/security impact
- **High risk**: `src/security/**`, `src/runtime/**`, `src/gateway/**`, `src/tools/**`, `.github/workflows/**`, access-control boundaries
When uncertain, classify as higher risk.
## Workflow
1. **Read before write** — inspect existing module, factory wiring, and adjacent tests before editing.
2. **One concern per PR** — avoid mixed feature+refactor+infra patches.
3. **Implement minimal patch** — no speculative abstractions, no config keys without a concrete use case.
4. **Validate by risk tier** — docs-only: lightweight checks. Code changes: full relevant checks.
5. **Document impact** — update PR notes for behavior, risk, side effects, and rollback.
6. **Queue hygiene** — stacked PR: declare `Depends on #...`. Replacing old PR: declare `Supersedes #...`.
Branch/commit/PR rules:
- Work from a non-`master` branch. Open a PR to `master`; do not push directly.
- Use conventional commit titles. Prefer small PRs (`size: XS/S/M`).
- Follow `.github/pull_request_template.md` fully.
- Never commit secrets, personal data, or real identity information (see `@docs/contributing/pr-discipline.md`).
## Anti-Patterns
- Do not add heavy dependencies for minor convenience.
- Do not silently weaken security policy or access constraints.
- Do not add speculative config/feature flags "just in case".
- Do not mix massive formatting-only changes with functional changes.
- Do not modify unrelated modules "while here".
- Do not bypass failing checks without explicit explanation.
- Do not hide behavior-changing side effects in refactor commits.
- Do not include personal identity or sensitive information in test data, examples, docs, or commits.
## Linked References
- `@docs/contributing/change-playbooks.md` — adding providers, channels, tools, peripherals; security/gateway changes; architecture boundaries
- `@docs/contributing/pr-discipline.md` — privacy rules, superseded-PR attribution/templates, handoff template
- `@docs/contributing/docs-contract.md` — docs system contract, i18n rules, locale parity
_No custom slash commands defined yet._

3233
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,10 +1,10 @@
[workspace]
members = [".", "crates/robot-kit", "crates/aardvark-sys"]
members = [".", "crates/robot-kit", "crates/aardvark-sys", "apps/tauri"]
resolver = "2"
[package]
name = "zeroclawlabs"
version = "0.5.6"
version = "0.6.1"
edition = "2021"
authors = ["theonlyhennygod"]
license = "MIT OR Apache-2.0"
@ -150,6 +150,7 @@ which = "8.0"
# WebSocket client channels (Discord/Lark/DingTalk/Nostr)
tokio-tungstenite = { version = "0.29", features = ["rustls-tls-webpki-roots"] }
tokio-socks = "0.5"
futures-util = { version = "0.3", default-features = false, features = ["sink"] }
nostr-sdk = { version = "0.44", default-features = false, features = ["nip04", "nip59"], optional = true }
regex = "1.10"
@ -224,15 +225,13 @@ landlock = { version = "0.4", optional = true }
libc = "0.2"
[features]
default = ["observability-prometheus", "channel-nostr", "skill-creation"]
default = ["observability-prometheus", "channel-nostr", "channel-lark", "skill-creation"]
channel-nostr = ["dep:nostr-sdk"]
hardware = ["nusb", "tokio-serial"]
channel-matrix = ["dep:matrix-sdk"]
channel-lark = ["dep:prost"]
channel-feishu = ["channel-lark"] # Alias for Feishu users (Lark and Feishu are the same platform)
memory-postgres = ["dep:postgres"]
# memory-mem0 = Mem0 (OpenMemory) memory backend via REST API
memory-mem0 = []
observability-prometheus = ["dep:prometheus"]
observability-otel = ["dep:opentelemetry", "dep:opentelemetry_sdk", "dep:opentelemetry-otlp"]
peripheral-rpi = ["rppal"]
@ -267,7 +266,6 @@ ci-all = [
"channel-matrix",
"channel-lark",
"memory-postgres",
"memory-mem0",
"observability-prometheus",
"observability-otel",
"peripheral-rpi",

View File

@ -12,7 +12,7 @@ RUN npm run build
FROM rust:1.94-slim@sha256:da9dab7a6b8dd428e71718402e97207bb3e54167d37b5708616050b1e8f60ed6 AS builder
WORKDIR /app
ARG ZEROCLAW_CARGO_FEATURES="memory-postgres"
ARG ZEROCLAW_CARGO_FEATURES="memory-postgres,channel-lark"
# Install build dependencies
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
@ -79,6 +79,10 @@ RUN mkdir -p /zeroclaw-data/.zeroclaw /zeroclaw-data/workspace && \
'port = 42617' \
'host = "[::]"' \
'allow_public_bind = true' \
'' \
'[autonomy]' \
'level = "supervised"' \
'auto_approve = ["file_read", "file_write", "file_edit", "memory_recall", "memory_store", "web_search_tool", "web_fetch", "calculator", "glob_search", "content_search", "image_info", "weather", "git_operations"]' \
> /zeroclaw-data/.zeroclaw/config.toml && \
chown -R 65534:65534 /zeroclaw-data

View File

@ -27,7 +27,7 @@ RUN npm run build
FROM rust:1.94-bookworm AS builder
WORKDIR /app
ARG ZEROCLAW_CARGO_FEATURES="memory-postgres"
ARG ZEROCLAW_CARGO_FEATURES="memory-postgres,channel-lark"
# Install build dependencies
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
@ -89,6 +89,10 @@ RUN mkdir -p /zeroclaw-data/.zeroclaw /zeroclaw-data/workspace && \
'port = 42617' \
'host = "[::]"' \
'allow_public_bind = true' \
'' \
'[autonomy]' \
'level = "supervised"' \
'auto_approve = ["file_read", "file_write", "file_edit", "memory_recall", "memory_store", "web_search_tool", "web_fetch", "calculator", "glob_search", "content_search", "image_info", "weather", "git_operations"]' \
> /zeroclaw-data/.zeroclaw/config.toml && \
chown -R 65534:65534 /zeroclaw-data

29
apps/tauri/Cargo.toml Normal file
View File

@ -0,0 +1,29 @@
[package]
name = "zeroclaw-desktop"
version = "0.1.0"
edition = "2021"
description = "ZeroClaw Desktop — Tauri-powered system tray app"
publish = false
[build-dependencies]
tauri-build = { version = "2.0", features = [] }
[dependencies]
tauri = { version = "2.0", features = ["tray-icon", "image-png"] }
tauri-plugin-shell = "2.0"
tauri-plugin-store = "2.0"
tauri-plugin-single-instance = "2.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
tokio = { version = "1.50", features = ["rt-multi-thread", "macros", "sync", "time"] }
anyhow = "1.0"
[target.'cfg(target_os = "macos")'.dependencies]
objc2 = "0.6"
objc2-app-kit = { version = "0.3", features = ["NSApplication", "NSImage", "NSRunningApplication"] }
objc2-foundation = { version = "0.3", features = ["NSData"] }
[features]
default = ["custom-protocol"]
custom-protocol = ["tauri/custom-protocol"]

3
apps/tauri/build.rs Normal file
View File

@ -0,0 +1,3 @@
fn main() {
tauri_build::build();
}

View File

@ -0,0 +1,14 @@
{
"$schema": "../gen/schemas/desktop-schema.json",
"identifier": "default",
"description": "Default capability set for ZeroClaw Desktop",
"windows": ["main"],
"permissions": [
"core:default",
"shell:allow-open",
"store:allow-get",
"store:allow-set",
"store:allow-save",
"store:allow-load"
]
}

View File

@ -0,0 +1,14 @@
{
"identifier": "desktop",
"description": "Desktop-specific permissions for ZeroClaw",
"windows": ["main"],
"permissions": [
"core:default",
"shell:allow-open",
"shell:allow-execute",
"store:allow-get",
"store:allow-set",
"store:allow-save",
"store:allow-load"
]
}

View File

@ -0,0 +1,8 @@
{
"identifier": "mobile",
"description": "Mobile-specific permissions for ZeroClaw",
"windows": ["main"],
"permissions": [
"core:default"
]
}

View File

View File

View File

Binary file not shown.

After

Width:  |  Height:  |  Size: 1002 B

BIN
apps/tauri/icons/32x32.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 243 B

BIN
apps/tauri/icons/icon.icns Normal file

Binary file not shown.

BIN
apps/tauri/icons/icon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 243 B

View File

@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<rect width="128" height="128" rx="16" fill="#DC322F"/>
<text x="64" y="80" font-size="64" font-family="monospace" font-weight="bold" fill="white" text-anchor="middle">Z</text>
</svg>

After

Width:  |  Height:  |  Size: 251 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 208 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 168 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 201 B

View File

@ -0,0 +1,17 @@
use crate::gateway_client::GatewayClient;
use crate::state::SharedState;
use tauri::State;
#[tauri::command]
pub async fn send_message(
state: State<'_, SharedState>,
message: String,
) -> Result<serde_json::Value, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client
.send_webhook_message(&message)
.await
.map_err(|e| e.to_string())
}

View File

@ -0,0 +1,11 @@
use crate::gateway_client::GatewayClient;
use crate::state::SharedState;
use tauri::State;
#[tauri::command]
pub async fn list_channels(state: State<'_, SharedState>) -> Result<serde_json::Value, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client.get_status().await.map_err(|e| e.to_string())
}

View File

@ -0,0 +1,19 @@
use crate::gateway_client::GatewayClient;
use crate::state::SharedState;
use tauri::State;
#[tauri::command]
pub async fn get_status(state: State<'_, SharedState>) -> Result<serde_json::Value, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client.get_status().await.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_health(state: State<'_, SharedState>) -> Result<bool, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client.get_health().await.map_err(|e| e.to_string())
}

View File

@ -0,0 +1,4 @@
pub mod agent;
pub mod channels;
pub mod gateway;
pub mod pairing;

View File

@ -0,0 +1,19 @@
use crate::gateway_client::GatewayClient;
use crate::state::SharedState;
use tauri::State;
#[tauri::command]
pub async fn initiate_pairing(state: State<'_, SharedState>) -> Result<serde_json::Value, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client.initiate_pairing().await.map_err(|e| e.to_string())
}
#[tauri::command]
pub async fn get_devices(state: State<'_, SharedState>) -> Result<serde_json::Value, String> {
let s = state.read().await;
let client = GatewayClient::new(&s.gateway_url, s.token.as_deref());
drop(s);
client.get_devices().await.map_err(|e| e.to_string())
}

View File

@ -0,0 +1,213 @@
//! HTTP client for communicating with the ZeroClaw gateway.
use anyhow::{Context, Result};
pub struct GatewayClient {
pub(crate) base_url: String,
pub(crate) token: Option<String>,
client: reqwest::Client,
}
impl GatewayClient {
pub fn new(base_url: &str, token: Option<&str>) -> Self {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self {
base_url: base_url.to_string(),
token: token.map(String::from),
client,
}
}
pub(crate) fn auth_header(&self) -> Option<String> {
self.token.as_ref().map(|t| format!("Bearer {t}"))
}
pub async fn get_status(&self) -> Result<serde_json::Value> {
let mut req = self.client.get(format!("{}/api/status", self.base_url));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req.send().await.context("status request failed")?;
Ok(resp.json().await?)
}
pub async fn get_health(&self) -> Result<bool> {
match self
.client
.get(format!("{}/health", self.base_url))
.send()
.await
{
Ok(resp) => Ok(resp.status().is_success()),
Err(_) => Ok(false),
}
}
pub async fn get_devices(&self) -> Result<serde_json::Value> {
let mut req = self.client.get(format!("{}/api/devices", self.base_url));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req.send().await.context("devices request failed")?;
Ok(resp.json().await?)
}
pub async fn initiate_pairing(&self) -> Result<serde_json::Value> {
let mut req = self
.client
.post(format!("{}/api/pairing/initiate", self.base_url));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req.send().await.context("pairing request failed")?;
Ok(resp.json().await?)
}
/// Check whether the gateway requires pairing.
pub async fn requires_pairing(&self) -> Result<bool> {
let resp = self
.client
.get(format!("{}/health", self.base_url))
.send()
.await
.context("health request failed")?;
let body: serde_json::Value = resp.json().await?;
Ok(body["require_pairing"].as_bool().unwrap_or(false))
}
/// Request a new pairing code from the gateway (localhost-only admin endpoint).
pub async fn request_new_paircode(&self) -> Result<String> {
let resp = self
.client
.post(format!("{}/admin/paircode/new", self.base_url))
.send()
.await
.context("paircode request failed")?;
let body: serde_json::Value = resp.json().await?;
body["pairing_code"]
.as_str()
.map(String::from)
.context("no pairing_code in response")
}
/// Exchange a pairing code for a bearer token.
pub async fn pair_with_code(&self, code: &str) -> Result<String> {
let resp = self
.client
.post(format!("{}/pair", self.base_url))
.header("X-Pairing-Code", code)
.send()
.await
.context("pair request failed")?;
if !resp.status().is_success() {
anyhow::bail!("pair request returned {}", resp.status());
}
let body: serde_json::Value = resp.json().await?;
body["token"]
.as_str()
.map(String::from)
.context("no token in pair response")
}
/// Validate an existing token by calling a protected endpoint.
pub async fn validate_token(&self) -> Result<bool> {
let mut req = self.client.get(format!("{}/api/status", self.base_url));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
match req.send().await {
Ok(resp) => Ok(resp.status().is_success()),
Err(_) => Ok(false),
}
}
/// Auto-pair with the gateway: request a new code and exchange it for a token.
pub async fn auto_pair(&self) -> Result<String> {
let code = self.request_new_paircode().await?;
self.pair_with_code(&code).await
}
pub async fn send_webhook_message(&self, message: &str) -> Result<serde_json::Value> {
let mut req = self
.client
.post(format!("{}/webhook", self.base_url))
.json(&serde_json::json!({ "message": message }));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req.send().await.context("webhook request failed")?;
Ok(resp.json().await?)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn client_creation_no_token() {
let client = GatewayClient::new("http://127.0.0.1:42617", None);
assert_eq!(client.base_url, "http://127.0.0.1:42617");
assert!(client.token.is_none());
assert!(client.auth_header().is_none());
}
#[test]
fn client_creation_with_token() {
let client = GatewayClient::new("http://localhost:8080", Some("test-token"));
assert_eq!(client.base_url, "http://localhost:8080");
assert_eq!(client.token.as_deref(), Some("test-token"));
assert_eq!(client.auth_header().unwrap(), "Bearer test-token");
}
#[test]
fn client_custom_url() {
let client = GatewayClient::new("https://zeroclaw.example.com:9999", None);
assert_eq!(client.base_url, "https://zeroclaw.example.com:9999");
}
#[test]
fn auth_header_format() {
let client = GatewayClient::new("http://localhost", Some("zc_abc123"));
assert_eq!(client.auth_header().unwrap(), "Bearer zc_abc123");
}
#[tokio::test]
async fn health_returns_false_for_unreachable_host() {
// Connect to a port that should not be listening.
let client = GatewayClient::new("http://127.0.0.1:1", None);
let result = client.get_health().await.unwrap();
assert!(!result, "health should be false for unreachable host");
}
#[tokio::test]
async fn status_fails_for_unreachable_host() {
let client = GatewayClient::new("http://127.0.0.1:1", None);
let result = client.get_status().await;
assert!(result.is_err(), "status should fail for unreachable host");
}
#[tokio::test]
async fn devices_fails_for_unreachable_host() {
let client = GatewayClient::new("http://127.0.0.1:1", None);
let result = client.get_devices().await;
assert!(result.is_err(), "devices should fail for unreachable host");
}
#[tokio::test]
async fn pairing_fails_for_unreachable_host() {
let client = GatewayClient::new("http://127.0.0.1:1", None);
let result = client.initiate_pairing().await;
assert!(result.is_err(), "pairing should fail for unreachable host");
}
#[tokio::test]
async fn webhook_fails_for_unreachable_host() {
let client = GatewayClient::new("http://127.0.0.1:1", None);
let result = client.send_webhook_message("hello").await;
assert!(result.is_err(), "webhook should fail for unreachable host");
}
}

40
apps/tauri/src/health.rs Normal file
View File

@ -0,0 +1,40 @@
//! Background health polling for the ZeroClaw gateway.
use crate::gateway_client::GatewayClient;
use crate::state::SharedState;
use crate::tray::icon;
use std::time::Duration;
use tauri::{AppHandle, Emitter, Runtime};
const POLL_INTERVAL: Duration = Duration::from_secs(5);
/// Spawn a background task that polls gateway health and updates state + tray.
pub fn spawn_health_poller<R: Runtime>(app: AppHandle<R>, state: SharedState) {
tauri::async_runtime::spawn(async move {
loop {
let (url, token) = {
let s = state.read().await;
(s.gateway_url.clone(), s.token.clone())
};
let client = GatewayClient::new(&url, token.as_deref());
let healthy = client.get_health().await.unwrap_or(false);
let (connected, agent_status) = {
let mut s = state.write().await;
s.connected = healthy;
(s.connected, s.agent_status)
};
// Update the tray icon and tooltip to reflect current state.
if let Some(tray) = app.tray_by_id("main") {
let _ = tray.set_icon(Some(icon::icon_for_state(connected, agent_status)));
let _ = tray.set_tooltip(Some(icon::tooltip_for_state(connected, agent_status)));
}
let _ = app.emit("zeroclaw://status-changed", healthy);
tokio::time::sleep(POLL_INTERVAL).await;
}
});
}

136
apps/tauri/src/lib.rs Normal file
View File

@ -0,0 +1,136 @@
//! ZeroClaw Desktop — Tauri application library.
pub mod commands;
pub mod gateway_client;
pub mod health;
pub mod state;
pub mod tray;
use gateway_client::GatewayClient;
use state::shared_state;
use tauri::{Manager, RunEvent};
/// Attempt to auto-pair with the gateway so the WebView has a valid token
/// before the React frontend mounts. Runs on localhost so the admin endpoints
/// are accessible without auth.
async fn auto_pair(state: &state::SharedState) -> Option<String> {
let url = {
let s = state.read().await;
s.gateway_url.clone()
};
let client = GatewayClient::new(&url, None);
// Check if gateway is reachable and requires pairing.
if !client.requires_pairing().await.unwrap_or(false) {
return None; // Pairing disabled — no token needed.
}
// Check if we already have a valid token in state.
{
let s = state.read().await;
if let Some(ref token) = s.token {
let authed = GatewayClient::new(&url, Some(token));
if authed.validate_token().await.unwrap_or(false) {
return Some(token.clone()); // Existing token is valid.
}
}
}
// No valid token — auto-pair by requesting a new code and exchanging it.
let client = GatewayClient::new(&url, None);
match client.auto_pair().await {
Ok(token) => {
let mut s = state.write().await;
s.token = Some(token.clone());
Some(token)
}
Err(_) => None, // Gateway may not be ready yet; health poller will retry.
}
}
/// Inject a bearer token into the WebView's localStorage so the React app
/// skips the pairing dialog. Uses Tauri's WebviewWindow scripting API.
fn inject_token_into_webview<R: tauri::Runtime>(window: &tauri::WebviewWindow<R>, token: &str) {
let escaped = token.replace('\\', "\\\\").replace('\'', "\\'");
let script = format!("localStorage.setItem('zeroclaw_token', '{escaped}')");
// WebviewWindow scripting is the standard Tauri API for running JS in the WebView.
let _ = window.eval(&script);
}
/// Set the macOS dock icon programmatically so it shows even in dev builds
/// (which don't have a proper .app bundle).
#[cfg(target_os = "macos")]
fn set_dock_icon() {
use objc2::{AnyThread, MainThreadMarker};
use objc2_app_kit::NSApplication;
use objc2_app_kit::NSImage;
use objc2_foundation::NSData;
let icon_bytes = include_bytes!("../icons/128x128.png");
// Safety: setup() runs on the main thread in Tauri.
let mtm = unsafe { MainThreadMarker::new_unchecked() };
let data = NSData::with_bytes(icon_bytes);
if let Some(image) = NSImage::initWithData(NSImage::alloc(), &data) {
let app = NSApplication::sharedApplication(mtm);
unsafe { app.setApplicationIconImage(Some(&image)) };
}
}
/// Configure and run the Tauri application.
pub fn run() {
let shared = shared_state();
tauri::Builder::default()
.plugin(tauri_plugin_shell::init())
.plugin(tauri_plugin_store::Builder::default().build())
.plugin(tauri_plugin_single_instance::init(|app, _args, _cwd| {
// When a second instance launches, focus the existing window.
if let Some(window) = app.get_webview_window("main") {
let _ = window.show();
let _ = window.set_focus();
}
}))
.manage(shared.clone())
.invoke_handler(tauri::generate_handler![
commands::gateway::get_status,
commands::gateway::get_health,
commands::channels::list_channels,
commands::pairing::initiate_pairing,
commands::pairing::get_devices,
commands::agent::send_message,
])
.setup(move |app| {
// Set macOS dock icon (needed for dev builds without .app bundle).
#[cfg(target_os = "macos")]
set_dock_icon();
// Set up the system tray.
let _ = tray::setup_tray(app);
// Auto-pair with gateway and inject token into the WebView.
let app_handle = app.handle().clone();
let pair_state = shared.clone();
tauri::async_runtime::spawn(async move {
if let Some(token) = auto_pair(&pair_state).await {
if let Some(window) = app_handle.get_webview_window("main") {
inject_token_into_webview(&window, &token);
}
}
});
// Start background health polling.
health::spawn_health_poller(app.handle().clone(), shared.clone());
Ok(())
})
.build(tauri::generate_context!())
.expect("error while building tauri application")
.run(|_app, event| {
// Keep the app running in the background when all windows are closed.
// This is the standard pattern for menu bar / tray apps.
if let RunEvent::ExitRequested { api, .. } = event {
api.prevent_exit();
}
});
}

8
apps/tauri/src/main.rs Normal file
View File

@ -0,0 +1,8 @@
//! ZeroClaw Desktop — main entry point.
//!
//! Prevents an additional console window on Windows in release.
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]
fn main() {
zeroclaw_desktop::run();
}

6
apps/tauri/src/mobile.rs Normal file
View File

@ -0,0 +1,6 @@
//! Mobile entry point for ZeroClaw Desktop (iOS/Android).
#[tauri::mobile_entry_point]
fn main() {
zeroclaw_desktop::run();
}

99
apps/tauri/src/state.rs Normal file
View File

@ -0,0 +1,99 @@
//! Shared application state for Tauri.
use std::sync::Arc;
use tokio::sync::RwLock;
/// Agent status as reported by the gateway.
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
#[serde(rename_all = "snake_case")]
pub enum AgentStatus {
Idle,
Working,
Error,
}
/// Shared application state behind an `Arc<RwLock<_>>`.
#[derive(Debug, Clone)]
pub struct AppState {
pub gateway_url: String,
pub token: Option<String>,
pub connected: bool,
pub agent_status: AgentStatus,
}
impl Default for AppState {
fn default() -> Self {
Self {
gateway_url: "http://127.0.0.1:42617".to_string(),
token: None,
connected: false,
agent_status: AgentStatus::Idle,
}
}
}
/// Thread-safe wrapper around `AppState`.
pub type SharedState = Arc<RwLock<AppState>>;
/// Create the default shared state.
pub fn shared_state() -> SharedState {
Arc::new(RwLock::new(AppState::default()))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn default_state() {
let state = AppState::default();
assert_eq!(state.gateway_url, "http://127.0.0.1:42617");
assert!(state.token.is_none());
assert!(!state.connected);
assert_eq!(state.agent_status, AgentStatus::Idle);
}
#[test]
fn shared_state_is_cloneable() {
let s1 = shared_state();
let s2 = s1.clone();
// Both references point to the same allocation.
assert!(Arc::ptr_eq(&s1, &s2));
}
#[tokio::test]
async fn shared_state_concurrent_read_write() {
let state = shared_state();
// Write from one handle.
{
let mut s = state.write().await;
s.connected = true;
s.agent_status = AgentStatus::Working;
s.token = Some("zc_test".to_string());
}
// Read from cloned handle.
let state2 = state.clone();
let s = state2.read().await;
assert!(s.connected);
assert_eq!(s.agent_status, AgentStatus::Working);
assert_eq!(s.token.as_deref(), Some("zc_test"));
}
#[test]
fn agent_status_serialization() {
assert_eq!(
serde_json::to_string(&AgentStatus::Idle).unwrap(),
"\"idle\""
);
assert_eq!(
serde_json::to_string(&AgentStatus::Working).unwrap(),
"\"working\""
);
assert_eq!(
serde_json::to_string(&AgentStatus::Error).unwrap(),
"\"error\""
);
}
}

View File

@ -0,0 +1,25 @@
//! Tray menu event handling.
use tauri::{menu::MenuEvent, AppHandle, Manager, Runtime};
pub fn handle_menu_event<R: Runtime>(app: &AppHandle<R>, event: MenuEvent) {
match event.id().as_ref() {
"show" => show_main_window(app, None),
"chat" => show_main_window(app, Some("/agent")),
"quit" => {
app.exit(0);
}
_ => {}
}
}
fn show_main_window<R: Runtime>(app: &AppHandle<R>, navigate_to: Option<&str>) {
if let Some(window) = app.get_webview_window("main") {
let _ = window.show();
let _ = window.set_focus();
if let Some(path) = navigate_to {
let script = format!("window.location.hash = '{path}'");
let _ = window.eval(&script);
}
}
}

105
apps/tauri/src/tray/icon.rs Normal file
View File

@ -0,0 +1,105 @@
//! Tray icon management — swap icon based on connection/agent status.
use crate::state::AgentStatus;
use tauri::image::Image;
/// Embedded tray icon PNGs (22x22, RGBA).
const ICON_IDLE: &[u8] = include_bytes!("../../icons/tray-idle.png");
const ICON_WORKING: &[u8] = include_bytes!("../../icons/tray-working.png");
const ICON_ERROR: &[u8] = include_bytes!("../../icons/tray-error.png");
const ICON_DISCONNECTED: &[u8] = include_bytes!("../../icons/tray-disconnected.png");
/// Select the appropriate tray icon for the current state.
pub fn icon_for_state(connected: bool, status: AgentStatus) -> Image<'static> {
let bytes: &[u8] = if !connected {
ICON_DISCONNECTED
} else {
match status {
AgentStatus::Idle => ICON_IDLE,
AgentStatus::Working => ICON_WORKING,
AgentStatus::Error => ICON_ERROR,
}
};
Image::from_bytes(bytes).expect("embedded tray icon is a valid PNG")
}
/// Tooltip text for the current state.
pub fn tooltip_for_state(connected: bool, status: AgentStatus) -> &'static str {
if !connected {
return "ZeroClaw — Disconnected";
}
match status {
AgentStatus::Idle => "ZeroClaw — Idle",
AgentStatus::Working => "ZeroClaw — Working",
AgentStatus::Error => "ZeroClaw — Error",
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn icon_disconnected_when_not_connected() {
// Should not panic — icon bytes are valid PNGs.
let _img = icon_for_state(false, AgentStatus::Idle);
let _img = icon_for_state(false, AgentStatus::Working);
let _img = icon_for_state(false, AgentStatus::Error);
}
#[test]
fn icon_connected_variants() {
let _idle = icon_for_state(true, AgentStatus::Idle);
let _working = icon_for_state(true, AgentStatus::Working);
let _error = icon_for_state(true, AgentStatus::Error);
}
#[test]
fn tooltip_disconnected() {
assert_eq!(
tooltip_for_state(false, AgentStatus::Idle),
"ZeroClaw — Disconnected"
);
// Agent status is irrelevant when disconnected.
assert_eq!(
tooltip_for_state(false, AgentStatus::Working),
"ZeroClaw — Disconnected"
);
assert_eq!(
tooltip_for_state(false, AgentStatus::Error),
"ZeroClaw — Disconnected"
);
}
#[test]
fn tooltip_connected_variants() {
assert_eq!(
tooltip_for_state(true, AgentStatus::Idle),
"ZeroClaw — Idle"
);
assert_eq!(
tooltip_for_state(true, AgentStatus::Working),
"ZeroClaw — Working"
);
assert_eq!(
tooltip_for_state(true, AgentStatus::Error),
"ZeroClaw — Error"
);
}
#[test]
fn embedded_icons_are_valid_png() {
// Verify the PNG signature (first 8 bytes) of each embedded icon.
let png_sig: &[u8] = &[0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A];
assert!(ICON_IDLE.starts_with(png_sig), "idle icon not valid PNG");
assert!(
ICON_WORKING.starts_with(png_sig),
"working icon not valid PNG"
);
assert!(ICON_ERROR.starts_with(png_sig), "error icon not valid PNG");
assert!(
ICON_DISCONNECTED.starts_with(png_sig),
"disconnected icon not valid PNG"
);
}
}

View File

@ -0,0 +1,19 @@
//! Tray menu construction.
use tauri::{
menu::{Menu, MenuItemBuilder, PredefinedMenuItem},
App, Runtime,
};
pub fn create_tray_menu<R: Runtime>(app: &App<R>) -> Result<Menu<R>, tauri::Error> {
let show = MenuItemBuilder::with_id("show", "Show Dashboard").build(app)?;
let chat = MenuItemBuilder::with_id("chat", "Agent Chat").build(app)?;
let sep1 = PredefinedMenuItem::separator(app)?;
let status = MenuItemBuilder::with_id("status", "Status: Checking...")
.enabled(false)
.build(app)?;
let sep2 = PredefinedMenuItem::separator(app)?;
let quit = MenuItemBuilder::with_id("quit", "Quit ZeroClaw").build(app)?;
Menu::with_items(app, &[&show, &chat, &sep1, &status, &sep2, &quit])
}

View File

@ -0,0 +1,34 @@
//! System tray integration for ZeroClaw Desktop.
pub mod events;
pub mod icon;
pub mod menu;
use tauri::{
tray::{TrayIcon, TrayIconBuilder, TrayIconEvent},
App, Manager, Runtime,
};
/// Set up the system tray icon and menu.
pub fn setup_tray<R: Runtime>(app: &App<R>) -> Result<TrayIcon<R>, tauri::Error> {
let menu = menu::create_tray_menu(app)?;
TrayIconBuilder::with_id("main")
.tooltip("ZeroClaw — Disconnected")
.icon(icon::icon_for_state(false, crate::state::AgentStatus::Idle))
.menu(&menu)
.show_menu_on_left_click(false)
.on_menu_event(events::handle_menu_event)
.on_tray_icon_event(|tray, event| {
if let TrayIconEvent::Click { button, .. } = event {
if button == tauri::tray::MouseButton::Left {
let app = tray.app_handle();
if let Some(window) = app.get_webview_window("main") {
let _ = window.show();
let _ = window.set_focus();
}
}
}
})
.build(app)
}

View File

@ -0,0 +1,35 @@
{
"$schema": "https://raw.githubusercontent.com/tauri-apps/tauri/dev/crates/tauri-cli/config.schema.json",
"productName": "ZeroClaw",
"version": "0.6.1",
"identifier": "ai.zeroclawlabs.desktop",
"build": {
"devUrl": "http://127.0.0.1:42617/_app/",
"frontendDist": "http://127.0.0.1:42617/_app/"
},
"app": {
"windows": [
{
"title": "ZeroClaw",
"width": 1200,
"height": 800,
"resizable": true,
"fullscreen": false,
"visible": false
}
],
"security": {
"csp": "default-src 'self' http://127.0.0.1:* ws://127.0.0.1:*; connect-src 'self' http://127.0.0.1:* ws://127.0.0.1:*; script-src 'self' 'unsafe-inline' http://127.0.0.1:*; style-src 'self' 'unsafe-inline' http://127.0.0.1:*; img-src 'self' http://127.0.0.1:* data:"
}
},
"bundle": {
"active": true,
"targets": "all",
"icon": [
"icons/32x32.png",
"icons/128x128.png",
"icons/icon.icns",
"icons/icon.ico"
]
}
}

View File

@ -1,80 +0,0 @@
#!/bin/bash
# Start mem0 + reranker GPU container for ZeroClaw memory backend.
#
# Required env vars:
# MEM0_LLM_API_KEY or ZAI_API_KEY — API key for the LLM used in fact extraction
#
# Optional env vars (with defaults):
# MEM0_LLM_PROVIDER — mem0 LLM provider (default: "openai" i.e. OpenAI-compatible)
# MEM0_LLM_MODEL — LLM model for fact extraction (default: "glm-5-turbo")
# MEM0_LLM_BASE_URL — LLM API base URL (default: "https://api.z.ai/api/coding/paas/v4")
# MEM0_EMBEDDER_MODEL — embedding model (default: "BAAI/bge-m3")
# MEM0_EMBEDDER_DIMS — embedding dimensions (default: "1024")
# MEM0_EMBEDDER_DEVICE — "cuda", "cpu", or "auto" (default: "cuda")
# MEM0_VECTOR_COLLECTION — Qdrant collection name (default: "zeroclaw_mem0")
# RERANKER_MODEL — reranker model (default: "BAAI/bge-reranker-v2-m3")
# RERANKER_DEVICE — "cuda" or "cpu" (default: "cuda")
# MEM0_PORT — mem0 server port (default: 8765)
# RERANKER_PORT — reranker server port (default: 8678)
# CONTAINER_IMAGE — base container image (default: docker.io/kyuz0/amd-strix-halo-comfyui:latest)
# CONTAINER_NAME — container name (default: mem0-gpu)
# DATA_DIR — host path for Qdrant data (default: ~/mem0-data)
# SCRIPT_DIR — host path for server scripts (default: directory of this script)
set -e
# Resolve script directory for mounting server scripts
SCRIPT_DIR="${SCRIPT_DIR:-$(cd "$(dirname "$0")" && pwd)}"
# API key — accept either name
export MEM0_LLM_API_KEY="${MEM0_LLM_API_KEY:-${ZAI_API_KEY:?MEM0_LLM_API_KEY or ZAI_API_KEY must be set}}"
# Defaults
MEM0_LLM_MODEL="${MEM0_LLM_MODEL:-glm-5-turbo}"
MEM0_LLM_BASE_URL="${MEM0_LLM_BASE_URL:-https://api.z.ai/api/coding/paas/v4}"
MEM0_PORT="${MEM0_PORT:-8765}"
RERANKER_PORT="${RERANKER_PORT:-8678}"
CONTAINER_IMAGE="${CONTAINER_IMAGE:-docker.io/kyuz0/amd-strix-halo-comfyui:latest}"
CONTAINER_NAME="${CONTAINER_NAME:-mem0-gpu}"
DATA_DIR="${DATA_DIR:-$HOME/mem0-data}"
# Stop existing CPU services (if any)
kill -9 $(pgrep -f "mem0-server.py") 2>/dev/null || true
kill -9 $(pgrep -f "reranker-server.py") 2>/dev/null || true
# Stop existing container
podman stop "$CONTAINER_NAME" 2>/dev/null || true
podman rm "$CONTAINER_NAME" 2>/dev/null || true
podman run -d --name "$CONTAINER_NAME" \
--device /dev/dri --device /dev/kfd \
--group-add video --group-add render \
--restart unless-stopped \
-p "$MEM0_PORT:$MEM0_PORT" -p "$RERANKER_PORT:$RERANKER_PORT" \
-v "$DATA_DIR":/root/mem0-data:Z \
-v "$SCRIPT_DIR/mem0-server.py":/app/mem0-server.py:ro,Z \
-v "$SCRIPT_DIR/reranker-server.py":/app/reranker-server.py:ro,Z \
-v "$HOME/.cache/huggingface":/root/.cache/huggingface:Z \
-e MEM0_LLM_API_KEY="$MEM0_LLM_API_KEY" \
-e ZAI_API_KEY="$MEM0_LLM_API_KEY" \
-e MEM0_LLM_MODEL="$MEM0_LLM_MODEL" \
-e MEM0_LLM_BASE_URL="$MEM0_LLM_BASE_URL" \
${MEM0_LLM_PROVIDER:+-e MEM0_LLM_PROVIDER="$MEM0_LLM_PROVIDER"} \
${MEM0_EMBEDDER_MODEL:+-e MEM0_EMBEDDER_MODEL="$MEM0_EMBEDDER_MODEL"} \
${MEM0_EMBEDDER_DIMS:+-e MEM0_EMBEDDER_DIMS="$MEM0_EMBEDDER_DIMS"} \
${MEM0_EMBEDDER_DEVICE:+-e MEM0_EMBEDDER_DEVICE="$MEM0_EMBEDDER_DEVICE"} \
${MEM0_VECTOR_COLLECTION:+-e MEM0_VECTOR_COLLECTION="$MEM0_VECTOR_COLLECTION"} \
${RERANKER_MODEL:+-e RERANKER_MODEL="$RERANKER_MODEL"} \
${RERANKER_DEVICE:+-e RERANKER_DEVICE="$RERANKER_DEVICE"} \
-e RERANKER_PORT="$RERANKER_PORT" \
-e RERANKER_URL="http://127.0.0.1:$RERANKER_PORT/rerank" \
-e TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 \
-e HOME=/root \
"$CONTAINER_IMAGE" \
bash -c "pip install -q FlagEmbedding mem0ai flask httpx qdrant-client 2>&1 | tail -3; echo '=== Starting reranker (GPU) on :$RERANKER_PORT ==='; python3 /app/reranker-server.py & sleep 3; echo '=== Starting mem0 (GPU) on :$MEM0_PORT ==='; exec python3 /app/mem0-server.py"
echo "Container started, waiting for init..."
sleep 15
echo "=== Container logs ==="
podman logs "$CONTAINER_NAME" 2>&1 | tail -25
echo "=== Port check ==="
ss -tlnp | grep "$MEM0_PORT\|$RERANKER_PORT" || echo "Ports not yet ready, check: podman logs $CONTAINER_NAME"

View File

@ -1,288 +0,0 @@
"""Minimal OpenMemory-compatible REST server wrapping mem0 Python SDK."""
import asyncio
import json, os, uuid, httpx
from datetime import datetime, timezone
from fastapi import FastAPI, Query
from pydantic import BaseModel
from typing import Optional
from mem0 import Memory
app = FastAPI()
RERANKER_URL = os.environ.get("RERANKER_URL", "http://127.0.0.1:8678/rerank")
CUSTOM_EXTRACTION_PROMPT = """You are a memory extraction specialist for a Cantonese/Chinese chat assistant.
Extract ONLY important, persistent facts from the conversation. Rules:
1. Extract personal preferences, habits, relationships, names, locations
2. Extract decisions, plans, and commitments people make
3. SKIP small talk, greetings, reactions ("ok", "哈哈", "係呀")
4. SKIP temporary states ("我依家食緊飯") unless they reveal a habit
5. Keep facts in the ORIGINAL language (Cantonese/Chinese/English)
6. For each fact, note WHO it's about (use their name or identifier if known)
7. Merge/update existing facts rather than creating duplicates
Return a list of facts in JSON format: {"facts": ["fact1", "fact2", ...]}
"""
PROCEDURAL_EXTRACTION_PROMPT = """You are a procedural memory specialist for an AI assistant.
Extract HOW-TO patterns and reusable procedures from the conversation trace. Rules:
1. Identify step-by-step procedures the assistant followed to accomplish a task
2. Extract tool usage patterns: which tools were called, in what order, with what arguments
3. Capture decision points: why the assistant chose one approach over another
4. Note error-recovery patterns: what failed, how it was fixed
5. Keep the procedure generic enough to apply to similar future tasks
6. Preserve technical details (commands, file paths, API calls) that are reusable
7. SKIP greetings, small talk, and conversational filler
8. Format each procedure as: "To [goal]: [step1] -> [step2] -> ... -> [result]"
Return a list of procedures in JSON format: {"facts": ["procedure1", "procedure2", ...]}
"""
# ── Configurable via environment variables ─────────────────────────
# LLM (for fact extraction when infer=true)
MEM0_LLM_PROVIDER = os.environ.get("MEM0_LLM_PROVIDER", "openai") # "openai" (compatible), "anthropic", etc.
MEM0_LLM_MODEL = os.environ.get("MEM0_LLM_MODEL", "glm-5-turbo")
MEM0_LLM_API_KEY = os.environ.get("MEM0_LLM_API_KEY") or os.environ.get("ZAI_API_KEY", "")
MEM0_LLM_BASE_URL = os.environ.get("MEM0_LLM_BASE_URL", "https://api.z.ai/api/coding/paas/v4")
# Embedder
MEM0_EMBEDDER_PROVIDER = os.environ.get("MEM0_EMBEDDER_PROVIDER", "huggingface") # "huggingface", "openai", etc.
MEM0_EMBEDDER_MODEL = os.environ.get("MEM0_EMBEDDER_MODEL", "BAAI/bge-m3")
MEM0_EMBEDDER_DIMS = int(os.environ.get("MEM0_EMBEDDER_DIMS", "1024"))
MEM0_EMBEDDER_DEVICE = os.environ.get("MEM0_EMBEDDER_DEVICE", "cuda") # "cuda", "cpu", "auto"
# Vector store
MEM0_VECTOR_PROVIDER = os.environ.get("MEM0_VECTOR_PROVIDER", "qdrant") # "qdrant", "chroma", etc.
MEM0_VECTOR_COLLECTION = os.environ.get("MEM0_VECTOR_COLLECTION", "zeroclaw_mem0")
MEM0_VECTOR_PATH = os.environ.get("MEM0_VECTOR_PATH", os.path.expanduser("~/mem0-data/qdrant"))
config = {
"llm": {
"provider": MEM0_LLM_PROVIDER,
"config": {
"model": MEM0_LLM_MODEL,
"api_key": MEM0_LLM_API_KEY,
"openai_base_url": MEM0_LLM_BASE_URL,
},
},
"embedder": {
"provider": MEM0_EMBEDDER_PROVIDER,
"config": {
"model": MEM0_EMBEDDER_MODEL,
"embedding_dims": MEM0_EMBEDDER_DIMS,
"model_kwargs": {"device": MEM0_EMBEDDER_DEVICE},
},
},
"vector_store": {
"provider": MEM0_VECTOR_PROVIDER,
"config": {
"collection_name": MEM0_VECTOR_COLLECTION,
"embedding_model_dims": MEM0_EMBEDDER_DIMS,
"path": MEM0_VECTOR_PATH,
},
},
"custom_fact_extraction_prompt": CUSTOM_EXTRACTION_PROMPT,
}
m = Memory.from_config(config)
def rerank_results(query: str, items: list, top_k: int = 10) -> list:
"""Rerank search results using bge-reranker-v2-m3."""
if not items:
return items
documents = [item.get("memory", "") for item in items]
try:
resp = httpx.post(
RERANKER_URL,
json={"query": query, "documents": documents, "top_k": top_k},
timeout=10.0,
)
resp.raise_for_status()
ranked = resp.json().get("results", [])
return [items[r["index"]] for r in ranked]
except Exception as e:
print(f"Reranker failed, using original order: {e}")
return items
class AddMemoryRequest(BaseModel):
user_id: str
text: str
metadata: Optional[dict] = None
infer: bool = True
app: Optional[str] = None
custom_instructions: Optional[str] = None
@app.post("/api/v1/memories/")
async def add_memory(req: AddMemoryRequest):
# Use client-supplied prompt, fall back to server default, then mem0 SDK default
prompt = req.custom_instructions or CUSTOM_EXTRACTION_PROMPT
result = await asyncio.to_thread(m.add, req.text, user_id=req.user_id, metadata=req.metadata or {}, prompt=prompt)
return {"id": str(uuid.uuid4()), "status": "ok", "result": result}
class ProceduralMemoryRequest(BaseModel):
user_id: str
messages: list[dict]
metadata: Optional[dict] = None
@app.post("/api/v1/memories/procedural")
async def add_procedural_memory(req: ProceduralMemoryRequest):
"""Store a conversation trace as procedural memory.
Accepts a list of messages (role/content dicts) representing a full
conversation turn including tool calls, then uses mem0's native
procedural memory extraction to learn reusable "how to" patterns.
"""
# Build metadata with procedural type marker
meta = {"type": "procedural"}
if req.metadata:
meta.update(req.metadata)
# Use mem0's native message list support + procedural prompt
result = await asyncio.to_thread(m.add,
req.messages,
user_id=req.user_id,
metadata=meta,
prompt=PROCEDURAL_EXTRACTION_PROMPT,
)
return {"id": str(uuid.uuid4()), "status": "ok", "result": result}
def _parse_mem0_results(raw_results) -> list:
raw = raw_results.get("results", raw_results) if isinstance(raw_results, dict) else raw_results
items = []
for r in raw:
item = r if isinstance(r, dict) else {"memory": str(r)}
items.append({
"id": item.get("id", str(uuid.uuid4())),
"memory": item.get("memory", item.get("text", "")),
"created_at": item.get("created_at", datetime.now(timezone.utc).isoformat()),
"metadata_": item.get("metadata", {}),
})
return items
def _parse_iso_timestamp(value: str) -> Optional[datetime]:
"""Parse an ISO 8601 timestamp string, returning None on failure."""
try:
dt = datetime.fromisoformat(value)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt
except (ValueError, TypeError):
return None
def _item_created_at(item: dict) -> Optional[datetime]:
"""Extract created_at from an item as a timezone-aware datetime."""
raw = item.get("created_at")
if raw is None:
return None
if isinstance(raw, datetime):
if raw.tzinfo is None:
raw = raw.replace(tzinfo=timezone.utc)
return raw
return _parse_iso_timestamp(str(raw))
def _apply_post_filters(
items: list,
created_after: Optional[str],
created_before: Optional[str],
) -> list:
"""Filter items by created_after / created_before timestamps (post-query)."""
after_dt = _parse_iso_timestamp(created_after) if created_after else None
before_dt = _parse_iso_timestamp(created_before) if created_before else None
if after_dt is None and before_dt is None:
return items
filtered = []
for item in items:
ts = _item_created_at(item)
if ts is None:
# Keep items without a parseable timestamp
filtered.append(item)
continue
if after_dt and ts < after_dt:
continue
if before_dt and ts > before_dt:
continue
filtered.append(item)
return filtered
@app.get("/api/v1/memories/")
async def list_or_search_memories(
user_id: str = Query(...),
search_query: Optional[str] = Query(None),
size: int = Query(10),
rerank: bool = Query(True),
created_after: Optional[str] = Query(None),
created_before: Optional[str] = Query(None),
metadata_filter: Optional[str] = Query(None),
):
# Build mem0 SDK filters dict from metadata_filter JSON param
sdk_filters = None
if metadata_filter:
try:
sdk_filters = json.loads(metadata_filter)
except json.JSONDecodeError:
sdk_filters = None
if search_query:
# Fetch more results than needed so reranker has candidates to work with
fetch_size = min(size * 3, 50)
results = await asyncio.to_thread(m.search,
search_query,
user_id=user_id,
limit=fetch_size,
filters=sdk_filters,
)
items = _parse_mem0_results(results)
items = _apply_post_filters(items, created_after, created_before)
if rerank and items:
items = rerank_results(search_query, items, top_k=size)
else:
items = items[:size]
return {"items": items, "total": len(items)}
else:
results = await asyncio.to_thread(m.get_all,user_id=user_id, filters=sdk_filters)
items = _parse_mem0_results(results)
items = _apply_post_filters(items, created_after, created_before)
return {"items": items, "total": len(items)}
@app.delete("/api/v1/memories/{memory_id}")
async def delete_memory(memory_id: str):
try:
await asyncio.to_thread(m.delete, memory_id)
except Exception:
pass
return {"status": "ok"}
@app.get("/api/v1/memories/{memory_id}/history")
async def get_memory_history(memory_id: str):
"""Return the edit history of a specific memory."""
try:
history = await asyncio.to_thread(m.history, memory_id)
# Normalize to list of dicts
entries = []
raw = history if isinstance(history, list) else history.get("results", history) if isinstance(history, dict) else [history]
for h in raw:
entry = h if isinstance(h, dict) else {"event": str(h)}
entries.append(entry)
return {"memory_id": memory_id, "history": entries}
except Exception as e:
return {"memory_id": memory_id, "history": [], "error": str(e)}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8765)

View File

@ -1,50 +0,0 @@
from flask import Flask, request, jsonify
from FlagEmbedding import FlagReranker
import os, torch
app = Flask(__name__)
reranker = None
# ── Configurable via environment variables ─────────────────────────
RERANKER_MODEL = os.environ.get("RERANKER_MODEL", "BAAI/bge-reranker-v2-m3")
RERANKER_DEVICE = os.environ.get("RERANKER_DEVICE", "cuda" if torch.cuda.is_available() else "cpu")
RERANKER_PORT = int(os.environ.get("RERANKER_PORT", "8678"))
def get_reranker():
global reranker
if reranker is None:
reranker = FlagReranker(RERANKER_MODEL, use_fp16=True, device=RERANKER_DEVICE)
return reranker
@app.route('/rerank', methods=['POST'])
def rerank():
data = request.json
query = data.get('query', '')
documents = data.get('documents', [])
top_k = data.get('top_k', len(documents))
if not query or not documents:
return jsonify({'error': 'query and documents required'}), 400
pairs = [[query, doc] for doc in documents]
scores = get_reranker().compute_score(pairs)
if isinstance(scores, float):
scores = [scores]
results = sorted(
[{'index': i, 'document': doc, 'score': score}
for i, (doc, score) in enumerate(zip(documents, scores))],
key=lambda x: x['score'], reverse=True
)[:top_k]
return jsonify({'results': results})
@app.route('/health', methods=['GET'])
def health():
return jsonify({'status': 'ok', 'model': RERANKER_MODEL, 'device': RERANKER_DEVICE})
if __name__ == '__main__':
print(f'Loading reranker model ({RERANKER_MODEL}) on {RERANKER_DEVICE}...')
get_reranker()
print(f'Reranker server ready on :{RERANKER_PORT}')
app.run(host='0.0.0.0', port=RERANKER_PORT)

View File

@ -10,3 +10,22 @@ default_temperature = 0.7
port = 42617
host = "[::]"
allow_public_bind = true
# Cost tracking and budget enforcement configuration
# Enable to track API usage costs and enforce spending limits
[cost]
enabled = false
daily_limit_usd = 10.0
monthly_limit_usd = 100.0
warn_at_percent = 80
allow_override = false
# Per-model pricing (USD per 1M tokens)
# Uncomment and customize to override default pricing
# [cost.prices."anthropic/claude-sonnet-4-20250514"]
# input = 3.0
# output = 15.0
#
# [cost.prices."openai/gpt-4o"]
# input = 5.0
# output = 15.0

4
dist/aur/.SRCINFO vendored
View File

@ -1,6 +1,6 @@
pkgbase = zeroclaw
pkgdesc = Zero overhead. Zero compromise. 100% Rust. The fastest, smallest AI assistant.
pkgver = 0.5.6
pkgver = 0.5.9
pkgrel = 1
url = https://github.com/zeroclaw-labs/zeroclaw
arch = x86_64
@ -10,7 +10,7 @@ pkgbase = zeroclaw
makedepends = git
depends = gcc-libs
depends = openssl
source = zeroclaw-0.5.6.tar.gz::https://github.com/zeroclaw-labs/zeroclaw/archive/refs/tags/v0.5.6.tar.gz
source = zeroclaw-0.5.9.tar.gz::https://github.com/zeroclaw-labs/zeroclaw/archive/refs/tags/v0.5.9.tar.gz
sha256sums = SKIP
pkgname = zeroclaw

2
dist/aur/PKGBUILD vendored
View File

@ -1,6 +1,6 @@
# Maintainer: zeroclaw-labs <bot@zeroclaw.dev>
pkgname=zeroclaw
pkgver=0.5.6
pkgver=0.5.9
pkgrel=1
pkgdesc="Zero overhead. Zero compromise. 100% Rust. The fastest, smallest AI assistant."
arch=('x86_64')

View File

@ -1,11 +1,11 @@
{
"version": "0.5.6",
"version": "0.5.9",
"description": "Zero overhead. Zero compromise. 100% Rust. The fastest, smallest AI assistant.",
"homepage": "https://github.com/zeroclaw-labs/zeroclaw",
"license": "MIT|Apache-2.0",
"architecture": {
"64bit": {
"url": "https://github.com/zeroclaw-labs/zeroclaw/releases/download/v0.5.6/zeroclaw-x86_64-pc-windows-msvc.zip",
"url": "https://github.com/zeroclaw-labs/zeroclaw/releases/download/v0.5.9/zeroclaw-x86_64-pc-windows-msvc.zip",
"hash": "",
"bin": "zeroclaw.exe"
}

View File

@ -0,0 +1,202 @@
# ADR-004: Tool Shared State Ownership Contract
**Status:** Accepted
**Date:** 2026-03-22
**Issue:** [#4057](https://github.com/zeroclaw/zeroclaw/issues/4057)
## Context
ZeroClaw tools execute in a multi-client environment where a single daemon
process serves requests from multiple connected clients simultaneously. Several
tools already maintain long-lived shared state:
- **`DelegateParentToolsHandle`** (`src/tools/mod.rs`):
`Arc<RwLock<Vec<Arc<dyn Tool>>>>` — holds parent tools for delegate agents
with no per-client isolation.
- **`ChannelMapHandle`** (`src/tools/reaction.rs`):
`Arc<RwLock<HashMap<String, Arc<dyn Channel>>>>` — global channel map shared
across all clients.
- **`CanvasStore`** (`src/tools/canvas.rs`):
`Arc<RwLock<HashMap<String, CanvasEntry>>>` — canvas IDs are plain strings
with no client namespace.
These patterns emerged organically. As the tool surface grows and more clients
connect concurrently, we need a clear contract governing ownership, identity,
isolation, lifecycle, and reload behavior for tool-held shared state. Without
this contract, new tools risk introducing data leaks between clients, stale
state after config reloads, or inconsistent initialization timing.
Additional context:
- The tool registry is immutable after startup, built once in
`all_tools_with_runtime()`.
- Client identity is currently derived from IP address only
(`src/gateway/mod.rs`), which is insufficient for reliable namespacing.
- `SecurityPolicy` is scoped per agent, not per client.
- `WorkspaceManager` provides some isolation but workspace switching is global.
## Decision
### 1. Ownership: May tools own long-lived shared state?
**Yes.** Tools MAY own long-lived shared state, provided they follow the
established **handle pattern**: wrap the state in `Arc<RwLock<T>>` (or
`Arc<parking_lot::RwLock<T>>`) and expose a cloneable handle type.
This pattern is already proven by three independent implementations:
| Handle | Location | Inner type |
|--------|----------|-----------|
| `DelegateParentToolsHandle` | `src/tools/mod.rs` | `Vec<Arc<dyn Tool>>` |
| `ChannelMapHandle` | `src/tools/reaction.rs` | `HashMap<String, Arc<dyn Channel>>` |
| `CanvasStore` | `src/tools/canvas.rs` | `HashMap<String, CanvasEntry>` |
Tools that need shared state MUST:
- Define a named handle type alias (e.g., `pub type FooHandle = Arc<RwLock<T>>`).
- Accept the handle at construction time rather than creating global state.
- Document the concurrency contract in the handle type's doc comment.
Tools MUST NOT use static mutable state (`lazy_static!`, `OnceCell` with
interior mutability) for per-request or per-client data.
### 2. Identity assignment: Who constructs identity keys?
**The daemon SHOULD provide identity.** Tools MUST NOT construct their own
client identity keys.
A new `ClientId` type should be introduced (opaque, `Clone + Eq + Hash + Send + Sync`)
that the daemon assigns at connection time. This replaces the current approach
of using raw IP addresses (`src/gateway/mod.rs:259-306`), which breaks when
multiple clients share a NAT address or when proxied connections arrive.
`ClientId` is passed to tools that require per-client state namespacing as part
of the tool execution context. Tools that do not need per-client isolation
(e.g., the immutable tool registry) may ignore it.
The `ClientId` contract:
- Generated by the gateway layer at connection establishment.
- Opaque to tools — tools must not parse or derive meaning from the value.
- Stable for the lifetime of a single client session.
- Passed through the execution context, not stored globally.
### 3. Lifecycle: When may tools run startup-style validation?
**Validation runs once at first registration, and again when config changes
are detected.**
The lifecycle phases are:
1. **Construction** — tool is instantiated with handles and config. No I/O or
validation occurs here.
2. **Registration** — tool is registered in the tool registry via
`all_tools_with_runtime()`. At this point the tool MAY perform one-time
startup validation (e.g., checking that required credentials exist, verifying
external service connectivity).
3. **Execution** — tool handles individual requests. No re-validation unless
the config-change signal fires (see Reload Semantics below).
4. **Shutdown** — daemon is stopping. Tools with open resources SHOULD clean up
gracefully via `Drop` or an explicit shutdown method.
Tools MUST NOT perform blocking validation during execution-phase calls.
Validation results SHOULD be cached in the tool's handle state and checked
via a fast path during execution.
### 4. Isolation: What must be isolated per client?
State falls into two categories with different isolation requirements:
**MUST be isolated per client:**
- Security-sensitive state: credentials, API keys, quotas, rate-limit counters,
per-client authorization decisions.
- User-specific session data: conversation context, user preferences,
workspace-scoped file paths.
Isolation mechanism: tools holding per-client state MUST key their internal
maps by `ClientId`. The handle pattern naturally supports this by using
`HashMap<ClientId, T>` inside the `RwLock`.
**MAY be shared across clients (with namespace prefixing):**
- Broadcast/display state: canvas frames (`CanvasStore`), notification channels
(`ChannelMapHandle`).
- Read-only reference data: tool registry, static configuration, model
metadata.
When shared state uses string keys (e.g., canvas IDs, channel names), tools
SHOULD support optional namespace prefixing (e.g., `{client_id}:{canvas_name}`)
to allow per-client isolation when needed without mandating it for broadcast
use cases.
Tools MUST NOT store per-client secrets in shared (non-isolated) state
structures.
### 5. Reload semantics: What invalidates prior shared state on config change?
**Config changes detected via hash comparison MUST invalidate cached
validation state.**
The reload contract:
- The daemon computes a hash of the tool-relevant config section at startup and
after each config reload event.
- When the hash changes, the daemon signals affected tools to re-run their
registration-phase validation.
- Tools MUST treat their cached validation result as stale when signaled and
re-validate before the next execution.
Specific invalidation rules:
| Config change | Invalidation scope |
|--------------|-------------------|
| Credential/secret rotation | Per-tool validation cache; per-client credential state |
| Tool enable/disable | Full tool registry rebuild via `all_tools_with_runtime()` |
| Security policy change | `SecurityPolicy` re-derivation; per-agent policy state |
| Workspace directory change | `WorkspaceManager` state; file-path-dependent tool state |
| Provider config change | Provider-dependent tools re-validate connectivity |
Tools MAY retain non-security shared state (e.g., canvas content, channel
subscriptions) across config reloads unless the reload explicitly affects that
state's validity.
## Consequences
### Positive
- **Consistency:** All new tools follow the same handle pattern, making shared
state discoverable and auditable.
- **Safety:** Per-client isolation of security-sensitive state prevents data
leaks in multi-tenant scenarios.
- **Clarity:** Explicit lifecycle phases eliminate ambiguity about when
validation runs.
- **Evolvability:** The `ClientId` abstraction decouples tools from transport
details, supporting future identity mechanisms (tokens, certificates).
### Negative
- **Migration cost:** Existing tools (`CanvasStore`, `ReactionTool`) may need
refactoring to accept `ClientId` and namespace their state.
- **Complexity:** Tools that were simple singletons now need to consider
multi-client semantics even if they currently have one client.
- **Performance:** Per-client keying adds a hash lookup on each access, though
this is negligible compared to I/O costs.
### Neutral
- The tool registry remains immutable after startup; this ADR does not change
that invariant.
- `SecurityPolicy` remains per-agent; this ADR documents that client isolation
is orthogonal to agent-level policy.
## References
- `src/tools/mod.rs``DelegateParentToolsHandle`, `all_tools_with_runtime()`
- `src/tools/reaction.rs``ChannelMapHandle`, `ReactionTool`
- `src/tools/canvas.rs``CanvasStore`, `CanvasEntry`
- `src/tools/traits.rs``Tool` trait
- `src/gateway/mod.rs` — client IP extraction (`forwarded_client_ip`, `resolve_client_ip`)
- `src/security/``SecurityPolicy`

215
docs/browser-setup.md Normal file
View File

@ -0,0 +1,215 @@
# Browser Automation Setup Guide
This guide covers setting up browser automation capabilities in ZeroClaw, including both headless automation and GUI access via VNC.
## Overview
ZeroClaw supports multiple browser access methods:
| Method | Use Case | Requirements |
|--------|----------|--------------|
| **agent-browser CLI** | Headless automation, AI agents | npm, Chrome |
| **VNC + noVNC** | GUI access, debugging | Xvfb, x11vnc, noVNC |
| **Chrome Remote Desktop** | Remote GUI via Google | XFCE, Google account |
## Quick Start: Headless Automation
### 1. Install agent-browser
```bash
# Install CLI
npm install -g agent-browser
# Download Chrome for Testing
agent-browser install --with-deps # Linux (includes system deps)
agent-browser install # macOS/Windows
```
### 2. Verify ZeroClaw Config
The browser tool is enabled by default. To verify or customize, edit
`~/.zeroclaw/config.toml`:
```toml
[browser]
enabled = true # default: true
allowed_domains = ["*"] # default: ["*"] (all public hosts)
backend = "agent_browser" # default: "agent_browser"
native_headless = true # default: true
```
To restrict domains or disable the browser tool:
```toml
[browser]
enabled = false # disable entirely
# or restrict to specific domains:
allowed_domains = ["example.com", "docs.example.com"]
```
### 3. Test
```bash
echo "Open https://example.com and tell me what it says" | zeroclaw agent
```
## VNC Setup (GUI Access)
For debugging or when you need visual browser access:
### Install Dependencies
```bash
# Ubuntu/Debian
apt-get install -y xvfb x11vnc fluxbox novnc websockify
# Optional: Desktop environment for Chrome Remote Desktop
apt-get install -y xfce4 xfce4-goodies
```
### Start VNC Server
```bash
#!/bin/bash
# Start virtual display with VNC access
DISPLAY_NUM=99
VNC_PORT=5900
NOVNC_PORT=6080
RESOLUTION=1920x1080x24
# Start Xvfb
Xvfb :$DISPLAY_NUM -screen 0 $RESOLUTION -ac &
sleep 1
# Start window manager
fluxbox -display :$DISPLAY_NUM &
sleep 1
# Start x11vnc
x11vnc -display :$DISPLAY_NUM -rfbport $VNC_PORT -forever -shared -nopw -bg
sleep 1
# Start noVNC (web-based VNC)
websockify --web=/usr/share/novnc $NOVNC_PORT localhost:$VNC_PORT &
echo "VNC available at:"
echo " VNC Client: localhost:$VNC_PORT"
echo " Web Browser: http://localhost:$NOVNC_PORT/vnc.html"
```
### VNC Access
- **VNC Client**: Connect to `localhost:5900`
- **Web Browser**: Open `http://localhost:6080/vnc.html`
### Start Browser on VNC Display
```bash
DISPLAY=:99 google-chrome --no-sandbox https://example.com &
```
## Chrome Remote Desktop
### Install
```bash
# Download and install
wget https://dl.google.com/linux/direct/chrome-remote-desktop_current_amd64.deb
apt-get install -y ./chrome-remote-desktop_current_amd64.deb
# Configure session
echo "xfce4-session" > ~/.chrome-remote-desktop-session
chmod +x ~/.chrome-remote-desktop-session
```
### Setup
1. Visit <https://remotedesktop.google.com/headless>
2. Copy the "Debian Linux" setup command
3. Run it on your server
4. Start the service: `systemctl --user start chrome-remote-desktop`
### Remote Access
Go to <https://remotedesktop.google.com/access> from any device.
## Testing
### CLI Tests
```bash
# Basic open and close
agent-browser open https://example.com
agent-browser get title
agent-browser close
# Snapshot with refs
agent-browser open https://example.com
agent-browser snapshot -i
agent-browser close
# Screenshot
agent-browser open https://example.com
agent-browser screenshot /tmp/test.png
agent-browser close
```
### ZeroClaw Integration Tests
```bash
# Content extraction
echo "Open https://example.com and summarize it" | zeroclaw agent
# Navigation
echo "Go to https://github.com/trending and list the top 3 repos" | zeroclaw agent
# Form interaction
echo "Go to Wikipedia, search for 'Rust programming language', and summarize" | zeroclaw agent
```
## Troubleshooting
### "Element not found"
The page may not be fully loaded. Add a wait:
```bash
agent-browser open https://slow-site.com
agent-browser wait --load networkidle
agent-browser snapshot -i
```
### Cookie dialogs blocking access
Handle cookie consent first:
```bash
agent-browser open https://site-with-cookies.com
agent-browser snapshot -i
agent-browser click @accept_cookies # Click the accept button
agent-browser snapshot -i # Now get the actual content
```
### Docker sandbox network restrictions
If `web_fetch` fails inside Docker sandbox, use agent-browser instead:
```bash
# Instead of web_fetch, use:
agent-browser open https://example.com
agent-browser get text body
```
## Security Notes
- `agent-browser` runs Chrome in headless mode with sandboxing
- For sensitive sites, use `--session-name` to persist auth state
- The `--allowed-domains` config restricts navigation to specific domains
- VNC ports (5900, 6080) should be behind a firewall or Tailscale
## Related
- [agent-browser Documentation](https://github.com/vercel-labs/agent-browser)
- [ZeroClaw Configuration Reference](./config-reference.md)
- [Skills Documentation](../skills/)

View File

@ -20,6 +20,7 @@ Selected allowlist (all actions currently used across Quality Gate, Release Beta
| `docker/setup-buildx-action@v3` | release, promote-release | Docker Buildx setup |
| `docker/login-action@v3` | release, promote-release | GHCR authentication |
| `docker/build-push-action@v6` | release, promote-release | Multi-platform Docker image build and push |
| `actions/labeler@v5` | pr-path-labeler | Apply path/scope labels from `labeler.yml` |
Equivalent allowlist patterns:
@ -36,6 +37,7 @@ Equivalent allowlist patterns:
| Quality Gate | `.github/workflows/checks-on-pr.yml` | Pull requests to `master` |
| Release Beta | `.github/workflows/release-beta-on-push.yml` | Push to `master` |
| Release Stable | `.github/workflows/release-stable-manual.yml` | Manual `workflow_dispatch` |
| PR Path Labeler | `.github/workflows/pr-path-labeler.yml` | `pull_request_target` (opened, synchronize, reopened) |
## Change Control
@ -62,6 +64,7 @@ gh api repos/zeroclaw-labs/zeroclaw/actions/permissions/selected-actions
## Change Log
- 2026-03-23: Added PR Path Labeler (`pr-path-labeler.yml`) using `actions/labeler@v5`. No allowlist change needed — covered by existing `actions/*` pattern.
- 2026-03-10: Renamed workflows — CI → Quality Gate (`checks-on-pr.yml`), Beta Release → Release Beta (`release-beta-on-push.yml`), Promote Release → Release Stable (`release-stable-manual.yml`). Added `lint` and `security` jobs to Quality Gate. Added Cross-Platform Build (`cross-platform-build-manual.yml`).
- 2026-03-05: Complete workflow overhaul — replaced 22 workflows with 3 (CI, Beta Release, Promote Release)
- Removed patterns no longer in use: `DavidAnson/markdownlint-cli2-action@*`, `lycheeverse/lychee-action@*`, `EmbarkStudios/cargo-deny-action@*`, `rustsec/audit-check@*`, `rhysd/actionlint@*`, `sigstore/cosign-installer@*`, `Checkmarx/vorpal-reviewdog-github-action@*`, `useblacksmith/*`

View File

@ -45,6 +45,15 @@ For complete code examples of each extension trait, see [extension-examples.md](
- Keep multilingual entry-point parity for all supported locales (`en`, `zh-CN`, `ja`, `ru`, `fr`, `vi`) when nav or key wording changes.
- When shared docs wording changes, sync corresponding localized docs in the same PR (or explicitly document deferral and follow-up PR).
## Tool Shared State
- Follow the `Arc<RwLock<T>>` handle pattern for any tool that owns long-lived shared state.
- Accept handles at construction; do not create global/static mutable state.
- Use `ClientId` (provided by the daemon) to namespace per-client state — never construct identity keys inside the tool.
- Isolate security-sensitive state (credentials, quotas) per client; broadcast/display state may be shared with optional namespace prefixing.
- Cached validation is invalidated on config change — tools must re-validate before the next execution when signaled.
- See [ADR-004: Tool Shared State Ownership](../architecture/adr-004-tool-shared-state-ownership.md) for the full contract.
## Architecture Boundary Rules
- Extend capabilities by adding trait implementations + factory wiring first; avoid cross-module rewrites for isolated features.

View File

@ -0,0 +1,213 @@
# Label Registry
Single reference for every label used on PRs and issues. Labels are grouped by category. Each entry lists the label name, definition, and how it is applied.
Sources consolidated here:
- `.github/labeler.yml` (path-label config for `actions/labeler`)
- `.github/label-policy.json` (contributor tier thresholds)
- `docs/contributing/pr-workflow.md` (size, risk, and triage label definitions)
- `docs/contributing/ci-map.md` (automation behavior and high-risk path heuristics)
Note: The CI was simplified to 4 workflows (`ci.yml`, `release.yml`, `ci-full.yml`, `promote-release.yml`). Workflows that previously automated size, risk, contributor tier, and triage labels (`pr-labeler.yml`, `pr-auto-response.yml`, `pr-check-stale.yml`, and supporting scripts) were removed. Only path labels via `pr-path-labeler.yml` are currently automated.
---
## Path labels
Applied automatically by `pr-path-labeler.yml` using `actions/labeler`. Matches changed files against glob patterns in `.github/labeler.yml`.
### Base scope labels
| Label | Matches |
|---|---|
| `docs` | `docs/**`, `**/*.md`, `**/*.mdx`, `LICENSE`, `.markdownlint-cli2.yaml` |
| `dependencies` | `Cargo.toml`, `Cargo.lock`, `deny.toml`, `.github/dependabot.yml` |
| `ci` | `.github/**`, `.githooks/**` |
| `core` | `src/*.rs` |
| `agent` | `src/agent/**` |
| `channel` | `src/channels/**` |
| `gateway` | `src/gateway/**` |
| `config` | `src/config/**` |
| `cron` | `src/cron/**` |
| `daemon` | `src/daemon/**` |
| `doctor` | `src/doctor/**` |
| `health` | `src/health/**` |
| `heartbeat` | `src/heartbeat/**` |
| `integration` | `src/integrations/**` |
| `memory` | `src/memory/**` |
| `security` | `src/security/**` |
| `runtime` | `src/runtime/**` |
| `onboard` | `src/onboard/**` |
| `provider` | `src/providers/**` |
| `service` | `src/service/**` |
| `skillforge` | `src/skillforge/**` |
| `skills` | `src/skills/**` |
| `tool` | `src/tools/**` |
| `tunnel` | `src/tunnel/**` |
| `observability` | `src/observability/**` |
| `tests` | `tests/**` |
| `scripts` | `scripts/**` |
| `dev` | `dev/**` |
### Per-component channel labels
Each channel gets a specific label in addition to the base `channel` label.
| Label | Matches |
|---|---|
| `channel:bluesky` | `bluesky.rs` |
| `channel:clawdtalk` | `clawdtalk.rs` |
| `channel:cli` | `cli.rs` |
| `channel:dingtalk` | `dingtalk.rs` |
| `channel:discord` | `discord.rs`, `discord_history.rs` |
| `channel:email` | `email_channel.rs`, `gmail_push.rs` |
| `channel:imessage` | `imessage.rs` |
| `channel:irc` | `irc.rs` |
| `channel:lark` | `lark.rs` |
| `channel:linq` | `linq.rs` |
| `channel:matrix` | `matrix.rs` |
| `channel:mattermost` | `mattermost.rs` |
| `channel:mochat` | `mochat.rs` |
| `channel:mqtt` | `mqtt.rs` |
| `channel:nextcloud-talk` | `nextcloud_talk.rs` |
| `channel:nostr` | `nostr.rs` |
| `channel:notion` | `notion.rs` |
| `channel:qq` | `qq.rs` |
| `channel:reddit` | `reddit.rs` |
| `channel:signal` | `signal.rs` |
| `channel:slack` | `slack.rs` |
| `channel:telegram` | `telegram.rs` |
| `channel:twitter` | `twitter.rs` |
| `channel:wati` | `wati.rs` |
| `channel:webhook` | `webhook.rs` |
| `channel:wecom` | `wecom.rs` |
| `channel:whatsapp` | `whatsapp.rs`, `whatsapp_storage.rs`, `whatsapp_web.rs` |
### Per-component provider labels
| Label | Matches |
|---|---|
| `provider:anthropic` | `anthropic.rs` |
| `provider:azure-openai` | `azure_openai.rs` |
| `provider:bedrock` | `bedrock.rs` |
| `provider:claude-code` | `claude_code.rs` |
| `provider:compatible` | `compatible.rs` |
| `provider:copilot` | `copilot.rs` |
| `provider:gemini` | `gemini.rs`, `gemini_cli.rs` |
| `provider:glm` | `glm.rs` |
| `provider:kilocli` | `kilocli.rs` |
| `provider:ollama` | `ollama.rs` |
| `provider:openai` | `openai.rs`, `openai_codex.rs` |
| `provider:openrouter` | `openrouter.rs` |
| `provider:telnyx` | `telnyx.rs` |
### Per-group tool labels
Tools are grouped by logical function rather than one label per file.
| Label | Matches |
|---|---|
| `tool:browser` | `browser.rs`, `browser_delegate.rs`, `browser_open.rs`, `text_browser.rs`, `screenshot.rs` |
| `tool:cloud` | `cloud_ops.rs`, `cloud_patterns.rs` |
| `tool:composio` | `composio.rs` |
| `tool:cron` | `cron_add.rs`, `cron_list.rs`, `cron_remove.rs`, `cron_run.rs`, `cron_runs.rs`, `cron_update.rs` |
| `tool:file` | `file_edit.rs`, `file_read.rs`, `file_write.rs`, `glob_search.rs`, `content_search.rs` |
| `tool:google-workspace` | `google_workspace.rs` |
| `tool:mcp` | `mcp_client.rs`, `mcp_deferred.rs`, `mcp_protocol.rs`, `mcp_tool.rs`, `mcp_transport.rs` |
| `tool:memory` | `memory_forget.rs`, `memory_recall.rs`, `memory_store.rs` |
| `tool:microsoft365` | `microsoft365/**` |
| `tool:security` | `security_ops.rs`, `verifiable_intent.rs` |
| `tool:shell` | `shell.rs`, `node_tool.rs`, `cli_discovery.rs` |
| `tool:sop` | `sop_advance.rs`, `sop_approve.rs`, `sop_execute.rs`, `sop_list.rs`, `sop_status.rs` |
| `tool:web` | `web_fetch.rs`, `web_search_tool.rs`, `web_search_provider_routing.rs`, `http_request.rs` |
---
## Size labels
Defined in `pr-workflow.md` §6.1. Based on effective changed line count, normalized for docs-only and lockfile-heavy PRs.
| Label | Threshold |
|---|---|
| `size: XS` | <= 80 lines |
| `size: S` | <= 250 lines |
| `size: M` | <= 500 lines |
| `size: L` | <= 1000 lines |
| `size: XL` | > 1000 lines |
**Applied by:** manual. The workflows that previously computed size labels (`pr-labeler.yml` and supporting scripts) were removed during CI simplification.
---
## Risk labels
Defined in `pr-workflow.md` §13.2 and `ci-map.md`. Based on a heuristic combining touched paths and change size.
| Label | Meaning |
|---|---|
| `risk: low` | No high-risk paths touched, small change |
| `risk: medium` | Behavioral `src/**` changes without boundary/security impact |
| `risk: high` | Touches high-risk paths (see below) or large security-adjacent change |
| `risk: manual` | Maintainer override that freezes automated risk recalculation |
High-risk paths: `src/security/**`, `src/runtime/**`, `src/gateway/**`, `src/tools/**`, `.github/workflows/**`.
The boundary between low and medium is not formally defined beyond "no high-risk paths."
**Applied by:** manual. Previously automated via `pr-labeler.yml`; removed during CI simplification.
---
## Contributor tier labels
Defined in `.github/label-policy.json`. Based on the author's merged PR count queried from the GitHub API.
| Label | Minimum merged PRs |
|---|---|
| `trusted contributor` | 5 |
| `experienced contributor` | 10 |
| `principal contributor` | 20 |
| `distinguished contributor` | 50 |
**Applied by:** manual. Previously automated via `pr-labeler.yml` and `pr-auto-response.yml`; removed during CI simplification.
---
## Response and triage labels
Defined in `pr-workflow.md` §8. Applied manually.
| Label | Purpose | Applied by |
|---|---|---|
| `r:needs-repro` | Incomplete bug report; request deterministic repro | Manual |
| `r:support` | Usage/help item better handled outside bug backlog | Manual |
| `invalid` | Not a valid bug/feature request | Manual |
| `duplicate` | Duplicate of existing issue | Manual |
| `stale-candidate` | Dormant PR/issue; candidate for closing | Manual |
| `superseded` | Replaced by a newer PR | Manual |
| `no-stale` | Exempt from stale automation; accepted but blocked work | Manual |
**Automation:** none currently. The workflows that handled label-driven issue closing (`pr-auto-response.yml`) and stale detection (`pr-check-stale.yml`) were removed during CI simplification.
---
## Implementation status
| Category | Count | Automated | Workflow |
|---|---|---|---|
| Path (base scope) | 27 | Yes | `pr-path-labeler.yml` |
| Path (per-component) | 52 | Yes | `pr-path-labeler.yml` |
| Size | 5 | No | Manual |
| Risk | 4 | No | Manual |
| Contributor tier | 4 | No | Manual |
| Response/triage | 7 | No | Manual |
| **Total** | **99** | | |
---
## Maintenance
- **Owner:** maintainers responsible for label policy and PR triage automation.
- **Update trigger:** new channels, providers, or tools added to the source tree; label policy changes; triage workflow changes.
- **Source of truth:** this document consolidates definitions from the four source files listed at the top. When definitions conflict, update the source file first, then sync this registry.

View File

@ -411,30 +411,6 @@ allowed_roots = [\"~/Desktop/projects\", \"/opt/shared-repo\"]
- 内存上下文注入忽略旧的 `assistant_resp*` 自动保存键,以防止旧模型生成的摘要被视为事实。
### `[memory.mem0]`
Mem0 (OpenMemory) 后端 — 连接自托管 mem0 服务器,提供基于向量的记忆存储和 LLM 事实提取。构建时需要 `memory-mem0` feature flag配置需设置 `backend = "mem0"`
| 键 | 默认值 | 环境变量 | 用途 |
|---|---|---|---|
| `url` | `http://localhost:8765` | `MEM0_URL` | OpenMemory 服务器地址 |
| `user_id` | `zeroclaw` | `MEM0_USER_ID` | 记忆作用域的用户 ID |
| `app_name` | `zeroclaw` | `MEM0_APP_NAME` | 在 mem0 中注册的应用名称 |
| `infer` | `true` | — | 使用 LLM 从存储文本中提取事实 (`true`) 或原样存储 (`false`) |
| `extraction_prompt` | 未设置 | `MEM0_EXTRACTION_PROMPT` | 自定义 LLM 事实提取提示词(如适用于非英文内容) |
```toml
[memory]
backend = "mem0"
[memory.mem0]
url = "http://192.168.0.171:8765"
user_id = "zeroclaw-bot"
extraction_prompt = "用原始语言提取事实..."
```
服务器部署脚本位于 `deploy/mem0/`
## `[[model_routes]]``[[embedding_routes]]`
使用路由提示,以便集成可以在模型 ID 演变时保持稳定的名称。

View File

@ -12,8 +12,6 @@ SOP 审计条目通过 `SopAuditLogger` 持久化到配置的内存后端的 `so
- `sop_step_{run_id}_{step_number}`:单步结果
- `sop_approval_{run_id}_{step_number}`:操作员审批记录
- `sop_timeout_approve_{run_id}_{step_number}`:超时自动审批记录
- `sop_gate_decision_{gate_id}_{timestamp_ms}`:门评估器决策记录(启用 `ampersona-gates` 时)
- `sop_phase_state`:持久化的信任阶段状态快照(启用 `ampersona-gates` 时)
## 2. 检查路径

View File

@ -508,30 +508,6 @@ Notes:
- Memory context injection ignores legacy `assistant_resp*` auto-save keys to prevent old model-authored summaries from being treated as facts.
### `[memory.mem0]`
Mem0 (OpenMemory) backend — connects to a self-hosted mem0 server for vector-based memory with LLM-powered fact extraction. Requires feature flag `memory-mem0` at build time and `backend = "mem0"` in config.
| Key | Default | Env var | Purpose |
|---|---|---|---|
| `url` | `http://localhost:8765` | `MEM0_URL` | OpenMemory server URL |
| `user_id` | `zeroclaw` | `MEM0_USER_ID` | User ID for scoping memories |
| `app_name` | `zeroclaw` | `MEM0_APP_NAME` | Application name registered in mem0 |
| `infer` | `true` | — | Use LLM to extract facts from stored text (`true`) or store raw (`false`) |
| `extraction_prompt` | unset | `MEM0_EXTRACTION_PROMPT` | Custom prompt for LLM fact extraction (e.g. for non-English content) |
```toml
[memory]
backend = "mem0"
[memory.mem0]
url = "http://192.168.0.171:8765"
user_id = "zeroclaw-bot"
extraction_prompt = "Extract facts in the original language..."
```
Server deployment scripts are in `deploy/mem0/`.
## `[[model_routes]]` and `[[embedding_routes]]`
Use route hints so integrations can keep stable names while model IDs evolve.

View File

@ -12,8 +12,6 @@ Common key patterns:
- `sop_step_{run_id}_{step_number}`: per-step result
- `sop_approval_{run_id}_{step_number}`: operator approval record
- `sop_timeout_approve_{run_id}_{step_number}`: timeout auto-approval record
- `sop_gate_decision_{gate_id}_{timestamp_ms}`: gate evaluator decision record (when `ampersona-gates` is enabled)
- `sop_phase_state`: persisted trust-phase state snapshot (when `ampersona-gates` is enabled)
## 2. Inspection Paths

View File

@ -337,30 +337,6 @@ Lưu ý:
- Chèn ngữ cảnh memory bỏ qua khóa auto-save `assistant_resp*` kiểu cũ để tránh tóm tắt do model tạo bị coi là sự thật.
### `[memory.mem0]`
Backend Mem0 (OpenMemory) — kết nối đến server mem0 tự host, cung cấp bộ nhớ vector với trích xuất sự kiện bằng LLM. Cần feature flag `memory-mem0` khi build và `backend = "mem0"` trong config.
| Khóa | Mặc định | Biến môi trường | Mục đích |
|---|---|---|---|
| `url` | `http://localhost:8765` | `MEM0_URL` | URL server OpenMemory |
| `user_id` | `zeroclaw` | `MEM0_USER_ID` | User ID để phân vùng memory |
| `app_name` | `zeroclaw` | `MEM0_APP_NAME` | Tên ứng dụng đăng ký trong mem0 |
| `infer` | `true` | — | Dùng LLM trích xuất sự kiện từ text (`true`) hoặc lưu nguyên (`false`) |
| `extraction_prompt` | chưa đặt | `MEM0_EXTRACTION_PROMPT` | Prompt tùy chỉnh cho trích xuất sự kiện LLM (vd: cho nội dung không phải tiếng Anh) |
```toml
[memory]
backend = "mem0"
[memory.mem0]
url = "http://192.168.0.171:8765"
user_id = "zeroclaw-bot"
extraction_prompt = "Trích xuất sự kiện bằng ngôn ngữ gốc..."
```
Script triển khai server nằm trong `deploy/mem0/`.
## `[[model_routes]]``[[embedding_routes]]`
Route hint giúp tên tích hợp ổn định khi model ID thay đổi.

View File

@ -38,3 +38,82 @@ allowed_tools = ["read", "edit", "exec"]
max_iterations = 15
# Optional: use longer timeout for complex coding tasks
agentic_timeout_secs = 600
# ── Cron Configuration ────────────────────────────────────────
[cron]
# Enable the cron subsystem. Default: true
enabled = true
# Run all overdue jobs at scheduler startup. Default: true
catch_up_on_startup = true
# Maximum number of historical cron run records to retain. Default: 50
max_run_history = 50
# ── Declarative Cron Jobs ─────────────────────────────────────
# Define cron jobs directly in config. These are synced to the database
# at scheduler startup. Each job needs a stable `id` for merge semantics.
# Shell job: runs a shell command on a cron schedule
[[cron.jobs]]
id = "daily-backup"
name = "Daily Backup"
job_type = "shell"
command = "tar czf /tmp/backup.tar.gz /data"
schedule = { kind = "cron", expr = "0 2 * * *" }
# Agent job: runs an agent prompt on an interval
[[cron.jobs]]
id = "health-check"
name = "Health Check"
job_type = "agent"
prompt = "Check server health: disk space, memory, CPU load"
model = "anthropic/claude-sonnet-4"
allowed_tools = ["shell", "file_read"]
schedule = { kind = "every", every_ms = 300000 }
# Cron job with timezone and delivery
# [[cron.jobs]]
# id = "morning-report"
# name = "Morning Report"
# job_type = "agent"
# prompt = "Generate a daily summary of system metrics"
# schedule = { kind = "cron", expr = "0 9 * * 1-5", tz = "America/New_York" }
# [cron.jobs.delivery]
# mode = "announce"
# channel = "telegram"
# to = "123456789"
# ── Cost Tracking Configuration ────────────────────────────────
[cost]
# Enable cost tracking and budget enforcement. Default: false
enabled = false
# Daily spending limit in USD. Default: 10.0
daily_limit_usd = 10.0
# Monthly spending limit in USD. Default: 100.0
monthly_limit_usd = 100.0
# Warn when spending reaches this percentage of limit. Default: 80
warn_at_percent = 80
# Allow requests to exceed budget with --override flag. Default: false
allow_override = false
# Per-model pricing (USD per 1M tokens).
# Built-in defaults exist for popular models; add overrides here.
# [cost.prices."anthropic/claude-opus-4-20250514"]
# input = 15.0
# output = 75.0
# [cost.prices."anthropic/claude-sonnet-4-20250514"]
# input = 3.0
# output = 15.0
# [cost.prices."openai/gpt-4o"]
# input = 5.0
# output = 15.0
# [cost.prices."openai/gpt-4o-mini"]
# input = 0.15
# output = 0.60
# ── Voice Transcription ─────────────────────────────────────────
# [transcription]
# enabled = true
# default_provider = "groq"
# Also transcribe non-PTT (forwarded / regular) audio on WhatsApp.
# Default: false (only voice notes are transcribed).
# transcribe_non_ptt_audio = false

View File

@ -230,6 +230,49 @@ detect_release_target() {
esac
}
detect_device_class() {
# Containers are never desktops
if _is_container_runtime; then
echo "container"
return
fi
# Termux / Android
if [[ -n "${TERMUX_VERSION:-}" || -d "/data/data/com.termux" ]]; then
echo "mobile"
return
fi
local os arch
os="$(uname -s)"
arch="$(uname -m)"
case "$os" in
Darwin)
# macOS is always a desktop
echo "desktop"
;;
Linux)
# Raspberry Pi / ARM SBCs — treat as embedded (typically headless)
case "$arch" in
armv6l|armv7l)
echo "embedded"
return
;;
esac
# Check for a display server (X11 or Wayland)
if [[ -n "${DISPLAY:-}" || -n "${WAYLAND_DISPLAY:-}" || -n "${XDG_SESSION_TYPE:-}" ]]; then
echo "desktop"
else
echo "server"
fi
;;
*)
echo "server"
;;
esac
}
should_attempt_prebuilt_for_resources() {
local workspace="${1:-.}"
local min_ram_mb min_disk_mb total_ram_mb free_disk_mb low_resource
@ -1155,6 +1198,9 @@ while [[ $# -gt 0 ]]; do
done
OS_NAME="$(uname -s)"
DEVICE_CLASS="$(detect_device_class)"
step_dot "Device: $OS_NAME/$(uname -m) ($DEVICE_CLASS)"
if [[ "$GUIDED_MODE" == "auto" ]]; then
if [[ "$OS_NAME" == "Linux" && "$ORIGINAL_ARG_COUNT" -eq 0 && -t 0 && -t 1 ]]; then
GUIDED_MODE="on"
@ -1416,8 +1462,20 @@ if [[ "$SKIP_BUILD" == false ]]; then
step_dot "Cleaning stale build cache (upgrade detected)"
cargo clean --release 2>/dev/null || true
fi
# Determine cargo feature flags — disable prometheus on 32-bit targets
# (prometheus crate requires AtomicU64, unavailable on armv7l/armv6l)
CARGO_FEATURE_FLAGS=""
_build_arch="$(uname -m)"
case "$_build_arch" in
armv7l|armv6l|armhf)
step_dot "32-bit ARM detected ($_build_arch) — disabling prometheus (requires 64-bit atomics)"
CARGO_FEATURE_FLAGS="--no-default-features --features channel-nostr,skill-creation"
;;
esac
step_dot "Building release binary"
cargo build --release --locked
cargo build --release --locked $CARGO_FEATURE_FLAGS
step_ok "Release binary built"
else
step_dot "Skipping build"
@ -1436,7 +1494,7 @@ if [[ "$SKIP_INSTALL" == false ]]; then
fi
fi
cargo install --path "$WORK_DIR" --force --locked
cargo install --path "$WORK_DIR" --force --locked $CARGO_FEATURE_FLAGS
step_ok "ZeroClaw installed"
# Sync binary to ~/.local/bin so PATH lookups find the fresh version
@ -1448,6 +1506,85 @@ else
step_dot "Skipping install"
fi
# --- Build web dashboard ---
if [[ "$SKIP_BUILD" == false && -d "$WORK_DIR/web" ]]; then
if have_cmd node && have_cmd npm; then
step_dot "Building web dashboard"
if (cd "$WORK_DIR/web" && npm ci --ignore-scripts 2>/dev/null && npm run build 2>/dev/null); then
step_ok "Web dashboard built"
else
warn "Web dashboard build failed — dashboard will not be available"
fi
else
warn "node/npm not found — skipping web dashboard build"
warn "Install Node.js (>=18) and re-run, or build manually: cd web && npm ci && npm run build"
fi
else
if [[ "$SKIP_BUILD" == true ]]; then
step_dot "Skipping web dashboard build"
fi
fi
# --- Companion desktop app (device-class-aware) ---
# The desktop app is a pre-built download from the website, not built from source.
# This keeps the one-liner install fast and the CLI binary small.
DESKTOP_DOWNLOAD_URL="https://www.zeroclawlabs.ai/download"
DESKTOP_APP_DETECTED=false
if [[ "$DEVICE_CLASS" == "desktop" ]]; then
# Check if the companion app is already installed
case "$OS_NAME" in
Darwin)
if [[ -d "/Applications/ZeroClaw.app" ]] || [[ -d "$HOME/Applications/ZeroClaw.app" ]]; then
DESKTOP_APP_DETECTED=true
step_ok "Companion app found (ZeroClaw.app)"
fi
;;
Linux)
if have_cmd zeroclaw-desktop; then
DESKTOP_APP_DETECTED=true
step_ok "Companion app found (zeroclaw-desktop)"
elif [[ -x "$HOME/.local/bin/zeroclaw-desktop" ]]; then
DESKTOP_APP_DETECTED=true
step_ok "Companion app found (~/.local/bin/zeroclaw-desktop)"
fi
;;
esac
if [[ "$DESKTOP_APP_DETECTED" == false ]]; then
echo
echo -e "${BOLD}Companion App${RESET}"
echo -e " Menu bar access to your ZeroClaw agent."
echo -e " Works alongside the CLI — connects to the same gateway."
echo
case "$OS_NAME" in
Darwin)
echo -e " ${BOLD}Download for macOS:${RESET} ${BLUE}${DESKTOP_DOWNLOAD_URL}${RESET}"
;;
Linux)
echo -e " ${BOLD}Download for Linux:${RESET} ${BLUE}${DESKTOP_DOWNLOAD_URL}${RESET}"
;;
esac
echo -e " ${DIM}Or run: zeroclaw desktop --install${RESET}"
fi
elif [[ "$DEVICE_CLASS" != "desktop" ]]; then
# Non-desktop device — explain why companion app is not offered
case "$DEVICE_CLASS" in
mobile)
step_dot "Mobile device — use the web dashboard at http://127.0.0.1:42617"
;;
embedded)
step_dot "Embedded device ($(uname -m)) — use the web dashboard"
;;
container)
step_dot "Container runtime — use the web dashboard"
;;
server)
step_dot "Headless server — use the web dashboard"
;;
esac
fi
ZEROCLAW_BIN=""
if [[ -x "$HOME/.cargo/bin/zeroclaw" ]]; then
ZEROCLAW_BIN="$HOME/.cargo/bin/zeroclaw"
@ -1614,6 +1751,13 @@ echo -e "${BOLD}Next steps:${RESET}"
echo -e " ${DIM}zeroclaw status${RESET}"
echo -e " ${DIM}zeroclaw agent -m \"Hello, ZeroClaw!\"${RESET}"
echo -e " ${DIM}zeroclaw gateway${RESET}"
if [[ "$DEVICE_CLASS" == "desktop" ]]; then
if [[ "$DESKTOP_APP_DETECTED" == true ]]; then
echo -e " ${DIM}zeroclaw desktop${RESET} ${DIM}# Launch the menu bar app${RESET}"
else
echo -e " ${DIM}zeroclaw desktop --install${RESET} ${DIM}# Download the companion app${RESET}"
fi
fi
echo
echo -e "${BOLD}Docs:${RESET} ${BLUE}https://www.zeroclawlabs.ai/docs${RESET}"
echo

View File

@ -0,0 +1,21 @@
#!/bin/bash
# Start a browser on a virtual display
# Usage: ./start-browser.sh [display_num] [url]
set -e
DISPLAY_NUM=${1:-99}
URL=${2:-"https://google.com"}
export DISPLAY=:$DISPLAY_NUM
# Check if display is running
if ! xdpyinfo -display :$DISPLAY_NUM &>/dev/null; then
echo "Error: Display :$DISPLAY_NUM not running."
echo "Start VNC first: ./start-vnc.sh"
exit 1
fi
google-chrome --no-sandbox --disable-gpu --disable-setuid-sandbox "$URL" &
echo "Chrome started on display :$DISPLAY_NUM"
echo "View via VNC or noVNC"

52
scripts/browser/start-vnc.sh Executable file
View File

@ -0,0 +1,52 @@
#!/bin/bash
# Start virtual display with VNC access for browser GUI
# Usage: ./start-vnc.sh [display_num] [vnc_port] [novnc_port] [resolution]
set -e
DISPLAY_NUM=${1:-99}
VNC_PORT=${2:-5900}
NOVNC_PORT=${3:-6080}
RESOLUTION=${4:-1920x1080x24}
echo "Starting virtual display :$DISPLAY_NUM at $RESOLUTION"
# Kill any existing sessions
pkill -f "Xvfb :$DISPLAY_NUM" 2>/dev/null || true
pkill -f "x11vnc.*:$DISPLAY_NUM" 2>/dev/null || true
pkill -f "websockify.*$NOVNC_PORT" 2>/dev/null || true
sleep 1
# Start Xvfb (virtual framebuffer)
Xvfb :$DISPLAY_NUM -screen 0 $RESOLUTION -ac &
XVFB_PID=$!
sleep 1
# Set DISPLAY
export DISPLAY=:$DISPLAY_NUM
# Start window manager
fluxbox -display :$DISPLAY_NUM 2>/dev/null &
sleep 1
# Start x11vnc
x11vnc -display :$DISPLAY_NUM -rfbport $VNC_PORT -forever -shared -nopw -bg 2>/dev/null
sleep 1
# Start noVNC (web-based VNC client)
websockify --web=/usr/share/novnc $NOVNC_PORT localhost:$VNC_PORT &
NOVNC_PID=$!
echo ""
echo "==================================="
echo "VNC Server started!"
echo "==================================="
echo "VNC Direct: localhost:$VNC_PORT"
echo "noVNC Web: http://localhost:$NOVNC_PORT/vnc.html"
echo "Display: :$DISPLAY_NUM"
echo "==================================="
echo ""
echo "To start a browser, run:"
echo " DISPLAY=:$DISPLAY_NUM google-chrome &"
echo ""
echo "To stop, run: pkill -f 'Xvfb :$DISPLAY_NUM'"

11
scripts/browser/stop-vnc.sh Executable file
View File

@ -0,0 +1,11 @@
#!/bin/bash
# Stop virtual display and VNC server
# Usage: ./stop-vnc.sh [display_num]
DISPLAY_NUM=${1:-99}
pkill -f "Xvfb :$DISPLAY_NUM" 2>/dev/null || true
pkill -f "x11vnc.*:$DISPLAY_NUM" 2>/dev/null || true
pkill -f "websockify.*6080" 2>/dev/null || true
echo "VNC server stopped"

View File

@ -77,7 +77,9 @@ echo "Created annotated tag: $TAG"
if [[ "$PUSH_TAG" == "true" ]]; then
git push origin "$TAG"
echo "Pushed tag to origin: $TAG"
echo "GitHub release pipeline will run via .github/workflows/pub-release.yml"
echo "Release Stable workflow will auto-trigger via tag push."
echo "Monitor: gh workflow view 'Release Stable' --web"
else
echo "Next step: git push origin $TAG"
echo "This will auto-trigger the Release Stable workflow (builds, Docker, crates.io, website, Scoop, AUR, Homebrew, tweet)."
fi

122
skills/browser/SKILL.md Normal file
View File

@ -0,0 +1,122 @@
---
name: browser
description: Headless browser automation using agent-browser CLI
metadata: {"zeroclaw":{"emoji":"🌐","requires":{"bins":["agent-browser"]}}}
---
# Browser Skill
Control a headless browser for web automation, scraping, and testing.
## Prerequisites
- `agent-browser` CLI installed globally (`npm install -g agent-browser`)
- Chrome downloaded (`agent-browser install`)
## Installation
```bash
# Install agent-browser CLI
npm install -g agent-browser
# Download Chrome for Testing
agent-browser install --with-deps # Linux
agent-browser install # macOS/Windows
```
## Usage
### Navigate and snapshot
```bash
agent-browser open https://example.com
agent-browser snapshot -i
```
### Interact with elements
```bash
agent-browser click @e1 # Click by ref
agent-browser fill @e2 "text" # Fill input
agent-browser press Enter # Press key
```
### Extract data
```bash
agent-browser get text @e1 # Get text content
agent-browser get url # Get current URL
agent-browser screenshot page.png # Take screenshot
```
### Session management
```bash
agent-browser close # Close browser
```
## Common Workflows
### Login flow
```bash
agent-browser open https://site.com/login
agent-browser snapshot -i
agent-browser fill @email "user@example.com"
agent-browser fill @password "secretpass"
agent-browser click @submit
agent-browser wait --text "Welcome"
```
### Scrape page content
```bash
agent-browser open https://news.ycombinator.com
agent-browser snapshot -i
agent-browser get text @e1
```
### Take screenshots
```bash
agent-browser open https://google.com
agent-browser screenshot --full page.png
```
## Options
- `--json` - JSON output for parsing
- `--headed` - Show browser window (for debugging)
- `--session-name <name>` - Persist session cookies
- `--profile <path>` - Use persistent browser profile
## Configuration
The browser tool is enabled by default with `allowed_domains = ["*"]` and
`backend = "agent_browser"`. To customize, edit `~/.zeroclaw/config.toml`:
```toml
[browser]
enabled = true # default: true
allowed_domains = ["*"] # default: ["*"] (all public hosts)
backend = "agent_browser" # default: "agent_browser"
native_headless = true # default: true
```
To restrict domains or disable the browser tool:
```toml
[browser]
enabled = false # disable entirely
# or restrict to specific domains:
allowed_domains = ["example.com", "docs.example.com"]
```
## Full Command Reference
Run `agent-browser --help` for all available commands.
## Related
- [agent-browser GitHub](https://github.com/vercel-labs/agent-browser)
- [VNC Setup Guide](../docs/browser-setup.md)

3
skills/browser/TEST.sh Normal file
View File

@ -0,0 +1,3 @@
# Browser skill tests
# Format: command | expected_exit_code | expected_output_pattern
echo "browser skill loaded" | 0 | browser skill loaded

View File

@ -12,11 +12,29 @@ use crate::runtime;
use crate::security::SecurityPolicy;
use crate::tools::{self, Tool, ToolSpec};
use anyhow::Result;
use chrono::{Datelike, Timelike};
use std::collections::HashMap;
use std::io::Write as IoWrite;
use std::sync::Arc;
use std::time::Instant;
/// Events emitted during a streamed agent turn.
///
/// Consumers receive these through a `tokio::sync::mpsc::Sender<TurnEvent>`
/// passed to [`Agent::turn_streamed`].
#[derive(Debug, Clone)]
pub enum TurnEvent {
/// A text chunk from the LLM response (may arrive many times).
Chunk { delta: String },
/// The agent is invoking a tool.
ToolCall {
name: String,
args: serde_json::Value,
},
/// A tool has returned a result.
ToolResult { name: String, output: String },
}
pub struct Agent {
provider: Box<dyn Provider>,
tools: Vec<Box<dyn Tool>>,
@ -359,22 +377,23 @@ impl Agent {
None
};
let (mut tools, delegate_handle, _reaction_handle) = tools::all_tools_with_runtime(
Arc::new(config.clone()),
&security,
runtime,
memory.clone(),
composio_key,
composio_entity_id,
&config.browser,
&config.http_request,
&config.web_fetch,
&config.workspace_dir,
&config.agents,
config.api_key.as_deref(),
config,
None,
);
let (mut tools, delegate_handle, _reaction_handle, _channel_map_handle, _ask_user_handle) =
tools::all_tools_with_runtime(
Arc::new(config.clone()),
&security,
runtime,
memory.clone(),
composio_key,
composio_entity_id,
&config.browser,
&config.http_request,
&config.web_fetch,
&config.workspace_dir,
&config.agents,
config.api_key.as_deref(),
config,
None,
);
// ── Wire MCP tools (non-fatal) ─────────────────────────────
// Replicates the same MCP initialization logic used in the CLI
@ -634,6 +653,24 @@ impl Agent {
return format!("hint:{}", decision.hint);
}
}
// Fallback: auto-classify by complexity when no rule matched.
if let Some(ref ac) = self.config.auto_classify {
let tier = super::eval::estimate_complexity(user_message);
if let Some(hint) = ac.hint_for(tier) {
if self.available_hints.contains(&hint.to_string()) {
tracing::info!(
target: "query_classification",
hint = hint,
complexity = ?tier,
message_length = user_message.len(),
"Auto-classified by complexity"
);
return format!("hint:{hint}");
}
}
}
self.model_name.clone()
}
@ -668,11 +705,17 @@ impl Agent {
.await;
}
let now = chrono::Local::now().format("%Y-%m-%d %H:%M:%S %Z");
let now = chrono::Local::now();
let (year, month, day) = (now.year(), now.month(), now.day());
let (hour, minute, second) = (now.hour(), now.minute(), now.second());
let tz = now.format("%Z");
let date_str =
format!("{year:04}-{month:02}-{day:02} {hour:02}:{minute:02}:{second:02} {tz}");
let enriched = if context.is_empty() {
format!("[{now}] {user_message}")
format!("[CURRENT DATE & TIME: {date_str}]\n\n{user_message}")
} else {
format!("{context}[{now}] {user_message}")
format!("[CURRENT DATE & TIME: {date_str}]\n\n{context}\n\n{user_message}")
};
self.history
@ -798,6 +841,254 @@ impl Agent {
)
}
/// Execute a single agent turn while streaming intermediate events.
///
/// Behaves identically to [`turn`](Self::turn) but forwards [`TurnEvent`]s
/// through the provided channel so callers (e.g. the WebSocket gateway)
/// can relay incremental updates to clients.
///
/// The returned `String` is the final, complete assistant response — the
/// same value that `turn` would return.
pub async fn turn_streamed(
&mut self,
user_message: &str,
event_tx: tokio::sync::mpsc::Sender<TurnEvent>,
) -> Result<String> {
// ── Preamble (identical to turn) ───────────────────────────────
if self.history.is_empty() {
let system_prompt = self.build_system_prompt()?;
self.history
.push(ConversationMessage::Chat(ChatMessage::system(
system_prompt,
)));
}
let context = self
.memory_loader
.load_context(
self.memory.as_ref(),
user_message,
self.memory_session_id.as_deref(),
)
.await
.unwrap_or_default();
if self.auto_save {
let _ = self
.memory
.store(
"user_msg",
user_message,
MemoryCategory::Conversation,
self.memory_session_id.as_deref(),
)
.await;
}
let now = chrono::Local::now().format("%Y-%m-%d %H:%M:%S %Z");
let enriched = if context.is_empty() {
format!("[{now}] {user_message}")
} else {
format!("{context}[{now}] {user_message}")
};
self.history
.push(ConversationMessage::Chat(ChatMessage::user(enriched)));
let effective_model = self.classify_model(user_message);
// ── Turn loop ──────────────────────────────────────────────────
for _ in 0..self.config.max_tool_iterations {
let messages = self.tool_dispatcher.to_provider_messages(&self.history);
// Response cache check (same as turn)
let cache_key = if self.temperature == 0.0 {
self.response_cache.as_ref().map(|_| {
let last_user = messages
.iter()
.rfind(|m| m.role == "user")
.map(|m| m.content.as_str())
.unwrap_or("");
let system = messages
.iter()
.find(|m| m.role == "system")
.map(|m| m.content.as_str());
crate::memory::response_cache::ResponseCache::cache_key(
&effective_model,
system,
last_user,
)
})
} else {
None
};
if let (Some(ref cache), Some(ref key)) = (&self.response_cache, &cache_key) {
if let Ok(Some(cached)) = cache.get(key) {
self.observer.record_event(&ObserverEvent::CacheHit {
cache_type: "response".into(),
tokens_saved: 0,
});
self.history
.push(ConversationMessage::Chat(ChatMessage::assistant(
cached.clone(),
)));
self.trim_history();
return Ok(cached);
}
self.observer.record_event(&ObserverEvent::CacheMiss {
cache_type: "response".into(),
});
}
// ── Streaming LLM call ────────────────────────────────────
// Try streaming first; if the provider returns content we
// forward deltas. Otherwise fall back to non-streaming chat.
use futures_util::StreamExt;
let stream_opts = crate::providers::traits::StreamOptions::new(true);
let mut stream = self.provider.stream_chat_with_history(
&messages,
&effective_model,
self.temperature,
stream_opts,
);
let mut streamed_text = String::new();
let mut got_stream = false;
while let Some(item) = stream.next().await {
match item {
Ok(chunk) => {
if !chunk.delta.is_empty() {
got_stream = true;
streamed_text.push_str(&chunk.delta);
let _ = event_tx.send(TurnEvent::Chunk { delta: chunk.delta }).await;
}
}
Err(_) => break,
}
}
// Drop the stream so we release the borrow on provider.
drop(stream);
// If streaming produced text, use it as the response and
// check for tool calls via the dispatcher.
let response = if got_stream {
// Build a synthetic ChatResponse from streamed text
crate::providers::ChatResponse {
text: Some(streamed_text),
tool_calls: Vec::new(),
usage: None,
reasoning_content: None,
}
} else {
// Fall back to non-streaming chat
match self
.provider
.chat(
ChatRequest {
messages: &messages,
tools: if self.tool_dispatcher.should_send_tool_specs() {
Some(&self.tool_specs)
} else {
None
},
},
&effective_model,
self.temperature,
)
.await
{
Ok(resp) => resp,
Err(err) => return Err(err),
}
};
let (text, calls) = self.tool_dispatcher.parse_response(&response);
if calls.is_empty() {
let final_text = if text.is_empty() {
response.text.unwrap_or_default()
} else {
text
};
// Store in response cache
if let (Some(ref cache), Some(ref key)) = (&self.response_cache, &cache_key) {
let token_count = response
.usage
.as_ref()
.and_then(|u| u.output_tokens)
.unwrap_or(0);
#[allow(clippy::cast_possible_truncation)]
let _ = cache.put(key, &effective_model, &final_text, token_count as u32);
}
// If we didn't stream, send the full response as a single chunk
if !got_stream && !final_text.is_empty() {
let _ = event_tx
.send(TurnEvent::Chunk {
delta: final_text.clone(),
})
.await;
}
self.history
.push(ConversationMessage::Chat(ChatMessage::assistant(
final_text.clone(),
)));
self.trim_history();
return Ok(final_text);
}
// ── Tool calls ─────────────────────────────────────────────
if !text.is_empty() {
self.history
.push(ConversationMessage::Chat(ChatMessage::assistant(
text.clone(),
)));
}
self.history.push(ConversationMessage::AssistantToolCalls {
text: response.text.clone(),
tool_calls: response.tool_calls.clone(),
reasoning_content: response.reasoning_content.clone(),
});
// Notify about each tool call
for call in &calls {
let _ = event_tx
.send(TurnEvent::ToolCall {
name: call.name.clone(),
args: call.arguments.clone(),
})
.await;
}
let results = self.execute_tools(&calls).await;
// Notify about each tool result
for result in &results {
let _ = event_tx
.send(TurnEvent::ToolResult {
name: result.name.clone(),
output: result.output.clone(),
})
.await;
}
let formatted = self.tool_dispatcher.format_results(&results);
self.history.push(formatted);
self.trim_history();
}
anyhow::bail!(
"Agent exceeded maximum tool iterations ({})",
self.config.max_tool_iterations
)
}
pub async fn run_single(&mut self, message: &str) -> Result<String> {
self.turn(message).await
}

View File

@ -0,0 +1,155 @@
use crate::providers::traits::ChatMessage;
use std::collections::HashSet;
/// Signals extracted from conversation context to guide tool filtering.
#[derive(Debug, Clone)]
pub struct ContextSignals {
/// Tool names likely needed. Empty vec means no filtering.
pub suggested_tools: Vec<String>,
/// Whether full history is relevant.
pub history_relevant: bool,
}
/// Analyze context to determine which tools are likely needed.
pub fn analyze_turn_context(
history: &[ChatMessage],
_user_message: &str,
iteration: usize,
last_tool_calls: &[String],
) -> ContextSignals {
if iteration == 0 {
return ContextSignals {
suggested_tools: Vec::new(),
history_relevant: true,
};
}
let mut tools: HashSet<String> = HashSet::new();
for tool in last_tool_calls {
tools.insert(tool.clone());
}
if let Some(last_assistant) = history.iter().rev().find(|m| m.role == "assistant") {
for word in last_assistant.content.split_whitespace() {
for tool_name in tools_for_keyword(word) {
tools.insert(tool_name.to_string());
}
}
}
let mut suggested: Vec<String> = tools.into_iter().collect();
suggested.sort();
ContextSignals {
suggested_tools: suggested,
history_relevant: true,
}
}
fn tools_for_keyword(keyword: &str) -> &'static [&'static str] {
match keyword.to_lowercase().as_str() {
"file" | "read" | "write" | "edit" | "path" | "directory" => {
&["file_read", "file_write", "file_edit", "glob_search"]
}
"shell" | "command" | "run" | "execute" | "install" | "build" => &["shell"],
"memory" | "remember" | "recall" | "store" | "forget" => &["memory_store", "memory_recall"],
"search" | "find" | "grep" | "look" => {
&["content_search", "glob_search", "web_search_tool"]
}
"browser" | "website" | "url" | "http" | "fetch" => &["web_fetch", "web_search_tool"],
"image" | "screenshot" | "picture" => &["image_info"],
"git" | "commit" | "branch" | "push" | "pull" => &["git_operations", "shell"],
_ => &[],
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_message(role: &str, content: &str) -> ChatMessage {
ChatMessage {
role: role.to_string(),
content: content.to_string(),
}
}
#[test]
fn iteration_zero_returns_empty_suggestions() {
let history = vec![make_message("user", "hello")];
let signals = analyze_turn_context(&history, "do something", 0, &[]);
assert!(signals.suggested_tools.is_empty());
assert!(signals.history_relevant);
}
#[test]
fn iteration_one_includes_last_tools() {
let history = vec![
make_message("user", "hello"),
make_message("assistant", "sure"),
];
let last_tools = vec!["shell".to_string(), "file_read".to_string()];
let signals = analyze_turn_context(&history, "next step", 1, &last_tools);
assert!(signals.suggested_tools.contains(&"shell".to_string()));
assert!(signals.suggested_tools.contains(&"file_read".to_string()));
}
#[test]
fn keyword_extraction_from_assistant_message() {
let history = vec![
make_message("user", "help me"),
make_message("assistant", "I will read the file at that path"),
];
let signals = analyze_turn_context(&history, "ok", 1, &[]);
assert!(signals.suggested_tools.contains(&"file_read".to_string()));
}
#[test]
fn shell_keywords_suggest_shell_tool() {
let history = vec![
make_message("user", "build the project"),
make_message("assistant", "I will run the build command"),
];
let signals = analyze_turn_context(&history, "go", 1, &[]);
assert!(signals.suggested_tools.contains(&"shell".to_string()));
}
#[test]
fn memory_keywords_suggest_memory_tools() {
let history = vec![
make_message("user", "save this"),
make_message("assistant", "I will store that in memory"),
];
let signals = analyze_turn_context(&history, "ok", 1, &[]);
assert!(signals
.suggested_tools
.contains(&"memory_store".to_string()));
assert!(signals
.suggested_tools
.contains(&"memory_recall".to_string()));
}
#[test]
fn combined_keywords_merge_tools() {
let history = vec![
make_message("user", "do stuff"),
make_message(
"assistant",
"I need to read the file and run a shell command to search",
),
];
let signals = analyze_turn_context(&history, "go", 1, &[]);
assert!(signals.suggested_tools.contains(&"file_read".to_string()));
assert!(signals.suggested_tools.contains(&"shell".to_string()));
assert!(signals
.suggested_tools
.contains(&"content_search".to_string()));
}
#[test]
fn empty_history_iteration_one() {
let history: Vec<ChatMessage> = vec![];
let signals = analyze_turn_context(&history, "hello", 1, &[]);
assert!(signals.suggested_tools.is_empty());
}
}

415
src/agent/eval.rs Normal file
View File

@ -0,0 +1,415 @@
use serde::{Deserialize, Serialize};
use schemars::JsonSchema;
// ── Complexity estimation ───────────────────────────────────────
/// Coarse complexity tier for a user message.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ComplexityTier {
/// Short, simple query (greetings, yes/no, lookups).
Simple,
/// Typical request — not trivially simple, not deeply complex.
Standard,
/// Long or reasoning-heavy request (code, multi-step, analysis).
Complex,
}
/// Heuristic keywords that signal reasoning complexity.
const REASONING_KEYWORDS: &[&str] = &[
"explain",
"why",
"analyze",
"compare",
"design",
"implement",
"refactor",
"debug",
"optimize",
"architecture",
"trade-off",
"tradeoff",
"reasoning",
"step by step",
"think through",
"evaluate",
"critique",
"pros and cons",
];
/// Estimate the complexity of a user message without an LLM call.
///
/// Rules (applied in order):
/// - **Complex**: message > 200 chars, OR contains a code fence, OR ≥ 2
/// reasoning keywords.
/// - **Simple**: message < 50 chars AND no reasoning keywords.
/// - **Standard**: everything else.
pub fn estimate_complexity(message: &str) -> ComplexityTier {
let lower = message.to_lowercase();
let len = message.len();
let keyword_count = REASONING_KEYWORDS
.iter()
.filter(|kw| lower.contains(**kw))
.count();
let has_code_fence = message.contains("```");
if len > 200 || has_code_fence || keyword_count >= 2 {
return ComplexityTier::Complex;
}
if len < 50 && keyword_count == 0 {
return ComplexityTier::Simple;
}
ComplexityTier::Standard
}
// ── Auto-classify config ────────────────────────────────────────
/// Configuration for automatic complexity-based classification.
///
/// When the rule-based classifier in `QueryClassificationConfig` produces no
/// match, the eval layer can fall back to `estimate_complexity` and map the
/// resulting tier to a routing hint.
#[derive(Debug, Clone, Default, Serialize, Deserialize, JsonSchema)]
pub struct AutoClassifyConfig {
/// Hint to use for `Simple` complexity tier (e.g. `"fast"`).
#[serde(default)]
pub simple_hint: Option<String>,
/// Hint to use for `Standard` complexity tier.
#[serde(default)]
pub standard_hint: Option<String>,
/// Hint to use for `Complex` complexity tier (e.g. `"reasoning"`).
#[serde(default)]
pub complex_hint: Option<String>,
}
impl AutoClassifyConfig {
/// Map a complexity tier to the configured hint, if any.
pub fn hint_for(&self, tier: ComplexityTier) -> Option<&str> {
match tier {
ComplexityTier::Simple => self.simple_hint.as_deref(),
ComplexityTier::Standard => self.standard_hint.as_deref(),
ComplexityTier::Complex => self.complex_hint.as_deref(),
}
}
}
// ── Post-response eval ──────────────────────────────────────────
/// Configuration for the post-response quality evaluator.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct EvalConfig {
/// Enable the eval quality gate.
#[serde(default)]
pub enabled: bool,
/// Minimum quality score (0.01.0) to accept a response.
/// Below this threshold, a retry with a higher-tier model is suggested.
#[serde(default = "default_min_quality_score")]
pub min_quality_score: f64,
/// Maximum retries with escalated models before accepting whatever we get.
#[serde(default = "default_max_retries")]
pub max_retries: u32,
}
fn default_min_quality_score() -> f64 {
0.5
}
fn default_max_retries() -> u32 {
1
}
impl Default for EvalConfig {
fn default() -> Self {
Self {
enabled: false,
min_quality_score: default_min_quality_score(),
max_retries: default_max_retries(),
}
}
}
/// Result of evaluating a response against quality heuristics.
#[derive(Debug, Clone)]
pub struct EvalResult {
/// Aggregate quality score from 0.0 (terrible) to 1.0 (excellent).
pub score: f64,
/// Individual check outcomes (for observability).
pub checks: Vec<EvalCheck>,
/// If score < threshold, the suggested higher-tier hint for retry.
pub retry_hint: Option<String>,
}
#[derive(Debug, Clone)]
pub struct EvalCheck {
pub name: &'static str,
pub passed: bool,
pub weight: f64,
}
/// Code-related keywords in user queries.
const CODE_KEYWORDS: &[&str] = &[
"code",
"function",
"implement",
"class",
"struct",
"module",
"script",
"program",
"bug",
"error",
"compile",
"syntax",
"refactor",
];
/// Evaluate a response against heuristic quality checks. No LLM call.
///
/// Checks:
/// 1. **Non-empty**: response must not be empty.
/// 2. **Not a cop-out**: response must not be just "I don't know" or similar.
/// 3. **Sufficient length**: response length should be proportional to query complexity.
/// 4. **Code presence**: if the query mentions code keywords, the response should
/// contain a code block.
pub fn evaluate_response(
query: &str,
response: &str,
complexity: ComplexityTier,
auto_classify: Option<&AutoClassifyConfig>,
) -> EvalResult {
let mut checks = Vec::new();
// Check 1: Non-empty
let non_empty = !response.trim().is_empty();
checks.push(EvalCheck {
name: "non_empty",
passed: non_empty,
weight: 0.3,
});
// Check 2: Not a cop-out
let lower_resp = response.to_lowercase();
let cop_out_phrases = [
"i don't know",
"i'm not sure",
"i cannot",
"i can't help",
"as an ai",
];
let is_cop_out = cop_out_phrases
.iter()
.any(|phrase| lower_resp.starts_with(phrase));
let not_cop_out = !is_cop_out || response.len() > 200; // long responses with caveats are fine
checks.push(EvalCheck {
name: "not_cop_out",
passed: not_cop_out,
weight: 0.25,
});
// Check 3: Sufficient length for complexity
let min_len = match complexity {
ComplexityTier::Simple => 5,
ComplexityTier::Standard => 20,
ComplexityTier::Complex => 50,
};
let sufficient_length = response.len() >= min_len;
checks.push(EvalCheck {
name: "sufficient_length",
passed: sufficient_length,
weight: 0.2,
});
// Check 4: Code presence when expected
let query_lower = query.to_lowercase();
let expects_code = CODE_KEYWORDS.iter().any(|kw| query_lower.contains(kw));
let has_code = response.contains("```") || response.contains(" "); // code block or indented
let code_check_passed = !expects_code || has_code;
checks.push(EvalCheck {
name: "code_presence",
passed: code_check_passed,
weight: 0.25,
});
// Compute weighted score
let total_weight: f64 = checks.iter().map(|c| c.weight).sum();
let earned: f64 = checks.iter().filter(|c| c.passed).map(|c| c.weight).sum();
let score = if total_weight > 0.0 {
earned / total_weight
} else {
1.0
};
// Determine retry hint: if score is low, suggest escalating
let retry_hint = if score <= default_min_quality_score() {
// Try to escalate: Simple→Standard→Complex
let next_tier = match complexity {
ComplexityTier::Simple => Some(ComplexityTier::Standard),
ComplexityTier::Standard => Some(ComplexityTier::Complex),
ComplexityTier::Complex => None, // already at max
};
next_tier.and_then(|tier| {
auto_classify
.and_then(|ac| ac.hint_for(tier))
.map(String::from)
})
} else {
None
};
EvalResult {
score,
checks,
retry_hint,
}
}
#[cfg(test)]
mod tests {
use super::*;
// ── estimate_complexity ─────────────────────────────────────
#[test]
fn simple_short_message() {
assert_eq!(estimate_complexity("hi"), ComplexityTier::Simple);
assert_eq!(estimate_complexity("hello"), ComplexityTier::Simple);
assert_eq!(estimate_complexity("yes"), ComplexityTier::Simple);
}
#[test]
fn complex_long_message() {
let long = "a".repeat(201);
assert_eq!(estimate_complexity(&long), ComplexityTier::Complex);
}
#[test]
fn complex_code_fence() {
let msg = "Here is some code:\n```rust\nfn main() {}\n```";
assert_eq!(estimate_complexity(msg), ComplexityTier::Complex);
}
#[test]
fn complex_multiple_reasoning_keywords() {
let msg = "Please explain why this design is better and analyze the trade-off";
assert_eq!(estimate_complexity(msg), ComplexityTier::Complex);
}
#[test]
fn standard_medium_message() {
// 50+ chars but no code fence, < 2 reasoning keywords
let msg = "Can you help me find a good restaurant in this area please?";
assert_eq!(estimate_complexity(msg), ComplexityTier::Standard);
}
#[test]
fn standard_short_with_one_keyword() {
// < 50 chars but has 1 reasoning keyword → still not Simple
let msg = "explain this";
assert_eq!(estimate_complexity(msg), ComplexityTier::Standard);
}
// ── auto_classify ───────────────────────────────────────────
#[test]
fn auto_classify_maps_tiers_to_hints() {
let ac = AutoClassifyConfig {
simple_hint: Some("fast".into()),
standard_hint: None,
complex_hint: Some("reasoning".into()),
};
assert_eq!(ac.hint_for(ComplexityTier::Simple), Some("fast"));
assert_eq!(ac.hint_for(ComplexityTier::Standard), None);
assert_eq!(ac.hint_for(ComplexityTier::Complex), Some("reasoning"));
}
// ── evaluate_response ───────────────────────────────────────
#[test]
fn empty_response_scores_low() {
let result = evaluate_response("hello", "", ComplexityTier::Simple, None);
assert!(result.score <= 0.5, "empty response should score low");
}
#[test]
fn good_response_scores_high() {
let result = evaluate_response(
"what is 2+2?",
"The answer is 4.",
ComplexityTier::Simple,
None,
);
assert!(
result.score >= 0.9,
"good simple response should score high, got {}",
result.score
);
}
#[test]
fn cop_out_response_penalized() {
let result = evaluate_response(
"explain quantum computing",
"I don't know much about that.",
ComplexityTier::Standard,
None,
);
assert!(
result.score < 1.0,
"cop-out should be penalized, got {}",
result.score
);
}
#[test]
fn code_query_without_code_response_penalized() {
let result = evaluate_response(
"write a function to sort an array",
"You should use a sorting algorithm.",
ComplexityTier::Standard,
None,
);
// "code_presence" check should fail
let code_check = result.checks.iter().find(|c| c.name == "code_presence");
assert!(
code_check.is_some() && !code_check.unwrap().passed,
"code check should fail"
);
}
#[test]
fn retry_hint_escalation() {
let ac = AutoClassifyConfig {
simple_hint: Some("fast".into()),
standard_hint: Some("default".into()),
complex_hint: Some("reasoning".into()),
};
// Empty response for a Simple query → should suggest Standard hint
let result = evaluate_response("hello", "", ComplexityTier::Simple, Some(&ac));
assert_eq!(result.retry_hint, Some("default".into()));
}
#[test]
fn no_retry_when_already_complex() {
let ac = AutoClassifyConfig {
simple_hint: Some("fast".into()),
standard_hint: Some("default".into()),
complex_hint: Some("reasoning".into()),
};
// Empty response for Complex → no escalation possible
let result =
evaluate_response("explain everything", "", ComplexityTier::Complex, Some(&ac));
assert_eq!(result.retry_hint, None);
}
#[test]
fn max_retries_defaults() {
let config = EvalConfig::default();
assert!(!config.enabled);
assert_eq!(config.max_retries, 1);
assert!((config.min_quality_score - 0.5).abs() < f64::EPSILON);
}
}

283
src/agent/history_pruner.rs Normal file
View File

@ -0,0 +1,283 @@
use crate::providers::traits::ChatMessage;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
// ---------------------------------------------------------------------------
// Config
// ---------------------------------------------------------------------------
fn default_max_tokens() -> usize {
8192
}
fn default_keep_recent() -> usize {
4
}
fn default_collapse() -> bool {
true
}
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct HistoryPrunerConfig {
/// Enable history pruning. Default: false.
#[serde(default)]
pub enabled: bool,
/// Maximum estimated tokens for message history. Default: 8192.
#[serde(default = "default_max_tokens")]
pub max_tokens: usize,
/// Keep the N most recent messages untouched. Default: 4.
#[serde(default = "default_keep_recent")]
pub keep_recent: usize,
/// Collapse old tool call/result pairs into short summaries. Default: true.
#[serde(default = "default_collapse")]
pub collapse_tool_results: bool,
}
impl Default for HistoryPrunerConfig {
fn default() -> Self {
Self {
enabled: false,
max_tokens: 8192,
keep_recent: 4,
collapse_tool_results: true,
}
}
}
// ---------------------------------------------------------------------------
// Stats
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PruneStats {
pub messages_before: usize,
pub messages_after: usize,
pub collapsed_pairs: usize,
pub dropped_messages: usize,
}
// ---------------------------------------------------------------------------
// Token estimation
// ---------------------------------------------------------------------------
fn estimate_tokens(messages: &[ChatMessage]) -> usize {
messages.iter().map(|m| m.content.len() / 4).sum()
}
// ---------------------------------------------------------------------------
// Protected-index helpers
// ---------------------------------------------------------------------------
fn protected_indices(messages: &[ChatMessage], keep_recent: usize) -> Vec<bool> {
let len = messages.len();
let mut protected = vec![false; len];
for (i, msg) in messages.iter().enumerate() {
if msg.role == "system" {
protected[i] = true;
}
}
let recent_start = len.saturating_sub(keep_recent);
for p in protected.iter_mut().skip(recent_start) {
*p = true;
}
protected
}
// ---------------------------------------------------------------------------
// Public entry point
// ---------------------------------------------------------------------------
pub fn prune_history(messages: &mut Vec<ChatMessage>, config: &HistoryPrunerConfig) -> PruneStats {
let messages_before = messages.len();
if !config.enabled || messages.is_empty() {
return PruneStats {
messages_before,
messages_after: messages_before,
collapsed_pairs: 0,
dropped_messages: 0,
};
}
let mut collapsed_pairs: usize = 0;
// Phase 1 collapse assistant+tool pairs
if config.collapse_tool_results {
let mut i = 0;
while i + 1 < messages.len() {
let protected = protected_indices(messages, config.keep_recent);
if messages[i].role == "assistant"
&& messages[i + 1].role == "tool"
&& !protected[i]
&& !protected[i + 1]
{
let tool_content = &messages[i + 1].content;
let truncated: String = tool_content.chars().take(100).collect();
let summary = format!("[Tool result: {truncated}...]");
messages[i] = ChatMessage {
role: "assistant".to_string(),
content: summary,
};
messages.remove(i + 1);
collapsed_pairs += 1;
} else {
i += 1;
}
}
}
// Phase 2 budget enforcement
let mut dropped_messages: usize = 0;
while estimate_tokens(messages) > config.max_tokens {
let protected = protected_indices(messages, config.keep_recent);
if let Some(idx) = protected
.iter()
.enumerate()
.find(|(_, &p)| !p)
.map(|(i, _)| i)
{
messages.remove(idx);
dropped_messages += 1;
} else {
break;
}
}
PruneStats {
messages_before,
messages_after: messages.len(),
collapsed_pairs,
dropped_messages,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn msg(role: &str, content: &str) -> ChatMessage {
ChatMessage {
role: role.to_string(),
content: content.to_string(),
}
}
#[test]
fn prune_disabled_is_noop() {
let mut messages = vec![
msg("system", "You are helpful."),
msg("user", "Hello"),
msg("assistant", "Hi there!"),
];
let config = HistoryPrunerConfig {
enabled: false,
..Default::default()
};
let stats = prune_history(&mut messages, &config);
assert_eq!(messages.len(), 3);
assert_eq!(messages[0].content, "You are helpful.");
assert_eq!(stats.messages_before, 3);
assert_eq!(stats.messages_after, 3);
assert_eq!(stats.collapsed_pairs, 0);
}
#[test]
fn prune_under_budget_no_change() {
let mut messages = vec![
msg("system", "You are helpful."),
msg("user", "Hello"),
msg("assistant", "Hi!"),
];
let config = HistoryPrunerConfig {
enabled: true,
max_tokens: 8192,
keep_recent: 2,
collapse_tool_results: false,
};
let stats = prune_history(&mut messages, &config);
assert_eq!(messages.len(), 3);
assert_eq!(stats.collapsed_pairs, 0);
assert_eq!(stats.dropped_messages, 0);
}
#[test]
fn prune_collapses_tool_pairs() {
let tool_result = "a".repeat(160);
let mut messages = vec![
msg("system", "sys"),
msg("assistant", "calling tool X"),
msg("tool", &tool_result),
msg("user", "thanks"),
msg("assistant", "done"),
];
let config = HistoryPrunerConfig {
enabled: true,
max_tokens: 100_000,
keep_recent: 2,
collapse_tool_results: true,
};
let stats = prune_history(&mut messages, &config);
assert_eq!(stats.collapsed_pairs, 1);
assert_eq!(messages.len(), 4);
assert_eq!(messages[1].role, "assistant");
assert!(messages[1].content.starts_with("[Tool result: "));
}
#[test]
fn prune_preserves_system_and_recent() {
let big = "x".repeat(40_000);
let mut messages = vec![
msg("system", "system prompt"),
msg("user", &big),
msg("assistant", "old reply"),
msg("user", "recent1"),
msg("assistant", "recent2"),
];
let config = HistoryPrunerConfig {
enabled: true,
max_tokens: 100,
keep_recent: 2,
collapse_tool_results: false,
};
let stats = prune_history(&mut messages, &config);
assert!(messages.iter().any(|m| m.role == "system"));
assert!(messages.iter().any(|m| m.content == "recent1"));
assert!(messages.iter().any(|m| m.content == "recent2"));
assert!(stats.dropped_messages > 0);
}
#[test]
fn prune_drops_oldest_when_over_budget() {
let filler = "y".repeat(400);
let mut messages = vec![
msg("system", "sys"),
msg("user", &filler),
msg("assistant", &filler),
msg("user", "recent-user"),
msg("assistant", "recent-assistant"),
];
let config = HistoryPrunerConfig {
enabled: true,
max_tokens: 150,
keep_recent: 2,
collapse_tool_results: false,
};
let stats = prune_history(&mut messages, &config);
assert!(stats.dropped_messages >= 1);
assert_eq!(messages[0].role, "system");
assert!(messages.iter().any(|m| m.content == "recent-user"));
assert!(messages.iter().any(|m| m.content == "recent-assistant"));
}
#[test]
fn prune_empty_messages() {
let mut messages: Vec<ChatMessage> = vec![];
let config = HistoryPrunerConfig {
enabled: true,
..Default::default()
};
let stats = prune_history(&mut messages, &config);
assert_eq!(stats.messages_before, 0);
assert_eq!(stats.messages_after, 0);
}
}

View File

@ -4,7 +4,7 @@ use crate::config::Config;
use crate::cost::types::{BudgetCheck, TokenUsage as CostTokenUsage};
use crate::cost::CostTracker;
use crate::i18n::ToolDescriptions;
use crate::memory::{self, Memory, MemoryCategory};
use crate::memory::{self, decay, Memory, MemoryCategory};
use crate::multimodal;
use crate::observability::{self, runtime_trace, Observer, ObserverEvent};
use crate::providers::{
@ -561,6 +561,7 @@ fn save_interactive_session_history(path: &Path, history: &[ChatMessage]) -> Res
/// Build context preamble by searching memory for relevant entries.
/// Entries with a hybrid score below `min_relevance_score` are dropped to
/// prevent unrelated memories from bleeding into the conversation.
/// Core memories are exempt from time decay (evergreen).
async fn build_context(
mem: &dyn Memory,
user_msg: &str,
@ -570,7 +571,10 @@ async fn build_context(
let mut context = String::new();
// Pull relevant memories for this message
if let Ok(entries) = mem.recall(user_msg, 5, session_id, None, None).await {
if let Ok(mut entries) = mem.recall(user_msg, 5, session_id, None, None).await {
// Apply time decay: older non-Core memories score lower
decay::apply_time_decay(&mut entries, decay::DEFAULT_HALF_LIFE_DAYS);
let relevant: Vec<_> = entries
.iter()
.filter(|e| match e.score {
@ -2659,6 +2663,14 @@ pub(crate) async fn run_tool_call_loop(
let mut consecutive_identical_outputs: usize = 0;
let mut last_tool_output_hash: Option<u64> = None;
let mut loop_detector = crate::agent::loop_detector::LoopDetector::new(
crate::agent::loop_detector::LoopDetectorConfig {
enabled: pacing.loop_detection_enabled,
window_size: pacing.loop_detection_window_size,
max_repeats: pacing.loop_detection_max_repeats,
},
);
for iteration in 0..max_iterations {
let mut seen_tool_signatures: HashSet<(String, String)> = HashSet::new();
@ -2707,16 +2719,53 @@ pub(crate) async fn run_tool_call_loop(
let use_native_tools = provider.supports_native_tools() && !tool_specs.is_empty();
let image_marker_count = multimodal::count_image_markers(history);
if image_marker_count > 0 && !provider.supports_vision() {
return Err(ProviderCapabilityError {
provider: provider_name.to_string(),
capability: "vision".to_string(),
message: format!(
"received {image_marker_count} image marker(s), but this provider does not support vision input"
),
// ── Vision provider routing ──────────────────────────
// When the default provider lacks vision support but a dedicated
// vision_provider is configured, create it on demand and use it
// for this iteration. Otherwise, preserve the original error.
let vision_provider_box: Option<Box<dyn Provider>> = if image_marker_count > 0
&& !provider.supports_vision()
{
if let Some(ref vp) = multimodal_config.vision_provider {
let vp_instance = providers::create_provider(vp, None)
.map_err(|e| anyhow::anyhow!("failed to create vision provider '{vp}': {e}"))?;
if !vp_instance.supports_vision() {
return Err(ProviderCapabilityError {
provider: vp.clone(),
capability: "vision".to_string(),
message: format!(
"configured vision_provider '{vp}' does not support vision input"
),
}
.into());
}
Some(vp_instance)
} else {
return Err(ProviderCapabilityError {
provider: provider_name.to_string(),
capability: "vision".to_string(),
message: format!(
"received {image_marker_count} image marker(s), but this provider does not support vision input"
),
}
.into());
}
.into());
}
} else {
None
};
let (active_provider, active_provider_name, active_model): (&dyn Provider, &str, &str) =
if let Some(ref vp_box) = vision_provider_box {
let vp_name = multimodal_config
.vision_provider
.as_deref()
.unwrap_or(provider_name);
let vm = multimodal_config.vision_model.as_deref().unwrap_or(model);
(vp_box.as_ref(), vp_name, vm)
} else {
(provider, provider_name, model)
};
let prepared_messages =
multimodal::prepare_messages_for_provider(history, multimodal_config).await?;
@ -2732,15 +2781,15 @@ pub(crate) async fn run_tool_call_loop(
}
observer.record_event(&ObserverEvent::LlmRequest {
provider: provider_name.to_string(),
model: model.to_string(),
provider: active_provider_name.to_string(),
model: active_model.to_string(),
messages_count: history.len(),
});
runtime_trace::record_event(
"llm_request",
Some(channel_name),
Some(provider_name),
Some(model),
Some(active_provider_name),
Some(active_model),
Some(&turn_id),
None,
None,
@ -2778,12 +2827,12 @@ pub(crate) async fn run_tool_call_loop(
None
};
let chat_future = provider.chat(
let chat_future = active_provider.chat(
ChatRequest {
messages: &prepared_messages.messages,
tools: request_tools,
},
model,
active_model,
temperature,
);
@ -2836,8 +2885,8 @@ pub(crate) async fn run_tool_call_loop(
.unwrap_or((None, None));
observer.record_event(&ObserverEvent::LlmResponse {
provider: provider_name.to_string(),
model: model.to_string(),
provider: active_provider_name.to_string(),
model: active_model.to_string(),
duration: llm_started_at.elapsed(),
success: true,
error_message: None,
@ -2846,10 +2895,9 @@ pub(crate) async fn run_tool_call_loop(
});
// Record cost via task-local tracker (no-op when not scoped)
let _ = resp
.usage
.as_ref()
.and_then(|usage| record_tool_loop_cost_usage(provider_name, model, usage));
let _ = resp.usage.as_ref().and_then(|usage| {
record_tool_loop_cost_usage(active_provider_name, active_model, usage)
});
let response_text = resp.text_or_empty().to_string();
// First try native structured tool calls (OpenAI-format).
@ -2872,8 +2920,8 @@ pub(crate) async fn run_tool_call_loop(
runtime_trace::record_event(
"tool_call_parse_issue",
Some(channel_name),
Some(provider_name),
Some(model),
Some(active_provider_name),
Some(active_model),
Some(&turn_id),
Some(false),
Some(&parse_issue),
@ -2890,8 +2938,8 @@ pub(crate) async fn run_tool_call_loop(
runtime_trace::record_event(
"llm_response",
Some(channel_name),
Some(provider_name),
Some(model),
Some(active_provider_name),
Some(active_model),
Some(&turn_id),
Some(true),
None,
@ -2940,8 +2988,8 @@ pub(crate) async fn run_tool_call_loop(
Err(e) => {
let safe_error = crate::providers::sanitize_api_error(&e.to_string());
observer.record_event(&ObserverEvent::LlmResponse {
provider: provider_name.to_string(),
model: model.to_string(),
provider: active_provider_name.to_string(),
model: active_model.to_string(),
duration: llm_started_at.elapsed(),
success: false,
error_message: Some(safe_error.clone()),
@ -2951,8 +2999,8 @@ pub(crate) async fn run_tool_call_loop(
runtime_trace::record_event(
"llm_response",
Some(channel_name),
Some(provider_name),
Some(model),
Some(active_provider_name),
Some(active_model),
Some(&turn_id),
Some(false),
Some(&safe_error),
@ -3036,7 +3084,11 @@ pub(crate) async fn run_tool_call_loop(
if !display_text.is_empty() {
if !native_tool_calls.is_empty() {
if let Some(ref tx) = on_delta {
let _ = tx.send(display_text.clone()).await;
let mut narration = display_text.clone();
if !narration.ends_with('\n') {
narration.push('\n');
}
let _ = tx.send(narration).await;
}
}
if !silent {
@ -3325,9 +3377,54 @@ pub(crate) async fn run_tool_call_loop(
// Collect tool results and build per-tool output for loop detection.
// Only non-ignored tool outputs contribute to the identical-output hash.
let mut detection_relevant_output = String::new();
for (tool_name, tool_call_id, outcome) in ordered_results.into_iter().flatten() {
// Use enumerate *before* filter_map so result_index stays aligned with
// tool_calls even when some ordered_results entries are None.
for (result_index, (tool_name, tool_call_id, outcome)) in ordered_results
.into_iter()
.enumerate()
.filter_map(|(i, opt)| opt.map(|v| (i, v)))
{
if !loop_ignore_tools.contains(tool_name.as_str()) {
detection_relevant_output.push_str(&outcome.output);
// Feed the pattern-based loop detector with name + args + result.
let args = tool_calls
.get(result_index)
.map(|c| &c.arguments)
.unwrap_or(&serde_json::Value::Null);
let det_result = loop_detector.record(&tool_name, args, &outcome.output);
match det_result {
crate::agent::loop_detector::LoopDetectionResult::Ok => {}
crate::agent::loop_detector::LoopDetectionResult::Warning(ref msg) => {
tracing::warn!(tool = %tool_name, %msg, "loop detector warning");
// Inject a system nudge so the LLM adjusts strategy.
history.push(ChatMessage::system(format!("[Loop Detection] {msg}")));
}
crate::agent::loop_detector::LoopDetectionResult::Block(ref msg) => {
tracing::warn!(tool = %tool_name, %msg, "loop detector blocked tool call");
// Replace the tool output with the block message.
// We still continue the loop so the LLM sees the block feedback.
history.push(ChatMessage::system(format!(
"[Loop Detection — BLOCKED] {msg}"
)));
}
crate::agent::loop_detector::LoopDetectionResult::Break(msg) => {
runtime_trace::record_event(
"loop_detector_circuit_breaker",
Some(channel_name),
Some(provider_name),
Some(model),
Some(&turn_id),
Some(false),
Some(&msg),
serde_json::json!({
"iteration": iteration + 1,
"tool": tool_name,
}),
);
anyhow::bail!("Agent loop aborted by loop detector: {msg}");
}
}
}
individual_results.push((tool_call_id, outcome.output.clone()));
let _ = writeln!(
@ -3525,7 +3622,13 @@ pub async fn run(
} else {
(None, None)
};
let (mut tools_registry, delegate_handle, _reaction_handle) = tools::all_tools_with_runtime(
let (
mut tools_registry,
delegate_handle,
_reaction_handle,
_channel_map_handle,
_ask_user_handle,
) = tools::all_tools_with_runtime(
Arc::new(config.clone()),
&security,
runtime,
@ -3701,6 +3804,11 @@ pub async fn run(
// ── Build system prompt from workspace MD files (OpenClaw framework) ──
let skills = crate::skills::load_skills_with_config(&config.workspace_dir, &config);
// Register skill-defined tools as callable tool specs in the tool registry
// so the LLM can invoke them via native function calling, not just XML prompts.
tools::register_skill_tools(&mut tools_registry, &skills, security.clone());
let mut tool_descs: Vec<(&str, &str)> = vec![
(
"shell",
@ -3865,17 +3973,45 @@ pub async fn run(
let mut final_output = String::new();
// Save the base system prompt before any thinking modifications so
// the interactive loop can restore it between turns.
let base_system_prompt = system_prompt.clone();
if let Some(msg) = message {
// ── Parse thinking directive from user message ─────────
let (thinking_directive, effective_msg) =
match crate::agent::thinking::parse_thinking_directive(&msg) {
Some((level, remaining)) => {
tracing::info!(thinking_level = ?level, "Thinking directive parsed from message");
(Some(level), remaining)
}
None => (None, msg.clone()),
};
let thinking_level = crate::agent::thinking::resolve_thinking_level(
thinking_directive,
None,
&config.agent.thinking,
);
let thinking_params = crate::agent::thinking::apply_thinking_level(thinking_level);
let effective_temperature = crate::agent::thinking::clamp_temperature(
temperature + thinking_params.temperature_adjustment,
);
// Prepend thinking system prompt prefix when present.
if let Some(ref prefix) = thinking_params.system_prompt_prefix {
system_prompt = format!("{prefix}\n\n{system_prompt}");
}
// Auto-save user message to memory (skip short/trivial messages)
if config.memory.auto_save
&& msg.chars().count() >= AUTOSAVE_MIN_MESSAGE_CHARS
&& !memory::should_skip_autosave_content(&msg)
&& effective_msg.chars().count() >= AUTOSAVE_MIN_MESSAGE_CHARS
&& !memory::should_skip_autosave_content(&effective_msg)
{
let user_key = autosave_memory_key("user_msg");
let _ = mem
.store(
&user_key,
&msg,
&effective_msg,
MemoryCategory::Conversation,
memory_session_id.as_deref(),
)
@ -3885,7 +4021,7 @@ pub async fn run(
// Inject memory + hardware RAG context into user message
let mem_context = build_context(
mem.as_ref(),
&msg,
&effective_msg,
config.memory.min_relevance_score,
memory_session_id.as_deref(),
)
@ -3893,14 +4029,14 @@ pub async fn run(
let rag_limit = if config.agent.compact_context { 2 } else { 5 };
let hw_context = hardware_rag
.as_ref()
.map(|r| build_hardware_context(r, &msg, &board_names, rag_limit))
.map(|r| build_hardware_context(r, &effective_msg, &board_names, rag_limit))
.unwrap_or_default();
let context = format!("{mem_context}{hw_context}");
let now = chrono::Local::now().format("%Y-%m-%d %H:%M:%S %Z");
let enriched = if context.is_empty() {
format!("[{now}] {msg}")
format!("[{now}] {effective_msg}")
} else {
format!("{context}[{now}] {msg}")
format!("{context}[{now}] {effective_msg}")
};
let mut history = vec![
@ -3908,9 +4044,20 @@ pub async fn run(
ChatMessage::user(&enriched),
];
// Prune history for token efficiency (when enabled).
if config.agent.history_pruning.enabled {
let _stats = crate::agent::history_pruner::prune_history(
&mut history,
&config.agent.history_pruning,
);
}
// Compute per-turn excluded MCP tools from tool_filter_groups.
let excluded_tools =
compute_excluded_mcp_tools(&tools_registry, &config.agent.tool_filter_groups, &msg);
let excluded_tools = compute_excluded_mcp_tools(
&tools_registry,
&config.agent.tool_filter_groups,
&effective_msg,
);
#[allow(unused_assignments)]
let mut response = String::new();
@ -3922,7 +4069,7 @@ pub async fn run(
observer.as_ref(),
&provider_name,
&model_name,
temperature,
effective_temperature,
false,
approval_manager.as_ref(),
channel_name,
@ -4042,9 +4189,10 @@ pub async fn run(
"/quit" | "/exit" => break,
"/help" => {
println!("Available commands:");
println!(" /help Show this help message");
println!(" /clear /new Clear conversation history");
println!(" /quit /exit Exit interactive mode\n");
println!(" /help Show this help message");
println!(" /clear /new Clear conversation history");
println!(" /quit /exit Exit interactive mode");
println!(" /think:<level> Set reasoning depth (off|minimal|low|medium|high|max)\n");
continue;
}
"/clear" | "/new" => {
@ -4096,16 +4244,47 @@ pub async fn run(
_ => {}
}
// ── Parse thinking directive from interactive input ───
let (thinking_directive, effective_input) =
match crate::agent::thinking::parse_thinking_directive(&user_input) {
Some((level, remaining)) => {
tracing::info!(thinking_level = ?level, "Thinking directive parsed");
(Some(level), remaining)
}
None => (None, user_input.clone()),
};
let thinking_level = crate::agent::thinking::resolve_thinking_level(
thinking_directive,
None,
&config.agent.thinking,
);
let thinking_params = crate::agent::thinking::apply_thinking_level(thinking_level);
let turn_temperature = crate::agent::thinking::clamp_temperature(
temperature + thinking_params.temperature_adjustment,
);
// For non-Medium levels, temporarily patch the system prompt with prefix.
let turn_system_prompt;
if let Some(ref prefix) = thinking_params.system_prompt_prefix {
turn_system_prompt = format!("{prefix}\n\n{system_prompt}");
// Update the system message in history for this turn.
if let Some(sys_msg) = history.first_mut() {
if sys_msg.role == "system" {
sys_msg.content = turn_system_prompt.clone();
}
}
}
// Auto-save conversation turns (skip short/trivial messages)
if config.memory.auto_save
&& user_input.chars().count() >= AUTOSAVE_MIN_MESSAGE_CHARS
&& !memory::should_skip_autosave_content(&user_input)
&& effective_input.chars().count() >= AUTOSAVE_MIN_MESSAGE_CHARS
&& !memory::should_skip_autosave_content(&effective_input)
{
let user_key = autosave_memory_key("user_msg");
let _ = mem
.store(
&user_key,
&user_input,
&effective_input,
MemoryCategory::Conversation,
memory_session_id.as_deref(),
)
@ -4115,7 +4294,7 @@ pub async fn run(
// Inject memory + hardware RAG context into user message
let mem_context = build_context(
mem.as_ref(),
&user_input,
&effective_input,
config.memory.min_relevance_score,
memory_session_id.as_deref(),
)
@ -4123,14 +4302,14 @@ pub async fn run(
let rag_limit = if config.agent.compact_context { 2 } else { 5 };
let hw_context = hardware_rag
.as_ref()
.map(|r| build_hardware_context(r, &user_input, &board_names, rag_limit))
.map(|r| build_hardware_context(r, &effective_input, &board_names, rag_limit))
.unwrap_or_default();
let context = format!("{mem_context}{hw_context}");
let now = chrono::Local::now().format("%Y-%m-%d %H:%M:%S %Z");
let enriched = if context.is_empty() {
format!("[{now}] {user_input}")
format!("[{now}] {effective_input}")
} else {
format!("{context}[{now}] {user_input}")
format!("{context}[{now}] {effective_input}")
};
history.push(ChatMessage::user(&enriched));
@ -4139,7 +4318,7 @@ pub async fn run(
let excluded_tools = compute_excluded_mcp_tools(
&tools_registry,
&config.agent.tool_filter_groups,
&user_input,
&effective_input,
);
let response = loop {
@ -4150,7 +4329,7 @@ pub async fn run(
observer.as_ref(),
&provider_name,
&model_name,
temperature,
turn_temperature,
false,
approval_manager.as_ref(),
channel_name,
@ -4235,6 +4414,15 @@ pub async fn run(
// Hard cap as a safety net.
trim_history(&mut history, config.agent.max_history_messages);
// Restore base system prompt (remove per-turn thinking prefix).
if thinking_params.system_prompt_prefix.is_some() {
if let Some(sys_msg) = history.first_mut() {
if sys_msg.role == "system" {
sys_msg.content.clone_from(&base_system_prompt);
}
}
}
if let Some(path) = session_state_file.as_deref() {
save_interactive_session_history(path, &history)?;
}
@ -4285,23 +4473,28 @@ pub async fn process_message(
} else {
(None, None)
};
let (mut tools_registry, delegate_handle_pm, _reaction_handle_pm) =
tools::all_tools_with_runtime(
Arc::new(config.clone()),
&security,
runtime,
mem.clone(),
composio_key,
composio_entity_id,
&config.browser,
&config.http_request,
&config.web_fetch,
&config.workspace_dir,
&config.agents,
config.api_key.as_deref(),
&config,
None,
);
let (
mut tools_registry,
delegate_handle_pm,
_reaction_handle_pm,
_channel_map_handle_pm,
_ask_user_handle_pm,
) = tools::all_tools_with_runtime(
Arc::new(config.clone()),
&security,
runtime,
mem.clone(),
composio_key,
composio_entity_id,
&config.browser,
&config.http_request,
&config.web_fetch,
&config.workspace_dir,
&config.agents,
config.api_key.as_deref(),
&config,
None,
);
let peripheral_tools: Vec<Box<dyn Tool>> =
crate::peripherals::create_peripheral_tools(&config.peripherals).await?;
tools_registry.extend(peripheral_tools);
@ -4415,6 +4608,10 @@ pub async fn process_message(
let i18n_descs = crate::i18n::ToolDescriptions::load(&i18n_locale, &i18n_search_dirs);
let skills = crate::skills::load_skills_with_config(&config.workspace_dir, &config);
// Register skill-defined tools as callable tool specs (process_message path).
tools::register_skill_tools(&mut tools_registry, &skills, security.clone());
let mut tool_descs: Vec<(&str, &str)> = vec![
("shell", "Execute terminal commands."),
("file_read", "Read file contents."),
@ -4508,9 +4705,34 @@ pub async fn process_message(
system_prompt.push_str(&deferred_section);
}
// ── Parse thinking directive from user message ─────────────
let (thinking_directive, effective_message) =
match crate::agent::thinking::parse_thinking_directive(message) {
Some((level, remaining)) => {
tracing::info!(thinking_level = ?level, "Thinking directive parsed from message");
(Some(level), remaining)
}
None => (None, message.to_string()),
};
let thinking_level = crate::agent::thinking::resolve_thinking_level(
thinking_directive,
None,
&config.agent.thinking,
);
let thinking_params = crate::agent::thinking::apply_thinking_level(thinking_level);
let effective_temperature = crate::agent::thinking::clamp_temperature(
config.default_temperature + thinking_params.temperature_adjustment,
);
// Prepend thinking system prompt prefix when present.
if let Some(ref prefix) = thinking_params.system_prompt_prefix {
system_prompt = format!("{prefix}\n\n{system_prompt}");
}
let effective_msg_ref = effective_message.as_str();
let mem_context = build_context(
mem.as_ref(),
message,
effective_msg_ref,
config.memory.min_relevance_score,
session_id,
)
@ -4518,22 +4740,25 @@ pub async fn process_message(
let rag_limit = if config.agent.compact_context { 2 } else { 5 };
let hw_context = hardware_rag
.as_ref()
.map(|r| build_hardware_context(r, message, &board_names, rag_limit))
.map(|r| build_hardware_context(r, effective_msg_ref, &board_names, rag_limit))
.unwrap_or_default();
let context = format!("{mem_context}{hw_context}");
let now = chrono::Local::now().format("%Y-%m-%d %H:%M:%S %Z");
let enriched = if context.is_empty() {
format!("[{now}] {message}")
format!("[{now}] {effective_message}")
} else {
format!("{context}[{now}] {message}")
format!("{context}[{now}] {effective_message}")
};
let mut history = vec![
ChatMessage::system(&system_prompt),
ChatMessage::user(&enriched),
];
let mut excluded_tools =
compute_excluded_mcp_tools(&tools_registry, &config.agent.tool_filter_groups, message);
let mut excluded_tools = compute_excluded_mcp_tools(
&tools_registry,
&config.agent.tool_filter_groups,
effective_msg_ref,
);
if config.autonomy.level != AutonomyLevel::Full {
excluded_tools.extend(config.autonomy.non_cli_excluded_tools.iter().cloned());
}
@ -4545,7 +4770,7 @@ pub async fn process_message(
observer.as_ref(),
provider_name,
&model_name,
config.default_temperature,
effective_temperature,
true,
"daemon",
None,
@ -5094,6 +5319,7 @@ mod tests {
max_images: 4,
max_image_size_mb: 1,
allow_remote_fetch: false,
..Default::default()
};
let err = run_tool_call_loop(
@ -5171,6 +5397,313 @@ mod tests {
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
/// When `vision_provider` is not set and the default provider lacks vision
/// support, the original `ProviderCapabilityError` should be returned.
#[tokio::test]
async fn run_tool_call_loop_no_vision_provider_config_preserves_error() {
let calls = Arc::new(AtomicUsize::new(0));
let provider = NonVisionProvider {
calls: Arc::clone(&calls),
};
let mut history = vec![ChatMessage::user(
"check [IMAGE:data:image/png;base64,iVBORw0KGgo=]".to_string(),
)];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
let err = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"mock-provider",
"mock-model",
0.0,
true,
None,
"cli",
None,
&crate::config::MultimodalConfig::default(),
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect_err("should fail without vision_provider config");
assert!(err.to_string().contains("capability=vision"));
assert_eq!(calls.load(Ordering::SeqCst), 0);
}
/// When `vision_provider` is set but the provider factory cannot resolve
/// the name, a descriptive error should be returned (not the generic
/// capability error).
#[tokio::test]
async fn run_tool_call_loop_vision_provider_creation_failure() {
let calls = Arc::new(AtomicUsize::new(0));
let provider = NonVisionProvider {
calls: Arc::clone(&calls),
};
let mut history = vec![ChatMessage::user(
"inspect [IMAGE:data:image/png;base64,iVBORw0KGgo=]".to_string(),
)];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
let multimodal = crate::config::MultimodalConfig {
vision_provider: Some("nonexistent-provider-xyz".to_string()),
vision_model: Some("some-model".to_string()),
..Default::default()
};
let err = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"mock-provider",
"mock-model",
0.0,
true,
None,
"cli",
None,
&multimodal,
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect_err("should fail when vision provider cannot be created");
assert!(
err.to_string().contains("failed to create vision provider"),
"expected creation failure error, got: {}",
err
);
assert_eq!(calls.load(Ordering::SeqCst), 0);
}
/// Messages without image markers should use the default provider even
/// when `vision_provider` is configured.
#[tokio::test]
async fn run_tool_call_loop_no_images_uses_default_provider() {
let provider = ScriptedProvider::from_text_responses(vec!["hello world"]);
let mut history = vec![ChatMessage::user("just text, no images".to_string())];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
let multimodal = crate::config::MultimodalConfig {
vision_provider: Some("nonexistent-provider-xyz".to_string()),
vision_model: Some("some-model".to_string()),
..Default::default()
};
// Even though vision_provider points to a nonexistent provider, this
// should succeed because there are no image markers to trigger routing.
let result = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"scripted",
"scripted-model",
0.0,
true,
None,
"cli",
None,
&multimodal,
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect("text-only messages should succeed with default provider");
assert_eq!(result, "hello world");
}
/// When `vision_provider` is set but `vision_model` is not, the default
/// model should be used as fallback for the vision provider.
#[tokio::test]
async fn run_tool_call_loop_vision_provider_without_model_falls_back() {
let calls = Arc::new(AtomicUsize::new(0));
let provider = NonVisionProvider {
calls: Arc::clone(&calls),
};
let mut history = vec![ChatMessage::user(
"look [IMAGE:data:image/png;base64,iVBORw0KGgo=]".to_string(),
)];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
// vision_provider set but vision_model is None — the code should
// fall back to the default model. Since the provider name is invalid,
// we just verify the error path references the correct provider.
let multimodal = crate::config::MultimodalConfig {
vision_provider: Some("nonexistent-provider-xyz".to_string()),
vision_model: None,
..Default::default()
};
let err = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"mock-provider",
"mock-model",
0.0,
true,
None,
"cli",
None,
&multimodal,
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect_err("should fail due to nonexistent vision provider");
// Verify the routing was attempted (not the generic capability error).
assert!(
err.to_string().contains("failed to create vision provider"),
"expected creation failure, got: {}",
err
);
}
/// Empty `[IMAGE:]` markers (which are preserved as literal text by the
/// parser) should not trigger vision provider routing.
#[tokio::test]
async fn run_tool_call_loop_empty_image_markers_use_default_provider() {
let provider = ScriptedProvider::from_text_responses(vec!["handled"]);
let mut history = vec![ChatMessage::user(
"empty marker [IMAGE:] should be ignored".to_string(),
)];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
let multimodal = crate::config::MultimodalConfig {
vision_provider: Some("nonexistent-provider-xyz".to_string()),
..Default::default()
};
let result = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"scripted",
"scripted-model",
0.0,
true,
None,
"cli",
None,
&multimodal,
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect("empty image markers should not trigger vision routing");
assert_eq!(result, "handled");
}
/// Multiple image markers should still trigger vision routing when
/// vision_provider is configured.
#[tokio::test]
async fn run_tool_call_loop_multiple_images_trigger_vision_routing() {
let calls = Arc::new(AtomicUsize::new(0));
let provider = NonVisionProvider {
calls: Arc::clone(&calls),
};
let mut history = vec![ChatMessage::user(
"two images [IMAGE:data:image/png;base64,aQ==] and [IMAGE:data:image/png;base64,bQ==]"
.to_string(),
)];
let tools_registry: Vec<Box<dyn Tool>> = Vec::new();
let observer = NoopObserver;
let multimodal = crate::config::MultimodalConfig {
vision_provider: Some("nonexistent-provider-xyz".to_string()),
vision_model: Some("llava:7b".to_string()),
..Default::default()
};
let err = run_tool_call_loop(
&provider,
&mut history,
&tools_registry,
&observer,
"mock-provider",
"mock-model",
0.0,
true,
None,
"cli",
None,
&multimodal,
3,
None,
None,
None,
&[],
&[],
None,
None,
&crate::config::PacingConfig::default(),
)
.await
.expect_err("should attempt vision provider creation for multiple images");
assert!(
err.to_string().contains("failed to create vision provider"),
"expected creation failure for multiple images, got: {}",
err
);
}
#[test]
fn should_execute_tools_in_parallel_returns_false_for_single_call() {
let calls = vec![ParsedToolCall {
@ -5849,7 +6382,7 @@ mod tests {
let explanation_idx = deltas
.iter()
.position(|delta| delta == "Task started. Waiting 30 seconds before checking status.")
.position(|delta| delta == "Task started. Waiting 30 seconds before checking status.\n")
.expect("native assistant text should be relayed to on_delta");
let clear_idx = deltas
.iter()
@ -7699,6 +8232,7 @@ Let me check the result."#;
mode: ToolFilterGroupMode::Always,
tools: vec!["mcp_filesystem_*".into()],
keywords: vec![],
filter_builtins: false,
}];
let result = filter_tool_specs_for_turn(specs, &groups, "anything");
let names: Vec<&str> = result.iter().map(|s| s.name.as_str()).collect();
@ -7717,6 +8251,7 @@ Let me check the result."#;
mode: ToolFilterGroupMode::Dynamic,
tools: vec!["mcp_browser_*".into()],
keywords: vec!["browse".into(), "website".into()],
filter_builtins: false,
}];
let result = filter_tool_specs_for_turn(specs, &groups, "please browse this page");
let names: Vec<&str> = result.iter().map(|s| s.name.as_str()).collect();
@ -7733,6 +8268,7 @@ Let me check the result."#;
mode: ToolFilterGroupMode::Dynamic,
tools: vec!["mcp_browser_*".into()],
keywords: vec!["browse".into(), "website".into()],
filter_builtins: false,
}];
let result = filter_tool_specs_for_turn(specs, &groups, "read the file /etc/hosts");
let names: Vec<&str> = result.iter().map(|s| s.name.as_str()).collect();
@ -7749,6 +8285,7 @@ Let me check the result."#;
mode: ToolFilterGroupMode::Dynamic,
tools: vec!["mcp_browser_*".into()],
keywords: vec!["Browse".into()],
filter_builtins: false,
}];
let result = filter_tool_specs_for_turn(specs, &groups, "BROWSE the site");
assert_eq!(result.len(), 1);

696
src/agent/loop_detector.rs Normal file
View File

@ -0,0 +1,696 @@
//! Loop detection guardrail for the agent tool-call loop.
//!
//! Monitors a sliding window of recent tool calls and their results to detect
//! three repetitive patterns that indicate the agent is stuck:
//!
//! 1. **Exact repeat** — same tool + args called 3+ times consecutively.
//! 2. **Ping-pong** — two tools alternating (A->B->A->B) for 4+ cycles.
//! 3. **No progress** — same tool called 5+ times with different args but
//! identical result hash each time.
//!
//! Detection triggers escalating responses: `Warning` -> `Block` -> `Break`.
use std::collections::hash_map::DefaultHasher;
use std::collections::VecDeque;
use std::hash::{Hash, Hasher};
// ── Configuration ────────────────────────────────────────────────
/// Configuration for the loop detector, typically derived from
/// `PacingConfig` fields at the call site.
#[derive(Debug, Clone)]
pub(crate) struct LoopDetectorConfig {
/// Master switch. When `false`, `record` always returns `Ok`.
pub enabled: bool,
/// Number of recent calls retained for pattern analysis.
pub window_size: usize,
/// How many consecutive exact-repeat calls before escalation starts.
pub max_repeats: usize,
}
impl Default for LoopDetectorConfig {
fn default() -> Self {
Self {
enabled: true,
window_size: 20,
max_repeats: 3,
}
}
}
// ── Result enum ──────────────────────────────────────────────────
/// Outcome of a loop-detection check after recording a tool call.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum LoopDetectionResult {
/// No pattern detected — continue normally.
Ok,
/// A suspicious pattern was detected; the caller should inject a
/// system-level nudge message into the conversation.
Warning(String),
/// The tool call should be refused (output replaced with an error).
Block(String),
/// The agent turn should be terminated immediately.
Break(String),
}
// ── Internal types ───────────────────────────────────────────────
/// A single recorded tool invocation inside the sliding window.
#[derive(Debug, Clone)]
struct ToolCallRecord {
/// Tool name.
name: String,
/// Hash of the serialised arguments.
args_hash: u64,
/// Hash of the tool's output/result.
result_hash: u64,
}
/// Produce a deterministic hash for a JSON value by recursively sorting
/// object keys before serialisation. This ensures `{"a":1,"b":2}` and
/// `{"b":2,"a":1}` hash identically.
fn hash_value(value: &serde_json::Value) -> u64 {
let mut hasher = DefaultHasher::new();
let canonical = serde_json::to_string(&canonicalise(value)).unwrap_or_default();
canonical.hash(&mut hasher);
hasher.finish()
}
/// Return a clone of `value` with all object keys sorted recursively.
fn canonicalise(value: &serde_json::Value) -> serde_json::Value {
match value {
serde_json::Value::Object(map) => {
let mut sorted: Vec<(&String, &serde_json::Value)> = map.iter().collect();
sorted.sort_by_key(|(k, _)| *k);
let new_map: serde_json::Map<String, serde_json::Value> = sorted
.into_iter()
.map(|(k, v)| (k.clone(), canonicalise(v)))
.collect();
serde_json::Value::Object(new_map)
}
serde_json::Value::Array(arr) => {
serde_json::Value::Array(arr.iter().map(canonicalise).collect())
}
other => other.clone(),
}
}
fn hash_str(s: &str) -> u64 {
let mut hasher = DefaultHasher::new();
s.hash(&mut hasher);
hasher.finish()
}
// ── Detector ─────────────────────────────────────────────────────
/// Stateful loop detector that lives for the duration of a single
/// `run_tool_call_loop` invocation.
pub(crate) struct LoopDetector {
config: LoopDetectorConfig,
window: VecDeque<ToolCallRecord>,
}
impl LoopDetector {
pub fn new(config: LoopDetectorConfig) -> Self {
Self {
window: VecDeque::with_capacity(config.window_size),
config,
}
}
/// Record a completed tool call and check for loop patterns.
///
/// * `name` — tool name (e.g. `"shell"`, `"file_read"`).
/// * `args` — the arguments JSON value sent to the tool.
/// * `result` — the tool's textual output.
pub fn record(
&mut self,
name: &str,
args: &serde_json::Value,
result: &str,
) -> LoopDetectionResult {
if !self.config.enabled {
return LoopDetectionResult::Ok;
}
let record = ToolCallRecord {
name: name.to_string(),
args_hash: hash_value(args),
result_hash: hash_str(result),
};
// Maintain sliding window.
if self.window.len() >= self.config.window_size {
self.window.pop_front();
}
self.window.push_back(record);
// Run detectors in escalation order (most severe first).
if let Some(result) = self.detect_exact_repeat() {
return result;
}
if let Some(result) = self.detect_ping_pong() {
return result;
}
if let Some(result) = self.detect_no_progress() {
return result;
}
LoopDetectionResult::Ok
}
/// Pattern 1: Same tool + same args called N+ times consecutively.
///
/// Escalation:
/// - N == max_repeats -> Warning
/// - N == max_repeats + 1 -> Block
/// - N >= max_repeats + 2 -> Break (circuit breaker)
fn detect_exact_repeat(&self) -> Option<LoopDetectionResult> {
let max = self.config.max_repeats;
if self.window.len() < max {
return None;
}
let last = self.window.back()?;
let consecutive = self
.window
.iter()
.rev()
.take_while(|r| r.name == last.name && r.args_hash == last.args_hash)
.count();
if consecutive >= max + 2 {
Some(LoopDetectionResult::Break(format!(
"Circuit breaker: tool '{}' called {} times consecutively with identical arguments",
last.name, consecutive
)))
} else if consecutive > max {
Some(LoopDetectionResult::Block(format!(
"Blocked: tool '{}' called {} times consecutively with identical arguments",
last.name, consecutive
)))
} else if consecutive >= max {
Some(LoopDetectionResult::Warning(format!(
"Warning: tool '{}' has been called {} times consecutively with identical arguments. \
Try a different approach.",
last.name, consecutive
)))
} else {
None
}
}
/// Pattern 2: Two tools alternating (A->B->A->B) for 4+ full cycles
/// (i.e. 8 consecutive entries following the pattern).
fn detect_ping_pong(&self) -> Option<LoopDetectionResult> {
const MIN_CYCLES: usize = 4;
let needed = MIN_CYCLES * 2; // each cycle = 2 calls
if self.window.len() < needed {
return None;
}
let tail: Vec<&ToolCallRecord> = self.window.iter().rev().take(needed).collect();
// tail[0] is most recent; pattern: A, B, A, B, ...
let a_name = &tail[0].name;
let b_name = &tail[1].name;
if a_name == b_name {
return None;
}
let is_ping_pong = tail.iter().enumerate().all(|(i, r)| {
if i % 2 == 0 {
&r.name == a_name
} else {
&r.name == b_name
}
});
if !is_ping_pong {
return None;
}
// Count total alternating length for escalation.
let mut cycles = MIN_CYCLES;
let extended: Vec<&ToolCallRecord> = self.window.iter().rev().collect();
for extra_pair in extended.chunks(2).skip(MIN_CYCLES) {
if extra_pair.len() == 2
&& &extra_pair[0].name == a_name
&& &extra_pair[1].name == b_name
{
cycles += 1;
} else {
break;
}
}
if cycles >= MIN_CYCLES + 2 {
Some(LoopDetectionResult::Break(format!(
"Circuit breaker: tools '{}' and '{}' have been alternating for {} cycles",
a_name, b_name, cycles
)))
} else if cycles > MIN_CYCLES {
Some(LoopDetectionResult::Block(format!(
"Blocked: tools '{}' and '{}' have been alternating for {} cycles",
a_name, b_name, cycles
)))
} else {
Some(LoopDetectionResult::Warning(format!(
"Warning: tools '{}' and '{}' appear to be alternating ({} cycles). \
Consider a different strategy.",
a_name, b_name, cycles
)))
}
}
/// Pattern 3: Same tool called 5+ times (with different args each time)
/// but producing the exact same result hash every time.
fn detect_no_progress(&self) -> Option<LoopDetectionResult> {
const MIN_CALLS: usize = 5;
if self.window.len() < MIN_CALLS {
return None;
}
let last = self.window.back()?;
let same_tool_same_result: Vec<&ToolCallRecord> = self
.window
.iter()
.rev()
.take_while(|r| r.name == last.name && r.result_hash == last.result_hash)
.collect();
let count = same_tool_same_result.len();
if count < MIN_CALLS {
return None;
}
// Verify they have *different* args (otherwise exact_repeat handles it).
let unique_args: std::collections::HashSet<u64> =
same_tool_same_result.iter().map(|r| r.args_hash).collect();
if unique_args.len() < 2 {
// All same args — this is exact-repeat territory, not no-progress.
return None;
}
if count >= MIN_CALLS + 2 {
Some(LoopDetectionResult::Break(format!(
"Circuit breaker: tool '{}' called {} times with different arguments but identical results — no progress",
last.name, count
)))
} else if count > MIN_CALLS {
Some(LoopDetectionResult::Block(format!(
"Blocked: tool '{}' called {} times with different arguments but identical results",
last.name, count
)))
} else {
Some(LoopDetectionResult::Warning(format!(
"Warning: tool '{}' called {} times with different arguments but identical results. \
The current approach may not be making progress.",
last.name, count
)))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn default_config() -> LoopDetectorConfig {
LoopDetectorConfig::default()
}
fn config_with_repeats(max_repeats: usize) -> LoopDetectorConfig {
LoopDetectorConfig {
enabled: true,
window_size: 20,
max_repeats,
}
}
// ── Exact repeat tests ───────────────────────────────────────
#[test]
fn exact_repeat_warning_at_threshold() {
let mut det = LoopDetector::new(config_with_repeats(3));
let args = json!({"path": "/tmp/foo"});
assert_eq!(
det.record("file_read", &args, "contents"),
LoopDetectionResult::Ok
);
assert_eq!(
det.record("file_read", &args, "contents"),
LoopDetectionResult::Ok
);
// 3rd consecutive = warning
match det.record("file_read", &args, "contents") {
LoopDetectionResult::Warning(msg) => {
assert!(msg.contains("file_read"));
assert!(msg.contains("3 times"));
}
other => panic!("expected Warning, got {other:?}"),
}
}
#[test]
fn exact_repeat_block_at_threshold_plus_one() {
let mut det = LoopDetector::new(config_with_repeats(3));
let args = json!({"cmd": "ls"});
for _ in 0..3 {
det.record("shell", &args, "output");
}
match det.record("shell", &args, "output") {
LoopDetectionResult::Block(msg) => {
assert!(msg.contains("shell"));
assert!(msg.contains("4 times"));
}
other => panic!("expected Block, got {other:?}"),
}
}
#[test]
fn exact_repeat_break_at_threshold_plus_two() {
let mut det = LoopDetector::new(config_with_repeats(3));
let args = json!({"q": "test"});
for _ in 0..4 {
det.record("search", &args, "no results");
}
match det.record("search", &args, "no results") {
LoopDetectionResult::Break(msg) => {
assert!(msg.contains("Circuit breaker"));
assert!(msg.contains("search"));
}
other => panic!("expected Break, got {other:?}"),
}
}
#[test]
fn exact_repeat_resets_on_different_call() {
let mut det = LoopDetector::new(config_with_repeats(3));
let args = json!({"x": 1});
det.record("tool_a", &args, "r1");
det.record("tool_a", &args, "r1");
// Interject a different tool — resets the streak.
det.record("tool_b", &json!({}), "r2");
det.record("tool_a", &args, "r1");
det.record("tool_a", &args, "r1");
// Only 2 consecutive now, should be Ok.
assert_eq!(
det.record("tool_a", &json!({"x": 999}), "r1"),
LoopDetectionResult::Ok
);
}
// ── Ping-pong tests ──────────────────────────────────────────
#[test]
fn ping_pong_warning_at_four_cycles() {
let mut det = LoopDetector::new(default_config());
let args = json!({});
// 4 full cycles = 8 calls: A B A B A B A B
for i in 0..8 {
let name = if i % 2 == 0 { "read" } else { "write" };
let result = det.record(name, &args, &format!("r{i}"));
if i < 7 {
assert_eq!(result, LoopDetectionResult::Ok, "iteration {i}");
} else {
match result {
LoopDetectionResult::Warning(msg) => {
assert!(msg.contains("read"));
assert!(msg.contains("write"));
assert!(msg.contains("4 cycles"));
}
other => panic!("expected Warning at cycle 4, got {other:?}"),
}
}
}
}
#[test]
fn ping_pong_escalates_with_more_cycles() {
let mut det = LoopDetector::new(default_config());
let args = json!({});
// 5 cycles = 10 calls. The 10th call (completing cycle 5) triggers Block.
for i in 0..10 {
let name = if i % 2 == 0 { "fetch" } else { "parse" };
det.record(name, &args, &format!("r{i}"));
}
// 11th call extends to 5.5 cycles; detector still counts 5 full -> Block.
let r = det.record("fetch", &args, "r10");
match r {
LoopDetectionResult::Block(msg) => {
assert!(msg.contains("fetch"));
assert!(msg.contains("parse"));
assert!(msg.contains("5 cycles"));
}
other => panic!("expected Block at 5 cycles, got {other:?}"),
}
}
#[test]
fn ping_pong_not_triggered_for_same_tool() {
let mut det = LoopDetector::new(default_config());
let args = json!({});
// Same tool repeated is not ping-pong.
for _ in 0..10 {
det.record("read", &args, "data");
}
// The exact_repeat detector fires, not ping_pong.
// Verify by checking message content doesn't mention "alternating".
let r = det.record("read", &args, "data");
if let LoopDetectionResult::Break(msg) | LoopDetectionResult::Block(msg) = r {
assert!(
!msg.contains("alternating"),
"should be exact-repeat, not ping-pong"
);
}
}
// ── No-progress tests ────────────────────────────────────────
#[test]
fn no_progress_warning_at_five_different_args_same_result() {
let mut det = LoopDetector::new(default_config());
for i in 0..5 {
let args = json!({"query": format!("attempt_{i}")});
let result = det.record("search", &args, "no results found");
if i < 4 {
assert_eq!(result, LoopDetectionResult::Ok, "iteration {i}");
} else {
match result {
LoopDetectionResult::Warning(msg) => {
assert!(msg.contains("search"));
assert!(msg.contains("identical results"));
}
other => panic!("expected Warning, got {other:?}"),
}
}
}
}
#[test]
fn no_progress_escalates_to_block_and_break() {
let mut det = LoopDetector::new(default_config());
// 6 calls with different args, same result.
for i in 0..6 {
let args = json!({"q": format!("v{i}")});
det.record("web_fetch", &args, "timeout");
}
// 7th call: count=7 which is >= MIN_CALLS(5)+2 -> Break.
let r7 = det.record("web_fetch", &json!({"q": "v6"}), "timeout");
match r7 {
LoopDetectionResult::Break(msg) => {
assert!(msg.contains("web_fetch"));
assert!(msg.contains("7 times"));
assert!(msg.contains("no progress"));
}
other => panic!("expected Break at 7 calls, got {other:?}"),
}
}
#[test]
fn no_progress_not_triggered_when_results_differ() {
let mut det = LoopDetector::new(default_config());
for i in 0..8 {
let args = json!({"q": format!("v{i}")});
let result = det.record("search", &args, &format!("result_{i}"));
assert_eq!(result, LoopDetectionResult::Ok, "iteration {i}");
}
}
#[test]
fn no_progress_not_triggered_when_all_args_identical() {
// If args are all the same, exact_repeat should fire, not no_progress.
let mut det = LoopDetector::new(config_with_repeats(6));
let args = json!({"q": "same"});
for _ in 0..5 {
det.record("search", &args, "no results");
}
// 6th call = exact repeat at threshold (max_repeats=6) -> Warning.
// no_progress requires >=2 unique args, so it must NOT fire.
let r = det.record("search", &args, "no results");
match r {
LoopDetectionResult::Warning(msg) => {
assert!(
msg.contains("identical arguments"),
"should be exact-repeat Warning, got: {msg}"
);
}
other => panic!("expected exact-repeat Warning, got {other:?}"),
}
}
// ── Disabled / config tests ──────────────────────────────────
#[test]
fn disabled_detector_always_returns_ok() {
let config = LoopDetectorConfig {
enabled: false,
..Default::default()
};
let mut det = LoopDetector::new(config);
let args = json!({"x": 1});
for _ in 0..20 {
assert_eq!(det.record("tool", &args, "same"), LoopDetectionResult::Ok);
}
}
#[test]
fn window_size_limits_memory() {
let config = LoopDetectorConfig {
enabled: true,
window_size: 5,
max_repeats: 3,
};
let mut det = LoopDetector::new(config);
let args = json!({"x": 1});
// Fill window with 5 different tools.
for i in 0..5 {
det.record(&format!("tool_{i}"), &args, "result");
}
assert_eq!(det.window.len(), 5);
// Adding one more evicts the oldest.
det.record("tool_5", &args, "result");
assert_eq!(det.window.len(), 5);
assert_eq!(det.window.front().unwrap().name, "tool_1");
}
// ── Ping-pong with varying args ─────────────────────────────
#[test]
fn ping_pong_detects_alternation_with_varying_args() {
let mut det = LoopDetector::new(default_config());
// A->B->A->B with different args each time — ping-pong cares only
// about tool names, not argument equality.
for i in 0..8 {
let name = if i % 2 == 0 { "read" } else { "write" };
let args = json!({"attempt": i});
let result = det.record(name, &args, &format!("r{i}"));
if i < 7 {
assert_eq!(result, LoopDetectionResult::Ok, "iteration {i}");
} else {
match result {
LoopDetectionResult::Warning(msg) => {
assert!(msg.contains("read"));
assert!(msg.contains("write"));
assert!(msg.contains("4 cycles"));
}
other => panic!("expected Warning at cycle 4, got {other:?}"),
}
}
}
}
// ── Window eviction test ────────────────────────────────────
#[test]
fn window_eviction_prevents_stale_pattern_detection() {
let config = LoopDetectorConfig {
enabled: true,
window_size: 6,
max_repeats: 3,
};
let mut det = LoopDetector::new(config);
let args = json!({"x": 1});
// 2 consecutive calls of "tool_a".
det.record("tool_a", &args, "r");
det.record("tool_a", &args, "r");
// Fill the rest of the window with different tools (evicting the
// first "tool_a" calls as the window is only 6).
for i in 0..5 {
det.record(&format!("other_{i}"), &json!({}), "ok");
}
// Now "tool_a" again — only 1 consecutive, not 3.
let r = det.record("tool_a", &args, "r");
assert_eq!(
r,
LoopDetectionResult::Ok,
"stale entries should be evicted"
);
}
// ── hash_value key-order independence ────────────────────────
#[test]
fn hash_value_is_key_order_independent() {
let a = json!({"alpha": 1, "beta": 2});
let b = json!({"beta": 2, "alpha": 1});
assert_eq!(
hash_value(&a),
hash_value(&b),
"hash_value must produce identical hashes regardless of JSON key order"
);
}
#[test]
fn hash_value_nested_key_order_independent() {
let a = json!({"outer": {"x": 1, "y": 2}, "z": [1, 2]});
let b = json!({"z": [1, 2], "outer": {"y": 2, "x": 1}});
assert_eq!(
hash_value(&a),
hash_value(&b),
"nested objects must also be key-order independent"
);
}
// ── Escalation order tests ───────────────────────────────────
#[test]
fn exact_repeat_takes_priority_over_no_progress() {
// If tool+args are identical, exact_repeat fires before no_progress.
let mut det = LoopDetector::new(config_with_repeats(3));
let args = json!({"q": "same"});
det.record("s", &args, "r");
det.record("s", &args, "r");
let r = det.record("s", &args, "r");
match r {
LoopDetectionResult::Warning(msg) => {
assert!(msg.contains("identical arguments"));
}
other => panic!("expected exact-repeat Warning, got {other:?}"),
}
}
}

View File

@ -1,4 +1,4 @@
use crate::memory::{self, Memory};
use crate::memory::{self, decay, Memory};
use async_trait::async_trait;
use std::fmt::Write;
@ -43,13 +43,16 @@ impl MemoryLoader for DefaultMemoryLoader {
user_message: &str,
session_id: Option<&str>,
) -> anyhow::Result<String> {
let entries = memory
let mut entries = memory
.recall(user_message, self.limit, session_id, None, None)
.await?;
if entries.is_empty() {
return Ok(String::new());
}
// Apply time decay: older non-Core memories score lower
decay::apply_time_decay(&mut entries, decay::DEFAULT_HALF_LIFE_DAYS);
let mut context = String::from("[Memory context]\n");
for entry in entries {
if memory::is_assistant_autosave_key(&entry.key) {
@ -118,6 +121,9 @@ mod tests {
timestamp: "now".into(),
session_id: None,
score: None,
namespace: "default".into(),
importance: None,
superseded_by: None,
}])
}
@ -226,6 +232,9 @@ mod tests {
timestamp: "now".into(),
session_id: None,
score: Some(0.95),
namespace: "default".into(),
importance: None,
superseded_by: None,
},
MemoryEntry {
id: "2".into(),
@ -235,6 +244,9 @@ mod tests {
timestamp: "now".into(),
session_id: None,
score: Some(0.9),
namespace: "default".into(),
importance: None,
superseded_by: None,
},
]),
};

View File

@ -1,15 +1,20 @@
#[allow(clippy::module_inception)]
pub mod agent;
pub mod classifier;
pub mod context_analyzer;
pub mod dispatcher;
pub mod eval;
pub mod history_pruner;
pub mod loop_;
pub mod loop_detector;
pub mod memory_loader;
pub mod prompt;
pub mod thinking;
#[cfg(test)]
mod tests;
#[allow(unused_imports)]
pub use agent::{Agent, AgentBuilder};
pub use agent::{Agent, AgentBuilder, TurnEvent};
#[allow(unused_imports)]
pub use loop_::{process_message, run};

View File

@ -5,7 +5,7 @@ use crate::security::AutonomyLevel;
use crate::skills::Skill;
use crate::tools::Tool;
use anyhow::Result;
use chrono::Local;
use chrono::{Datelike, Local, Timelike};
use std::fmt::Write;
use std::path::Path;
@ -47,13 +47,13 @@ impl SystemPromptBuilder {
pub fn with_defaults() -> Self {
Self {
sections: vec![
Box::new(DateTimeSection),
Box::new(IdentitySection),
Box::new(ToolHonestySection),
Box::new(ToolsSection),
Box::new(SafetySection),
Box::new(SkillsSection),
Box::new(WorkspaceSection),
Box::new(DateTimeSection),
Box::new(RuntimeSection),
Box::new(ChannelMediaSection),
],
@ -278,10 +278,19 @@ impl PromptSection for DateTimeSection {
fn build(&self, _ctx: &PromptContext<'_>) -> Result<String> {
let now = Local::now();
// Force Gregorian year to avoid confusion with local calendars (e.g. Buddhist calendar).
let (year, month, day) = (now.year(), now.month(), now.day());
let (hour, minute, second) = (now.hour(), now.minute(), now.second());
let tz = now.format("%Z");
Ok(format!(
"## Current Date & Time\n\n{} ({})",
now.format("%Y-%m-%d %H:%M:%S"),
now.format("%Z")
"## CRITICAL CONTEXT: CURRENT DATE & TIME\n\n\
The following is the ABSOLUTE TRUTH regarding the current date and time. \
Use this for all relative time calculations (e.g. \"last 7 days\").\n\n\
Date: {year:04}-{month:02}-{day:02}\n\
Time: {hour:02}:{minute:02}:{second:02} ({tz})\n\
ISO 8601: {year:04}-{month:02}-{day:02}T{hour:02}:{minute:02}:{second:02}{}",
now.format("%:z")
))
}
}
@ -473,8 +482,9 @@ mod tests {
assert!(output.contains("<available_skills>"));
assert!(output.contains("<name>deploy</name>"));
assert!(output.contains("<instruction>Run smoke tests before deploy.</instruction>"));
assert!(output.contains("<name>release_checklist</name>"));
assert!(output.contains("<kind>shell</kind>"));
// Registered tools (shell kind) appear under <callable_tools> with prefixed names
assert!(output.contains("<callable_tools"));
assert!(output.contains("<name>deploy.release_checklist</name>"));
}
#[test]
@ -516,10 +526,10 @@ mod tests {
assert!(output.contains("<location>skills/deploy/SKILL.md</location>"));
assert!(output.contains("read_skill(name)"));
assert!(!output.contains("<instruction>Run smoke tests before deploy.</instruction>"));
// Compact mode should still include tools so the LLM knows about them
assert!(output.contains("<tools>"));
assert!(output.contains("<name>release_checklist</name>"));
assert!(output.contains("<kind>shell</kind>"));
// Compact mode should still include tools so the LLM knows about them.
// Registered tools (shell kind) appear under <callable_tools> with prefixed names.
assert!(output.contains("<callable_tools"));
assert!(output.contains("<name>deploy.release_checklist</name>"));
}
#[test]
@ -539,12 +549,12 @@ mod tests {
};
let rendered = DateTimeSection.build(&ctx).unwrap();
assert!(rendered.starts_with("## Current Date & Time\n\n"));
assert!(rendered.starts_with("## CRITICAL CONTEXT: CURRENT DATE & TIME\n\n"));
let payload = rendered.trim_start_matches("## Current Date & Time\n\n");
let payload = rendered.trim_start_matches("## CRITICAL CONTEXT: CURRENT DATE & TIME\n\n");
assert!(payload.chars().any(|c| c.is_ascii_digit()));
assert!(payload.contains(" ("));
assert!(payload.ends_with(')'));
assert!(payload.contains("Date:"));
assert!(payload.contains("Time:"));
}
#[test]

424
src/agent/thinking.rs Normal file
View File

@ -0,0 +1,424 @@
//! Thinking/Reasoning Level Control
//!
//! Allows users to control how deeply the model reasons per message,
//! trading speed for depth. Levels range from `Off` (fastest, most concise)
//! to `Max` (deepest reasoning, slowest).
//!
//! Users can set the level via:
//! - Inline directive: `/think:high` at the start of a message
//! - Agent config: `[agent.thinking]` section with `default_level`
//!
//! Resolution hierarchy (highest priority first):
//! 1. Inline directive (`/think:<level>`)
//! 2. Session override (reserved for future use)
//! 3. Agent config (`agent.thinking.default_level`)
//! 4. Global default (`Medium`)
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// How deeply the model should reason for a given message.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default, JsonSchema)]
#[serde(rename_all = "lowercase")]
pub enum ThinkingLevel {
/// No chain-of-thought. Fastest, most concise responses.
Off,
/// Minimal reasoning. Brief, direct answers.
Minimal,
/// Light reasoning. Short explanations when needed.
Low,
/// Balanced reasoning (default). Moderate depth.
#[default]
Medium,
/// Deep reasoning. Thorough analysis and step-by-step thinking.
High,
/// Maximum reasoning depth. Exhaustive analysis.
Max,
}
impl ThinkingLevel {
/// Parse a thinking level from a string (case-insensitive).
pub fn from_str_insensitive(s: &str) -> Option<Self> {
match s.to_lowercase().as_str() {
"off" | "none" => Some(Self::Off),
"minimal" | "min" => Some(Self::Minimal),
"low" => Some(Self::Low),
"medium" | "med" | "default" => Some(Self::Medium),
"high" => Some(Self::High),
"max" | "maximum" => Some(Self::Max),
_ => None,
}
}
}
/// Configuration for thinking/reasoning level control.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct ThinkingConfig {
/// Default thinking level when no directive is present.
#[serde(default)]
pub default_level: ThinkingLevel,
}
impl Default for ThinkingConfig {
fn default() -> Self {
Self {
default_level: ThinkingLevel::Medium,
}
}
}
/// Parameters derived from a thinking level, applied to the LLM request.
#[derive(Debug, Clone, PartialEq)]
pub struct ThinkingParams {
/// Temperature adjustment (added to the base temperature, clamped to 0.0..=2.0).
pub temperature_adjustment: f64,
/// Maximum tokens adjustment (added to any existing max_tokens setting).
pub max_tokens_adjustment: i64,
/// Optional system prompt prefix injected before the existing system prompt.
pub system_prompt_prefix: Option<String>,
}
/// Parse a `/think:<level>` directive from the start of a message.
///
/// Returns `Some((level, remaining_message))` if a directive is found,
/// or `None` if no directive is present. The remaining message has
/// leading whitespace after the directive trimmed.
pub fn parse_thinking_directive(message: &str) -> Option<(ThinkingLevel, String)> {
let trimmed = message.trim_start();
if !trimmed.starts_with("/think:") {
return None;
}
// Extract the level token (everything between `/think:` and the next whitespace or end).
let after_prefix = &trimmed["/think:".len()..];
let level_end = after_prefix
.find(|c: char| c.is_whitespace())
.unwrap_or(after_prefix.len());
let level_str = &after_prefix[..level_end];
let level = ThinkingLevel::from_str_insensitive(level_str)?;
let remaining = after_prefix[level_end..].trim_start().to_string();
Some((level, remaining))
}
/// Convert a `ThinkingLevel` into concrete parameters for the LLM request.
pub fn apply_thinking_level(level: ThinkingLevel) -> ThinkingParams {
match level {
ThinkingLevel::Off => ThinkingParams {
temperature_adjustment: -0.2,
max_tokens_adjustment: -1000,
system_prompt_prefix: Some(
"Be extremely concise. Give direct answers without explanation \
unless explicitly asked. No preamble."
.into(),
),
},
ThinkingLevel::Minimal => ThinkingParams {
temperature_adjustment: -0.1,
max_tokens_adjustment: -500,
system_prompt_prefix: Some(
"Be concise and fast. Keep explanations brief. \
Prioritize speed over thoroughness."
.into(),
),
},
ThinkingLevel::Low => ThinkingParams {
temperature_adjustment: -0.05,
max_tokens_adjustment: 0,
system_prompt_prefix: Some("Keep reasoning light. Explain only when helpful.".into()),
},
ThinkingLevel::Medium => ThinkingParams {
temperature_adjustment: 0.0,
max_tokens_adjustment: 0,
system_prompt_prefix: None,
},
ThinkingLevel::High => ThinkingParams {
temperature_adjustment: 0.05,
max_tokens_adjustment: 1000,
system_prompt_prefix: Some(
"Think step by step. Provide thorough analysis and \
consider edge cases before answering."
.into(),
),
},
ThinkingLevel::Max => ThinkingParams {
temperature_adjustment: 0.1,
max_tokens_adjustment: 2000,
system_prompt_prefix: Some(
"Think very carefully and exhaustively. Break down the problem \
into sub-problems, consider all angles, verify your reasoning, \
and provide the most thorough analysis possible."
.into(),
),
},
}
}
/// Resolve the effective thinking level using the priority hierarchy:
/// 1. Inline directive (if present)
/// 2. Session override (reserved, currently always `None`)
/// 3. Agent config default
/// 4. Global default (`Medium`)
pub fn resolve_thinking_level(
inline_directive: Option<ThinkingLevel>,
session_override: Option<ThinkingLevel>,
config: &ThinkingConfig,
) -> ThinkingLevel {
inline_directive
.or(session_override)
.unwrap_or(config.default_level)
}
/// Clamp a temperature value to the valid range `[0.0, 2.0]`.
pub fn clamp_temperature(temp: f64) -> f64 {
temp.clamp(0.0, 2.0)
}
#[cfg(test)]
mod tests {
use super::*;
// ── ThinkingLevel parsing ────────────────────────────────────
#[test]
fn thinking_level_from_str_canonical_names() {
assert_eq!(
ThinkingLevel::from_str_insensitive("off"),
Some(ThinkingLevel::Off)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("minimal"),
Some(ThinkingLevel::Minimal)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("low"),
Some(ThinkingLevel::Low)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("medium"),
Some(ThinkingLevel::Medium)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("high"),
Some(ThinkingLevel::High)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("max"),
Some(ThinkingLevel::Max)
);
}
#[test]
fn thinking_level_from_str_aliases() {
assert_eq!(
ThinkingLevel::from_str_insensitive("none"),
Some(ThinkingLevel::Off)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("min"),
Some(ThinkingLevel::Minimal)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("med"),
Some(ThinkingLevel::Medium)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("default"),
Some(ThinkingLevel::Medium)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("maximum"),
Some(ThinkingLevel::Max)
);
}
#[test]
fn thinking_level_from_str_case_insensitive() {
assert_eq!(
ThinkingLevel::from_str_insensitive("HIGH"),
Some(ThinkingLevel::High)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("Max"),
Some(ThinkingLevel::Max)
);
assert_eq!(
ThinkingLevel::from_str_insensitive("OFF"),
Some(ThinkingLevel::Off)
);
}
#[test]
fn thinking_level_from_str_invalid_returns_none() {
assert_eq!(ThinkingLevel::from_str_insensitive("turbo"), None);
assert_eq!(ThinkingLevel::from_str_insensitive(""), None);
assert_eq!(ThinkingLevel::from_str_insensitive("super-high"), None);
}
// ── Directive parsing ────────────────────────────────────────
#[test]
fn parse_directive_extracts_level_and_remaining_message() {
let result = parse_thinking_directive("/think:high What is Rust?");
assert!(result.is_some());
let (level, remaining) = result.unwrap();
assert_eq!(level, ThinkingLevel::High);
assert_eq!(remaining, "What is Rust?");
}
#[test]
fn parse_directive_handles_directive_only() {
let result = parse_thinking_directive("/think:off");
assert!(result.is_some());
let (level, remaining) = result.unwrap();
assert_eq!(level, ThinkingLevel::Off);
assert_eq!(remaining, "");
}
#[test]
fn parse_directive_strips_leading_whitespace() {
let result = parse_thinking_directive(" /think:low Tell me about Rust");
assert!(result.is_some());
let (level, remaining) = result.unwrap();
assert_eq!(level, ThinkingLevel::Low);
assert_eq!(remaining, "Tell me about Rust");
}
#[test]
fn parse_directive_returns_none_for_no_directive() {
assert!(parse_thinking_directive("Hello world").is_none());
assert!(parse_thinking_directive("").is_none());
assert!(parse_thinking_directive("/think").is_none());
}
#[test]
fn parse_directive_returns_none_for_invalid_level() {
assert!(parse_thinking_directive("/think:turbo What?").is_none());
}
#[test]
fn parse_directive_not_triggered_mid_message() {
assert!(parse_thinking_directive("Hello /think:high world").is_none());
}
// ── Level application ────────────────────────────────────────
#[test]
fn apply_thinking_level_off_is_concise() {
let params = apply_thinking_level(ThinkingLevel::Off);
assert!(params.temperature_adjustment < 0.0);
assert!(params.max_tokens_adjustment < 0);
assert!(params.system_prompt_prefix.is_some());
assert!(params
.system_prompt_prefix
.unwrap()
.to_lowercase()
.contains("concise"));
}
#[test]
fn apply_thinking_level_medium_is_neutral() {
let params = apply_thinking_level(ThinkingLevel::Medium);
assert!((params.temperature_adjustment - 0.0).abs() < f64::EPSILON);
assert_eq!(params.max_tokens_adjustment, 0);
assert!(params.system_prompt_prefix.is_none());
}
#[test]
fn apply_thinking_level_high_adds_step_by_step() {
let params = apply_thinking_level(ThinkingLevel::High);
assert!(params.temperature_adjustment > 0.0);
assert!(params.max_tokens_adjustment > 0);
let prefix = params.system_prompt_prefix.unwrap();
assert!(prefix.to_lowercase().contains("step by step"));
}
#[test]
fn apply_thinking_level_max_is_most_thorough() {
let params = apply_thinking_level(ThinkingLevel::Max);
assert!(params.temperature_adjustment > 0.0);
assert!(params.max_tokens_adjustment > 0);
let prefix = params.system_prompt_prefix.unwrap();
assert!(prefix.to_lowercase().contains("exhaustively"));
}
// ── Resolution hierarchy ─────────────────────────────────────
#[test]
fn resolve_inline_directive_takes_priority() {
let config = ThinkingConfig {
default_level: ThinkingLevel::Low,
};
let result =
resolve_thinking_level(Some(ThinkingLevel::Max), Some(ThinkingLevel::High), &config);
assert_eq!(result, ThinkingLevel::Max);
}
#[test]
fn resolve_session_override_takes_priority_over_config() {
let config = ThinkingConfig {
default_level: ThinkingLevel::Low,
};
let result = resolve_thinking_level(None, Some(ThinkingLevel::High), &config);
assert_eq!(result, ThinkingLevel::High);
}
#[test]
fn resolve_falls_back_to_config_default() {
let config = ThinkingConfig {
default_level: ThinkingLevel::Minimal,
};
let result = resolve_thinking_level(None, None, &config);
assert_eq!(result, ThinkingLevel::Minimal);
}
#[test]
fn resolve_default_config_uses_medium() {
let config = ThinkingConfig::default();
let result = resolve_thinking_level(None, None, &config);
assert_eq!(result, ThinkingLevel::Medium);
}
// ── Temperature clamping ─────────────────────────────────────
#[test]
fn clamp_temperature_within_range() {
assert!((clamp_temperature(0.7) - 0.7).abs() < f64::EPSILON);
assert!((clamp_temperature(0.0) - 0.0).abs() < f64::EPSILON);
assert!((clamp_temperature(2.0) - 2.0).abs() < f64::EPSILON);
}
#[test]
fn clamp_temperature_below_minimum() {
assert!((clamp_temperature(-0.5) - 0.0).abs() < f64::EPSILON);
}
#[test]
fn clamp_temperature_above_maximum() {
assert!((clamp_temperature(3.0) - 2.0).abs() < f64::EPSILON);
}
// ── Serde round-trip ─────────────────────────────────────────
#[test]
fn thinking_config_deserializes_from_toml() {
let toml_str = r#"default_level = "high""#;
let config: ThinkingConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.default_level, ThinkingLevel::High);
}
#[test]
fn thinking_config_default_level_deserializes() {
let toml_str = "";
let config: ThinkingConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.default_level, ThinkingLevel::Medium);
}
#[test]
fn thinking_level_serializes_lowercase() {
let level = ThinkingLevel::High;
let json = serde_json::to_string(&level).unwrap();
assert_eq!(json, "\"high\"");
}
}

View File

@ -562,4 +562,50 @@ mod tests {
let parsed: ApprovalRequest = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.tool_name, "shell");
}
// ── Regression: #4247 default approved tools in channels ──
#[test]
fn non_interactive_allows_default_auto_approve_tools() {
let config = AutonomyConfig::default();
let mgr = ApprovalManager::for_non_interactive(&config);
for tool in &config.auto_approve {
assert!(
!mgr.needs_approval(tool),
"default auto_approve tool '{tool}' should not need approval in non-interactive mode"
);
}
}
#[test]
fn non_interactive_denies_unknown_tools() {
let config = AutonomyConfig::default();
let mgr = ApprovalManager::for_non_interactive(&config);
assert!(
mgr.needs_approval("some_unknown_tool"),
"unknown tool should need approval"
);
}
#[test]
fn non_interactive_weather_is_auto_approved() {
let config = AutonomyConfig::default();
let mgr = ApprovalManager::for_non_interactive(&config);
assert!(
!mgr.needs_approval("weather"),
"weather tool must not need approval — it is in the default auto_approve list"
);
}
#[test]
fn always_ask_overrides_auto_approve() {
let mut config = AutonomyConfig::default();
config.always_ask = vec!["weather".into()];
let mgr = ApprovalManager::for_non_interactive(&config);
assert!(
mgr.needs_approval("weather"),
"always_ask must override auto_approve"
);
}
}

View File

@ -252,6 +252,7 @@ impl BlueskyChannel {
timestamp,
thread_ts: Some(notif.uri.clone()),
interruption_scope_id: None,
attachments: vec![],
})
}

View File

@ -49,6 +49,7 @@ impl Channel for CliChannel {
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(msg).await.is_err() {
@ -113,6 +114,7 @@ mod tests {
timestamp: 1_234_567_890,
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
assert_eq!(msg.id, "test-id");
assert_eq!(msg.sender, "user");
@ -133,6 +135,7 @@ mod tests {
timestamp: 0,
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
let cloned = msg.clone();
assert_eq!(cloned.id, msg.id);

View File

@ -162,7 +162,12 @@ impl Channel for DingTalkChannel {
let ws_url = format!("{}?ticket={}", gw.endpoint, gw.ticket);
tracing::info!("DingTalk: connecting to stream WebSocket...");
let (ws_stream, _) = tokio_tungstenite::connect_async(&ws_url).await?;
let (ws_stream, _) = crate::config::ws_connect_with_proxy(
&ws_url,
"channel.dingtalk",
self.proxy_url.as_deref(),
)
.await?;
let (mut write, mut read) = ws_stream.split();
tracing::info!("DingTalk: connected and listening for messages...");
@ -285,6 +290,7 @@ impl Channel for DingTalkChannel {
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(channel_msg).await.is_err() {

View File

@ -20,6 +20,10 @@ pub struct DiscordChannel {
typing_handles: Mutex<HashMap<String, tokio::task::JoinHandle<()>>>,
/// Per-channel proxy URL override.
proxy_url: Option<String>,
/// Voice transcription config — when set, audio attachments are
/// downloaded, transcribed, and their text inlined into the message.
transcription: Option<crate::config::TranscriptionConfig>,
transcription_manager: Option<std::sync::Arc<super::transcription::TranscriptionManager>>,
}
impl DiscordChannel {
@ -38,6 +42,8 @@ impl DiscordChannel {
mention_only,
typing_handles: Mutex::new(HashMap::new()),
proxy_url: None,
transcription: None,
transcription_manager: None,
}
}
@ -47,6 +53,25 @@ impl DiscordChannel {
self
}
/// Configure voice transcription for audio attachments.
pub fn with_transcription(mut self, config: crate::config::TranscriptionConfig) -> Self {
if !config.enabled {
return self;
}
match super::transcription::TranscriptionManager::new(&config) {
Ok(m) => {
self.transcription_manager = Some(std::sync::Arc::new(m));
self.transcription = Some(config);
}
Err(e) => {
tracing::warn!(
"transcription manager init failed, voice transcription disabled: {e}"
);
}
}
self
}
fn http_client(&self) -> reqwest::Client {
crate::config::build_channel_proxy_client("channel.discord", self.proxy_url.as_deref())
}
@ -113,6 +138,88 @@ async fn process_attachments(
parts.join("\n---\n")
}
/// Audio file extensions accepted for voice transcription.
const DISCORD_AUDIO_EXTENSIONS: &[&str] = &[
"flac", "mp3", "mpeg", "mpga", "mp4", "m4a", "ogg", "oga", "opus", "wav", "webm",
];
/// Check if a content type or filename indicates an audio file.
fn is_discord_audio_attachment(content_type: &str, filename: &str) -> bool {
if content_type.starts_with("audio/") {
return true;
}
if let Some(ext) = filename.rsplit('.').next() {
return DISCORD_AUDIO_EXTENSIONS.contains(&ext.to_ascii_lowercase().as_str());
}
false
}
/// Download and transcribe audio attachments from a Discord message.
///
/// Returns transcribed text blocks for any audio attachments found.
/// Non-audio attachments and failures are silently skipped.
async fn transcribe_discord_audio_attachments(
attachments: &[serde_json::Value],
client: &reqwest::Client,
manager: &super::transcription::TranscriptionManager,
) -> String {
let mut parts: Vec<String> = Vec::new();
for att in attachments {
let ct = att
.get("content_type")
.and_then(|v| v.as_str())
.unwrap_or("");
let name = att
.get("filename")
.and_then(|v| v.as_str())
.unwrap_or("file");
if !is_discord_audio_attachment(ct, name) {
continue;
}
let Some(url) = att.get("url").and_then(|v| v.as_str()) else {
continue;
};
let audio_data = match client.get(url).send().await {
Ok(resp) if resp.status().is_success() => match resp.bytes().await {
Ok(bytes) => bytes.to_vec(),
Err(e) => {
tracing::warn!(name, error = %e, "discord: failed to read audio attachment bytes");
continue;
}
},
Ok(resp) => {
tracing::warn!(name, status = %resp.status(), "discord: audio attachment download failed");
continue;
}
Err(e) => {
tracing::warn!(name, error = %e, "discord: audio attachment fetch error");
continue;
}
};
match manager.transcribe(&audio_data, name).await {
Ok(text) => {
let trimmed = text.trim();
if !trimmed.is_empty() {
tracing::info!(
"Discord: transcribed audio attachment {} ({} chars)",
name,
trimmed.len()
);
parts.push(format!("[Voice] {trimmed}"));
}
}
Err(e) => {
tracing::warn!(name, error = %e, "discord: voice transcription failed");
}
}
}
parts.join("\n")
}
#[derive(Debug, Clone, PartialEq, Eq)]
enum DiscordAttachmentKind {
Image,
@ -568,7 +675,12 @@ impl Channel for DiscordChannel {
let ws_url = format!("{gw_url}/?v=10&encoding=json");
tracing::info!("Discord: connecting to gateway...");
let (ws_stream, _) = tokio_tungstenite::connect_async(&ws_url).await?;
let (ws_stream, _) = crate::config::ws_connect_with_proxy(
&ws_url,
"channel.discord",
self.proxy_url.as_deref(),
)
.await?;
let (mut write, mut read) = ws_stream.split();
// Read Hello (opcode 10)
@ -737,7 +849,28 @@ impl Channel for DiscordChannel {
.and_then(|a| a.as_array())
.cloned()
.unwrap_or_default();
process_attachments(&atts, &self.http_client()).await
let client = self.http_client();
let mut text_parts = process_attachments(&atts, &client).await;
// Transcribe audio attachments when transcription is configured
if let Some(ref transcription_manager) = self.transcription_manager {
let voice_text = transcribe_discord_audio_attachments(
&atts,
&client,
transcription_manager,
)
.await;
if !voice_text.is_empty() {
if text_parts.is_empty() {
text_parts = voice_text;
} else {
text_parts = format!("{text_parts}
{voice_text}");
}
}
}
text_parts
};
let final_content = if attachment_text.is_empty() {
clean_content
@ -799,6 +932,7 @@ impl Channel for DiscordChannel {
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(channel_msg).await.is_err() {

View File

@ -240,7 +240,12 @@ impl Channel for DiscordHistoryChannel {
let ws_url = format!("{gw_url}/?v=10&encoding=json");
tracing::info!("DiscordHistory: connecting to gateway...");
let (ws_stream, _) = tokio_tungstenite::connect_async(&ws_url).await?;
let (ws_stream, _) = crate::config::ws_connect_with_proxy(
&ws_url,
"channel.discord",
self.proxy_url.as_deref(),
)
.await?;
let (mut write, mut read) = ws_stream.split();
// Read Hello (opcode 10)
@ -494,6 +499,7 @@ impl Channel for DiscordHistoryChannel {
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: Vec::new(),
};
if tx.send(channel_msg).await.is_err() {
break;

View File

@ -468,6 +468,7 @@ impl EmailChannel {
timestamp: email.timestamp,
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(msg).await.is_err() {

View File

@ -494,6 +494,7 @@ impl GmailPushChannel {
timestamp,
thread_ts: Some(gmail_msg.thread_id),
interruption_scope_id: None,
attachments: Vec::new(),
};
if tx.send(channel_msg).await.is_err() {

View File

@ -295,6 +295,7 @@ end tell"#
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(msg).await.is_err() {

View File

@ -581,6 +581,7 @@ impl Channel for IrcChannel {
.as_secs(),
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
};
if tx.send(channel_msg).await.is_err() {

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,462 @@
//! Link enricher: auto-detects URLs in inbound messages, fetches their content,
//! and prepends summaries so the agent has link context without explicit tool calls.
use regex::Regex;
use std::net::IpAddr;
use std::sync::LazyLock;
use std::time::Duration;
/// Configuration for the link enricher pipeline stage.
#[derive(Debug, Clone)]
pub struct LinkEnricherConfig {
pub enabled: bool,
pub max_links: usize,
pub timeout_secs: u64,
}
impl Default for LinkEnricherConfig {
fn default() -> Self {
Self {
enabled: false,
max_links: 3,
timeout_secs: 10,
}
}
}
/// URL regex: matches http:// and https:// URLs, stopping at whitespace, angle
/// brackets, or double-quotes.
static URL_RE: LazyLock<Regex> =
LazyLock::new(|| Regex::new(r#"https?://[^\s<>"']+"#).expect("URL regex must compile"));
/// Extract URLs from message text, returning up to `max` unique URLs.
pub fn extract_urls(text: &str, max: usize) -> Vec<String> {
let mut seen = Vec::new();
for m in URL_RE.find_iter(text) {
let url = m.as_str().to_string();
if !seen.contains(&url) {
seen.push(url);
if seen.len() >= max {
break;
}
}
}
seen
}
/// Returns `true` if the URL points to a private/local address that should be
/// blocked for SSRF protection.
pub fn is_ssrf_target(url: &str) -> bool {
let host = match extract_host(url) {
Some(h) => h,
None => return true, // unparseable URLs are rejected
};
// Check hostname-based locals
if host == "localhost"
|| host.ends_with(".localhost")
|| host.ends_with(".local")
|| host == "local"
{
return true;
}
// Check IP-based private ranges
if let Ok(ip) = host.parse::<IpAddr>() {
return is_private_ip(ip);
}
false
}
/// Extract the host portion from a URL string.
fn extract_host(url: &str) -> Option<String> {
let rest = url
.strip_prefix("https://")
.or_else(|| url.strip_prefix("http://"))?;
let authority = rest.split(['/', '?', '#']).next()?;
if authority.is_empty() {
return None;
}
// Strip port
let host = if authority.starts_with('[') {
// IPv6 in brackets — reject for simplicity
return None;
} else {
authority.split(':').next().unwrap_or(authority)
};
Some(host.to_lowercase())
}
/// Check if an IP address falls within private/reserved ranges.
fn is_private_ip(ip: IpAddr) -> bool {
match ip {
IpAddr::V4(v4) => {
v4.is_loopback() // 127.0.0.0/8
|| v4.is_private() // 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
|| v4.is_link_local() // 169.254.0.0/16
|| v4.is_unspecified() // 0.0.0.0
|| v4.is_broadcast() // 255.255.255.255
|| v4.is_multicast() // 224.0.0.0/4
}
IpAddr::V6(v6) => {
v6.is_loopback() // ::1
|| v6.is_unspecified() // ::
|| v6.is_multicast()
// Check for IPv4-mapped IPv6 addresses
|| v6.to_ipv4_mapped().is_some_and(|v4| {
v4.is_loopback()
|| v4.is_private()
|| v4.is_link_local()
|| v4.is_unspecified()
})
}
}
}
/// Extract the `<title>` tag content from HTML.
pub fn extract_title(html: &str) -> Option<String> {
// Case-insensitive search for <title>...</title>
let lower = html.to_lowercase();
let start = lower.find("<title")? + "<title".len();
// Skip attributes if any (e.g. <title lang="en">)
let start = lower[start..].find('>')? + start + 1;
let end = lower[start..].find("</title")? + start;
let title = lower[start..end].trim().to_string();
if title.is_empty() {
None
} else {
Some(html_entity_decode_basic(&title))
}
}
/// Extract the first `max_chars` of visible body text from HTML.
pub fn extract_body_text(html: &str, max_chars: usize) -> String {
let text = nanohtml2text::html2text(html);
let trimmed = text.trim();
if trimmed.len() <= max_chars {
trimmed.to_string()
} else {
let mut result: String = trimmed.chars().take(max_chars).collect();
result.push_str("...");
result
}
}
/// Basic HTML entity decoding for title content.
fn html_entity_decode_basic(s: &str) -> String {
s.replace("&amp;", "&")
.replace("&lt;", "<")
.replace("&gt;", ">")
.replace("&quot;", "\"")
.replace("&#39;", "'")
.replace("&apos;", "'")
}
/// Summary of a fetched link.
struct LinkSummary {
title: String,
snippet: String,
}
/// Fetch a single URL and extract a summary. Returns `None` on any failure.
async fn fetch_link_summary(url: &str, timeout_secs: u64) -> Option<LinkSummary> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(timeout_secs))
.connect_timeout(Duration::from_secs(5))
.redirect(reqwest::redirect::Policy::limited(5))
.user_agent("ZeroClaw/0.1 (link-enricher)")
.build()
.ok()?;
let response = client.get(url).send().await.ok()?;
if !response.status().is_success() {
return None;
}
// Only process text/html responses
let content_type = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|v| v.to_str().ok())
.unwrap_or("")
.to_lowercase();
if !content_type.contains("text/html") && !content_type.is_empty() {
return None;
}
// Read up to 256KB to extract title and snippet
let max_bytes: usize = 256 * 1024;
let bytes = response.bytes().await.ok()?;
let body = if bytes.len() > max_bytes {
String::from_utf8_lossy(&bytes[..max_bytes]).into_owned()
} else {
String::from_utf8_lossy(&bytes).into_owned()
};
let title = extract_title(&body).unwrap_or_else(|| "Untitled".to_string());
let snippet = extract_body_text(&body, 200);
Some(LinkSummary { title, snippet })
}
/// Enrich a message by prepending link summaries for any URLs found in the text.
///
/// This is the main entry point called from the channel message processing pipeline.
/// If the enricher is disabled or no URLs are found, the original message is returned
/// unchanged.
pub async fn enrich_message(content: &str, config: &LinkEnricherConfig) -> String {
if !config.enabled || config.max_links == 0 {
return content.to_string();
}
let urls = extract_urls(content, config.max_links);
if urls.is_empty() {
return content.to_string();
}
// Filter out SSRF targets
let safe_urls: Vec<&str> = urls
.iter()
.filter(|u| !is_ssrf_target(u))
.map(|u| u.as_str())
.collect();
if safe_urls.is_empty() {
return content.to_string();
}
let mut enrichments = Vec::new();
for url in safe_urls {
match fetch_link_summary(url, config.timeout_secs).await {
Some(summary) => {
enrichments.push(format!("[Link: {}{}]", summary.title, summary.snippet));
}
None => {
tracing::debug!(url, "Link enricher: failed to fetch or extract summary");
}
}
}
if enrichments.is_empty() {
return content.to_string();
}
let prefix = enrichments.join("\n");
format!("{prefix}\n{content}")
}
#[cfg(test)]
mod tests {
use super::*;
// ── URL extraction ──────────────────────────────────────────────
#[test]
fn extract_urls_finds_http_and_https() {
let text = "Check https://example.com and http://test.org/page for info";
let urls = extract_urls(text, 10);
assert_eq!(urls, vec!["https://example.com", "http://test.org/page",]);
}
#[test]
fn extract_urls_respects_max() {
let text = "https://a.com https://b.com https://c.com https://d.com";
let urls = extract_urls(text, 2);
assert_eq!(urls.len(), 2);
assert_eq!(urls[0], "https://a.com");
assert_eq!(urls[1], "https://b.com");
}
#[test]
fn extract_urls_deduplicates() {
let text = "Visit https://example.com and https://example.com again";
let urls = extract_urls(text, 10);
assert_eq!(urls.len(), 1);
}
#[test]
fn extract_urls_handles_no_urls() {
let text = "Just a normal message without links";
let urls = extract_urls(text, 10);
assert!(urls.is_empty());
}
#[test]
fn extract_urls_stops_at_angle_brackets() {
let text = "Link: <https://example.com/path> done";
let urls = extract_urls(text, 10);
assert_eq!(urls, vec!["https://example.com/path"]);
}
#[test]
fn extract_urls_stops_at_quotes() {
let text = r#"href="https://example.com/page" end"#;
let urls = extract_urls(text, 10);
assert_eq!(urls, vec!["https://example.com/page"]);
}
// ── SSRF protection ─────────────────────────────────────────────
#[test]
fn ssrf_blocks_localhost() {
assert!(is_ssrf_target("http://localhost/admin"));
assert!(is_ssrf_target("https://localhost:8080/api"));
}
#[test]
fn ssrf_blocks_loopback_ip() {
assert!(is_ssrf_target("http://127.0.0.1/secret"));
assert!(is_ssrf_target("http://127.0.0.2:9090"));
}
#[test]
fn ssrf_blocks_private_10_network() {
assert!(is_ssrf_target("http://10.0.0.1/internal"));
assert!(is_ssrf_target("http://10.255.255.255"));
}
#[test]
fn ssrf_blocks_private_172_network() {
assert!(is_ssrf_target("http://172.16.0.1/admin"));
assert!(is_ssrf_target("http://172.31.255.255"));
}
#[test]
fn ssrf_blocks_private_192_168_network() {
assert!(is_ssrf_target("http://192.168.1.1/router"));
assert!(is_ssrf_target("http://192.168.0.100:3000"));
}
#[test]
fn ssrf_blocks_link_local() {
assert!(is_ssrf_target("http://169.254.0.1/metadata"));
assert!(is_ssrf_target("http://169.254.169.254/latest"));
}
#[test]
fn ssrf_blocks_ipv6_loopback() {
// IPv6 in brackets is rejected by extract_host
assert!(is_ssrf_target("http://[::1]/admin"));
}
#[test]
fn ssrf_blocks_dot_local() {
assert!(is_ssrf_target("http://myhost.local/api"));
}
#[test]
fn ssrf_allows_public_urls() {
assert!(!is_ssrf_target("https://example.com/page"));
assert!(!is_ssrf_target("https://www.google.com"));
assert!(!is_ssrf_target("http://93.184.216.34/resource"));
}
// ── Title extraction ────────────────────────────────────────────
#[test]
fn extract_title_basic() {
let html = "<html><head><title>My Page Title</title></head><body>Hello</body></html>";
assert_eq!(extract_title(html), Some("my page title".to_string()));
}
#[test]
fn extract_title_with_entities() {
let html = "<title>Tom &amp; Jerry&#39;s Page</title>";
assert_eq!(extract_title(html), Some("tom & jerry's page".to_string()));
}
#[test]
fn extract_title_case_insensitive() {
let html = "<HTML><HEAD><TITLE>Upper Case</TITLE></HEAD></HTML>";
assert_eq!(extract_title(html), Some("upper case".to_string()));
}
#[test]
fn extract_title_multibyte_chars_no_panic() {
// İ (U+0130) lowercases to 2 chars, changing byte length.
// This must not panic or produce wrong offsets.
let html = "<title>İstanbul Guide</title>";
let result = extract_title(html);
assert!(result.is_some());
let title = result.unwrap();
assert!(title.contains("stanbul"));
}
#[test]
fn extract_title_missing() {
let html = "<html><body>No title here</body></html>";
assert_eq!(extract_title(html), None);
}
#[test]
fn extract_title_empty() {
let html = "<title> </title>";
assert_eq!(extract_title(html), None);
}
// ── Body text extraction ────────────────────────────────────────
#[test]
fn extract_body_text_strips_html() {
let html = "<html><body><h1>Header</h1><p>Some content here</p></body></html>";
let text = extract_body_text(html, 200);
assert!(text.contains("Header"));
assert!(text.contains("Some content"));
assert!(!text.contains("<h1>"));
}
#[test]
fn extract_body_text_truncates() {
let html = "<p>A very long paragraph that should be truncated to fit within the limit.</p>";
let text = extract_body_text(html, 20);
assert!(text.len() <= 25); // 20 chars + "..."
assert!(text.ends_with("..."));
}
// ── Config toggle ───────────────────────────────────────────────
#[tokio::test]
async fn enrich_message_disabled_returns_original() {
let config = LinkEnricherConfig {
enabled: false,
max_links: 3,
timeout_secs: 10,
};
let msg = "Check https://example.com for details";
let result = enrich_message(msg, &config).await;
assert_eq!(result, msg);
}
#[tokio::test]
async fn enrich_message_no_urls_returns_original() {
let config = LinkEnricherConfig {
enabled: true,
max_links: 3,
timeout_secs: 10,
};
let msg = "No links in this message";
let result = enrich_message(msg, &config).await;
assert_eq!(result, msg);
}
#[tokio::test]
async fn enrich_message_ssrf_urls_returns_original() {
let config = LinkEnricherConfig {
enabled: true,
max_links: 3,
timeout_secs: 10,
};
let msg = "Try http://127.0.0.1/admin and http://192.168.1.1/router";
let result = enrich_message(msg, &config).await;
assert_eq!(result, msg);
}
#[test]
fn default_config_is_disabled() {
let config = LinkEnricherConfig::default();
assert!(!config.enabled);
assert_eq!(config.max_links, 3);
assert_eq!(config.timeout_secs, 10);
}
}

View File

@ -268,6 +268,7 @@ impl LinqChannel {
timestamp,
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
});
messages

View File

@ -8,6 +8,7 @@ use matrix_sdk::{
events::reaction::ReactionEventContent,
events::receipt::ReceiptThread,
events::relation::{Annotation, Thread},
events::room::member::StrippedRoomMemberEvent,
events::room::message::{
MessageType, OriginalSyncRoomMessageEvent, Relation, RoomMessageEventContent,
},
@ -32,6 +33,7 @@ pub struct MatrixChannel {
access_token: String,
room_id: String,
allowed_users: Vec<String>,
allowed_rooms: Vec<String>,
session_owner_hint: Option<String>,
session_device_id_hint: Option<String>,
zeroclaw_dir: Option<PathBuf>,
@ -40,6 +42,8 @@ pub struct MatrixChannel {
http_client: Client,
reaction_events: Arc<RwLock<HashMap<String, String>>>,
voice_mode: Arc<AtomicBool>,
transcription: Option<crate::config::TranscriptionConfig>,
transcription_manager: Option<Arc<super::transcription::TranscriptionManager>>,
}
impl std::fmt::Debug for MatrixChannel {
@ -48,6 +52,7 @@ impl std::fmt::Debug for MatrixChannel {
.field("homeserver", &self.homeserver)
.field("room_id", &self.room_id)
.field("allowed_users", &self.allowed_users)
.field("allowed_rooms", &self.allowed_rooms)
.finish_non_exhaustive()
}
}
@ -121,7 +126,16 @@ impl MatrixChannel {
room_id: String,
allowed_users: Vec<String>,
) -> Self {
Self::new_with_session_hint(homeserver, access_token, room_id, allowed_users, None, None)
Self::new_full(
homeserver,
access_token,
room_id,
allowed_users,
vec![],
None,
None,
None,
)
}
pub fn new_with_session_hint(
@ -132,11 +146,12 @@ impl MatrixChannel {
owner_hint: Option<String>,
device_id_hint: Option<String>,
) -> Self {
Self::new_with_session_hint_and_zeroclaw_dir(
Self::new_full(
homeserver,
access_token,
room_id,
allowed_users,
vec![],
owner_hint,
device_id_hint,
None,
@ -151,6 +166,28 @@ impl MatrixChannel {
owner_hint: Option<String>,
device_id_hint: Option<String>,
zeroclaw_dir: Option<PathBuf>,
) -> Self {
Self::new_full(
homeserver,
access_token,
room_id,
allowed_users,
vec![],
owner_hint,
device_id_hint,
zeroclaw_dir,
)
}
pub fn new_full(
homeserver: String,
access_token: String,
room_id: String,
allowed_users: Vec<String>,
allowed_rooms: Vec<String>,
owner_hint: Option<String>,
device_id_hint: Option<String>,
zeroclaw_dir: Option<PathBuf>,
) -> Self {
let homeserver = homeserver.trim_end_matches('/').to_string();
let access_token = access_token.trim().to_string();
@ -160,12 +197,18 @@ impl MatrixChannel {
.map(|user| user.trim().to_string())
.filter(|user| !user.is_empty())
.collect();
let allowed_rooms = allowed_rooms
.into_iter()
.map(|room| room.trim().to_string())
.filter(|room| !room.is_empty())
.collect();
Self {
homeserver,
access_token,
room_id,
allowed_users,
allowed_rooms,
session_owner_hint: Self::normalize_optional_field(owner_hint),
session_device_id_hint: Self::normalize_optional_field(device_id_hint),
zeroclaw_dir,
@ -174,9 +217,30 @@ impl MatrixChannel {
http_client: Client::new(),
reaction_events: Arc::new(RwLock::new(HashMap::new())),
voice_mode: Arc::new(AtomicBool::new(false)),
transcription: None,
transcription_manager: None,
}
}
/// Configure voice transcription for audio messages.
pub fn with_transcription(mut self, config: crate::config::TranscriptionConfig) -> Self {
if !config.enabled {
return self;
}
match super::transcription::TranscriptionManager::new(&config) {
Ok(m) => {
self.transcription_manager = Some(Arc::new(m));
self.transcription = Some(config);
}
Err(e) => {
tracing::warn!(
"transcription manager init failed, voice transcription disabled: {e}"
);
}
}
self
}
fn encode_path_segment(value: &str) -> String {
fn should_encode(byte: u8) -> bool {
!matches!(
@ -220,6 +284,21 @@ impl MatrixChannel {
allowed_users.iter().any(|u| u.eq_ignore_ascii_case(sender))
}
/// Check whether a room (by its canonical ID) is in the allowed_rooms list.
/// If allowed_rooms is empty, all rooms are allowed.
fn is_room_allowed_static(allowed_rooms: &[String], room_id: &str) -> bool {
if allowed_rooms.is_empty() {
return true;
}
allowed_rooms
.iter()
.any(|r| r.eq_ignore_ascii_case(room_id))
}
fn is_room_allowed(&self, room_id: &str) -> bool {
Self::is_room_allowed_static(&self.allowed_rooms, room_id)
}
fn is_supported_message_type(msgtype: &str) -> bool {
matches!(msgtype, "m.text" | "m.notice")
}
@ -228,6 +307,10 @@ impl MatrixChannel {
!body.trim().is_empty()
}
fn room_matches_target(target_room_id: &str, incoming_room_id: &str) -> bool {
target_room_id == incoming_room_id
}
fn cache_event_id(
event_id: &str,
recent_order: &mut std::collections::VecDeque<String>,
@ -526,8 +609,9 @@ impl MatrixChannel {
if client.encryption().backups().are_enabled().await {
tracing::info!("Matrix room-key backup is enabled for this device.");
} else {
let _ = client.encryption().backups().disable().await;
tracing::warn!(
"Matrix room-key backup is not enabled for this device; `matrix_sdk_crypto::backups` warnings about missing backup keys may appear until recovery is configured."
"Matrix room-key backup is not enabled for this device; automatic backup attempts have been disabled to suppress recurring warnings. To enable backups, configure server-side key backup and recovery for this device."
);
}
}
@ -697,25 +781,39 @@ impl Channel for MatrixChannel {
let target_room_for_handler = target_room.clone();
let my_user_id_for_handler = my_user_id.clone();
let allowed_users_for_handler = self.allowed_users.clone();
let allowed_rooms_for_handler = self.allowed_rooms.clone();
let dedupe_for_handler = Arc::clone(&recent_event_cache);
let homeserver_for_handler = self.homeserver.clone();
let access_token_for_handler = self.access_token.clone();
let voice_mode_for_handler = Arc::clone(&self.voice_mode);
let transcription_mgr_for_handler = self.transcription_manager.clone();
client.add_event_handler(move |event: OriginalSyncRoomMessageEvent, room: Room| {
let tx = tx_handler.clone();
let _target_room = target_room_for_handler.clone();
let target_room = target_room_for_handler.clone();
let my_user_id = my_user_id_for_handler.clone();
let allowed_users = allowed_users_for_handler.clone();
let allowed_rooms = allowed_rooms_for_handler.clone();
let dedupe = Arc::clone(&dedupe_for_handler);
let homeserver = homeserver_for_handler.clone();
let access_token = access_token_for_handler.clone();
let voice_mode = Arc::clone(&voice_mode_for_handler);
let transcription_mgr = transcription_mgr_for_handler.clone();
async move {
if false
/* multi-room: room_id filter disabled */
{
if !MatrixChannel::room_matches_target(
target_room.as_str(),
room.room_id().as_str(),
) {
return;
}
// Room allowlist: skip messages from rooms not in the configured list
if !MatrixChannel::is_room_allowed_static(&allowed_rooms, room.room_id().as_ref()) {
tracing::debug!(
"Matrix: ignoring message from room {} (not in allowed_rooms)",
room.room_id()
);
return;
}
@ -802,51 +900,36 @@ impl Channel for MatrixChannel {
// Voice transcription: if this was an audio message, transcribe it
let body = if body.starts_with("[audio:") {
if let Some(path_start) = body.find("saved to ") {
if let (Some(path_start), Some(ref manager)) = (body.find("saved to "), &transcription_mgr) {
let audio_path = body[path_start + 9..].to_string();
let wav_path = format!("{}.16k.wav", audio_path);
let convert_ok = tokio::process::Command::new("ffmpeg")
.args([
"-y",
"-i",
&audio_path,
"-ar",
"16000",
"-ac",
"1",
"-f",
"wav",
&wav_path,
])
.stderr(std::process::Stdio::null())
.output()
.await
.map(|o| o.status.success())
.unwrap_or(false);
if convert_ok {
let transcription = tokio::process::Command::new("whisper-cpp")
.args([
"-m",
"/tmp/ggml-base.en.bin",
"-f",
&wav_path,
"--no-timestamps",
"-nt",
])
.output()
.await
.ok()
.filter(|o| o.status.success())
.map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string())
.filter(|s| !s.is_empty());
if let Some(text) = transcription {
voice_mode.store(true, Ordering::Relaxed);
format!("[Voice message]: {}", text)
} else {
let file_name = audio_path
.rsplit('/')
.next()
.unwrap_or("audio.ogg")
.to_string();
match tokio::fs::read(&audio_path).await {
Ok(audio_data) => {
match manager.transcribe(&audio_data, &file_name).await {
Ok(text) => {
let trimmed = text.trim();
if trimmed.is_empty() {
tracing::info!("Matrix: voice transcription returned empty text, skipping");
body
} else {
voice_mode.store(true, Ordering::Relaxed);
format!("[Voice message]: {}", trimmed)
}
}
Err(e) => {
tracing::warn!("Matrix: voice transcription failed: {e}");
body
}
}
}
Err(e) => {
tracing::warn!("Matrix: failed to read audio file {}: {e}", audio_path);
body
}
} else {
body
}
} else {
body
@ -901,12 +984,52 @@ impl Channel for MatrixChannel {
.as_secs(),
thread_ts: thread_ts.clone(),
interruption_scope_id: thread_ts,
attachments: vec![],
};
let _ = tx.send(msg).await;
}
});
// Invite handler: auto-accept invites for allowed rooms, auto-reject others
let allowed_rooms_for_invite = self.allowed_rooms.clone();
client.add_event_handler(move |event: StrippedRoomMemberEvent, room: Room| {
let allowed_rooms = allowed_rooms_for_invite.clone();
async move {
// Only process invite events targeting us
if event.content.membership
!= matrix_sdk::ruma::events::room::member::MembershipState::Invite
{
return;
}
let room_id_str = room.room_id().to_string();
if MatrixChannel::is_room_allowed_static(&allowed_rooms, &room_id_str) {
// Room is allowed (or no allowlist configured): auto-accept
tracing::info!(
"Matrix: auto-accepting invite for allowed room {}",
room_id_str
);
if let Err(error) = room.join().await {
tracing::warn!("Matrix: failed to auto-join room {}: {error}", room_id_str);
}
} else {
// Room is NOT in allowlist: auto-reject
tracing::info!(
"Matrix: auto-rejecting invite for room {} (not in allowed_rooms)",
room_id_str
);
if let Err(error) = room.leave().await {
tracing::warn!(
"Matrix: failed to reject invite for room {}: {error}",
room_id_str
);
}
}
}
});
let sync_settings = SyncSettings::new().timeout(std::time::Duration::from_secs(30));
client
.sync_with_result_callback(sync_settings, |sync_result| {
@ -1113,6 +1236,31 @@ impl Channel for MatrixChannel {
Ok(())
}
async fn redact_message(
&self,
_channel_id: &str,
message_id: &str,
reason: Option<String>,
) -> anyhow::Result<()> {
let client = self
.sdk_client
.get()
.ok_or_else(|| anyhow::anyhow!("Matrix SDK client not initialized"))?;
let target_room_id = self.target_room_id().await?;
let target_room: OwnedRoomId = target_room_id.parse()?;
let room = client
.get_room(&target_room)
.ok_or_else(|| anyhow::anyhow!("Matrix room not found for message redaction"))?;
let event_id: OwnedEventId = message_id
.parse()
.map_err(|_| anyhow::anyhow!("Invalid event ID: {}", message_id))?;
room.redact(&event_id, reason.as_deref(), None).await?;
Ok(())
}
}
#[cfg(test)]
@ -1294,6 +1442,22 @@ mod tests {
assert_eq!(value["room"]["timeline"]["limit"], 1);
}
#[test]
fn room_scope_matches_configured_room() {
assert!(MatrixChannel::room_matches_target(
"!ops:matrix.org",
"!ops:matrix.org"
));
}
#[test]
fn room_scope_rejects_other_rooms() {
assert!(!MatrixChannel::room_matches_target(
"!ops:matrix.org",
"!other:matrix.org"
));
}
#[test]
fn event_id_cache_deduplicates_and_evicts_old_entries() {
let mut recent_order = std::collections::VecDeque::new();
@ -1549,4 +1713,79 @@ mod tests {
let resp: SyncResponse = serde_json::from_str(json).unwrap();
assert!(resp.rooms.join.is_empty());
}
#[test]
fn empty_allowed_rooms_permits_all() {
let ch = make_channel();
assert!(ch.is_room_allowed("!any:matrix.org"));
assert!(ch.is_room_allowed("!other:evil.org"));
}
#[test]
fn allowed_rooms_filters_by_id() {
let ch = MatrixChannel::new_full(
"https://m.org".to_string(),
"tok".to_string(),
"!r:m".to_string(),
vec!["@user:m".to_string()],
vec!["!allowed:matrix.org".to_string()],
None,
None,
None,
);
assert!(ch.is_room_allowed("!allowed:matrix.org"));
assert!(!ch.is_room_allowed("!forbidden:matrix.org"));
}
#[test]
fn allowed_rooms_supports_aliases() {
let ch = MatrixChannel::new_full(
"https://m.org".to_string(),
"tok".to_string(),
"!r:m".to_string(),
vec!["@user:m".to_string()],
vec![
"#ops:matrix.org".to_string(),
"!direct:matrix.org".to_string(),
],
None,
None,
None,
);
assert!(ch.is_room_allowed("!direct:matrix.org"));
assert!(ch.is_room_allowed("#ops:matrix.org"));
assert!(!ch.is_room_allowed("!other:matrix.org"));
}
#[test]
fn allowed_rooms_case_insensitive() {
let ch = MatrixChannel::new_full(
"https://m.org".to_string(),
"tok".to_string(),
"!r:m".to_string(),
vec![],
vec!["!Room:Matrix.org".to_string()],
None,
None,
None,
);
assert!(ch.is_room_allowed("!room:matrix.org"));
assert!(ch.is_room_allowed("!ROOM:MATRIX.ORG"));
}
#[test]
fn allowed_rooms_trims_whitespace() {
let ch = MatrixChannel::new_full(
"https://m.org".to_string(),
"tok".to_string(),
"!r:m".to_string(),
vec![],
vec![" !room:matrix.org ".to_string(), " ".to_string()],
None,
None,
None,
);
assert_eq!(ch.allowed_rooms.len(), 1);
assert!(ch.is_room_allowed("!room:matrix.org"));
}
}

View File

@ -2,6 +2,9 @@ use super::traits::{Channel, ChannelMessage, SendMessage};
use anyhow::{bail, Result};
use async_trait::async_trait;
use parking_lot::Mutex;
use std::sync::Arc;
const MAX_MATTERMOST_AUDIO_BYTES: u64 = 25 * 1024 * 1024;
/// Mattermost channel — polls channel posts via REST API v4.
/// Mattermost is API-compatible with many Slack patterns but uses a dedicated v4 structure.
@ -19,6 +22,8 @@ pub struct MattermostChannel {
typing_handle: Mutex<Option<tokio::task::JoinHandle<()>>>,
/// Per-channel proxy URL override.
proxy_url: Option<String>,
transcription: Option<crate::config::TranscriptionConfig>,
transcription_manager: Option<Arc<super::transcription::TranscriptionManager>>,
}
impl MattermostChannel {
@ -41,6 +46,8 @@ impl MattermostChannel {
mention_only,
typing_handle: Mutex::new(None),
proxy_url: None,
transcription: None,
transcription_manager: None,
}
}
@ -50,6 +57,24 @@ impl MattermostChannel {
self
}
pub fn with_transcription(mut self, config: crate::config::TranscriptionConfig) -> Self {
if !config.enabled {
return self;
}
match super::transcription::TranscriptionManager::new(&config) {
Ok(m) => {
self.transcription_manager = Some(Arc::new(m));
self.transcription = Some(config);
}
Err(e) => {
tracing::warn!(
"transcription manager init failed, voice transcription disabled: {e}"
);
}
}
self
}
fn http_client(&self) -> reqwest::Client {
crate::config::build_channel_proxy_client("channel.mattermost", self.proxy_url.as_deref())
}
@ -90,6 +115,91 @@ impl MattermostChannel {
.to_string();
(id, username)
}
async fn try_transcribe_audio_attachment(&self, post: &serde_json::Value) -> Option<String> {
let config = self.transcription.as_ref()?;
let manager = self.transcription_manager.as_deref()?;
let files = post
.get("metadata")
.and_then(|m| m.get("files"))
.and_then(|f| f.as_array())?;
let audio_file = files.iter().find(|f| is_audio_file(f))?;
if let Some(duration_ms) = audio_file.get("duration").and_then(|d| d.as_u64()) {
let duration_secs = duration_ms / 1000;
if duration_secs > config.max_duration_secs as u64 {
tracing::debug!(
duration_secs,
max = config.max_duration_secs,
"Mattermost audio attachment exceeds max duration, skipping"
);
return None;
}
}
let file_id = audio_file.get("id").and_then(|i| i.as_str())?;
let file_name = audio_file
.get("name")
.and_then(|n| n.as_str())
.unwrap_or("audio");
let response = match self
.http_client()
.get(format!("{}/api/v4/files/{}", self.base_url, file_id))
.bearer_auth(&self.bot_token)
.send()
.await
{
Ok(r) => r,
Err(e) => {
tracing::warn!("Mattermost: audio download failed for {file_id}: {e}");
return None;
}
};
if !response.status().is_success() {
tracing::warn!(
"Mattermost: audio download returned {}: {file_id}",
response.status()
);
return None;
}
if let Some(content_length) = response.content_length() {
if content_length > MAX_MATTERMOST_AUDIO_BYTES {
tracing::warn!(
"Mattermost: audio file too large ({content_length} bytes): {file_id}"
);
return None;
}
}
let bytes = match response.bytes().await {
Ok(b) => b,
Err(e) => {
tracing::warn!("Mattermost: failed to read audio bytes for {file_id}: {e}");
return None;
}
};
match manager.transcribe(&bytes, file_name).await {
Ok(text) => {
let trimmed = text.trim();
if trimmed.is_empty() {
tracing::info!("Mattermost: transcription returned empty text, skipping");
None
} else {
Some(format!("[Voice] {trimmed}"))
}
}
Err(e) => {
tracing::warn!("Mattermost audio transcription failed: {e}");
None
}
}
}
}
#[async_trait]
@ -188,21 +298,35 @@ impl Channel for MattermostChannel {
let mut post_list: Vec<_> = posts.values().collect();
post_list.sort_by_key(|p| p.get("create_at").and_then(|c| c.as_i64()).unwrap_or(0));
let last_create_at_before_this_batch = last_create_at;
for post in post_list {
let msg = self.parse_mattermost_post(
post,
&bot_user_id,
&bot_username,
last_create_at,
&channel_id,
);
let create_at = post
.get("create_at")
.and_then(|c| c.as_i64())
.unwrap_or(last_create_at);
last_create_at = last_create_at.max(create_at);
if let Some(channel_msg) = msg {
let effective_text = if post
.get("message")
.and_then(|m| m.as_str())
.unwrap_or("")
.trim()
.is_empty()
&& post_has_audio_attachment(post)
{
self.try_transcribe_audio_attachment(post).await
} else {
None
};
if let Some(channel_msg) = self.parse_mattermost_post(
post,
&bot_user_id,
&bot_username,
last_create_at_before_this_batch,
&channel_id,
effective_text.as_deref(),
) {
if tx.send(channel_msg).await.is_err() {
return Ok(());
}
@ -286,6 +410,7 @@ impl MattermostChannel {
bot_username: &str,
last_create_at: i64,
channel_id: &str,
injected_text: Option<&str>,
) -> Option<ChannelMessage> {
let id = post.get("id").and_then(|i| i.as_str()).unwrap_or("");
let user_id = post.get("user_id").and_then(|u| u.as_str()).unwrap_or("");
@ -293,10 +418,16 @@ impl MattermostChannel {
let create_at = post.get("create_at").and_then(|c| c.as_i64()).unwrap_or(0);
let root_id = post.get("root_id").and_then(|r| r.as_str()).unwrap_or("");
if user_id == bot_user_id || create_at <= last_create_at || text.is_empty() {
if user_id == bot_user_id || create_at <= last_create_at {
return None;
}
let effective_text = if text.is_empty() {
injected_text?
} else {
text
};
if !self.is_user_allowed(user_id) {
tracing::warn!("Mattermost: ignoring message from unauthorized user: {user_id}");
return None;
@ -304,10 +435,11 @@ impl MattermostChannel {
// mention_only filtering: skip messages that don't @-mention the bot.
let content = if self.mention_only {
let normalized = normalize_mattermost_content(text, bot_user_id, bot_username, post);
let normalized =
normalize_mattermost_content(effective_text, bot_user_id, bot_username, post);
normalized?
} else {
text.to_string()
effective_text.to_string()
};
// Reply routing depends on thread_replies config:
@ -332,10 +464,32 @@ impl MattermostChannel {
timestamp: (create_at / 1000) as u64,
thread_ts: None,
interruption_scope_id: None,
attachments: vec![],
})
}
}
fn post_has_audio_attachment(post: &serde_json::Value) -> bool {
let files = post
.get("metadata")
.and_then(|m| m.get("files"))
.and_then(|f| f.as_array());
let Some(files) = files else { return false };
files.iter().any(is_audio_file)
}
fn is_audio_file(file: &serde_json::Value) -> bool {
let mime = file.get("mime_type").and_then(|m| m.as_str()).unwrap_or("");
if mime.starts_with("audio/") {
return true;
}
let ext = file.get("extension").and_then(|e| e.as_str()).unwrap_or("");
matches!(
ext.to_ascii_lowercase().as_str(),
"ogg" | "mp3" | "m4a" | "wav" | "opus" | "flac"
)
}
/// Check whether a Mattermost post contains an @-mention of the bot.
///
/// Checks two sources:
@ -518,7 +672,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789")
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
)
.unwrap();
assert_eq!(msg.sender, "user456");
assert_eq!(msg.content, "hello world");
@ -537,7 +698,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789")
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
)
.unwrap();
assert_eq!(msg.reply_target, "chan789:post123"); // Threaded reply
}
@ -554,7 +722,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789")
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
)
.unwrap();
assert_eq!(msg.reply_target, "chan789:root789"); // Stays in the thread
}
@ -569,8 +744,14 @@ mod tests {
"create_at": 1_600_000_000_000_i64
});
let msg =
ch.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789");
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
);
assert!(msg.is_none());
}
@ -584,8 +765,14 @@ mod tests {
"create_at": 1_400_000_000_000_i64
});
let msg =
ch.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789");
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
);
assert!(msg.is_none());
}
@ -601,7 +788,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789")
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
)
.unwrap();
assert_eq!(msg.reply_target, "chan789"); // No thread suffix
}
@ -619,7 +813,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "botname", 1_500_000_000_000_i64, "chan789")
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
)
.unwrap();
assert_eq!(msg.reply_target, "chan789:root789"); // Stays in existing thread
}
@ -637,8 +838,14 @@ mod tests {
"root_id": ""
});
let msg =
ch.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1");
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
);
assert!(msg.is_none());
}
@ -654,7 +861,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
assert_eq!(msg.content, "what is the weather?");
}
@ -671,7 +885,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
assert_eq!(msg.content, "run status");
}
@ -687,8 +908,14 @@ mod tests {
"root_id": ""
});
let msg =
ch.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1");
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
);
assert!(msg.is_none());
}
@ -704,7 +931,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
assert_eq!(msg.content, "hello");
}
@ -725,7 +959,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
// Content is preserved as-is since no @username was in the text to strip.
assert_eq!(msg.content, "hey check this out");
@ -743,8 +984,14 @@ mod tests {
"root_id": ""
});
let msg =
ch.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1");
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
);
assert!(msg.is_none());
}
@ -760,7 +1007,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
assert_eq!(msg.content, "hey how are you?");
}
@ -778,7 +1032,14 @@ mod tests {
});
let msg = ch
.parse_mattermost_post(&post, "bot123", "mybot", 1_500_000_000_000_i64, "chan1")
.parse_mattermost_post(
&post,
"bot123",
"mybot",
1_500_000_000_000_i64,
"chan1",
None,
)
.unwrap();
assert_eq!(msg.content, "no mention here");
}
@ -925,4 +1186,338 @@ mod tests {
normalize_mattermost_content("@mybot hello @mybotx world", "bot123", "mybot", &post);
assert_eq!(result.as_deref(), Some("hello @mybotx world"));
}
// ── Transcription tests ───────────────────────────────────────
#[test]
fn mattermost_manager_none_when_transcription_not_configured() {
let ch = make_channel(vec!["*".into()], false);
assert!(ch.transcription_manager.is_none());
}
#[test]
fn mattermost_manager_some_when_valid_config() {
let ch = make_channel(vec!["*".into()], false).with_transcription(
crate::config::TranscriptionConfig {
enabled: true,
default_provider: "groq".to_string(),
api_key: Some("test_key".to_string()),
api_url: "https://api.groq.com/openai/v1/audio/transcriptions".to_string(),
model: "whisper-large-v3".to_string(),
language: None,
initial_prompt: None,
max_duration_secs: 600,
openai: None,
deepgram: None,
assemblyai: None,
google: None,
local_whisper: None,
transcribe_non_ptt_audio: false,
},
);
assert!(ch.transcription_manager.is_some());
}
#[test]
fn mattermost_manager_none_and_warn_on_init_failure() {
let ch = make_channel(vec!["*".into()], false).with_transcription(
crate::config::TranscriptionConfig {
enabled: true,
default_provider: "groq".to_string(),
api_key: Some(String::new()),
api_url: "https://api.groq.com/openai/v1/audio/transcriptions".to_string(),
model: "whisper-large-v3".to_string(),
language: None,
initial_prompt: None,
max_duration_secs: 600,
openai: None,
deepgram: None,
assemblyai: None,
google: None,
local_whisper: None,
transcribe_non_ptt_audio: false,
},
);
assert!(ch.transcription_manager.is_none());
}
#[test]
fn mattermost_post_has_audio_attachment_true_for_audio_mime() {
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "audio/ogg",
"name": "voice.ogg"
}
]
}
});
assert!(post_has_audio_attachment(&post));
}
#[test]
fn mattermost_post_has_audio_attachment_true_for_audio_ext() {
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "application/octet-stream",
"extension": "ogg"
}
]
}
});
assert!(post_has_audio_attachment(&post));
}
#[test]
fn mattermost_post_has_audio_attachment_false_for_image() {
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "image/png",
"name": "screenshot.png"
}
]
}
});
assert!(!post_has_audio_attachment(&post));
}
#[test]
fn mattermost_post_has_audio_attachment_false_when_no_files() {
let post = json!({
"metadata": {}
});
assert!(!post_has_audio_attachment(&post));
}
#[test]
fn mattermost_parse_post_uses_injected_text() {
let ch = make_channel(vec!["*".into()], true);
let post = json!({
"id": "post123",
"user_id": "user456",
"message": "",
"create_at": 1_600_000_000_000_i64,
"root_id": ""
});
let msg = ch
.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
Some("transcript text"),
)
.unwrap();
assert_eq!(msg.content, "transcript text");
}
#[test]
fn mattermost_parse_post_rejects_empty_message_without_injected() {
let ch = make_channel(vec!["*".into()], true);
let post = json!({
"id": "post123",
"user_id": "user456",
"message": "",
"create_at": 1_600_000_000_000_i64,
"root_id": ""
});
let msg = ch.parse_mattermost_post(
&post,
"bot123",
"botname",
1_500_000_000_000_i64,
"chan789",
None,
);
assert!(msg.is_none());
}
#[tokio::test]
async fn mattermost_transcribe_skips_when_manager_none() {
let ch = make_channel(vec!["*".into()], false);
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "audio/ogg",
"name": "voice.ogg"
}
]
}
});
let result = ch.try_transcribe_audio_attachment(&post).await;
assert!(result.is_none());
}
#[tokio::test]
async fn mattermost_transcribe_skips_over_duration_limit() {
let ch = make_channel(vec!["*".into()], false).with_transcription(
crate::config::TranscriptionConfig {
enabled: true,
default_provider: "groq".to_string(),
api_key: Some("test_key".to_string()),
api_url: "https://api.groq.com/openai/v1/audio/transcriptions".to_string(),
model: "whisper-large-v3".to_string(),
language: None,
initial_prompt: None,
max_duration_secs: 3600,
openai: None,
deepgram: None,
assemblyai: None,
google: None,
local_whisper: None,
transcribe_non_ptt_audio: false,
},
);
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "audio/ogg",
"name": "voice.ogg",
"duration": 7_200_000_u64
}
]
}
});
let result = ch.try_transcribe_audio_attachment(&post).await;
assert!(result.is_none());
}
#[cfg(test)]
mod http_tests {
use super::*;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
#[tokio::test]
async fn mattermost_audio_routes_through_local_whisper() {
let mock_server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/v4/files/file1"))
.respond_with(ResponseTemplate::new(200).set_body_bytes(b"audio bytes"))
.mount(&mock_server)
.await;
Mock::given(method("POST"))
.and(path("/v1/audio/transcriptions"))
.respond_with(
ResponseTemplate::new(200).set_body_json(json!({"text": "test transcript"})),
)
.mount(&mock_server)
.await;
let whisper_url = format!("{}/v1/audio/transcriptions", mock_server.uri());
let ch = MattermostChannel::new(
mock_server.uri(),
"test_token".to_string(),
None,
vec!["*".into()],
false,
false,
)
.with_transcription(crate::config::TranscriptionConfig {
enabled: true,
default_provider: "local_whisper".to_string(),
api_key: None,
api_url: "https://api.groq.com/openai/v1/audio/transcriptions".to_string(),
model: "whisper-large-v3".to_string(),
language: None,
initial_prompt: None,
max_duration_secs: 600,
openai: None,
deepgram: None,
assemblyai: None,
google: None,
local_whisper: Some(crate::config::LocalWhisperConfig {
url: whisper_url,
bearer_token: "test_token".to_string(),
max_audio_bytes: 25_000_000,
timeout_secs: 300,
}),
transcribe_non_ptt_audio: false,
});
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "audio/ogg",
"name": "voice.ogg"
}
]
}
});
let result = ch.try_transcribe_audio_attachment(&post).await;
assert_eq!(result.as_deref(), Some("[Voice] test transcript"));
}
#[tokio::test]
async fn mattermost_audio_skips_non_audio_attachment() {
let mock_server = MockServer::start().await;
let ch = MattermostChannel::new(
mock_server.uri(),
"test_token".to_string(),
None,
vec!["*".into()],
false,
false,
)
.with_transcription(crate::config::TranscriptionConfig {
enabled: true,
default_provider: "local_whisper".to_string(),
api_key: None,
api_url: "https://api.groq.com/openai/v1/audio/transcriptions".to_string(),
model: "whisper-large-v3".to_string(),
language: None,
initial_prompt: None,
max_duration_secs: 600,
openai: None,
deepgram: None,
assemblyai: None,
google: None,
local_whisper: Some(crate::config::LocalWhisperConfig {
url: mock_server.uri(),
bearer_token: "test_token".to_string(),
max_audio_bytes: 25_000_000,
timeout_secs: 300,
}),
transcribe_non_ptt_audio: false,
});
let post = json!({
"metadata": {
"files": [
{
"id": "file1",
"mime_type": "image/png",
"name": "screenshot.png"
}
]
}
});
let result = ch.try_transcribe_audio_attachment(&post).await;
assert!(result.is_none());
}
}
}

View File

@ -0,0 +1,409 @@
//! Automatic media understanding pipeline for inbound channel messages.
//!
//! Pre-processes media attachments (audio, images, video) before the agent sees
//! the message, enriching the text with human-readable annotations:
//!
//! - **Audio**: transcribed via the existing [`super::transcription`] infrastructure,
//! prepended as `[Audio transcription: ...]`.
//! - **Images**: when a vision-capable provider is active, described as `[Image: <description>]`.
//! Falls back to `[Image: attached]` when vision is unavailable.
//! - **Video**: summarised as `[Video summary: ...]` when an API is available,
//! otherwise `[Video: attached]`.
//!
//! The pipeline is **opt-in** via `[media_pipeline] enabled = true` in config.
use crate::config::{MediaPipelineConfig, TranscriptionConfig};
/// Classifies an attachment by MIME type or file extension.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum MediaKind {
Audio,
Image,
Video,
Unknown,
}
/// A single media attachment on an inbound message.
#[derive(Debug, Clone)]
pub struct MediaAttachment {
/// Original file name (e.g. `voice.ogg`, `photo.jpg`).
pub file_name: String,
/// Raw bytes of the attachment.
pub data: Vec<u8>,
/// MIME type if known (e.g. `audio/ogg`, `image/jpeg`).
pub mime_type: Option<String>,
}
impl MediaAttachment {
/// Classify this attachment into a [`MediaKind`].
pub fn kind(&self) -> MediaKind {
// Try MIME type first.
if let Some(ref mime) = self.mime_type {
let lower = mime.to_ascii_lowercase();
if lower.starts_with("audio/") {
return MediaKind::Audio;
}
if lower.starts_with("image/") {
return MediaKind::Image;
}
if lower.starts_with("video/") {
return MediaKind::Video;
}
}
// Fall back to file extension.
let ext = self
.file_name
.rsplit_once('.')
.map(|(_, e)| e.to_ascii_lowercase())
.unwrap_or_default();
match ext.as_str() {
"flac" | "mp3" | "mpeg" | "mpga" | "m4a" | "ogg" | "oga" | "opus" | "wav" | "webm" => {
MediaKind::Audio
}
"png" | "jpg" | "jpeg" | "gif" | "bmp" | "webp" | "heic" | "tiff" | "svg" => {
MediaKind::Image
}
"mp4" | "mkv" | "avi" | "mov" | "wmv" | "flv" => MediaKind::Video,
_ => MediaKind::Unknown,
}
}
}
/// The media understanding pipeline.
///
/// Consumes a message's text and attachments, returning enriched text with
/// media annotations prepended.
pub struct MediaPipeline<'a> {
config: &'a MediaPipelineConfig,
transcription_config: &'a TranscriptionConfig,
vision_available: bool,
}
impl<'a> MediaPipeline<'a> {
/// Create a new pipeline. `vision_available` indicates whether the current
/// provider supports vision (image description).
pub fn new(
config: &'a MediaPipelineConfig,
transcription_config: &'a TranscriptionConfig,
vision_available: bool,
) -> Self {
Self {
config,
transcription_config,
vision_available,
}
}
/// Process a message's attachments and return enriched text.
///
/// If the pipeline is disabled via config, returns `original_text` unchanged.
pub async fn process(&self, original_text: &str, attachments: &[MediaAttachment]) -> String {
if !self.config.enabled || attachments.is_empty() {
return original_text.to_string();
}
let mut annotations = Vec::new();
for attachment in attachments {
match attachment.kind() {
MediaKind::Audio if self.config.transcribe_audio => {
let annotation = self.process_audio(attachment).await;
annotations.push(annotation);
}
MediaKind::Image if self.config.describe_images => {
let annotation = self.process_image(attachment);
annotations.push(annotation);
}
MediaKind::Video if self.config.summarize_video => {
let annotation = self.process_video(attachment);
annotations.push(annotation);
}
_ => {}
}
}
if annotations.is_empty() {
return original_text.to_string();
}
let mut enriched = String::with_capacity(
annotations.iter().map(|a| a.len() + 1).sum::<usize>() + original_text.len() + 2,
);
for annotation in &annotations {
enriched.push_str(annotation);
enriched.push('\n');
}
if !original_text.is_empty() {
enriched.push('\n');
enriched.push_str(original_text);
}
enriched.trim().to_string()
}
/// Transcribe an audio attachment using the existing transcription infra.
async fn process_audio(&self, attachment: &MediaAttachment) -> String {
if !self.transcription_config.enabled {
return "[Audio: attached]".to_string();
}
match super::transcription::transcribe_audio(
attachment.data.clone(),
&attachment.file_name,
self.transcription_config,
)
.await
{
Ok(text) => {
let trimmed = text.trim();
if trimmed.is_empty() {
"[Audio transcription: (empty)]".to_string()
} else {
format!("[Audio transcription: {trimmed}]")
}
}
Err(err) => {
tracing::warn!(
file = %attachment.file_name,
error = %err,
"Media pipeline: audio transcription failed"
);
"[Audio: transcription failed]".to_string()
}
}
}
/// Describe an image attachment.
///
/// When vision is available, the image will be passed through to the
/// provider as an `[IMAGE:]` marker and described by the model in the
/// normal flow. Here we only add a placeholder annotation so the agent
/// knows an image is present.
fn process_image(&self, attachment: &MediaAttachment) -> String {
if self.vision_available {
format!(
"[Image: {} attached, will be processed by vision model]",
attachment.file_name
)
} else {
format!("[Image: {} attached]", attachment.file_name)
}
}
/// Summarize a video attachment.
///
/// Video analysis requires external APIs not currently integrated.
/// For now we add a placeholder annotation.
fn process_video(&self, attachment: &MediaAttachment) -> String {
format!("[Video: {} attached]", attachment.file_name)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn default_pipeline_config(enabled: bool) -> MediaPipelineConfig {
MediaPipelineConfig {
enabled,
transcribe_audio: true,
describe_images: true,
summarize_video: true,
}
}
fn sample_audio() -> MediaAttachment {
MediaAttachment {
file_name: "voice.ogg".to_string(),
data: vec![0u8; 100],
mime_type: Some("audio/ogg".to_string()),
}
}
fn sample_image() -> MediaAttachment {
MediaAttachment {
file_name: "photo.jpg".to_string(),
data: vec![0u8; 50],
mime_type: Some("image/jpeg".to_string()),
}
}
fn sample_video() -> MediaAttachment {
MediaAttachment {
file_name: "clip.mp4".to_string(),
data: vec![0u8; 200],
mime_type: Some("video/mp4".to_string()),
}
}
#[test]
fn media_kind_from_mime() {
let audio = MediaAttachment {
file_name: "file".to_string(),
data: vec![],
mime_type: Some("audio/ogg".to_string()),
};
assert_eq!(audio.kind(), MediaKind::Audio);
let image = MediaAttachment {
file_name: "file".to_string(),
data: vec![],
mime_type: Some("image/png".to_string()),
};
assert_eq!(image.kind(), MediaKind::Image);
let video = MediaAttachment {
file_name: "file".to_string(),
data: vec![],
mime_type: Some("video/mp4".to_string()),
};
assert_eq!(video.kind(), MediaKind::Video);
}
#[test]
fn media_kind_from_extension() {
let audio = MediaAttachment {
file_name: "voice.ogg".to_string(),
data: vec![],
mime_type: None,
};
assert_eq!(audio.kind(), MediaKind::Audio);
let image = MediaAttachment {
file_name: "photo.png".to_string(),
data: vec![],
mime_type: None,
};
assert_eq!(image.kind(), MediaKind::Image);
let video = MediaAttachment {
file_name: "clip.mp4".to_string(),
data: vec![],
mime_type: None,
};
assert_eq!(video.kind(), MediaKind::Video);
let unknown = MediaAttachment {
file_name: "data.bin".to_string(),
data: vec![],
mime_type: None,
};
assert_eq!(unknown.kind(), MediaKind::Unknown);
}
#[tokio::test]
async fn disabled_pipeline_returns_original_text() {
let config = default_pipeline_config(false);
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, false);
let result = pipeline.process("hello", &[sample_audio()]).await;
assert_eq!(result, "hello");
}
#[tokio::test]
async fn empty_attachments_returns_original_text() {
let config = default_pipeline_config(true);
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, false);
let result = pipeline.process("hello", &[]).await;
assert_eq!(result, "hello");
}
#[tokio::test]
async fn image_annotation_with_vision() {
let config = default_pipeline_config(true);
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, true);
let result = pipeline.process("check this", &[sample_image()]).await;
assert!(
result.contains("[Image: photo.jpg attached, will be processed by vision model]"),
"expected vision annotation, got: {result}"
);
assert!(result.contains("check this"));
}
#[tokio::test]
async fn image_annotation_without_vision() {
let config = default_pipeline_config(true);
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, false);
let result = pipeline.process("check this", &[sample_image()]).await;
assert!(
result.contains("[Image: photo.jpg attached]"),
"expected basic image annotation, got: {result}"
);
}
#[tokio::test]
async fn video_annotation() {
let config = default_pipeline_config(true);
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, false);
let result = pipeline.process("watch", &[sample_video()]).await;
assert!(
result.contains("[Video: clip.mp4 attached]"),
"expected video annotation, got: {result}"
);
}
#[tokio::test]
async fn audio_without_transcription_enabled() {
let config = default_pipeline_config(true);
let mut tc = TranscriptionConfig::default();
tc.enabled = false;
let pipeline = MediaPipeline::new(&config, &tc, false);
let result = pipeline.process("", &[sample_audio()]).await;
assert_eq!(result, "[Audio: attached]");
}
#[tokio::test]
async fn multiple_attachments_produce_multiple_annotations() {
let config = default_pipeline_config(true);
let mut tc = TranscriptionConfig::default();
tc.enabled = false;
let pipeline = MediaPipeline::new(&config, &tc, false);
let attachments = vec![sample_audio(), sample_image(), sample_video()];
let result = pipeline.process("context", &attachments).await;
assert!(
result.contains("[Audio: attached]"),
"missing audio annotation"
);
assert!(
result.contains("[Image: photo.jpg attached]"),
"missing image annotation"
);
assert!(
result.contains("[Video: clip.mp4 attached]"),
"missing video annotation"
);
assert!(result.contains("context"), "missing original text");
}
#[tokio::test]
async fn disabled_sub_features_skip_processing() {
let config = MediaPipelineConfig {
enabled: true,
transcribe_audio: false,
describe_images: false,
summarize_video: false,
};
let tc = TranscriptionConfig::default();
let pipeline = MediaPipeline::new(&config, &tc, false);
let attachments = vec![sample_audio(), sample_image(), sample_video()];
let result = pipeline.process("hello", &attachments).await;
assert_eq!(result, "hello");
}
}

Some files were not shown because too many files have changed in this diff Show More