zeroclaw/tests
Aleksandr Prilipko 2df4e902f6
feat(providers): normalize image paths to data URIs in OpenAI Codex
Fix OpenAI Codex vision support by converting file paths to data URIs
before sending requests to the API.

## Problem

OpenAI Codex API was rejecting vision requests with 400 error:
"Invalid 'input[0].content[1].image_url'. Expected a valid URL,
but got a value with an invalid format."

Root cause: provider was sending raw file paths (e.g. `/tmp/test.png`)
instead of data URIs (e.g. `data:image/png;base64,...`).

## Solution

Add image normalization in both `chat_with_system` and `chat_with_history`:
- Call `multimodal::prepare_messages_for_provider()` before building request
- Converts file paths to base64 data URIs
- Validates image size and MIME type
- Works with both local files and remote URLs

## Changes

- `src/providers/openai_codex.rs`:
  - Normalize images in `chat_with_system()`
  - Normalize images in `chat_with_history()`
  - Simplify `ResponsesInputContent.image_url` from nested object to String
  - Fix unit test assertion for flat image_url structure

- `tests/openai_codex_vision_e2e.rs`:
  - Add E2E test for second profile vision support
  - Validates capabilities, request success, and response content

## Verification

 Unit tests pass: `cargo test --lib openai_codex`
 E2E test passes: `cargo test openai_codex_second_vision -- --ignored`
 Second profile accepts vision requests (200 OK)
 Returns correct image descriptions

## Impact

- Enables vision support for all OpenAI Codex profiles
- Second profile works without rate limits
- Fallback chain: default → second → gemini
- No breaking changes to existing non-vision flows

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 16:03:00 +08:00
..
fixtures fix(tools): file_read binary file support — PDF extraction + lossy fallback 2026-02-21 13:03:13 +08:00
agent_e2e.rs fix(provider): preserve reasoning_content in tool-call conversation history 2026-02-22 17:40:48 +08:00
agent_loop_robustness.rs fix(provider): preserve reasoning_content in tool-call conversation history 2026-02-22 17:40:48 +08:00
channel_routing.rs fix(channels): interrupt in-flight telegram requests on newer sender messages 2026-02-20 01:54:07 +08:00
config_persistence.rs fix(channels): interrupt in-flight telegram requests on newer sender messages 2026-02-20 01:54:07 +08:00
config_schema.rs feat(channel): split lark and feishu providers 2026-02-22 14:10:34 +08:00
dockerignore_test.rs chore: Remove blocking read strings 2026-02-19 14:52:29 +08:00
gemini_fallback_oauth_refresh.rs feat(providers): auto-refresh expired Gemini OAuth tokens in warmup 2026-02-24 16:03:00 +08:00
hooks_integration.rs chore(hooks): fix formatting and clippy warnings 2026-02-21 13:34:09 +08:00
memory_comparison.rs feat(memory): add session_id isolation to Memory trait (#530) 2026-02-17 07:44:05 -05:00
memory_restart.rs fix(channels): interrupt in-flight telegram requests on newer sender messages 2026-02-20 01:54:07 +08:00
openai_codex_vision_e2e.rs feat(providers): normalize image paths to data URIs in OpenAI Codex 2026-02-24 16:03:00 +08:00
otel_dependency_feature_regression.rs fix(observability): prevent otel reactor panic in non-tokio contexts 2026-02-20 16:07:50 +08:00
provider_resolution.rs chore(fmt): align formatting after main rebase 2026-02-21 14:54:19 +08:00
provider_schema.rs fix(provider): preserve reasoning_content in tool-call conversation history 2026-02-22 17:40:48 +08:00
reply_target_field_regression.rs fix(channels): enforce reply_target naming consistency 2026-02-18 19:56:31 +08:00
telegram_attachment_fallback.rs fix(telegram): fall back to text link when media-by-URL fails 2026-02-21 12:34:33 +08:00
whatsapp_webhook_security.rs fix(memory): prevent autosave key collisions across runtime flows 2026-02-15 22:55:52 -05:00