cc13fec16d
The first API request after daemon startup consistently timed out (120s) when using channels (Telegram, Discord, etc.), requiring a retry before succeeding. This happened because the reqwest HTTP client's connection pool was cold — no TLS handshake, DNS resolution, or HTTP/2 negotiation had occurred yet. The fix adds a `warmup()` method to the Provider trait that establishes the connection pool on startup by hitting a lightweight endpoint (`/api/v1/auth/key` for OpenRouter). The channel server calls this immediately after creating the provider, before entering the message processing loop. Tested on Raspberry Pi 5 (aarch64) with OpenRouter + DeepSeek v3.2 via Telegram channel. Before: first message took 2-7 minutes (120s timeout + retries). After: first message responds in <30s with no retries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
24 lines
696 B
Rust
24 lines
696 B
Rust
use async_trait::async_trait;
|
|
|
|
#[async_trait]
|
|
pub trait Provider: Send + Sync {
|
|
async fn chat(&self, message: &str, model: &str, temperature: f64) -> anyhow::Result<String> {
|
|
self.chat_with_system(None, message, model, temperature)
|
|
.await
|
|
}
|
|
|
|
async fn chat_with_system(
|
|
&self,
|
|
system_prompt: Option<&str>,
|
|
message: &str,
|
|
model: &str,
|
|
temperature: f64,
|
|
) -> anyhow::Result<String>;
|
|
|
|
/// Warm up the HTTP connection pool (TLS handshake, DNS, HTTP/2 setup).
|
|
/// Default implementation is a no-op; providers with HTTP clients should override.
|
|
async fn warmup(&self) -> anyhow::Result<()> {
|
|
Ok(())
|
|
}
|
|
}
|