polymech/mono
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gui - cleanup

Browse Source
This commit is contained in:
babayaga 2025-09-18 10:50:07 +02:00
parent de926c429d
commit 18d5c6f8f5
16 changed files with 1724 additions and 1659 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
packages/kbot/dev-kbot.code-workspace Normal file
View File

@ -0,0 +1,14 @@
{
"folders": [
{
"path": "."
},
{
"path": "../../../site2"
},
{
"path": "../../../polymech-astro/packages/polymech"
}
],
"settings": {}
}

16
packages/kbot/dist-in/commands/images.js
View File

File diff suppressed because one or more lines are too long

BIN
packages/kbot/dist/win-64/tauri-app.exe vendored
View File

Binary file not shown.

89
packages/kbot/docs/images-gui.md Normal file
View File

@ -0,0 +1,89 @@
# Image Command GUI (`--gui`)
## Overview
The `images` command includes a powerful Graphical User Interface (GUI) for interactive image generation and editing. By adding the `--gui` flag to your command, you launch a desktop application that provides a rich, user-friendly environment for working with images.
This mode is designed for an iterative workflow, allowing you to refine prompts, swap source images, and see results in real-time, which we refer to as "Chat Mode".
To launch the GUI, simply add the `--gui` flag:
```bash
kbot image --dst "my_artwork.png" --gui
```
## Features
- **Interactive Prompting**: A large text area to write and refine your image descriptions.
- **Source Image Management**: Easily add, view, and remove source images for editing tasks.
- **Image Gallery**: A comprehensive gallery that displays both your source images and all generated images from the current session. You can select any image to view it larger or use it as a source for the next generation.
- **Interactive Chat Mode**: Continuously generate images without restarting the command. Each generated image is added to the gallery, allowing you to build upon your ideas.
- **Output Configuration**: Specify the destination file path for the final image.
- **Debug Panel**: An advanced panel to inspect the Inter-Process Communication (IPC) messages between the GUI and the command-line tool.
## Workflow: Interactive Chat Mode
The GUI operates in a persistent "Chat Mode", which facilitates an iterative creation process. Here’s a typical workflow:
1. **Launch**: Start the GUI with initial parameters. For example, to start with a specific prompt and some source images:
```bash
kbot image --prompt "a cat wearing a wizard hat" --include "cat.jpg" --dst "wizard_cat.png" --gui
```
2. **Generate**: The GUI will open, pre-filled with your prompt and images. Click the **✨ Generate Image** button.
3. **Review**: The newly generated image appears in the image gallery. You can click on its thumbnail to view it in the main display.
4. **Iterate**: Now, you can:
* Modify the prompt (e.g., "a cat wearing a blue wizard hat with stars").
* Select the newly generated image from the gallery to use it as a source for the next edit.
* Add or remove other source images.
5. **Re-generate**: Click **✨ Generate Image** again. A new image will be generated and added to the gallery.
6. **Repeat**: Continue this cycle of refining and generating until you are satisfied with the result.
This loop of generating, reviewing, and refining is the core of the Chat Mode experience.
```mermaid
sequenceDiagram
participant User
participant GUI as React Frontend
participant CLI as images.ts
User->>GUI: Modifies prompt / selects images
User->>GUI: Clicks "Generate Image"
GUI->>CLI: Sends 'generate_request' (prompt, files, dst)
CLI->>CLI: Calls Image Generation API
CLI-->>GUI: Sends generated image back ('image-received' event)
GUI->>GUI: Adds new image to gallery
GUI->>User: Displays new image
User->>GUI: Continues iteration...
```
## Parameters & Configuration
The GUI can be pre-configured using arguments from the command line.
- `--prompt <string>`: Sets the initial text in the prompt box.
- `--include <file...>`: Populates the image gallery with one or more source images.
- `--dst <file>`: Sets the initial value for the output file path.
- `--api_key <key>`: Provides the necessary API key for image generation. If not provided, it will be loaded from your config file.
When the GUI starts, it sends a request to the `images.ts` process, which then provides this initial configuration data.
## Finalizing and Saving
Once you have a generated image you're happy with, you have two options:
1. **Simple Mode (Generation only)**: If you are only generating images and don't need to return a specific file to the calling process, you can save images directly from the gallery and close the GUI when you're done. *Note: Direct saving from the GUI is not fully implemented yet.*
2. **Submitting a Final Result**: To complete the `images` command and save the final output, click the **💾 Save Last Generated Image and Close** button. This action:
* Identifies the most recently generated image.
* Sends a final payload containing the prompt, source files, and destination path back to the `images.ts` process.
* Closes the GUI application.
The `images.ts` command then saves the final image to the specified `--dst` path and exits cleanly.
## Communication Protocol (IPC)
The GUI and the `images.ts` CLI process communicate using an Inter-Process Communication (IPC) system that sends JSON messages over the standard input/output streams (`stdin`/`stdout`).
- **CLI → GUI**: The CLI sends initial configuration, source images, and newly generated images to the GUI.
- **GUI → CLI**: The GUI sends requests to generate images and, finally, sends the chosen prompt and settings when the user clicks "Save and Close".
For a detailed breakdown of the IPC message formats and communication flow, please see the [IPC Communication Documentation](./ipc.md).

BIN
packages/kbot/fat_cat.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 MiB

844
packages/kbot/gui/tauri-app/src-tauri/src/lib.rs
View File

@ -1,418 +1,426 @@
use tauri::{Manager, Emitter};
use serde::{Serialize, Deserialize};
use dirs;
struct Counter(std::sync::Mutex<u32>);
struct DebugMessages(std::sync::Mutex<Vec<DebugPayload>>);
#[derive(Serialize, Deserialize)]
struct Payload {
prompt: String,
files: Vec<String>,
dst: String,
}
#[derive(Serialize, Deserialize)]
struct IPCMessage {
#[serde(rename = "type")]
message_type: String,
data: serde_json::Value,
timestamp: Option<u64>,
id: Option<String>,
}
#[derive(Serialize, Deserialize)]
struct CounterPayload {
count: u32,
message: Option<String>,
}
#[derive(Serialize, Deserialize, Clone)]
struct DebugPayload {
level: String,
message: String,
data: Option<serde_json::Value>,
}
#[derive(Serialize, Deserialize)]
struct ImagePayload {
base64: String,
#[serde(rename = "mimeType")]
mime_type: String,
filename: Option<String>,
}
// Learn more about Tauri commands at https://tauri.app/develop/calling-rust/
#[tauri::command]
fn submit_prompt(prompt: &str, files: Vec<String>, dst: &str, window: tauri::Window) {
// Use eprintln! for debug logs so they go to stderr, not stdout
eprintln!("[RUST LOG]: submit_prompt command called.");
eprintln!("[RUST LOG]: - Prompt: {}", prompt);
eprintln!("[RUST LOG]: - Files: {:?}", files);
eprintln!("[RUST LOG]: - Dst: {}", dst);
let payload = Payload {
prompt: prompt.to_string(),
files,
dst: dst.to_string(),
};
let json_payload = serde_json::to_string(&payload).unwrap();
eprintln!("[RUST LOG]: - Sending JSON payload to stdout: {}", json_payload);
println!("{}", json_payload); // The actual payload - ONLY this should go to stdout
let _ = window.app_handle().exit(0);
}
#[tauri::command]
fn log_error_to_console(error: &str) {
eprintln!("[WebView ERROR forwarded from JS]: {}", error);
}
#[tauri::command]
fn resolve_path_relative_to_home(absolute_path: String) -> Result<String, String> {
eprintln!("[RUST LOG]: resolve_path_relative_to_home command called.");
eprintln!("[RUST LOG]: - Received absolute path: {}", absolute_path);
let home_dir = dirs::home_dir().ok_or_else(|| "Could not find home directory".to_string())?;
let path_to_resolve = std::path::Path::new(&absolute_path);
let relative_path = pathdiff::diff_paths(path_to_resolve, home_dir)
.ok_or_else(|| "Failed to calculate relative path from home directory".to_string())?;
let result = relative_path.to_string_lossy().to_string();
eprintln!("[RUST LOG]: - Resolved to path relative to home: {}", result);
Ok(result)
}
#[tauri::command]
fn increment_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: increment_counter command called.");
let mut counter = state.0.lock().unwrap();
*counter += 1;
let current_value = *counter;
eprintln!("[RUST LOG]: - Counter incremented to: {}", current_value);
Ok(current_value)
}
#[tauri::command]
fn get_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: get_counter command called.");
let counter = state.0.lock().unwrap();
let current_value = *counter;
eprintln!("[RUST LOG]: - Current counter value: {}", current_value);
Ok(current_value)
}
#[tauri::command]
fn reset_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: reset_counter command called.");
let mut counter = state.0.lock().unwrap();
*counter = 0;
eprintln!("[RUST LOG]: - Counter reset to: 0");
Ok(0)
}
#[tauri::command]
fn add_debug_message(message: String, level: String, data: Option<serde_json::Value>, state: tauri::State<'_, DebugMessages>) -> Result<(), String> {
eprintln!("[RUST LOG]: add_debug_message command called.");
eprintln!("[RUST LOG]: - Level: {}", level);
eprintln!("[RUST LOG]: - Message: {}", message);
let debug_payload = DebugPayload {
level,
message,
data,
};
let mut messages = state.0.lock().unwrap();
messages.push(debug_payload);
// Keep only the last 100 messages to prevent memory issues
if messages.len() > 100 {
let len = messages.len();
messages.drain(0..len - 100);
}
eprintln!("[RUST LOG]: - Debug message added. Total messages: {}", messages.len());
Ok(())
}
#[tauri::command]
fn get_debug_messages(state: tauri::State<'_, DebugMessages>) -> Result<Vec<DebugPayload>, String> {
eprintln!("[RUST LOG]: get_debug_messages command called.");
let messages = state.0.lock().unwrap();
let result = messages.clone();
eprintln!("[RUST LOG]: - Returning {} debug messages", result.len());
Ok(result)
}
#[tauri::command]
fn clear_debug_messages(state: tauri::State<'_, DebugMessages>) -> Result<(), String> {
eprintln!("[RUST LOG]: clear_debug_messages command called.");
let mut messages = state.0.lock().unwrap();
messages.clear();
eprintln!("[RUST LOG]: - Debug messages cleared");
Ok(())
}
#[tauri::command]
fn send_ipc_message(message_type: String, data: serde_json::Value, _window: tauri::Window) -> Result<(), String> {
eprintln!("[RUST LOG]: send_ipc_message command called.");
eprintln!("[RUST LOG]: - Type: {}", message_type);
eprintln!("[RUST LOG]: - Data: {}", data);
let ipc_message = IPCMessage {
message_type,
data,
timestamp: Some(std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64),
id: Some(format!("msg_{}_{}",
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis(),
rand::random::<u32>())),
};
let json_message = serde_json::to_string(&ipc_message).unwrap();
eprintln!("[RUST LOG]: - Sending IPC message to stdout: {}", json_message);
println!("{}", json_message);
Ok(())
}
#[tauri::command]
fn send_message_to_stdout(message: String) -> Result<(), String> {
eprintln!("[RUST LOG]: send_message_to_stdout command called.");
eprintln!("[RUST LOG]: - Message: {}", message);
// Send directly to stdout (this will be captured by images.ts)
println!("{}", message);
Ok(())
}
#[tauri::command]
fn generate_image_via_backend(prompt: String, files: Vec<String>, dst: String) -> Result<(), String> {
eprintln!("[RUST LOG]: generate_image_via_backend called");
eprintln!("[RUST LOG]: - Prompt: {}", prompt);
eprintln!("[RUST LOG]: - Files: {:?}", files);
eprintln!("[RUST LOG]: - Dst: {}", dst);
// Send generation request to images.ts via stdout
let request = serde_json::json!({
"type": "generate_request",
"prompt": prompt,
"files": files,
"dst": dst,
"timestamp": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis()
});
println!("{}", serde_json::to_string(&request).unwrap());
eprintln!("[RUST LOG]: Generation request sent to images.ts");
Ok(())
}
#[tauri::command]
fn request_config_from_images(_app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: request_config_from_images called");
// Send request to images.ts via stdout
let request = serde_json::json!({
"type": "config_request",
"timestamp": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis()
});
println!("{}", serde_json::to_string(&request).unwrap());
eprintln!("[RUST LOG]: Config request sent to images.ts");
Ok(())
}
#[tauri::command]
fn forward_config_to_frontend(prompt: Option<String>, dst: Option<String>, api_key: Option<String>, files: Vec<String>, app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: forward_config_to_frontend called");
let config_data = serde_json::json!({
"prompt": prompt,
"dst": dst,
"apiKey": api_key,
"files": files
});
if let Err(e) = app.emit("config-received", &config_data) {
eprintln!("[RUST LOG]: Failed to emit config-received: {}", e);
return Err(format!("Failed to emit config: {}", e));
}
eprintln!("[RUST LOG]: Config forwarded to frontend successfully");
Ok(())
}
#[tauri::command]
fn forward_image_to_frontend(base64: String, mime_type: String, filename: String, app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: forward_image_to_frontend called for {}", filename);
let image_data = serde_json::json!({
"base64": base64,
"mimeType": mime_type,
"filename": filename
});
if let Err(e) = app.emit("image-received", &image_data) {
eprintln!("[RUST LOG]: Failed to emit image-received: {}", e);
return Err(format!("Failed to emit image: {}", e));
}
eprintln!("[RUST LOG]: Image forwarded to frontend successfully");
Ok(())
}
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
let app = tauri::Builder::default()
.manage(Counter(std::sync::Mutex::new(0)))
.manage(DebugMessages(std::sync::Mutex::new(Vec::new())))
.plugin(tauri_plugin_opener::init())
.plugin(tauri_plugin_dialog::init())
.plugin(tauri_plugin_fs::init())
.plugin(tauri_plugin_http::init())
.invoke_handler(tauri::generate_handler![
submit_prompt,
log_error_to_console,
resolve_path_relative_to_home,
increment_counter,
get_counter,
reset_counter,
add_debug_message,
get_debug_messages,
clear_debug_messages,
send_message_to_stdout,
send_ipc_message,
request_config_from_images,
forward_config_to_frontend,
forward_image_to_frontend,
generate_image_via_backend
])
.setup(|app| {
let app_handle = app.handle().clone();
// Listen for stdin commands from images.ts
std::thread::spawn(move || {
use std::io::{self, BufRead, BufReader};
let stdin = io::stdin();
let reader = BufReader::new(stdin);
eprintln!("[RUST LOG]: Stdin listener thread started");
for line in reader.lines() {
if let Ok(line_content) = line {
if line_content.trim().is_empty() {
continue;
}
// Log stdin command but hide binary data
let log_content = if line_content.contains("\"base64\"") {
format!("[COMMAND WITH BASE64 DATA - {} chars]", line_content.len())
} else {
line_content.clone()
};
eprintln!("[RUST LOG]: Received stdin command: {}", log_content);
// Parse command from images.ts
if let Ok(command) = serde_json::from_str::<serde_json::Value>(&line_content) {
if let Some(cmd) = command.get("cmd").and_then(|v| v.as_str()) {
eprintln!("[RUST LOG]: Processing command: {}", cmd);
match cmd {
"forward_config_to_frontend" => {
eprintln!("[RUST LOG]: Forwarding config to frontend");
eprintln!("[RUST LOG]: - prompt: {:?}", command.get("prompt"));
eprintln!("[RUST LOG]: - dst: {:?}", command.get("dst"));
eprintln!("[RUST LOG]: - apiKey: {:?}", command.get("apiKey").map(|_| "[REDACTED]"));
eprintln!("[RUST LOG]: - files: {:?}", command.get("files"));
let config_data = serde_json::json!({
"prompt": command.get("prompt"),
"dst": command.get("dst"),
"apiKey": command.get("apiKey"),
"files": command.get("files")
});
if let Err(e) = app_handle.emit("config-received", &config_data) {
eprintln!("[RUST LOG]: Failed to emit config-received: {}", e);
} else {
eprintln!("[RUST LOG]: Config emitted successfully to frontend");
}
}
"forward_image_to_frontend" => {
if let (Some(filename), Some(base64), Some(mime_type)) = (
command.get("filename").and_then(|v| v.as_str()),
command.get("base64").and_then(|v| v.as_str()),
command.get("mimeType").and_then(|v| v.as_str())
) {
eprintln!("[RUST LOG]: Forwarding image to frontend: {}", filename);
let image_data = serde_json::json!({
"base64": base64,
"mimeType": mime_type,
"filename": filename
});
if let Err(e) = app_handle.emit("image-received", &image_data) {
eprintln!("[RUST LOG]: Failed to emit image-received: {}", e);
} else {
eprintln!("[RUST LOG]: Image emitted successfully: {}", filename);
}
}
}
"generation_result" => {
eprintln!("[RUST LOG]: Forwarding generation result to frontend");
if let Err(e) = app_handle.emit("generation-result", &command) {
eprintln!("[RUST LOG]: Failed to emit generation-result: {}", e);
} else {
eprintln!("[RUST LOG]: Generation result emitted successfully");
}
}
"generation_error" => {
eprintln!("[RUST LOG]: Forwarding generation error to frontend");
if let Err(e) = app_handle.emit("generation-error", &command) {
eprintln!("[RUST LOG]: Failed to emit generation-error: {}", e);
} else {
eprintln!("[RUST LOG]: Generation error emitted successfully");
}
}
_ => {
eprintln!("[RUST LOG]: Unknown command: {}", cmd);
}
}
}
} else {
eprintln!("[RUST LOG]: Failed to parse command as JSON");
}
}
}
eprintln!("[RUST LOG]: Stdin listener thread ended");
});
Ok(())
})
.build(tauri::generate_context!())
.expect("error while building tauri application");
app.run(|_app_handle, event| match event {
tauri::RunEvent::ExitRequested { api, .. } => {
api.prevent_exit();
}
_ => {}
});
}
use tauri::{Manager, Emitter};
use serde::{Serialize, Deserialize};
use dirs;
struct Counter(std::sync::Mutex<u32>);
struct DebugMessages(std::sync::Mutex<Vec<DebugPayload>>);
#[derive(Serialize, Deserialize)]
struct Payload {
prompt: String,
files: Vec<String>,
dst: String,
}
#[derive(Serialize, Deserialize)]
struct IPCMessage {
#[serde(rename = "type")]
message_type: String,
data: serde_json::Value,
timestamp: Option<u64>,
id: Option<String>,
}
#[derive(Serialize, Deserialize)]
struct CounterPayload {
count: u32,
message: Option<String>,
}
#[derive(Serialize, Deserialize, Clone)]
struct DebugPayload {
level: String,
message: String,
data: Option<serde_json::Value>,
}
#[derive(Serialize, Deserialize)]
struct ImagePayload {
base64: String,
#[serde(rename = "mimeType")]
mime_type: String,
filename: Option<String>,
}
// Learn more about Tauri commands at https://tauri.app/develop/calling-rust/
#[tauri::command]
fn submit_prompt(prompt: &str, files: Vec<String>, dst: &str, window: tauri::Window) {
// Use eprintln! for debug logs so they go to stderr, not stdout
eprintln!("[RUST LOG]: submit_prompt command called.");
eprintln!("[RUST LOG]: - Prompt: {}", prompt);
eprintln!("[RUST LOG]: - Files: {:?}", files);
eprintln!("[RUST LOG]: - Dst: {}", dst);
let payload = Payload {
prompt: prompt.to_string(),
files,
dst: dst.to_string(),
};
let json_payload = serde_json::to_string(&payload).unwrap();
eprintln!("[RUST LOG]: - Sending JSON payload to stdout: {}", json_payload);
println!("{}", json_payload); // The actual payload - ONLY this should go to stdout
let _ = window.app_handle().exit(0);
}
#[tauri::command]
fn log_error_to_console(error: &str) {
eprintln!("[WebView ERROR forwarded from JS]: {}", error);
}
#[tauri::command]
fn resolve_path_relative_to_home(absolute_path: String) -> Result<String, String> {
eprintln!("[RUST LOG]: resolve_path_relative_to_home command called.");
eprintln!("[RUST LOG]: - Received absolute path: {}", absolute_path);
let home_dir = dirs::home_dir().ok_or_else(|| "Could not find home directory".to_string())?;
let path_to_resolve = std::path::Path::new(&absolute_path);
let relative_path = pathdiff::diff_paths(path_to_resolve, home_dir)
.ok_or_else(|| "Failed to calculate relative path from home directory".to_string())?;
let result = relative_path.to_string_lossy().to_string();
eprintln!("[RUST LOG]: - Resolved to path relative to home: {}", result);
Ok(result)
}
#[tauri::command]
fn increment_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: increment_counter command called.");
let mut counter = state.0.lock().unwrap();
*counter += 1;
let current_value = *counter;
eprintln!("[RUST LOG]: - Counter incremented to: {}", current_value);
Ok(current_value)
}
#[tauri::command]
fn get_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: get_counter command called.");
let counter = state.0.lock().unwrap();
let current_value = *counter;
eprintln!("[RUST LOG]: - Current counter value: {}", current_value);
Ok(current_value)
}
#[tauri::command]
fn reset_counter(state: tauri::State<'_, Counter>) -> Result<u32, String> {
eprintln!("[RUST LOG]: reset_counter command called.");
let mut counter = state.0.lock().unwrap();
*counter = 0;
eprintln!("[RUST LOG]: - Counter reset to: 0");
Ok(0)
}
#[tauri::command]
fn add_debug_message(message: String, level: String, data: Option<serde_json::Value>, state: tauri::State<'_, DebugMessages>) -> Result<(), String> {
eprintln!("[RUST LOG]: add_debug_message command called.");
eprintln!("[RUST LOG]: - Level: {}", level);
eprintln!("[RUST LOG]: - Message: {}", message);
let debug_payload = DebugPayload {
level,
message,
data,
};
let mut messages = state.0.lock().unwrap();
messages.push(debug_payload);
// Keep only the last 100 messages to prevent memory issues
if messages.len() > 100 {
let len = messages.len();
messages.drain(0..len - 100);
}
eprintln!("[RUST LOG]: - Debug message added. Total messages: {}", messages.len());
Ok(())
}
#[tauri::command]
fn get_debug_messages(state: tauri::State<'_, DebugMessages>) -> Result<Vec<DebugPayload>, String> {
eprintln!("[RUST LOG]: get_debug_messages command called.");
let messages = state.0.lock().unwrap();
let result = messages.clone();
eprintln!("[RUST LOG]: - Returning {} debug messages", result.len());
Ok(result)
}
#[tauri::command]
fn clear_debug_messages(state: tauri::State<'_, DebugMessages>) -> Result<(), String> {
eprintln!("[RUST LOG]: clear_debug_messages command called.");
let mut messages = state.0.lock().unwrap();
messages.clear();
eprintln!("[RUST LOG]: - Debug messages cleared");
Ok(())
}
#[tauri::command]
fn send_ipc_message(message_type: String, data: serde_json::Value, _window: tauri::Window) -> Result<(), String> {
eprintln!("[RUST LOG]: send_ipc_message command called.");
eprintln!("[RUST LOG]: - Type: {}", message_type);
eprintln!("[RUST LOG]: - Data: {}", data);
let ipc_message = IPCMessage {
message_type,
data,
timestamp: Some(std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64),
id: Some(format!("msg_{}_{}",
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis(),
rand::random::<u32>())),
};
let json_message = serde_json::to_string(&ipc_message).unwrap();
eprintln!("[RUST LOG]: - Sending IPC message to stdout: {}", json_message);
println!("{}", json_message);
Ok(())
}
#[tauri::command]
fn send_message_to_stdout(message: String) -> Result<(), String> {
eprintln!("[RUST LOG]: send_message_to_stdout command called.");
eprintln!("[RUST LOG]: - Message: {}", message);
// Send directly to stdout (this will be captured by images.ts)
println!("{}", message);
Ok(())
}
#[tauri::command]
fn generate_image_via_backend(prompt: String, files: Vec<String>, dst: String) -> Result<(), String> {
eprintln!("[RUST LOG]: generate_image_via_backend called");
eprintln!("[RUST LOG]: - Prompt: {}", prompt);
eprintln!("[RUST LOG]: - Files: {:?}", files);
eprintln!("[RUST LOG]: - Dst: {}", dst);
// Send generation request to images.ts via stdout
let request = serde_json::json!({
"type": "generate_request",
"prompt": prompt,
"files": files,
"dst": dst,
"timestamp": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis()
});
println!("{}", serde_json::to_string(&request).unwrap());
eprintln!("[RUST LOG]: Generation request sent to images.ts");
Ok(())
}
#[tauri::command]
fn request_config_from_images(_app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: request_config_from_images called");
// Send request to images.ts via stdout
let request = serde_json::json!({
"type": "config_request",
"timestamp": std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis()
});
println!("{}", serde_json::to_string(&request).unwrap());
eprintln!("[RUST LOG]: Config request sent to images.ts");
Ok(())
}
#[tauri::command]
fn forward_config_to_frontend(prompt: Option<String>, dst: Option<String>, api_key: Option<String>, files: Vec<String>, app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: forward_config_to_frontend called");
let config_data = serde_json::json!({
"prompt": prompt,
"dst": dst,
"apiKey": api_key,
"files": files
});
if let Err(e) = app.emit("config-received", &config_data) {
eprintln!("[RUST LOG]: Failed to emit config-received: {}", e);
return Err(format!("Failed to emit config: {}", e));
}
eprintln!("[RUST LOG]: Config forwarded to frontend successfully");
Ok(())
}
#[tauri::command]
fn forward_image_to_frontend(base64: String, mime_type: String, filename: String, app: tauri::AppHandle) -> Result<(), String> {
eprintln!("[RUST LOG]: forward_image_to_frontend called for {}", filename);
let image_data = serde_json::json!({
"base64": base64,
"mimeType": mime_type,
"filename": filename
});
if let Err(e) = app.emit("image-received", &image_data) {
eprintln!("[RUST LOG]: Failed to emit image-received: {}", e);
return Err(format!("Failed to emit image: {}", e));
}
eprintln!("[RUST LOG]: Image forwarded to frontend successfully");
Ok(())
}
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
let app = tauri::Builder::default()
.manage(Counter(std::sync::Mutex::new(0)))
.manage(DebugMessages(std::sync::Mutex::new(Vec::new())))
.plugin(tauri_plugin_opener::init())
.plugin(tauri_plugin_dialog::init())
.plugin(tauri_plugin_fs::init())
.plugin(tauri_plugin_http::init())
.invoke_handler(tauri::generate_handler![
submit_prompt,
log_error_to_console,
resolve_path_relative_to_home,
increment_counter,
get_counter,
reset_counter,
add_debug_message,
get_debug_messages,
clear_debug_messages,
send_message_to_stdout,
send_ipc_message,
request_config_from_images,
forward_config_to_frontend,
forward_image_to_frontend,
generate_image_via_backend
])
.setup(|app| {
let app_handle = app.handle().clone();
// Listen for stdin commands from images.ts
std::thread::spawn(move || {
use std::io::{self, BufRead, BufReader};
let stdin = io::stdin();
let reader = BufReader::new(stdin);
eprintln!("[RUST LOG]: Stdin listener thread started");
for line in reader.lines() {
if let Ok(line_content) = line {
if line_content.trim().is_empty() {
continue;
}
// Log stdin command but hide binary data
let log_content = if line_content.contains("\"base64\"") {
format!("[COMMAND WITH BASE64 DATA - {} chars]", line_content.len())
} else {
line_content.clone()
};
eprintln!("[RUST LOG]: Received stdin command: {}", log_content);
// Parse command from images.ts
if let Ok(command) = serde_json::from_str::<serde_json::Value>(&line_content) {
if let Some(cmd) = command.get("cmd").and_then(|v| v.as_str()) {
eprintln!("[RUST LOG]: Processing command: {}", cmd);
match cmd {
"forward_config_to_frontend" => {
eprintln!("[RUST LOG]: Forwarding config to frontend");
eprintln!("[RUST LOG]: - prompt: {:?}", command.get("prompt"));
eprintln!("[RUST LOG]: - dst: {:?}", command.get("dst"));
eprintln!("[RUST LOG]: - apiKey: {:?}", command.get("apiKey").map(|_| "[REDACTED]"));
eprintln!("[RUST LOG]: - files: {:?}", command.get("files"));
let config_data = serde_json::json!({
"prompt": command.get("prompt"),
"dst": command.get("dst"),
"apiKey": command.get("apiKey"),
"files": command.get("files")
});
if let Err(e) = app_handle.emit("config-received", &config_data) {
eprintln!("[RUST LOG]: Failed to emit config-received: {}", e);
} else {
eprintln!("[RUST LOG]: Config emitted successfully to frontend");
}
}
"forward_image_to_frontend" => {
if let (Some(filename), Some(base64), Some(mime_type)) = (
command.get("filename").and_then(|v| v.as_str()),
command.get("base64").and_then(|v| v.as_str()),
command.get("mimeType").and_then(|v| v.as_str())
) {
eprintln!("[RUST LOG]: Forwarding image to frontend: {}", filename);
let image_data = serde_json::json!({
"base64": base64,
"mimeType": mime_type,
"filename": filename
});
if let Err(e) = app_handle.emit("image-received", &image_data) {
eprintln!("[RUST LOG]: Failed to emit image-received: {}", e);
} else {
eprintln!("[RUST LOG]: Image emitted successfully: {}", filename);
}
}
}
"generation_result" => {
eprintln!("[RUST LOG]: Forwarding generation result to frontend");
if let Err(e) = app_handle.emit("generation-result", &command) {
eprintln!("[RUST LOG]: Failed to emit generation-result: {}", e);
} else {
eprintln!("[RUST LOG]: Generation result emitted successfully");
}
}
"generation_error" => {
eprintln!("[RUST LOG]: Forwarding generation error to frontend");
if let Err(e) = app_handle.emit("generation-error", &command) {
eprintln!("[RUST LOG]: Failed to emit generation-error: {}", e);
} else {
eprintln!("[RUST LOG]: Generation error emitted successfully");
}
}
"generation_complete" => {
eprintln!("[RUST LOG]: Generation completed successfully");
if let Err(e) = app_handle.emit("generation-complete", &command) {
eprintln!("[RUST LOG]: Failed to emit generation-complete: {}", e);
} else {
eprintln!("[RUST LOG]: Generation complete emitted successfully");
}
}
_ => {
eprintln!("[RUST LOG]: Unknown command: {}", cmd);
}
}
}
} else {
eprintln!("[RUST LOG]: Failed to parse command as JSON");
}
}
}
eprintln!("[RUST LOG]: Stdin listener thread ended");
});
Ok(())
})
.build(tauri::generate_context!())
.expect("error while building tauri application");
app.run(|_app_handle, event| match event {
tauri::RunEvent::ExitRequested { api, .. } => {
api.prevent_exit();
}
_ => {}
});
}

1406
packages/kbot/gui/tauri-app/src/App.tsx
View File

File diff suppressed because it is too large Load Diff

154
packages/kbot/gui/tauri-app/src/components/DebugPanel.tsx Normal file
View File

@ -0,0 +1,154 @@
import React from 'react';
interface DebugPanelProps {
debugMessages: any[];
addDebugMessage: (level: 'info' | 'warn' | 'error' | 'debug', message: string, data?: any) => void;
sendIPCMessage: (messageType: string, data: any) => void;
clearDebugMessages: () => void;
ipcInitialized: boolean;
messageToSend: string;
setMessageToSend: (message: string) => void;
sendMessageToImages: () => void;
}
const DebugPanel: React.FC<DebugPanelProps> = ({
debugMessages,
addDebugMessage,
sendIPCMessage,
clearDebugMessages,
ipcInitialized,
messageToSend,
setMessageToSend,
sendMessageToImages,
}) => {
return (
<div className="w-full mt-12">
<div className="glass-card p-6 glass-shimmer shadow-xl">
<div className="flex justify-between items-center mb-6">
<h2 className="text-2xl font-bold accent-text">Debug Panel</h2>
<div className="flex gap-3">
<button
onClick={() => addDebugMessage('info', 'Test info message', { test: 'data' })}
className="glass-button text-sm px-4 py-2 rounded-lg"
>
Test Info
</button>
<button
onClick={() => addDebugMessage('error', 'Test error message')}
className="glass-button text-sm px-4 py-2 rounded-lg border-red-400/50 text-red-600 hover:bg-red-500/20"
>
Test Error
</button>
<button
onClick={() => sendIPCMessage('test-message', { content: 'Hello from GUI', timestamp: Date.now() })}
className="glass-button text-sm px-4 py-2 rounded-lg border-blue-400/50 text-blue-600 hover:bg-blue-500/20"
>
Send IPC
</button>
<button
onClick={clearDebugMessages}
className="glass-button text-sm px-4 py-2 rounded-lg border-gray-400/50 text-gray-600 hover:bg-gray-500/20"
>
Clear
</button>
</div>
</div>
<div className="glass-card max-h-96 overflow-y-auto">
{debugMessages.length === 0 ? (
<div className="p-4 text-center text-slate-500 dark:text-slate-400">
No debug messages yet. Click the test buttons above to generate some.
</div>
) : (
<div className="divide-y divide-slate-200/50 dark:divide-slate-700/50">
{debugMessages.map((msg, index) => (
<div key={index} className={`p-3 hover:bg-slate-50/50 dark:hover:bg-slate-800/50 ${msg.isIPC ? 'border-l-4 border-blue-400 bg-blue-50/30 dark:bg-blue-900/10' : ''}`}>
<div className="flex items-start justify-between gap-3">
<div className="flex-1">
<div className="flex items-center gap-2 mb-1">
<span className={`text-xs px-2 py-1 rounded-full font-semibold uppercase ${msg.isIPC ? 'bg-blue-100 text-blue-700 dark:bg-blue-900/30 dark:text-blue-400 border border-blue-300' : msg.level === 'error' ? 'bg-red-100 text-red-700 dark:bg-red-900/30 dark:text-red-400' : msg.level === 'warn' ? 'bg-yellow-100 text-yellow-700 dark:bg-yellow-900/30 dark:text-yellow-400' : msg.level === 'debug' ? 'bg-purple-100 text-purple-700 dark:bg-purple-900/30 dark:text-purple-400' : 'bg-blue-100 text-blue-700 dark:bg-blue-900/30 dark:text-blue-400'}`}>
{msg.isIPC ? '📨 IPC' : msg.level}
</span>
<span className="text-xs text-slate-500 dark:text-slate-400">{msg.timestamp}</span>
{msg.isIPC && msg.data?.id && (
<span className="text-xs text-blue-500 dark:text-blue-400 bg-blue-100/50 dark:bg-blue-900/20 px-1 rounded font-mono">
{msg.data.id.split('_').pop()}
</span>
)}
</div>
<div className="text-sm text-slate-700 dark:text-slate-300 mb-1">{msg.message}</div>
{msg.data && (
<div className="text-xs text-slate-500 dark:text-slate-400 bg-slate-100/50 dark:bg-slate-800/50 p-2 rounded font-mono max-h-40 overflow-y-auto">
{JSON.stringify(msg.data, null, 2)}
</div>
)}
</div>
</div>
</div>
))}
</div>
)}
</div>
<div className="mt-6 glass-card p-4">
<div className="flex justify-between items-center mb-3">
<h3 className="text-lg font-semibold accent-text">Send Message to images.ts</h3>
<span className={`text-xs text-slate-500 dark:text-slate-400 bg-slate-100/50 dark:bg-slate-800/50 px-2 py-1 rounded ${ipcInitialized ? 'text-green-600' : 'text-red-600'}`}>
{ipcInitialized ? '🟢 Connected' : '🔴 Disconnected'}
</span>
</div>
<div className="flex gap-3">
<textarea
value={messageToSend}
onChange={(e) => setMessageToSend(e.target.value)}
placeholder="Type a message to send to images.ts process..."
className="flex-1 glass-input p-3 rounded-lg min-h-[80px] resize-none text-sm"
rows={3}
onKeyDown={(e) => {
if (e.key === 'Enter' && (e.ctrlKey || e.metaKey)) {
e.preventDefault();
sendMessageToImages();
}
}}
/>
<div className="flex flex-col gap-2">
<button
onClick={sendMessageToImages}
disabled={!messageToSend.trim()}
className="glass-button px-4 py-2 rounded-lg border-blue-400/50 text-blue-600 hover:bg-blue-500/20 disabled:opacity-50 disabled:cursor-not-allowed text-sm font-semibold"
title="Send message (Ctrl+Enter)"
>
📤 Send
</button>
<button
onClick={() => {
setMessageToSend('Hello from GUI!');
}}
className="glass-button px-4 py-2 rounded-lg border-green-400/50 text-green-600 hover:bg-green-500/20 text-xs"
title="Quick test message"
>
Test
</button>
<button
onClick={() => {
const echoMsg = `Echo: ${Date.now()}`;
setMessageToSend(echoMsg);
setTimeout(() => sendMessageToImages(), 100);
}}
className="glass-button px-4 py-2 rounded-lg border-orange-400/50 text-orange-600 hover:bg-orange-500/20 text-xs"
title="Send echo test"
>
Echo
</button>
</div>
</div>
<div className="mt-2 text-xs text-slate-500 dark:text-slate-400">
💡 Press <kbd className="px-1 py-0.5 bg-slate-200 dark:bg-slate-700 rounded text-xs">Ctrl+Enter</kbd> to send quickly
</div>
</div>
</div>
</div>
);
};
export default DebugPanel;

73
packages/kbot/gui/tauri-app/src/components/Header.tsx Normal file
View File

@ -0,0 +1,73 @@
import React from 'react';
interface HeaderProps {
showDebugPanel: boolean;
setShowDebugPanel: (show: boolean) => void;
isDarkMode: boolean;
toggleTheme: () => void;
apiKey: string;
}
const Header: React.FC<HeaderProps> = ({
showDebugPanel,
setShowDebugPanel,
isDarkMode,
toggleTheme,
apiKey,
}) => {
return (
<div className="flex justify-between items-center mb-8">
<h1 className="text-5xl font-bold accent-text drop-shadow-sm">Image Prompt</h1>
<div className="flex items-center gap-4">
{/* Debug Panel Toggle */}
<button
onClick={() => setShowDebugPanel(!showDebugPanel)}
className="glass-button p-3 rounded-xl hover:shadow-lg transition-all duration-300"
title="Toggle Debug Panel"
>
<svg className="w-5 h-5 text-slate-700 dark:text-slate-300" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M18 10a8 8 0 11-16 0 8 8 0 0116 0zm-8-3a1 1 0 00-.867.5 1 1 0 11-1.731-1A3 3 0 0113 8a3.001 3.001 0 01-2 2.83V11a1 1 0 11-2 0v-1a1 1 0 011-1 1 1 0 100-2zm0 8a1 1 0 100-2 1 1 0 000 2z"
clipRule="evenodd"
/>
</svg>
</button>
{/* Theme Toggle */}
<button
onClick={toggleTheme}
className="glass-button p-3 rounded-xl hover:shadow-lg transition-all duration-300"
title={isDarkMode ? 'Switch to Light Mode' : 'Switch to Dark Mode'}
>
{isDarkMode ? (
<svg className="w-5 h-5 text-slate-300" fill="currentColor" viewBox="0 0 20 20">
<path
fillRule="evenodd"
d="M10 2a1 1 0 011 1v1a1 1 0 11-2 0V3a1 1 0 011-1zm4 8a4 4 0 11-8 0 4 4 0 018 0zm-.464 4.95l.707.707a1 1 0 001.414-1.414l-.707-.707a1 1 0 00-1.414 1.414zm2.12-10.607a1 1 0 010 1.414l-.706.707a1 1 0 11-1.414-1.414l.707-.707a1 1 0 011.414 0zM17 11a1 1 0 100-2h-1a1 1 0 100 2h1zm-7 4a1 1 0 011 1v1a1 1 0 11-2 0v-1a1 1 0 011-1zM5.05 6.464A1 1 0 106.465 5.05l-.708-.707a1 1 0 00-1.414 1.414l.707.707zm1.414 8.486l-.707.707a1 1 0 01-1.414-1.414l.707-.707a1 1 0 011.414 1.414zM4 11a1 1 0 100-2H3a1 1 0 000 2h1z"
clipRule="evenodd"
/>
</svg>
) : (
<svg className="w-5 h-5 text-slate-700" fill="currentColor" viewBox="0 0 20 20">
<path d="M17.293 13.293A8 8 0 016.707 2.707a8.001 8.001 0 1010.586 10.586z" />
</svg>
)}
</button>
{/* Status Indicator */}
<div className="flex items-center gap-4 glass-card p-4 shadow-lg">
<span className="text-sm font-semibold text-slate-800 dark:text-slate-200">Interactive Mode</span>
<div className={`h-3 w-3 rounded-full ${apiKey ? 'bg-green-500' : 'bg-red-500'}`}></div>
{!apiKey && (
<span className="text-xs text-slate-400 dark:text-slate-500 bg-slate-100/50 dark:bg-slate-800/50 px-2 py-1 rounded-full">
(API key required)
</span>
)}
</div>
</div>
</div>
);
};
export default Header;

292
packages/kbot/gui/tauri-app/src/components/ImageGallery.tsx Normal file
View File

@ -0,0 +1,292 @@
import { useState, useEffect } from 'react';
import { ImageFile } from '../types';
interface ImageGalleryProps {
images: ImageFile[];
onImageSelect?: (imagePath: string) => void;
onImageRemove?: (imagePath: string) => void;
showSelection?: boolean;
}
export default function ImageGallery({
images,
onImageSelect,
onImageRemove,
showSelection = false
}: ImageGalleryProps) {
const [currentIndex, setCurrentIndex] = useState(0);
const [lightboxOpen, setLightboxOpen] = useState(false);
const [lightboxLoaded, setLightboxLoaded] = useState(false);
// Reset current index if images change
useEffect(() => {
if (currentIndex >= images.length && images.length > 0) {
setCurrentIndex(0);
}
}, [images.length, currentIndex]);
// ESC key handler for lightbox
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
if (!lightboxOpen) return;
if (event.key === 'Escape') {
setLightboxOpen(false);
} else if (event.key === 'ArrowRight' && currentIndex < images.length - 1) {
const newIndex = currentIndex + 1;
setCurrentIndex(newIndex);
preloadImage(newIndex);
} else if (event.key === 'ArrowLeft' && currentIndex > 0) {
const newIndex = currentIndex - 1;
setCurrentIndex(newIndex);
preloadImage(newIndex);
}
};
if (lightboxOpen) {
document.addEventListener('keydown', handleKeyDown);
return () => document.removeEventListener('keydown', handleKeyDown);
}
}, [lightboxOpen, currentIndex, images.length]);
const preloadImage = (index: number) => {
if (images.length === 0 || index < 0 || index >= images.length) return;
setLightboxLoaded(false);
const img = new Image();
img.src = images[index].src;
img.onload = () => {
setLightboxLoaded(true);
};
img.onerror = () => {
setLightboxLoaded(true); // Show even if failed to load
};
};
const openLightbox = (index: number) => {
setCurrentIndex(index);
setLightboxOpen(true);
preloadImage(index);
};
const handleThumbnailClick = (imagePath: string, index: number) => {
// Always change the main image first
setCurrentIndex(index);
// If it's a generated image and selection is enabled, also toggle selection
const isGenerated = imagePath.startsWith('generated_');
if (showSelection && isGenerated && onImageSelect) {
onImageSelect(imagePath);
}
};
if (images.length === 0) {
return (
<div className="flex flex-col items-center justify-center p-8 text-center">
<div className="w-64 h-64 bg-gradient-to-br from-slate-100 to-slate-200 dark:from-slate-700 dark:to-slate-800 rounded-xl border-2 border-dashed border-slate-300 dark:border-slate-600 flex items-center justify-center">
<div className="text-slate-500 dark:text-slate-400">
<svg className="w-16 h-16 mx-auto mb-4" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M4 3a2 2 0 00-2 2v10a2 2 0 002 2h12a2 2 0 002-2V5a2 2 0 00-2-2H4zm12 12H4l4-8 3 6 2-4 3 6z" clipRule="evenodd" />
</svg>
<p className="text-lg font-medium">No images yet</p>
<p className="text-sm">Add images to get started</p>
</div>
</div>
</div>
);
}
const currentImage = images[currentIndex];
const isGenerated = currentImage?.path.startsWith('generated_');
const isSelected = currentImage?.selected || false;
return (
<div className="flex flex-col">
{/* Main Image Display - Compact */}
<div className="flex items-center justify-center rounded-lg mb-4">
<div className="relative max-w-full max-h-[200px] flex items-center justify-center">
<img
src={currentImage.src}
alt={currentImage.path}
className={`max-w-full max-h-full object-contain rounded-lg shadow-lg border-2 transition-all duration-300 cursor-pointer ${
isSelected
? 'border-blue-400 shadow-blue-400/30'
: isGenerated
? 'border-green-300'
: 'border-white/30'
}`}
onDoubleClick={() => openLightbox(currentIndex)}
title="Double-click for fullscreen"
/>
{/* Compact overlays */}
{isGenerated && isSelected && (
<div className="absolute top-2 left-2 bg-blue-500 text-white px-2 py-1 rounded text-xs font-semibold shadow-lg flex items-center gap-1">
<svg className="w-3 h-3" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M16.707 5.293a1 1 0 010 1.414l-8 8a1 1 0 01-1.414 0l-4-4a1 1 0 011.414-1.414L8 12.586l7.293-7.293a1 1 0 011.414 0z" clipRule="evenodd" />
</svg>
✓
</div>
)}
{isGenerated && !isSelected && (
<div className="absolute top-2 left-2 bg-green-500 text-white px-2 py-1 rounded text-xs font-semibold shadow-lg">
✨
</div>
)}
</div>
</div>
{/* Compact Image Info */}
<div className="text-center mb-3">
<p className="text-xs text-slate-600 dark:text-slate-400">
{currentImage.path.split(/[/\\]/).pop()} • {currentIndex + 1}/{images.length}
</p>
</div>
{/* Thumbnails Grid - Compact */}
<div className="grid grid-cols-4 md:grid-cols-6 lg:grid-cols-8 gap-2">
{images.map((image, index) => {
const thumbIsGenerating = image.path.startsWith('generating_');
const thumbIsGenerated = image.path.startsWith('generated_');
const thumbIsSelected = image.selected || false;
return (
<button
key={image.path}
onClick={() => handleThumbnailClick(image.path, index)}
onDoubleClick={() => openLightbox(index)}
className={`relative aspect-square rounded-lg overflow-hidden transition-all duration-300 border-2 ${
currentIndex === index
? 'ring-2 ring-orange-500 border-orange-500'
: thumbIsSelected
? 'border-blue-400 ring-2 ring-blue-300'
: thumbIsGenerated
? 'border-green-300 hover:border-blue-300'
: 'border-slate-300 hover:border-slate-400'
}`}
title={
thumbIsGenerated
? "Generated image - click to select/view"
: "Click to view"
}
>
<img
src={image.src}
alt={image.path}
className="w-full h-full object-cover"
/>
{/* Selection indicator */}
{thumbIsGenerated && thumbIsSelected && (
<div className="absolute top-1 left-1 bg-blue-500 text-white rounded-full w-5 h-5 flex items-center justify-center">
<svg className="w-3 h-3" fill="currentColor" viewBox="0 0 20 20">
<path fillRule="evenodd" d="M16.707 5.293a1 1 0 010 1.414l-8 8a1 1 0 01-1.414 0l-4-4a1 1 0 011.414-1.414L8 12.586l7.293-7.293a1 1 0 011.414 0z" clipRule="evenodd" />
</svg>
</div>
)}
{/* Generated indicator */}
{thumbIsGenerated && !thumbIsSelected && (
<div className="absolute top-1 left-1 bg-green-500 text-white rounded-full w-5 h-5 flex items-center justify-center text-xs">
✨
</div>
)}
{/* Remove button */}
{!thumbIsGenerating && onImageRemove && (
<button
onClick={(e) => {
e.stopPropagation();
onImageRemove(image.path);
}}
className="absolute top-1 right-1 bg-red-500/90 hover:bg-red-600 text-white rounded-full w-5 h-5 flex items-center justify-center text-xs opacity-0 group-hover:opacity-100 transition-all duration-200"
title="Remove"
>
×
</button>
)}
</button>
);
})}
</div>
{/* Lightbox Modal */}
{lightboxOpen && (
<div
className="fixed inset-0 bg-black/95 z-[9999] flex items-center justify-center"
onClick={() => setLightboxOpen(false)}
>
<div className="relative max-w-[95vw] max-h-[95vh] flex items-center justify-center">
{lightboxLoaded ? (
<img
src={images[currentIndex].src}
alt={images[currentIndex].path}
className="max-w-full max-h-full object-contain"
onClick={(e) => e.stopPropagation()}
/>
) : (
<div className="flex items-center justify-center">
<div className="w-12 h-12 border-4 border-white/30 border-t-white rounded-full animate-spin"></div>
</div>
)}
{/* Close Button */}
<button
onClick={(e) => {
e.stopPropagation();
setLightboxOpen(false);
}}
className="absolute top-4 right-4 text-white text-2xl p-4 bg-black/75 rounded-lg hover:bg-black/90 transition-all duration-200"
title="Close (ESC)"
>
×
</button>
{/* Navigation Buttons */}
{currentIndex > 0 && (
<button
onClick={(e) => {
e.stopPropagation();
setCurrentIndex(currentIndex - 1);
setLightboxLoaded(false);
const img = new Image();
img.src = images[currentIndex - 1].src;
img.onload = () => setLightboxLoaded(true);
}}
className="absolute left-4 top-1/2 transform -translate-y-1/2 p-4 text-white text-3xl bg-black/75 rounded-lg hover:bg-black/90 transition-all duration-200"
title="Previous (←)"
>
‹
</button>
)}
{currentIndex < images.length - 1 && (
<button
onClick={(e) => {
e.stopPropagation();
setCurrentIndex(currentIndex + 1);
setLightboxLoaded(false);
const img = new Image();
img.src = images[currentIndex + 1].src;
img.onload = () => setLightboxLoaded(true);
}}
className="absolute right-4 top-1/2 transform -translate-y-1/2 p-4 text-white text-3xl bg-black/75 rounded-lg hover:bg-black/90 transition-all duration-200"
title="Next (→)"
>
›
</button>
)}
{/* Info */}
{lightboxLoaded && (
<div className="absolute bottom-4 left-1/2 transform -translate-x-1/2 bg-black/75 text-white px-4 py-2 rounded-lg text-sm">
{images[currentIndex].path.split(/[/\\]/).pop()} • {currentIndex + 1} of {images.length} • ESC to close
</div>
)}
</div>
</div>
)}
</div>
);
}

170
packages/kbot/gui/tauri-app/src/components/PromptForm.tsx Normal file
View File

@ -0,0 +1,170 @@
import React from 'react';
import { ImageFile } from '../types';
import ImageGallery from './ImageGallery';
interface PromptFormProps {
prompt: string;
setPrompt: (prompt: string) => void;
dst: string;
setDst: (dst: string) => void;
openSaveDialog: () => void;
openFilePicker: () => void;
files: ImageFile[];
getSelectedImages: () => ImageFile[];
clearAllFiles: () => void;
toggleImageSelection: (path: string) => void;
removeFile: (path: string) => void;
isGenerating: boolean;
saveAndClose: () => void;
submit: () => void;
}
const PromptForm: React.FC<PromptFormProps> = ({
prompt,
setPrompt,
dst,
setDst,
openSaveDialog,
openFilePicker,
files,
getSelectedImages,
clearAllFiles,
toggleImageSelection,
removeFile,
isGenerating,
saveAndClose,
submit,
}) => {
return (
<form
className="flex flex-col items-center glass-card p-8 glass-shimmer shadow-2xl"
onSubmit={(e) => {
e.preventDefault();
submit();
}}
>
<div className="w-full space-y-6">
<div>
<label htmlFor="prompt-input" className="block text-sm font-semibold text-slate-700 dark:text-slate-300 mb-2">
Image Description
</label>
<textarea
id="prompt-input"
value={prompt}
onChange={(e) => setPrompt(e.currentTarget.value)}
placeholder="Describe the image you want to generate or edit..."
className="w-full glass-input p-4 rounded-xl min-h-[120px] resize-none"
rows={4}
/>
</div>
<div>
<label htmlFor="output-path" className="block text-sm font-semibold text-slate-700 dark:text-slate-300 mb-2">
Output File Path
</label>
<div className="flex gap-3">
<input
id="output-path"
type="text"
value={dst}
onChange={(e) => setDst(e.target.value)}
placeholder="output.png"
className="flex-1 glass-input p-4 rounded-xl"
/>
<button
type="button"
onClick={openSaveDialog}
className="glass-button font-semibold py-4 px-5 rounded-xl whitespace-nowrap"
title="Browse for save location"
>
📁 Browse
</button>
</div>
</div>
<div>
<label className="block text-sm font-semibold text-slate-700 dark:text-slate-300 mb-2">
Source Images
</label>
<button
type="button"
onClick={openFilePicker}
className="w-full glass-button font-semibold py-4 px-6 rounded-xl border-dashed border-2 border-slate-300/50 dark:border-slate-600/50 hover:border-slate-400/60 dark:hover:border-slate-500/60"
>
📸 Select Images to Edit
</button>
</div>
</div>
{files.length > 0 && (
<div className="w-full mt-6">
<div className="flex justify-between items-center mb-4">
<h3 className="text-sm font-semibold text-slate-700 dark:text-slate-300">
Images ({files.length})
{getSelectedImages().length > 0 && (
<span className="ml-2 text-blue-600 dark:text-blue-400">
• {getSelectedImages().length} selected
</span>
)}
</h3>
<button
type="button"
onClick={clearAllFiles}
className="glass-button border-red-400/50 text-red-600 hover:bg-red-500/20 text-sm px-3 py-2 rounded-lg"
title="Remove all images"
>
Clear All
</button>
</div>
<div className="glass-card p-4">
<ImageGallery
images={files}
onImageSelect={toggleImageSelection}
onImageRemove={removeFile}
showSelection={true}
/>
</div>
</div>
)}
<div className="w-full mt-8 space-y-3">
<button
type="submit"
className={`w-full font-bold py-4 px-8 rounded-xl transition-all duration-300 shadow-lg ${
isGenerating
? 'bg-slate-400 dark:bg-slate-600 cursor-not-allowed'
: 'status-gradient-connected hover:shadow-xl hover:scale-[1.02]'
}`}
disabled={isGenerating}
>
{isGenerating ? (
<span className="flex items-center justify-center gap-2">
<div className="w-5 h-5 border-2 border-current/30 border-t-current rounded-full animate-spin"></div>
<span>Generating...</span>
</span>
) : (
<span className="flex items-center justify-center gap-2">
✨ Generate Image
</span>
)}
</button>
{files.some(file => file.path.startsWith('generated_')) && (
<button
type="button"
onClick={saveAndClose}
className="w-full glass-button border-green-400/50 text-green-700 dark:text-green-400 hover:bg-green-500/20 font-semibold py-3 px-6 rounded-xl shadow-md hover:shadow-lg transition-all duration-300"
disabled={isGenerating}
>
<span className="flex items-center justify-center gap-2">
💾 Save Last Generated Image and Close
</span>
</button>
)}
</div>
</form>
);
};
export default PromptForm;

21
packages/kbot/gui/tauri-app/src/constants.ts Normal file
View File

@ -0,0 +1,21 @@
export enum TauriCommand {
RESOLVE_PATH_RELATIVE_TO_HOME = 'resolve_path_relative_to_home',
LOG_ERROR_TO_CONSOLE = 'log_error_to_console',
SUBMIT_PROMPT = 'submit_prompt',
GENERATE_IMAGE_VIA_BACKEND = 'generate_image_via_backend',
REQUEST_CONFIG_FROM_IMAGES = 'request_config_from_images',
INCREMENT_COUNTER = 'increment_counter',
RESET_COUNTER = 'reset_counter',
GET_COUNTER = 'get_counter',
ADD_DEBUG_MESSAGE = 'add_debug_message',
CLEAR_DEBUG_MESSAGES = 'clear_debug_messages',
SEND_IPC_MESSAGE = 'send_ipc_message',
SEND_MESSAGE_TO_STDOUT = 'send_message_to_stdout',
}
export enum TauriEvent {
CONFIG_RECEIVED = 'config-received',
IMAGE_RECEIVED = 'image-received',
GENERATION_ERROR = 'generation-error',
GENERATION_COMPLETE = 'generation-complete',
}

144
packages/kbot/gui/tauri-app/src/hooks/useTauriListeners.ts Normal file
View File

@ -0,0 +1,144 @@
import { useEffect } from 'react';
import { tauriApi } from '../lib/tauriApi';
import { TauriEvent } from '../constants';
import { ImageFile } from '../types';
interface TauriListenersProps {
setPrompt: (prompt: string) => void;
setDst: (dst: string) => void;
setApiKey: (key: string) => void;
setIpcInitialized: (initialized: boolean) => void;
addDebugMessage: (level: 'info' | 'warn' | 'error' | 'debug', message: string, data?: any) => void;
setFiles: React.Dispatch<React.SetStateAction<ImageFile[]>>;
isGenerating: boolean;
generationTimeoutId: NodeJS.Timeout | null;
setGenerationTimeoutId: (id: NodeJS.Timeout | null) => void;
setIsGenerating: (generating: boolean) => void;
prompt: string;
}
export function useTauriListeners({
setPrompt,
setDst,
setApiKey,
setIpcInitialized,
addDebugMessage,
setFiles,
isGenerating,
generationTimeoutId,
setGenerationTimeoutId,
setIsGenerating,
prompt
}: TauriListenersProps) {
useEffect(() => {
const initializeApp = async () => {
await tauriApi.ensureTauriApi();
if (tauriApi.isTauri()) {
addDebugMessage('info', 'IPC system initialized successfully');
}
const setupTauriEventListeners = async () => {
if (tauriApi.isTauri()) {
try {
await tauriApi.listen(TauriEvent.CONFIG_RECEIVED, (event: any) => {
const data = event.payload;
if (data.prompt) setPrompt(data.prompt);
if (data.dst) setDst(data.dst);
if (data.apiKey) setApiKey(data.apiKey);
setIpcInitialized(true);
addDebugMessage('info', '📨 Config received from images.ts', {
hasPrompt: !!data.prompt,
hasDst: !!data.dst,
hasApiKey: !!data.apiKey,
fileCount: data.files?.length || 0,
});
});
await tauriApi.listen(TauriEvent.IMAGE_RECEIVED, (event: any) => {
const imageData = event.payload;
addDebugMessage('debug', '🖼️ Processing image data', {
filename: imageData.filename,
mimeType: imageData.mimeType,
base64Length: imageData.base64?.length,
hasValidData: !!(imageData.base64 && imageData.mimeType && imageData.filename),
});
if (imageData.base64 && imageData.mimeType && imageData.filename) {
const src = `data:${imageData.mimeType};base64,${imageData.base64}`;
const hasGeneratingPlaceholder = document.querySelector('[src^="data:image/svg+xml"]'); // A bit hacky, but avoids depending on files state
const isGeneratedImage = isGenerating || hasGeneratingPlaceholder || imageData.filename.includes('_out') || imageData.filename.includes('generated_');
if (isGeneratedImage) {
const generatedImageFile: ImageFile = { path: `generated_${imageData.filename}`, src };
setFiles(prev => {
const withoutPlaceholder = prev.filter(file => !file.path.startsWith('generating_') && !file.path.endsWith(imageData.filename) && file.path !== `generated_${imageData.filename}`);
return [...withoutPlaceholder, generatedImageFile];
});
if (generationTimeoutId) {
clearTimeout(generationTimeoutId);
setGenerationTimeoutId(null);
}
setIsGenerating(false);
addDebugMessage('info', '✅ Generated image added to files', { filename: imageData.filename, prompt });
} else {
const newImageFile = { path: imageData.filename, src };
setFiles(prevFiles => {
const exists = prevFiles.some(f => f.path === imageData.filename);
if (!exists) {
addDebugMessage('info', `📁 Adding input image: ${imageData.filename}`);
return [...prevFiles, newImageFile];
}
addDebugMessage('warn', `🔄 Image already exists: ${imageData.filename}`);
return prevFiles;
});
}
} else {
addDebugMessage('error', '❌ Invalid image data received', {
hasBase64: !!imageData.base64,
hasMimeType: !!imageData.mimeType,
hasFilename: !!imageData.filename,
});
}
});
await tauriApi.listen(TauriEvent.GENERATION_ERROR, (event: any) => {
const errorData = event.payload;
addDebugMessage('error', '❌ Generation failed', errorData);
setIsGenerating(false);
setFiles(prev => prev.filter(file => !file.path.startsWith('generating_')));
});
await tauriApi.listen(TauriEvent.GENERATION_COMPLETE, (event: any) => {
const completionData = event.payload;
addDebugMessage('info', '✅ Simple mode: Image generation completed', {
dst: completionData.dst,
prompt: completionData.prompt
});
setIsGenerating(false);
setFiles(prev => prev.filter(file => !file.path.startsWith('generating_')));
});
addDebugMessage('info', 'Tauri event listeners set up');
try {
await tauriApi.requestConfigFromImages();
addDebugMessage('info', 'Config request sent to images.ts');
} catch (e) {
addDebugMessage('error', `Failed to request config: ${e}`);
}
} catch (error) {
addDebugMessage('error', `Failed to set up event listeners: ${error}`);
}
} else {
addDebugMessage('warn', 'Tauri event listeners not available - running in browser mode');
}
};
setTimeout(setupTauriEventListeners, 500);
};
setTimeout(initializeApp, 200);
}, []); // Empty dependency array to run only once on mount
}

131
packages/kbot/gui/tauri-app/src/lib/tauriApi.ts Normal file
View File

@ -0,0 +1,131 @@
import { TauriCommand } from '../constants';
// Dynamically import Tauri APIs
let invoke: any;
let open: any;
let save: any;
let readFile: any;
let writeFile: any;
let BaseDirectory: any;
let listen: any;
let getCurrentWindow: any;
let isTauri = false;
const isBrowser = typeof window !== 'undefined';
const apiInitializationPromise = (async () => {
if (!isBrowser) return;
try {
const windowApi = await import('@tauri-apps/api/window');
getCurrentWindow = windowApi.getCurrentWindow;
isTauri = true;
console.log('✅ Tauri window API loaded');
const coreApi = await import('@tauri-apps/api/core');
invoke = coreApi.invoke;
const eventApi = await import('@tauri-apps/api/event');
listen = eventApi.listen;
const dialogApi = await import('@tauri-apps/plugin-dialog');
open = dialogApi.open;
save = dialogApi.save;
const fsApi = await import('@tauri-apps/plugin-fs');
readFile = fsApi.readFile;
writeFile = fsApi.writeFile;
BaseDirectory = fsApi.BaseDirectory;
} catch (e) {
console.warn('Tauri APIs not available, running in browser mode.');
isTauri = false;
}
})();
export const ensureTauriApi = async () => {
await apiInitializationPromise;
};
// Safe invoke function
export const safeInvoke = async <T>(command: TauriCommand, args?: any): Promise<T | null> => {
await ensureTauriApi();
if (isTauri && typeof invoke === 'function') {
try {
return await invoke(command, args);
} catch (error) {
console.error(`Invoke failed for command "${command}":`, error);
return null;
}
} else {
console.log(`[Browser Mode] Would invoke: ${command}`, args);
return null;
}
};
export { invoke, isTauri };
// Typed API wrappers
export const tauriApi = {
ensureTauriApi,
isTauri: () => isTauri,
listen: async (...args: Parameters<typeof listen>) => {
await ensureTauriApi();
return listen ? listen(...args) : () => {};
},
fs: {
readFile: async (...args: Parameters<typeof readFile>) => {
await ensureTauriApi();
return readFile ? readFile(...args) : new Uint8Array();
},
writeFile: async (...args: Parameters<typeof writeFile>) => {
await ensureTauriApi();
if (writeFile) {
return writeFile(...args);
}
},
BaseDirectory: () => BaseDirectory,
},
dialog: {
open: async (...args: Parameters<typeof open>) => {
await ensureTauriApi();
return open ? open(...args) : null;
},
save: async (...args: Parameters<typeof save>) => {
await ensureTauriApi();
return save ? save(...args) : null;
},
},
window: {
getCurrent: async () => {
await ensureTauriApi();
return getCurrentWindow ? getCurrentWindow() : null;
},
},
// Add typed wrappers for your app's specific commands
resolvePathRelativeToHome: (absolutePath: string) =>
safeInvoke<string>(TauriCommand.RESOLVE_PATH_RELATIVE_TO_HOME, { absolutePath }),
submitPrompt: (data: { prompt: string; files: string[]; dst: string }) =>
safeInvoke(TauriCommand.SUBMIT_PROMPT, data),
generateImageViaBackend: (data: { prompt: string; files: string[]; dst: string }) =>
safeInvoke(TauriCommand.GENERATE_IMAGE_VIA_BACKEND, data),
requestConfigFromImages: () =>
safeInvoke(TauriCommand.REQUEST_CONFIG_FROM_IMAGES),
addDebugMessage: (message: string, level: string, data?: any) =>
safeInvoke(TauriCommand.ADD_DEBUG_MESSAGE, { message, level, data }),
clearDebugMessages: () =>
safeInvoke(TauriCommand.CLEAR_DEBUG_MESSAGES),
sendMessageToStdout: (message: string) =>
safeInvoke(TauriCommand.SEND_MESSAGE_TO_STDOUT, { message }),
logErrorToConsole: (error: string) =>
safeInvoke(TauriCommand.LOG_ERROR_TO_CONSOLE, { error }),
sendIPCMessage: (messageType: string, data: any) =>
safeInvoke(TauriCommand.SEND_IPC_MESSAGE, { messageType, data }),
};

15
packages/kbot/gui/tauri-app/src/types.ts Normal file
View File

@ -0,0 +1,15 @@
export interface ImageFile {
path: string;
src: string;
selected?: boolean;
}
export interface GeneratedImage {
id: string;
src: string;
prompt: string;
timestamp: number;
saved?: boolean;
selectedForNext?: boolean;
timeoutId?: NodeJS.Timeout;
}

14
packages/kbot/src/commands/images.ts
View File

@ -3,6 +3,8 @@ import * as path from 'node:path';
import { sync as write } from '@polymech/fs/write';
import { sync as exists } from '@polymech/fs/exists';
import { readFileSync } from 'node:fs';
import { variables } from '../variables.js';
import { resolve } from '@polymech/commons';
import { isArray, isString } from '@polymech/core/primitives';
@ -10,8 +12,6 @@ import { OptionsSchema } from '../zod_schema.js';
import { createImage, editImage } from '../lib/images-google.js';
import { getLogger } from '../index.js';
import { prompt as resolvePrompt } from '../prompt.js';
import { variables } from '../variables.js';
import { resolve } from '@polymech/commons';
import { spawn } from 'node:child_process';
import { loadConfig } from '../config.js';
@ -73,7 +73,7 @@ export const ImageOptionsSchema = () => {
async function launchGuiAndGetPrompt(argv: any): Promise<string | null> {
const logger = getLogger(argv);
return new Promise((resolve, reject) => {
return new Promise((_resolve, reject) => {
const guiAppPath = getGuiAppPath();
console.log('guiAppPath', guiAppPath);
if (!exists(guiAppPath)) {
@ -207,17 +207,17 @@ async function launchGuiAndGetPrompt(argv: any): Promise<string | null> {
}
if (imageBuffer) {
// Send the generated image back to the GUI (chat mode)
const base64Result = imageBuffer.toString('base64');
// Send the generated image back to the GUI
const resultResponse = {
const imageResponse = {
cmd: 'forward_image_to_frontend',
base64: base64Result,
mimeType: 'image/png',
filename: path.basename(genDst)
};
tauriProcess.stdin?.write(JSON.stringify(resultResponse) + '\n');
tauriProcess.stdin?.write(JSON.stringify(imageResponse) + '\n');
logger.info(`✅ Generated image sent to GUI: ${genDst}`);
} else {
logger.error('❌ Failed to generate image');
@ -262,7 +262,7 @@ async function launchGuiAndGetPrompt(argv: any): Promise<string | null> {
if (code === 0) {
const trimmedOutput = output.trim();
console.log('Attempting to parse JSON:', JSON.stringify(trimmedOutput));
resolve(trimmedOutput || null);
_resolve(trimmedOutput || null);
} else {
reject(new Error(`Tauri app exited with code ${code}. stderr: ${errorOutput}`));
}