mono/packages/media/docs/ui.md

# UI Proposal: Electron-based Dynamic Form Renderer

## Overview

This proposal outlines the implementation of a dynamic UI system for rendering Zod schema-based forms using Electron, React JSON Schema Form (RJSF), and shadcn/ui components. The goal is to replace the current static HTML template with a flexible, schema-driven form generator.

## Current State Analysis

### Existing Implementation
- **Template System**: `web-ui/_prompt.html` with basic variable substitution
- **Electron Setup**: Basic main process in `lib/ui/electron/main.ts` 
- **Schema System**: `ZodMetaMap` class with `getUISchema()` method
- **File Handling**: Basic file picker functionality via IPC

### Limitations
- Static HTML template requiring manual updates for new options
- Basic variable substitution not suitable for complex schemas
- No type safety between schema and UI
- Limited file picker functionality

## Proposed Architecture

### 1. React-based UI Application

**New Structure:**
```
media/
├── web-ui/
│   ├── src/
│   │   ├── components/
│   │   │   ├── SchemaForm.tsx          # Main RJSF form component
│   │   │   ├── FilePickerField.tsx     # Custom file picker widget
│   │   │   ├── ui/                     # shadcn/ui components
│   │   │   └── FormTheme.tsx           # RJSF shadcn theme
│   │   ├── hooks/
│   │   │   ├── useElectronIPC.ts       # IPC communication
│   │   │   └── useSchemaForm.ts        # Form state management
│   │   ├── types/
│   │   │   └── electron.d.ts           # Electron API types
│   │   ├── App.tsx                     # Main app component
│   │   └── main.tsx                    # React entry point
│   ├── package.json                    # React app dependencies
│   ├── vite.config.ts                  # Vite build config
│   └── dist/                           # Built React app
```

### 2. Enhanced Electron Integration

**Updated Electron Structure:**
```typescript
// lib/ui/electron/main.ts
interface UIOptions {
  schema: any;           // JSON schema from ZodMetaMap
  uiSchema: any;         // UI schema from getUISchema()
  formData?: any;        // Default form values
  title?: string;        // Window title
  width?: number;        // Window dimensions
  height?: number;
}

export const renderSchemaForm = async (options: UIOptions): Promise<any> => {
  // Create Electron window with React app
  // Pass schema and configuration via IPC
  // Return form submission result
}
```

### 3. ZodMetaMap Enhancements

**Enhanced Schema Generation:**
```typescript
// In ZodMetaMap class
getJSONSchema(): any {
  // Convert Zod schema to JSON Schema for RJSF
}

getUISchemaEnhanced(): any {
  // Enhanced UI schema with:
  // - File picker widgets for file/directory fields
  // - Custom validators
  // - Field dependencies
  // - Layout hints
}
```

### 4. File Picker Integration

**Custom RJSF Widget:**
```typescript
// components/FilePickerField.tsx
interface FilePickerProps {
  value: string | string[];
  onChange: (value: string | string[]) => void;
  schema: any;
  uiSchema: any;
}

const FilePickerField: React.FC<FilePickerProps> = ({
  value,
  onChange,
  schema,
  uiSchema
}) => {
  // Determine picker type from schema:
  // - Single file vs multiple files
  // - File vs directory
  // - File filters based on extensions
  
  const handleFilePicker = async () => {
    const result = await window.electronAPI.showFileDialog({
      properties: getPickerProperties(schema, uiSchema),
      filters: getFileFilters(schema)
    });
    onChange(result);
  };
  
  return (
    <div className="space-y-2">
      <Button onClick={handleFilePicker}>
        {getPickerButtonText(schema, uiSchema)}
      </Button>
      <FileList files={value} onRemove={handleRemoveFile} />
    </div>
  );
};
```

## Implementation Plan

### Phase 1: Core Infrastructure (Week 1)

1. **Setup React Application**
   ```bash
   cd media/web-ui
   npm init -y
   npm install react react-dom @rjsf/core @rjsf/utils @rjsf/validator-ajv8
   npm install -D vite @vitejs/plugin-react typescript @types/react @types/react-dom
   ```

2. **Install shadcn/ui**
   ```bash
   npx shadcn-ui@latest init
   npx shadcn-ui@latest add button input textarea select checkbox switch
   ```

3. **Create RJSF Theme**
   - Map RJSF widgets to shadcn/ui components
   - Implement custom field templates
   - Style validation errors and help text

### Phase 2: Schema Integration (Week 2)

1. **Enhance ZodMetaMap**
   - Add `toJSONSchema()` method
   - Enhance `getUISchema()` with file picker hints
   - Add field dependency support

2. **Custom Widgets**
   - File picker widget for `src` field
   - Directory picker widget for `dst` field  
   - Number input with validation
   - Enum select with descriptions

3. **IPC Communication**
   - Schema passing from main to renderer
   - Form submission handling
   - File dialog integration

### Phase 3: Advanced Features (Week 3)

1. **Dynamic Schema Loading**
   ```typescript
   // lib/ui/schema-renderer.ts
   export const renderSchema = async (
     schemaFn: () => any,
     options: RenderOptions = {}
   ): Promise<any> => {
     const schema = schemaFn();
     const zodMap = new ZodMetaMap();
     // Configure zodMap based on schema
     const jsonSchema = zodMap.toJSONSchema();
     const uiSchema = zodMap.getUISchemaEnhanced();
     
     return renderSchemaForm({
       schema: jsonSchema,
       uiSchema,
       title: options.title || 'Configuration',
       ...options
     });
   };
   ```

2. **File Management**
   - Drag & drop support
   - File preview thumbnails
   - Batch file selection
   - Recent files/directories

3. **Validation & Preview**
   - Real-time validation feedback
   - Command preview generation
   - Estimated processing time
   - Storage space calculations

### Phase 4: Integration (Week 4)

1. **Command Integration**
   ```typescript
   // commands/resize.ts - Updated handler
   export async function handler(argv: CLI.Arguments) {
     if (argv.ui || (!argv.src && process.env.NODE_ENV !== 'test')) {
       // Launch UI mode
       const result = await renderSchema(ResizeOptionsSchema, {
         title: 'Resize Images',
         width: 900,
         height: 700
       });
       
       if (!result) return; // User cancelled
       
       // Merge CLI args with UI result
       const options = { ...argv, ...result };
       await resize(options);
     } else {
       // Traditional CLI mode
       const options = sanitize(argv) as IOptions;
       await resize(options);
     }
   }
   ```

2. **CLI Flag Addition**
   ```typescript
   // Add --ui flag to all commands
   .option('ui', {
     boolean: true,
     default: false,
     describe: 'Launch interactive UI'
   })
   ```

## Technical Specifications

### Dependencies

**New Dependencies:**
```json
{
  "dependencies": {
    "@rjsf/core": "^5.15.0",
    "@rjsf/utils": "^5.15.0", 
    "@rjsf/validator-ajv8": "^5.15.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "class-variance-authority": "^0.7.0",
    "clsx": "^2.0.0",
    "tailwind-merge": "^2.0.0",
    "lucide-react": "^0.294.0"
  },
  "devDependencies": {
    "vite": "^5.0.0",
    "@vitejs/plugin-react": "^4.2.0",
    "tailwindcss": "^3.3.0",
    "autoprefixer": "^10.4.16",
    "postcss": "^8.4.32"
  }
}
```

### File Structure Changes

**Updated web_prompt Function:**
```typescript
// lib/ui/electron.ts
export const web_prompt = async (
  schema: any,
  uiSchema: any,
  options: {
    title?: string;
    formData?: any;
    width?: number;
    height?: number;
  } = {}
): Promise<any> => {
  // Write schema and config to temporary files
  const configPath = path.join(os.tmpdir(), `schema-${Date.now()}.json`);
  write(configPath, JSON.stringify({
    schema,
    uiSchema,
    formData: options.formData || {},
    title: options.title || 'Configuration'
  }));

  const electronPath = path.join(process.cwd(), './node_modules/electron/dist/electron.exe');
  const mainPath = path.join(process.cwd(), 'lib/ui/electron/main.js');
  
  const p = await Helper.runBin(
    '',
    electronPath,
    '',
    [`"${mainPath}"`, `--config="${configPath}"`]
  );

  const content = p.messages.find((m) => m.type === 'content');
  
  // Cleanup
  if (exists(configPath)) {
    fs.unlinkSync(configPath);
  }
  
  return content?.data;
};
```

## Benefits

### For Developers
- **Type Safety**: Full TypeScript support from schema to UI
- **Maintainability**: Single source of truth for form structure
- **Extensibility**: Easy to add new field types and validation
- **Consistency**: Uniform UI across all commands

### For Users
- **Intuitive Interface**: Modern, responsive UI with proper validation
- **File Management**: Drag & drop, thumbnails, batch operations
- **Real-time Feedback**: Immediate validation and preview
- **Accessibility**: Screen reader support, keyboard navigation

### For the Project
- **Reduced Maintenance**: No manual HTML template updates
- **Faster Development**: New commands automatically get UI support
- **Better UX**: Professional interface increases adoption
- **Future-Proof**: Extensible architecture for new features

## Migration Strategy

1. **Parallel Development**: Build new system alongside existing one
2. **Gradual Migration**: Start with resize command, then expand
3. **Backward Compatibility**: Keep CLI-only mode working
4. **User Testing**: Beta test with existing users
5. **Documentation**: Update all docs with UI screenshots

## Challenges & Mitigations

### Complex Schema Mapping
- **Challenge**: Converting Zod schemas to JSON Schema
- **Mitigation**: Use `zod-to-json-schema` library or custom converter

### File Picker UX
- **Challenge**: Handling complex file selection scenarios  
- **Mitigation**: Custom widgets with clear visual feedback

### Electron Bundle Size
- **Challenge**: React app increases bundle size
- **Mitigation**: Code splitting, lazy loading, production builds

### Cross-Platform Compatibility
- **Challenge**: File dialogs behave differently across OS
- **Mitigation**: Extensive testing, OS-specific adaptations

## Success Metrics

- **Developer Experience**: Time to add UI to new command < 10 minutes
- **User Adoption**: >50% of users prefer UI mode over CLI
- **Error Reduction**: 80% fewer invalid parameter combinations
- **Maintenance**: 90% reduction in UI-related bug reports

## Future Enhancements

- **Presets**: Save/load common configurations
- **Batch Processing**: Queue multiple operations
- **Progress Tracking**: Real-time operation progress
- **Plugin System**: Third-party UI extensions
- **Web Version**: Browser-based version for remote access

This proposal provides a comprehensive roadmap for transforming the current static UI into a dynamic, schema-driven system that will significantly improve both developer and user experience.