mono/packages/media/docs/ui.md

UI Proposal: Electron-based Dynamic Form Renderer

Overview

This proposal outlines the implementation of a dynamic UI system for rendering Zod schema-based forms using Electron, React JSON Schema Form (RJSF), and shadcn/ui components. The goal is to replace the current static HTML template with a flexible, schema-driven form generator.

Current State Analysis

Existing Implementation

• Template System: web-ui/_prompt.html with basic variable substitution
• Electron Setup: Basic main process in lib/ui/electron/main.ts
• Schema System: ZodMetaMap class with getUISchema() method
• File Handling: Basic file picker functionality via IPC

Limitations

• Static HTML template requiring manual updates for new options
• Basic variable substitution not suitable for complex schemas
• No type safety between schema and UI
• Limited file picker functionality

Proposed Architecture

1. React-based UI Application

New Structure:

media/
├── web-ui/
│   ├── src/
│   │   ├── components/
│   │   │   ├── SchemaForm.tsx          # Main RJSF form component
│   │   │   ├── FilePickerField.tsx     # Custom file picker widget
│   │   │   ├── ui/                     # shadcn/ui components
│   │   │   └── FormTheme.tsx           # RJSF shadcn theme
│   │   ├── hooks/
│   │   │   ├── useElectronIPC.ts       # IPC communication
│   │   │   └── useSchemaForm.ts        # Form state management
│   │   ├── types/
│   │   │   └── electron.d.ts           # Electron API types
│   │   ├── App.tsx                     # Main app component
│   │   └── main.tsx                    # React entry point
│   ├── package.json                    # React app dependencies
│   ├── vite.config.ts                  # Vite build config
│   └── dist/                           # Built React app

2. Enhanced Electron Integration

Updated Electron Structure:

// lib/ui/electron/main.ts
interface UIOptions {
  schema: any;           // JSON schema from ZodMetaMap
  uiSchema: any;         // UI schema from getUISchema()
  formData?: any;        // Default form values
  title?: string;        // Window title
  width?: number;        // Window dimensions
  height?: number;
}

export const renderSchemaForm = async (options: UIOptions): Promise<any> => {
  // Create Electron window with React app
  // Pass schema and configuration via IPC
  // Return form submission result
}

3. ZodMetaMap Enhancements

Enhanced Schema Generation:

// In ZodMetaMap class
getJSONSchema(): any {
  // Convert Zod schema to JSON Schema for RJSF
}

getUISchemaEnhanced(): any {
  // Enhanced UI schema with:
  // - File picker widgets for file/directory fields
  // - Custom validators
  // - Field dependencies
  // - Layout hints
}

4. File Picker Integration

Custom RJSF Widget:

// components/FilePickerField.tsx
interface FilePickerProps {
  value: string | string[];
  onChange: (value: string | string[]) => void;
  schema: any;
  uiSchema: any;
}

const FilePickerField: React.FC<FilePickerProps> = ({
  value,
  onChange,
  schema,
  uiSchema
}) => {
  // Determine picker type from schema:
  // - Single file vs multiple files
  // - File vs directory
  // - File filters based on extensions
  
  const handleFilePicker = async () => {
    const result = await window.electronAPI.showFileDialog({
      properties: getPickerProperties(schema, uiSchema),
      filters: getFileFilters(schema)
    });
    onChange(result);
  };
  
  return (
    <div className="space-y-2">
      <Button onClick={handleFilePicker}>
        {getPickerButtonText(schema, uiSchema)}
      </Button>
      <FileList files={value} onRemove={handleRemoveFile} />
    </div>
  );
};

Implementation Plan

Phase 1: Core Infrastructure (Week 1)

1. Setup React Application

   cd media/web-ui
   npm init -y
   npm install react react-dom @rjsf/core @rjsf/utils @rjsf/validator-ajv8
   npm install -D vite @vitejs/plugin-react typescript @types/react @types/react-dom
   

2. Install shadcn/ui

   npx shadcn-ui@latest init
   npx shadcn-ui@latest add button input textarea select checkbox switch
   

3. Create RJSF Theme

   • Map RJSF widgets to shadcn/ui components
   • Implement custom field templates
   • Style validation errors and help text

Phase 2: Schema Integration (Week 2)

1. Enhance ZodMetaMap

   • Add toJSONSchema() method
   • Enhance getUISchema() with file picker hints
   • Add field dependency support

2. Custom Widgets

   • File picker widget for src field
   • Directory picker widget for dst field
   • Number input with validation
   • Enum select with descriptions

3. IPC Communication

   • Schema passing from main to renderer
   • Form submission handling
   • File dialog integration

Phase 3: Advanced Features (Week 3)

1. Dynamic Schema Loading

   // lib/ui/schema-renderer.ts
   export const renderSchema = async (
     schemaFn: () => any,
     options: RenderOptions = {}
   ): Promise<any> => {
     const schema = schemaFn();
     const zodMap = new ZodMetaMap();
     // Configure zodMap based on schema
     const jsonSchema = zodMap.toJSONSchema();
     const uiSchema = zodMap.getUISchemaEnhanced();
   
     return renderSchemaForm({
       schema: jsonSchema,
       uiSchema,
       title: options.title || 'Configuration',
       ...options
     });
   };
   

2. File Management

   • Drag & drop support
   • File preview thumbnails
   • Batch file selection
   • Recent files/directories

3. Validation & Preview

   • Real-time validation feedback
   • Command preview generation
   • Estimated processing time
   • Storage space calculations

Phase 4: Integration (Week 4)

1. Command Integration

   // commands/resize.ts - Updated handler
   export async function handler(argv: CLI.Arguments) {
     if (argv.ui || (!argv.src && process.env.NODE_ENV !== 'test')) {
       // Launch UI mode
       const result = await renderSchema(ResizeOptionsSchema, {
         title: 'Resize Images',
         width: 900,
         height: 700
       });
   
       if (!result) return; // User cancelled
   
       // Merge CLI args with UI result
       const options = { ...argv, ...result };
       await resize(options);
     } else {
       // Traditional CLI mode
       const options = sanitize(argv) as IOptions;
       await resize(options);
     }
   }
   

2. CLI Flag Addition

   // Add --ui flag to all commands
   .option('ui', {
     boolean: true,
     default: false,
     describe: 'Launch interactive UI'
   })
   

Technical Specifications

Dependencies

New Dependencies:

{
  "dependencies": {
    "@rjsf/core": "^5.15.0",
    "@rjsf/utils": "^5.15.0", 
    "@rjsf/validator-ajv8": "^5.15.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "class-variance-authority": "^0.7.0",
    "clsx": "^2.0.0",
    "tailwind-merge": "^2.0.0",
    "lucide-react": "^0.294.0"
  },
  "devDependencies": {
    "vite": "^5.0.0",
    "@vitejs/plugin-react": "^4.2.0",
    "tailwindcss": "^3.3.0",
    "autoprefixer": "^10.4.16",
    "postcss": "^8.4.32"
  }
}

File Structure Changes

Updated web_prompt Function:

// lib/ui/electron.ts
export const web_prompt = async (
  schema: any,
  uiSchema: any,
  options: {
    title?: string;
    formData?: any;
    width?: number;
    height?: number;
  } = {}
): Promise<any> => {
  // Write schema and config to temporary files
  const configPath = path.join(os.tmpdir(), `schema-${Date.now()}.json`);
  write(configPath, JSON.stringify({
    schema,
    uiSchema,
    formData: options.formData || {},
    title: options.title || 'Configuration'
  }));

  const electronPath = path.join(process.cwd(), './node_modules/electron/dist/electron.exe');
  const mainPath = path.join(process.cwd(), 'lib/ui/electron/main.js');
  
  const p = await Helper.runBin(
    '',
    electronPath,
    '',
    [`"${mainPath}"`, `--config="${configPath}"`]
  );

  const content = p.messages.find((m) => m.type === 'content');
  
  // Cleanup
  if (exists(configPath)) {
    fs.unlinkSync(configPath);
  }
  
  return content?.data;
};

Benefits

For Developers

• Type Safety: Full TypeScript support from schema to UI
• Maintainability: Single source of truth for form structure
• Extensibility: Easy to add new field types and validation
• Consistency: Uniform UI across all commands

For Users

• Intuitive Interface: Modern, responsive UI with proper validation
• File Management: Drag & drop, thumbnails, batch operations
• Real-time Feedback: Immediate validation and preview
• Accessibility: Screen reader support, keyboard navigation

For the Project

• Reduced Maintenance: No manual HTML template updates
• Faster Development: New commands automatically get UI support
• Better UX: Professional interface increases adoption
• Future-Proof: Extensible architecture for new features

Migration Strategy

1. Parallel Development: Build new system alongside existing one
2. Gradual Migration: Start with resize command, then expand
3. Backward Compatibility: Keep CLI-only mode working
4. User Testing: Beta test with existing users
5. Documentation: Update all docs with UI screenshots

Challenges & Mitigations

Complex Schema Mapping

• Challenge: Converting Zod schemas to JSON Schema
• Mitigation: Use zod-to-json-schema library or custom converter

File Picker UX

• Challenge: Handling complex file selection scenarios
• Mitigation: Custom widgets with clear visual feedback

Electron Bundle Size

• Challenge: React app increases bundle size
• Mitigation: Code splitting, lazy loading, production builds

Cross-Platform Compatibility

• Challenge: File dialogs behave differently across OS
• Mitigation: Extensive testing, OS-specific adaptations

Success Metrics

• Developer Experience: Time to add UI to new command < 10 minutes
• User Adoption: >50% of users prefer UI mode over CLI
• Error Reduction: 80% fewer invalid parameter combinations
• Maintenance: 90% reduction in UI-related bug reports

Future Enhancements

• Presets: Save/load common configurations
• Batch Processing: Queue multiple operations
• Progress Tracking: Real-time operation progress
• Plugin System: Third-party UI extensions
• Web Version: Browser-based version for remote access

This proposal provides a comprehensive roadmap for transforming the current static UI into a dynamic, schema-driven system that will significantly improve both developer and user experience.