# kbot Test Overview

This document outlines the categories of tests implemented for the kbot system, along with potential new tests that could be added.

## Existing Test Categories

- **Basic Operations**: Simple operations to test basic functionality (addition, multiplication, division)
- **Math Operations**: More complex mathematical operations (factorials, square roots, equations)
- **Language Operations**: Tests focused on language processing (translation, grammar, summarization)
- **Format Operations**: Tests for formatting different data types (JSON, markdown, code, dates)
- **Coding Capabilities**: Tests that generate and execute code (factorial, bubble sort)
- **File Operations**: Tests that process files (single file, multiple files, glob patterns)
- **LLM Tools Operations**: Tests that use LLM tools (reading/writing files, directory operations)

## Potential New Tests

### Option-Specific Tests

These tests would validate specific options from `zod_schema.ts`:

1. **Router Selection**
   - **File**: `tests/unit/router.test.ts`
   - **Prompt**: `Use the openai router to generate a haiku about AI. Return only the haiku, no explanation.`
   - **Expected**: A valid haiku about AI
   - **Options**: `{ router: 'openai' }`

2. **Mode Selection**
   - **File**: `tests/unit/mode.test.ts`
   - **Prompt**: `What files are in the current directory? Use tools to help you.`
   - **Expected**: A JSON array of files
   - **Options**: `{ mode: 'tools' }`

3. **Filter Applications**
   - **File**: `tests/unit/filters.test.ts`
   - **Prompt**: `Write a JavaScript function that calculates the factorial of a number. Include comments.`
   - **Expected**: Clean code without markdown formatting
   - **Options**: `{ mode: 'completion', filters: 'code', dst: './output/factorial.js' }`

4. **Variable Substitution**
   - **File**: `tests/unit/variables.test.ts`
   - **Prompt**: `Tell me about ${TOPIC} in one sentence.`
   - **Expected**: A sentence about the specified topic
   - **Options**: `{ variables: { TOPIC: 'artificial intelligence' } }`

5. **Include Patterns**
   - **File**: `tests/unit/include.test.ts`
   - **Prompt**: `Analyze the code in these files and list all function names as a JSON array.`
   - **Expected**: A JSON array of function names
   - **Options**: `{ include: ['src/*.ts', 'tests/*.ts'] }`

### Integration Tests

1. **Multi-File Processing**
   - **File**: `tests/integration/multi_file.test.ts`
   - **Prompt**: `Compare the implementations in these two files and suggest optimizations.`
   - **Expected**: Meaningful comparison and optimization suggestions
   - **Options**: `{ include: ['src/file1.ts', 'src/file2.ts'] }`

2. **Tool Combination**
   - **File**: `tests/integration/tool_combo.test.ts`
   - **Prompt**: `Analyze this codebase, identify performance bottlenecks, and suggest fixes.`
   - **Expected**: Analysis with specific file references and code suggestions
   - **Options**: `{ mode: 'tools', tools: ['read', 'write', 'analyze'] }`

3. **End-to-End Workflow**
   - **File**: `tests/integration/e2e.test.ts`
   - **Prompt**: `Create a simple Node.js application that fetches data from an API and saves it to a file.`
   - **Expected**: Full application with multiple files created
   - **Options**: `{ mode: 'tools', output: './output/app/' }`

## Implementation Considerations

When implementing these tests:

1. Use `runTest` from `commons.ts` for consistency
2. Support multiple models using `getDefaultModels()`
3. Generate reports using `generateTestReport`
4. Set appropriate timeouts for API calls
5. Check both expected outputs and side effects (file creation, etc.)