Add comprehensive conversation system design documentation

- Document conversation system architecture and data models - Explain agent loop implementation and tool integration - Cover state management, authentication flow, and UI components - Provide development guidelines and troubleshooting guide - Include performance, security, and testing considerations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 18:28:06 +01:00 · 2025-06-11 18:28:06 +01:00 · e689acee53
commit e689acee53
parent aed7a25ea7
1 changed files with 400 additions and 0 deletions
--- a/docs/conversation_system_design.md
+++ b/docs/conversation_system_design.md
@ -0,0 +1,400 @@
+# Conversation System & Agent Loop Architecture
+
+## Overview
+
+The conversation system implements a complete conversational AI interface with tool calling capabilities, multi-conversation management, and persistence. It integrates with both inference providers (OpenRouter) and MCP (Model Context Protocol) servers to provide a unified tool ecosystem.
+
+## System Architecture
+
+### Core Components
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    ConversationApp                          │
+├─────────────────────┬───────────────────────────────────────┤
+│  ConversationSidebar│           ChatInterface               │
+│  - Conversation List│  - MessageList                        │
+│  - MCP Status       │  - MessageInput                       │
+│  - Management       │  - Authentication                     │
+└─────────────────────┴───────────────────────────────────────┘
+```
+
+### Context Providers
+
+The system uses a layered context provider architecture:
+
+```typescript
+<InferenceProvider>          // Authentication & model management
+  <MCPProvider>              // MCP server connections
+    <ConversationProvider>   // Conversation state & agent loops
+      <ConversationApp />
+    </ConversationProvider>
+  </MCPProvider>
+</InferenceProvider>
+```
+
+## Data Models
+
+### Conversation Structure
+
+```typescript
+interface Conversation {
+  id: string;
+  title: string;
+  messages: ConversationMessage[];
+  createdAt: Date;
+  updatedAt: Date;
+  status: 'idle' | 'thinking' | 'calling_tools' | 'error';
+  error?: string;
+  pendingToolCalls?: PendingToolCall[];
+}
+```
+
+### Message Format
+
+Messages use a structured content block system that supports text, tool use, and tool results:
+
+```typescript
+interface ConversationMessage {
+  id: string;
+  role: 'user' | 'assistant' | 'tool';
+  content: ConversationContentBlock[];
+  timestamp: Date;
+}
+
+type ConversationContentBlock = 
+  | { type: 'text'; text: string }
+  | { type: 'tool_use'; id: string; name: string; input: Record<string, any> }
+  | { type: 'tool_result'; tool_use_id: string; content: string | ContentItem[]; is_error?: boolean }
+```
+
+This design aligns with the Anthropic/Claude message format and supports rich tool interactions.
+
+## Agent Loop Implementation
+
+### Hook Architecture
+
+The agent loop is implemented as a custom React hook (`useAgentLoop`) that provides:
+
+```typescript
+interface UseAgentLoopReturn {
+  executeAgentLoop: (conversation: Conversation, onUpdate: (conversation: Conversation) => void) => Promise<void>;
+  stopLoop: (conversationId: string) => void;
+  getLoopState: (conversationId: string) => AgentLoopState | undefined;
+}
+```
+
+### Execution Flow
+
+1. **Conversation Processing**: Takes a full conversation and processes the latest user message
+2. **Tool Discovery**: Aggregates tools from test tools and connected MCP servers
+3. **Inference Request**: Sends messages + available tools to the inference provider
+4. **Tool Execution**: Executes any tool calls returned by the model
+5. **Result Integration**: Adds tool results back to the conversation
+6. **Iteration**: Continues until no more tool calls or max iterations reached
+
+```typescript
+for (let iteration = 0; iteration < maxIterations; iteration++) {
+  // 1. Convert conversation to inference format
+  const inferenceMessages = toInferenceMessages(conversation.messages);
+  
+  // 2. Get available tools (test + MCP)
+  const allTools = [...testTools, ...mcpTools];
+  
+  // 3. Generate response with tools
+  const response = await currentProvider.generateResponse({
+    messages: inferenceMessages,
+    tools: allTools
+  });
+  
+  // 4. Add assistant message to conversation
+  conversation.messages.push(fromInferenceResponse(response.message));
+  
+  // 5. Execute tool calls if present
+  if (response.message.toolCalls) {
+    for (const toolCall of response.message.toolCalls) {
+      const result = await executeTool(toolCall);
+      conversation.messages.push(createToolResultMessage(result));
+    }
+  } else {
+    break; // No more tool calls, done
+  }
+}
+```
+
+### Tool Integration
+
+#### Test Tools
+Built-in tools for development and testing:
+- `get_weather`: Mock weather data
+- `calculate`: Basic arithmetic evaluation
+- `get_current_time`: Current timestamp with timezone support
+
+#### MCP Tools
+Tools from connected MCP servers are:
+- **Name-prefixed**: `${serverName}.${toolName}` to avoid conflicts
+- **Dynamically routed**: Tool calls extract server name and route to correct connection
+- **Auto-discovered**: Available tools update when servers connect/disconnect
+
+```typescript
+// Tool execution routing
+if (toolCall.function.name.includes('.')) {
+  const [serverName] = toolCall.function.name.split('.');
+  const connection = connections.find(conn => conn.name === serverName);
+  return await callMCPTool(connection.id, toolCall.function.name, toolCall.function.arguments);
+}
+```
+
+## State Management
+
+### Conversation Persistence
+
+Conversations are persisted to `localStorage` with automatic serialization/deserialization:
+
+```typescript
+// Save on changes
+useEffect(() => {
+  if (hasLoadedPersisted.current && conversations.length > 0) {
+    localStorage.setItem('conversations', JSON.stringify(conversations));
+  }
+}, [conversations]);
+
+// Load on mount
+useEffect(() => {
+  const persistedData = localStorage.getItem('conversations');
+  if (persistedData) {
+    const restored = JSON.parse(persistedData).map(conv => ({
+      ...conv,
+      createdAt: new Date(conv.createdAt),
+      updatedAt: new Date(conv.updatedAt),
+      messages: conv.messages.map(msg => ({
+        ...msg,
+        timestamp: new Date(msg.timestamp)
+      }))
+    }));
+    setConversations(restored);
+  }
+}, []);
+```
+
+### Authentication State Management
+
+The system uses a reactive authentication pattern to ensure UI consistency:
+
+```typescript
+// InferenceContext tracks auth state changes
+const [authStateVersion, setAuthStateVersion] = useState(0);
+
+const refreshAuthState = useCallback(() => {
+  setAuthStateVersion(prev => prev + 1);
+}, []);
+
+// Components use context's reactive isAuthenticated value
+const contextValue = {
+  isAuthenticated: provider?.isAuthenticated || false,
+  refreshAuthState,
+  // ...
+};
+```
+
+**Critical Pattern**: Always use `isAuthenticated` from context, not `provider?.isAuthenticated` directly, to ensure reactive updates.
+
+## User Interface Components
+
+### ConversationSidebar
+- **Conversation list** with title, timestamp, status indicators
+- **New conversation** creation
+- **Conversation management** (delete, switch active)
+- **MCP status toggle** for debugging
+
+### ChatInterface
+- **Authentication gate**: Shows inline login if not authenticated
+- **Message display** with tool call visualization
+- **Input handling** with disabled states
+- **Real-time status** showing agent loop progress
+
+### MessageList
+- **Role-based styling** (user, assistant, tool)
+- **Tool call blocks** showing tool name and input parameters
+- **Tool result blocks** with success/error states and formatted output
+- **Timestamp display** with relative formatting
+
+### MessageInput
+- **Auto-resize textarea** with Enter/Shift+Enter handling
+- **Send button** with proper disabled states and tooltips
+- **Character count** for long messages
+
+## Authentication Flow
+
+### Inline Authentication
+Users authenticate directly in the chat interface without leaving the conversation view:
+
+1. **Provider Selection**: Choose between API key and OAuth providers
+2. **Authentication**: Enter API key or complete OAuth flow
+3. **Model Loading**: Automatically load available models
+4. **Context Update**: Refresh authentication state across all components
+5. **Chat Enablement**: Message input becomes active
+
+```typescript
+// After successful authentication
+await selectedProvider.authenticate({ type: 'api_key', apiKey });
+await selectedProvider.loadModels();
+setProvider(selectedProvider);
+refreshAuthState(); // Critical: Force reactive update
+```
+
+## Development Guidelines
+
+### Adding New Tools
+
+1. **Test Tools**: Add to `testTools` array in `useAgentLoop.ts`
+2. **MCP Tools**: Connect MCP server via MCP Provider Test tab
+
+```typescript
+const newTestTool: TestTool = {
+  type: 'function',
+  function: {
+    name: 'my_tool',
+    description: 'Description of what the tool does',
+    parameters: {
+      type: 'object',
+      properties: {
+        param1: { type: 'string', description: 'Parameter description' }
+      },
+      required: ['param1']
+    }
+  },
+  execute: async (args) => {
+    // Tool implementation
+    return { result: 'tool output' };
+  }
+};
+```
+
+### Adding New Inference Providers
+
+1. **Implement InferenceProvider abstract class**
+2. **Add to available providers** in `InferenceLogin` component
+3. **Handle authentication methods** (API key, OAuth)
+
+### Message Format Extensions
+
+To add new content block types:
+
+1. **Extend ConversationContentBlock union type**
+2. **Add rendering logic** in `MessageList.renderContentBlock()`
+3. **Update conversion helpers** in `useAgentLoop`
+
+### Debugging
+
+Use the MCP Status panel (toggle in sidebar) to monitor:
+- **Server connections** and their status
+- **Available tools** from each server
+- **Connection errors** and retry attempts
+
+Debug conversation state in browser DevTools:
+- **localStorage['conversations']**: Persisted conversation data
+- **React DevTools**: Component state and context values
+
+## Error Handling
+
+### Agent Loop Errors
+- **Tool execution failures**: Captured and shown as error tool results
+- **Network failures**: Retry with exponential backoff
+- **Authentication failures**: Graceful fallback to login screen
+- **Max iterations**: Configurable limit to prevent infinite loops
+
+### State Recovery
+- **Conversation persistence**: Automatic save/restore across sessions
+- **Connection restoration**: MCP servers automatically reconnect on app load
+- **Authentication persistence**: Tokens stored securely in localStorage
+
+## Performance Considerations
+
+### Optimization Strategies
+- **Message virtualization**: Consider for very long conversations
+- **Tool result caching**: Avoid re-executing identical tool calls
+- **Connection pooling**: Reuse MCP connections across conversations
+- **Lazy loading**: Load conversation history on demand
+
+### Memory Management
+- **Conversation limits**: Consider archiving old conversations
+- **Message trimming**: Truncate very long conversations for inference
+- **Tool result size**: Limit large tool outputs in UI display
+
+## Security Considerations
+
+### Authentication
+- **API keys**: Stored in localStorage, never sent to third parties
+- **OAuth tokens**: Proper PKCE flow with secure token storage
+- **MCP connections**: Support both authenticated and unauthenticated servers
+
+### Tool Safety
+- **Input validation**: Sanitize tool inputs before execution
+- **Output sanitization**: Escape tool outputs in UI display
+- **Execution limits**: Prevent long-running or resource-intensive tools
+
+### Data Privacy
+- **Local storage**: All conversation data stays on client
+- **No analytics**: No conversation data sent to analytics services
+- **MCP isolation**: Each server connection is isolated
+
+## Future Enhancements
+
+### Planned Features
+- **Streaming responses**: Real-time message generation
+- **File attachments**: Support for images and documents
+- **Voice input/output**: Speech-to-text and text-to-speech
+- **Advanced tool calling**: Parallel tool execution, tool composition
+- **Conversation export**: Export conversations to various formats
+- **Collaborative editing**: Shared conversations between users
+
+### Architecture Improvements
+- **Message streaming**: WebSocket-based real-time updates
+- **Background processing**: Web Workers for tool execution
+- **Offline support**: Service Worker for offline conversation access
+- **Database migration**: Move from localStorage to IndexedDB
+- **Plugin system**: Third-party tool and provider extensions
+
+## Testing Strategy
+
+### Unit Tests
+- **Tool execution**: Verify tool calling and result handling
+- **Message formatting**: Test content block serialization/deserialization
+- **Authentication flows**: Mock provider authentication
+
+### Integration Tests
+- **Agent loop**: End-to-end conversation flow with tool calls
+- **MCP integration**: Real MCP server connections and tool execution
+- **Persistence**: Conversation save/restore functionality
+
+### User Testing
+- **Authentication UX**: Smooth login flow without friction
+- **Tool usage**: Intuitive tool call visualization and results
+- **Multi-conversation**: Easy conversation management and switching
+
+## Troubleshooting Guide
+
+### Common Issues
+
+**"No authenticated inference provider available"**
+- Verify `isAuthenticated` from context, not `provider?.isAuthenticated`
+- Ensure `refreshAuthState()` called after authentication
+- Check if provider is properly set in context
+
+**Tool calls not working**
+- Verify MCP server connections in MCP Status panel
+- Check tool name prefixing for MCP tools (`server.tool_name`)
+- Ensure tool schemas match expected format
+
+**Conversations not persisting**
+- Check localStorage quota and permissions
+- Verify conversation serialization doesn't fail
+- Check browser's localStorage support
+
+**UI not updating after authentication**
+- Ensure components use context's `isAuthenticated` value
+- Verify `refreshAuthState()` is called after auth changes
+- Check React DevTools for proper context updates
+
+This system provides a robust foundation for conversational AI with extensible tool calling capabilities, proper state management, and a clean user experience.