example-remote-client/docs/conversation_system_design.md

Conversation System & Agent Loop Architecture

Overview

The conversation system implements a complete conversational AI interface with tool calling capabilities, multi-conversation management, and persistence. It integrates with both inference providers (OpenRouter) and MCP (Model Context Protocol) servers to provide a unified tool ecosystem.

System Architecture

Core Components

┌─────────────────────────────────────────────────────────────┐
│                    ConversationApp                          │
├─────────────────────┬───────────────────────────────────────┤
│  ConversationSidebar│           ChatInterface               │
│  - Conversation List│  - MessageList                        │
│  - MCP Status       │  - MessageInput                       │
│  - Management       │  - Authentication                     │
└─────────────────────┴───────────────────────────────────────┘

Context Providers

The system uses a layered context provider architecture:

<InferenceProvider>          // Authentication & model management
  <MCPProvider>              // MCP server connections
    <ConversationProvider>   // Conversation state & agent loops
      <ConversationApp />
    </ConversationProvider>
  </MCPProvider>
</InferenceProvider>

Data Models

Conversation Structure

interface Conversation {
  id: string;
  title: string;
  messages: ConversationMessage[];
  createdAt: Date;
  updatedAt: Date;
  status: 'idle' | 'thinking' | 'calling_tools' | 'error';
  error?: string;
  pendingToolCalls?: PendingToolCall[];
}

Message Format

Messages use a structured content block system that supports text, tool use, and tool results:

interface ConversationMessage {
  id: string;
  role: 'user' | 'assistant' | 'tool';
  content: ConversationContentBlock[];
  timestamp: Date;
}

type ConversationContentBlock = 
  | { type: 'text'; text: string }
  | { type: 'tool_use'; id: string; name: string; input: Record<string, any> }
  | { type: 'tool_result'; tool_use_id: string; content: string | ContentItem[]; is_error?: boolean }

This design aligns with the Anthropic/Claude message format and supports rich tool interactions.

Agent Loop Implementation

Hook Architecture

The agent loop is implemented as a custom React hook (useAgentLoop) that provides:

interface UseAgentLoopReturn {
  executeAgentLoop: (conversation: Conversation, onUpdate: (conversation: Conversation) => void) => Promise<void>;
  stopLoop: (conversationId: string) => void;
  getLoopState: (conversationId: string) => AgentLoopState | undefined;
}

Execution Flow

1. Conversation Processing: Takes a full conversation and processes the latest user message
2. Tool Discovery: Aggregates tools from test tools and connected MCP servers
3. Inference Request: Sends messages + available tools to the inference provider
4. Tool Execution: Executes any tool calls returned by the model
5. Result Integration: Adds tool results back to the conversation
6. Iteration: Continues until no more tool calls or max iterations reached

for (let iteration = 0; iteration < maxIterations; iteration++) {
  // 1. Convert conversation to inference format
  const inferenceMessages = toInferenceMessages(conversation.messages);
  
  // 2. Get available tools (test + MCP)
  const allTools = [...testTools, ...mcpTools];
  
  // 3. Generate response with tools
  const response = await currentProvider.generateResponse({
    messages: inferenceMessages,
    tools: allTools
  });
  
  // 4. Add assistant message to conversation
  conversation.messages.push(fromInferenceResponse(response.message));
  
  // 5. Execute tool calls if present
  if (response.message.toolCalls) {
    for (const toolCall of response.message.toolCalls) {
      const result = await executeTool(toolCall);
      conversation.messages.push(createToolResultMessage(result));
    }
  } else {
    break; // No more tool calls, done
  }
}

Tool Integration

Test Tools

Built-in tools for development and testing:

• get_weather: Mock weather data
• calculate: Basic arithmetic evaluation
• get_current_time: Current timestamp with timezone support

MCP Tools

Tools from connected MCP servers are:

• Name-prefixed: ${serverName}__${toolName} to avoid conflicts and comply with API requirements
• Dynamically routed: Tool calls extract server name and route to correct connection
• Auto-discovered: Available tools update when servers connect/disconnect

// Tool execution routing
if (toolCall.function.name.includes('__')) {
  const [serverName] = toolCall.function.name.split('__');
  const connection = connections.find(conn => conn.name === serverName);
  return await callMCPTool(connection.id, toolCall.function.name, toolCall.function.arguments);
}

Tool Naming Requirements

Important: Tool names must comply with OpenRouter API requirements:

• Pattern: Must match ^[a-zA-Z0-9_-]{1,64}$
• No dots allowed: This is why we use double underscore (__) as separator
• Length limit: Maximum 64 characters total

Example tool names:

• ✅ test__echo (valid)
• ✅ weather_service__get_forecast (valid)
• ❌ test.echo (invalid - contains dot)
• ❌ server__very_long_tool_name_that_exceeds_the_sixty_four_character_limit (invalid - too long)

State Management

Conversation Persistence

Conversations are persisted to localStorage with automatic serialization/deserialization:

// Save on changes
useEffect(() => {
  if (hasLoadedPersisted.current && conversations.length > 0) {
    localStorage.setItem('conversations', JSON.stringify(conversations));
  }
}, [conversations]);

// Load on mount
useEffect(() => {
  const persistedData = localStorage.getItem('conversations');
  if (persistedData) {
    const restored = JSON.parse(persistedData).map(conv => ({
      ...conv,
      createdAt: new Date(conv.createdAt),
      updatedAt: new Date(conv.updatedAt),
      messages: conv.messages.map(msg => ({
        ...msg,
        timestamp: new Date(msg.timestamp)
      }))
    }));
    setConversations(restored);
  }
}, []);

Authentication State Management

The system uses a reactive authentication pattern to ensure UI consistency:

// InferenceContext tracks auth state changes
const [authStateVersion, setAuthStateVersion] = useState(0);

const refreshAuthState = useCallback(() => {
  setAuthStateVersion(prev => prev + 1);
}, []);

// Components use context's reactive isAuthenticated value
const contextValue = {
  isAuthenticated: provider?.isAuthenticated || false,
  refreshAuthState,
  // ...
};

Critical Pattern: Always use isAuthenticated from context, not provider?.isAuthenticated directly, to ensure reactive updates.

User Interface Components

ConversationSidebar

• Conversation list with title, timestamp, status indicators
• New conversation creation
• Conversation management (delete, switch active)
• MCP status toggle for debugging

ChatInterface

• Authentication gate: Shows inline login if not authenticated
• Message display with tool call visualization
• Input handling with disabled states
• Real-time status showing agent loop progress

MessageList

• Role-based styling (user, assistant, tool)
• Tool call blocks showing tool name and input parameters
• Tool result blocks with success/error states and formatted output
• Timestamp display with relative formatting

MessageInput

• Auto-resize textarea with Enter/Shift+Enter handling
• Send button with proper disabled states and tooltips
• Character count for long messages

Authentication Flow

Inline Authentication

Users authenticate directly in the chat interface without leaving the conversation view:

1. Provider Selection: Choose between API key and OAuth providers
2. Authentication: Enter API key or complete OAuth flow
3. Model Loading: Automatically load available models
4. Context Update: Refresh authentication state across all components
5. Chat Enablement: Message input becomes active

// After successful authentication
await selectedProvider.authenticate({ type: 'api_key', apiKey });
await selectedProvider.loadModels();
setProvider(selectedProvider);
refreshAuthState(); // Critical: Force reactive update

Development Guidelines

Adding New Tools

1. Test Tools: Add to testTools array in useAgentLoop.ts
2. MCP Tools: Connect MCP server via MCP Provider Test tab

const newTestTool: TestTool = {
  type: 'function',
  function: {
    name: 'my_tool',
    description: 'Description of what the tool does',
    parameters: {
      type: 'object',
      properties: {
        param1: { type: 'string', description: 'Parameter description' }
      },
      required: ['param1']
    }
  },
  execute: async (args) => {
    // Tool implementation
    return { result: 'tool output' };
  }
};

Adding New Inference Providers

1. Implement InferenceProvider abstract class
2. Add to available providers in InferenceLogin component
3. Handle authentication methods (API key, OAuth)

Message Format Extensions

To add new content block types:

1. Extend ConversationContentBlock union type
2. Add rendering logic in MessageList.renderContentBlock()
3. Update conversion helpers in useAgentLoop

Debugging

Use the MCP Status panel (toggle in sidebar) to monitor:

• Server connections and their status
• Available tools from each server
• Connection errors and retry attempts

Debug conversation state in browser DevTools:

• localStorage['conversations']: Persisted conversation data
• React DevTools: Component state and context values

Error Handling

Agent Loop Errors

• Tool execution failures: Captured and shown as error tool results
• Network failures: Retry with exponential backoff
• Authentication failures: Graceful fallback to login screen
• Max iterations: Configurable limit to prevent infinite loops

State Recovery

• Conversation persistence: Automatic save/restore across sessions
• Connection restoration: MCP servers automatically reconnect on app load
• Authentication persistence: Tokens stored securely in localStorage

Performance Considerations

Optimization Strategies

• Message virtualization: Consider for very long conversations
• Tool result caching: Avoid re-executing identical tool calls
• Connection pooling: Reuse MCP connections across conversations
• Lazy loading: Load conversation history on demand

Memory Management

• Conversation limits: Consider archiving old conversations
• Message trimming: Truncate very long conversations for inference
• Tool result size: Limit large tool outputs in UI display

Security Considerations

Authentication

• API keys: Stored in localStorage, never sent to third parties
• OAuth tokens: Proper PKCE flow with secure token storage
• MCP connections: Support both authenticated and unauthenticated servers

Tool Safety

• Input validation: Sanitize tool inputs before execution
• Output sanitization: Escape tool outputs in UI display
• Execution limits: Prevent long-running or resource-intensive tools

Data Privacy

• Local storage: All conversation data stays on client
• No analytics: No conversation data sent to analytics services
• MCP isolation: Each server connection is isolated

Future Enhancements

Planned Features

• Streaming responses: Real-time message generation
• File attachments: Support for images and documents
• Voice input/output: Speech-to-text and text-to-speech
• Advanced tool calling: Parallel tool execution, tool composition
• Conversation export: Export conversations to various formats
• Collaborative editing: Shared conversations between users

Architecture Improvements

• Message streaming: WebSocket-based real-time updates
• Background processing: Web Workers for tool execution
• Offline support: Service Worker for offline conversation access
• Database migration: Move from localStorage to IndexedDB
• Plugin system: Third-party tool and provider extensions

Testing Strategy

Unit Tests

• Tool execution: Verify tool calling and result handling
• Message formatting: Test content block serialization/deserialization
• Authentication flows: Mock provider authentication

Integration Tests

• Agent loop: End-to-end conversation flow with tool calls
• MCP integration: Real MCP server connections and tool execution
• Persistence: Conversation save/restore functionality

User Testing

• Authentication UX: Smooth login flow without friction
• Tool usage: Intuitive tool call visualization and results
• Multi-conversation: Easy conversation management and switching

Troubleshooting Guide

Common Issues

"No authenticated inference provider available"

• Verify isAuthenticated from context, not provider?.isAuthenticated
• Ensure refreshAuthState() called after authentication
• Check if provider is properly set in context

Tool calls not working

• Verify MCP server connections in MCP Status panel
• Check tool name prefixing for MCP tools (server__tool_name)
• Ensure tool schemas match expected format
• Verify tool names comply with API requirements (alphanumeric, underscore, hyphen only)

Conversations not persisting

• Check localStorage quota and permissions
• Verify conversation serialization doesn't fail
• Check browser's localStorage support

UI not updating after authentication

• Ensure components use context's isAuthenticated value
• Verify refreshAuthState() is called after auth changes
• Check React DevTools for proper context updates

Agent loop hangs with MCP tools connected

• Check browser console for OpenRouter API errors about tool names
• Verify tool names don't contain dots or other invalid characters
• Ensure tool names are under 64 characters total length
• Look for pattern validation errors: String should match pattern '^[a-zA-Z0-9_-]{1,64}'

This system provides a robust foundation for conversational AI with extensible tool calling capabilities, proper state management, and a clean user experience.