Description

We'd like to propose introducing a generic model for representing interactions with Language Models (LLMs) in Theia. Currently, we have the ChatModel which is optimized specifically for chat-based interactions, but this doesn't adequately cover more complex use cases or non-chat agents.

Problems with Current Approach

1. Limited Scope: The current ChatModel is designed specifically for chat-based interactions and doesn't accommodate other interaction patterns well.

2. Complex Use Cases Unsupported: More advanced scenarios - like an agent making multiple LLM calls based on a single user request - aren't cleanly represented in the current model.

3. Fragmented History Recording: We currently have an inconsistent approach to history recording, with some parts automated and others requiring manual intervention.

4. Lack of Centralization: There's no single source of truth for all LLM interactions, making it difficult to implement cross-cutting concerns.

Key Principles

• We want a single place where "raw" language model interactions are stored, including tool calls and categorizations like thinking
• At the moment we have "ChatModel" but this is optimized for the chat use case and does not cover well more complicated use cases, like invoking many requests within an agent based on a single user request. it also does not cover non-chat agents at all
• At the moment we have a weird mix of history recording, parts are automated, parts are manual. This would be solved with a single source of truth for all interactions
• This would be useful for cross cutting concerns like tracing (logging) and testing

Proposed Solution

Introduce a generic LLMInteractionModel (or another name) that would serve as the foundation for all LLM interactions in Theia, with the current ChatModel becoming a presentation layer on top of this core model.

Architecture Overview

The proposed architecture would consist of:

1. Core LLMInteractionModel: Represents the fundamental components of any LLM interaction
2. Specialized Models: Including the current ChatModel, which would now be built on top of the core model
3. Common Services: For cross-cutting concerns like logging and testing