Thoughts on Auto mode can be found here

CAPI will be picking up this work in July, and here are the things we probably need to do on the Client side to have a nice Auto experience. I might be missing something - so do chime in and let me know:

• Figure out business case (what is the multiplier) and available models for Auto @isidorn
• Change model dropdown visuals such that there is no "Standard / Premium models". There is no included, it is 0x. so basically each model just has a multiplier, and that's it.
• Chat needs to call into CAPI before request sent to figure out what model will actually be used. Or what is the set of models that have capacity. CAPI can provide a new API for this. The question is how often does Chat call into CAPI to figure this out? Once per agent session? What is sent as part of the request (prompt + history?) What is the actual API contract?
• Once CAPI gives us the model to use via previous request, we construct the prompt and everything for that model, and then Chat sends "auto request but use model A that you just told me"
• Auto needs to be transparent (for users to love it):
  • Figure out with CAPI how do we show a set of models Auto chooses from. Is this just a details that we show on hover in model dropdown, or something else. The goal is that the user is aware of what models Auto might pick before the request is made
  • On response done, we need to render what model was actually used. I personally like this LMStudio UI (but we can use whatever we find best

I do not know how this would work in Inline Chat. Ideally same API flow, but as step 1 transparent UI on model used does not have to be there.