Skip to content

Puter AI refactor #1360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 28 commits into
base: main
Choose a base branch
from
Draft

Puter AI refactor #1360

wants to merge 28 commits into from

Conversation

KernelDeimos
Copy link
Contributor

No description provided.

This commit has cross-concern because I forgot to commit the cleanup
portion previously, and separating the changes did not seem worthwhile.
This could be done retroactively (i.e. during a rebase) but that's
probably not worthwhile either because this is likely to be squash
merged.

This commit implements coercion of puter_path using the interface
provided by the new module (content parts with type=data).
This is a possible idea for how various LLMs are registered and also
includes a change that should have been in previous commits when adding
Anthropic.
The challenging part here was figuring out how to organize code
responsible for conventions common across multiple AI providers that
were put in place by other AI providers. These will go in a directory
called "convention" to distinguish from code related specifically to
particular providers.
- create new package
- copy streaming utilities to new package
CompletionWriter is a copy of AIChatStream with a more accurate name
that leaves room for stream writers for other AI formats (since
"completion" is just one of many)
The Messages library had overloaded concerns and has now been split into
UniversalPromptNormalizer (normalization for the "universal" input
format), and NormalizedPromptUtil (functions that can be used on
normalized prompt input for convenient inference).
I keep forgetting I can't do this and it only works the other way
around, but I want the new AI module to use ES imports syntax. Luckily
backend services have an async _construct lifecycle method that I can
use to import the module and assign members to the instance variable.
Lines accessing `model` instead of `options.model` were causing an
exception to be thrown. This was not detected earlier because
`max_tokens` is usually already provided and we don't reach the
conditional expression that does this. However, this code was tested on
a different branch that erroneously skips this logic which brought this
bug to surface.
I did not test this part in my initial implementation and found a couple
of bugs when I did. These changes get handle_files to the functionality
it had prior to refactoring.
There were a lot of moving parts here; these changes were tested by
comparing responses between the current `main` branch and the branch
where this refactor is taking place.
This commit adds a registry for functions that implement whatever
behavior is necessary to produce a specified output value with the given
input values. This simplifies a few different things:
- trims off unnecessary constructs; now everything can be one of these
  definitions
- makes it possible to have different entrypoints without adding more
  code, so Puter can continue to pass its own `completionWriter`

Potential caveat is that the way the an interaction with an LLM is
invoked using this interface is unconventional compared to existing SDKs
, but this doesn't need to be the interface a consumer of this module
uses; we can employ the facade pattern to make it easier to use.
This changes adds a branch of behavior to Registry. Of the output type
being requested has no provider for which there are sufficient input
values, it then tries to obtain the missing input values recursively.
OpenAI has now been changed to use the Registry. It was not possible to
move OpenAIToolsAdapter because it is being used by other providers
which use the OpenAI SDK. OpenAIStreamAdapter remains as being
implemented like a utility class because it seems to be simpler that
way.
DeekSeek is mostly compatible with the OpenAI SDK. We used to have some
changes to deal with a quirk in tool calls but it never worked great and
the issue is reported to have been fixed on their end.
Claude didn't do a great job, but it didn't do a terrible job. I'm
committing this code to give it a little bit of time in the limelight to
see if it's something I can make work. For now, it's promising, but if
it turns out to be more grief than it's worth it will be removed
promptly and without hesitation.
so that it uses "obtain" in the airouter package
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant