-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Puter AI refactor #1360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
KernelDeimos
wants to merge
28
commits into
main
Choose a base branch
from
eric/ai-router
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Puter AI refactor #1360
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit has cross-concern because I forgot to commit the cleanup portion previously, and separating the changes did not seem worthwhile. This could be done retroactively (i.e. during a rebase) but that's probably not worthwhile either because this is likely to be squash merged. This commit implements coercion of puter_path using the interface provided by the new module (content parts with type=data).
This is a possible idea for how various LLMs are registered and also includes a change that should have been in previous commits when adding Anthropic.
The challenging part here was figuring out how to organize code responsible for conventions common across multiple AI providers that were put in place by other AI providers. These will go in a directory called "convention" to distinguish from code related specifically to particular providers.
- create new package - copy streaming utilities to new package
CompletionWriter is a copy of AIChatStream with a more accurate name that leaves room for stream writers for other AI formats (since "completion" is just one of many)
The Messages library had overloaded concerns and has now been split into UniversalPromptNormalizer (normalization for the "universal" input format), and NormalizedPromptUtil (functions that can be used on normalized prompt input for convenient inference).
I keep forgetting I can't do this and it only works the other way around, but I want the new AI module to use ES imports syntax. Luckily backend services have an async _construct lifecycle method that I can use to import the module and assign members to the instance variable.
Lines accessing `model` instead of `options.model` were causing an exception to be thrown. This was not detected earlier because `max_tokens` is usually already provided and we don't reach the conditional expression that does this. However, this code was tested on a different branch that erroneously skips this logic which brought this bug to surface.
I did not test this part in my initial implementation and found a couple of bugs when I did. These changes get handle_files to the functionality it had prior to refactoring.
There were a lot of moving parts here; these changes were tested by comparing responses between the current `main` branch and the branch where this refactor is taking place.
This commit adds a registry for functions that implement whatever behavior is necessary to produce a specified output value with the given input values. This simplifies a few different things: - trims off unnecessary constructs; now everything can be one of these definitions - makes it possible to have different entrypoints without adding more code, so Puter can continue to pass its own `completionWriter` Potential caveat is that the way the an interaction with an LLM is invoked using this interface is unconventional compared to existing SDKs , but this doesn't need to be the interface a consumer of this module uses; we can employ the facade pattern to make it easier to use.
This changes adds a branch of behavior to Registry. Of the output type being requested has no provider for which there are sufficient input values, it then tries to obtain the missing input values recursively.
OpenAI has now been changed to use the Registry. It was not possible to move OpenAIToolsAdapter because it is being used by other providers which use the OpenAI SDK. OpenAIStreamAdapter remains as being implemented like a utility class because it seems to be simpler that way.
DeekSeek is mostly compatible with the OpenAI SDK. We used to have some changes to deal with a quirk in tool calls but it never worked great and the issue is reported to have been fixed on their end.
Claude didn't do a great job, but it didn't do a terrible job. I'm committing this code to give it a little bit of time in the limelight to see if it's something I can make work. For now, it's promising, but if it turns out to be more grief than it's worth it will be removed promptly and without hesitation.
so that it uses "obtain" in the airouter package
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.