Skip to content

Conversation

ngxson
Copy link
Owner

@ngxson ngxson commented Mar 7, 2025

Example usage:

const messages: WllamaChatMessage[] = [
  { role: 'system', content: 'You are helpful.' },
  { role: 'user', content: 'Hi!' },
  { role: 'assistant', content: 'Hello!' },
  { role: 'user', content: 'How are you?' },
];
const stream = await wllama.createChatCompletionGenerator(messages, {
  nPredict: 10,
  sampling: {
    temp: 0.0,
  },
});

for await (const chunk of stream) {
  console.log(chunk.currentText);
}

@ngxson ngxson merged commit cb8857d into master Mar 7, 2025
5 checks passed
@ngxson
Copy link
Owner Author

ngxson commented Mar 7, 2025

TODO: also add test for AbortSignal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant