Skip to content

Conversation

zhyncs
Copy link
Member

@zhyncs zhyncs commented Sep 1, 2024

Motivation

add Llama 3.1 8B for test

This PR will continue to be improved upon.

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@zhyncs zhyncs merged commit 6487ef6 into main Sep 1, 2024
1 check passed
@zhyncs zhyncs deleted the ci branch September 1, 2024 17:19
@zhyncs
Copy link
Member Author

zhyncs commented Sep 1, 2024

@zhyncs
Copy link
Member Author

zhyncs commented Sep 1, 2024

ref
main...ci
#1272

TODO

  • add more models

TP1:
meta-llama/Meta-Llama-3.1-8B-Instruct
mistralai/Mistral-7B-Instruct-v0.3
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct
google/gemma-2-27b-it

TP2:
meta-llama/Meta-Llama-3.1-70B-Instruct
mistralai/Mistral-7B-Instruct-v0.3
Qwen/Qwen2-72B-Instruct
Qwen/Qwen2-57B-A14B-Instruct

  • add result threshold verification

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant