Skip to content

chore: default use 12.9.0 docker image #4235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 12, 2025
Merged

chore: default use 12.9.0 docker image #4235

merged 1 commit into from
May 12, 2025

Conversation

zwpaper
Copy link
Member

@zwpaper zwpaper commented May 12, 2025

Test CI: https://github.com/zwpaper/tabby/actions/runs/14973940325/job/42061314783

Test image:

  • zwpaper/tabby:0.105.0
  • zwpaper/tabby:0.105.0-cuda11

test command:

nerdctl run --gpus all --rm   -p 0.0.0.0:8080:8080 \                                                                                              (base)
                         -v $HOME/.tabby:/data \
                         zwpaper/tabby:0.105.0 \
                         serve \
                         --model Qwen2.5-Coder-1.5B \
                         --chat-model Qwen2.5-Coder-1.5B-Instruct \
                         --device cuda --chat-device cuda
docker.io/zwpaper/tabby:0.105.0:                                                  resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:8f17e50141a54023942160fd391861187b5ad10e52e14600ee22726a9d6d6569:    exists         |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:9992a93a56f111429b95c5e4545b7db6e6360eca24c98686544add7e6287849c: exists         |++++++++++++++++++++++++++++++++++++++|
config-sha256:ab75996a992f4ecf99243dbd71a4c2bfe6a479801bbde256847e4b3f44d8ec25:   exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:215ed5a638430309375291c48a01872859a8dbf1331e54ba0af221918eb8ce2e:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:8e257e2f61b3bb5efa27861c9f528c087effec0ae0df0bc85b425ff12a4628b2:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:6ec52169f4075cf80de2f49076cd5b8db43a64c663efc8ec39579b970071bb7c:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:b4d600b977438c1e74729cafa7158dfe98928d53ec5a0eb189fcd3739752cc20:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:12e96d9ed36f18f96cfa98e2351f018f104a9a51d980407af1fc22e60e4e778c:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:5ada09cfb5af365e37f3ac41858d0e72994953474937dbc460ae5ed1b2d9bba3:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:39a549996d3590b8193689c2f43e8af440a7136026e85e43e46df3a0fbed2732:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:01a77ecc44d681faf2e4514567d1ec0733a24aa52177c3ad36883df1e1e6eb16:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:fade6560457cf3fba6dfb3d8280309ce9e5e0eded9c748f8fa6e611cf6b2a96c:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:38cf2a08e5b04c3cbc6bbe9b2195d9bbc13f38ed602d0daa2e7b2d865e92bbfc:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:1bba15468fcc82586ce921fdc422649a192dcc635f587390cd01beb95ca6dfb5:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:1b1871cb04175218ddf8d02234c35a63e77b6c6ce7c2676ddfdde575c996ee2c:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:47e265d4453d154b2fbda7a482131d7fb31e9c25323117dac51d4315a4aa676a:    exists         |++++++++++++++++++++++++++++++++++++++|
layer-sha256:69dedd74cc89ea89b937e703728a3d57049ad6e4561a3da9e8ef6c0a58638651:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 227.1s                                                                   total:   0.0 B (0.0 B/s)
2025-05-12T15:00:06.691277Z  WARN tabby::serve: crates/tabby/src/serve.rs:384: Overriding completion model from config.toml. The overriding behavior might surprise you. Consider setting the model in config.toml directly.
2025-05-12T15:00:06.691292Z  WARN tabby::serve: crates/tabby/src/serve.rs:391: Overriding chat model from config.toml. The overriding behavior might surprise you. Consider setting the model in config.toml directly.
⠇     3.050 s   Starting...
████████╗ █████╗ ██████╗ ██████╗ ██╗   ██╗
╚══██╔══╝██╔══██╗██╔══██╗██╔══██╗╚██╗ ██╔╝
   ██║   ███████║██████╔╝██████╔╝ ╚████╔╝
   ██║   ██╔══██║██╔══██╗██╔══██╗  ╚██╔╝
   ██║   ██║  ██║██████╔╝██████╔╝   ██║
   ╚═╝   ╚═╝  ╚═╝╚═════╝ ╚═════╝    ╚═╝

📄 Version 0.28.0-dev.0
🚀 Listening at http://0.0.0.0:8080



  JWT secret is not set

  Tabby server will generate a one-time (non-persisted) JWT secret for the current process.
  Please set the TABBY_WEBSERVER_JWT_TOKEN_SECRET environment variable for production usage.

Working as expected:
CleanShot 2025-05-12 at 23 00 43@2x

@zwpaper zwpaper requested review from Copilot and wsxiaoys and removed request for Copilot May 12, 2025 15:03
Copilot
Copilot AI reviewed May 12, 2025
View reviewed changes
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the default Docker image to use CUDA version 12.9.0 while adding additional workflow steps to support legacy CUDA 11 image builds. Key changes include:

  • Introducing a new CUDA_VERSION environment variable with a default value of 12.9.0.
  • Updating the main Docker build step to pass CUDA_VERSION from the environment.
  • Adding separate steps for generating Docker metadata and building/pushing a CUDA 11 image with CUDA_VERSION explicitly set to 11.7.1.

.github/workflows/docker.yml
cache-to: ${{ steps.cache.outputs.cache-to }}
build-args: |
RUST_TOOLCHAIN=${{ env.RUST_TOOLCHAIN }}
CUDA_VERSION=11.7.1
Copy link
Preview

Copilot AI May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider refactoring common build arguments such as RUST_TOOLCHAIN into a shared configuration to reduce duplication between build steps, thereby improving maintainability.

Suggested change
CUDA_VERSION=11.7.1
CUDA_VERSION=${{ env.CUDA_VERSION_11 }}

Copilot uses AI. Check for mistakes.

@wsxiaoys wsxiaoys merged commit b2ae3fa into TabbyML:main May 12, 2025
1 check passed
zwpaper added a commit to zwpaper/tabby that referenced this pull request May 15, 2025
@zwpaper
chore: default use 12.9.0 docker image (TabbyML#4235)
42ae996
@dmbeta dmbeta mentioned this pull request Jul 7, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

Copilot code review Copilot Copilot left review comments

@wsxiaoys wsxiaoys Awaiting requested review from wsxiaoys

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zwpaper @wsxiaoys