Skip to content

fix vulkan building #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: vulkan
Choose a base branch
from
Open

fix vulkan building #14

wants to merge 2 commits into from

Conversation

Dts0
Copy link

@Dts0 Dts0 commented Feb 13, 2025

Fix #7

Now you can build it without make -f Makefile.sync clean sync

cmd:

# Build for CPU
cmake --preset CPU
cmake --build --parallel --preset CPU
cmake --install build --component CPU --strip

# Build for Vulkan
cmake --preset Vulkan
cmake --build --parallel --preset Vulkan
cmake --install build --component Vulkan --strip

# Build Ollama binary
go build -trimpath -buildmode=pie -o dist/bin/ollama

If a error occur while running, you can run it with sudo or enable CAP_PERFMON .

Dts0 added 2 commits February 13, 2025 19:30
@Dts0
Add amd rx580 support
f2c17aa
@Dts0
fix: fix vulkan building
3c22d5a 
1. Add preset for vulkan.
2. Add backend ggml-vulkan.
3. Add some log info.
@Dts0 Dts0 mentioned this pull request Feb 13, 2025
Open
@isoos
Copy link

isoos commented Feb 19, 2025

Thank you for doing this, I was able to run ollama + vulkan locally with ~21% improvement over CPU inference with an AMD iGPU (8700G vs 780M).

@isoos isoos mentioned this pull request Feb 19, 2025
Open
@pmonck
Copy link

pmonck commented Feb 21, 2025

I compiled this on 22.04, but it's still not working for me. Vulkan is detected on both of my GPUS, but ollama looks for rocm and falls back to CPU when it doesn't find the rocm library. Do I need to install rocm even if I don't intend to use it?

time=2025-02-18T22:11:00.296Z level=INFO source=routes.go:1237 msg="Listening on 127.0.0.1:11434 (version 0.0.0)" time=2025-02-18T22:11:00.296Z level=INFO source=gpu.go:255 msg="looking for compatible GPUs" time=2025-02-18T22:11:00.514Z level=INFO source=gpu.go:200 msg="vulkan: load libvulkan and libcap ok" time=2025-02-18T22:11:00.698Z level=INFO source=gpu.go:422 msg="error looking up vulkan GPU memory" error="device is a CPU" time=2025-02-18T22:11:00.698Z level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2025-02-18T22:11:00.699Z level=WARN source=amd_linux.go:443 msg="amdgpu detected, but no compatible rocm library found. Either install rocm v6, or follow manual install instructions at https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install" time=2025-02-18T22:11:00.699Z level=WARN source=amd_linux.go:348 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=GPU-2e542f98-b4e2-1707-d3e8-aa04138f97bc library=cuda variant=v12 compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" total="23.6 GiB" available="4.1 GiB" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=0 library=vulkan variant="" compute=1.4 driver=1.4 name="NVIDIA GeForce RTX 3090" total="24.2 GiB" available="4.6 GiB" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=1 library=vulkan variant="" compute=1.3 driver=1.3 name="AMD Radeon Graphics (RADV VEGA20)" total="32.0 GiB" available="31.4 GiB

@isoos
Copy link

isoos commented Feb 21, 2025

@pmonck: running it as root helped me

@pmonck
Copy link

pmonck commented Feb 21, 2025

@pmonck: running it as root helped me

Thanks, that worked.
Does anyone know how to make it work running as 'ollama'?
I've tried "sudo setcap cap_perfmon=ep /usr/bin/ollama" , but it looks like there's another permissions issue somewhere.

@jhemmond
Copy link

jhemmond commented Mar 8, 2025

@Dts0 Are these instructions valid for a Windows build?

@Dts0
Copy link
Author

Dts0 commented Mar 9, 2025

If you're looking for Windows, binaries, newer versions, or more detailed introductions, see :
#7

@jhemmond
Copy link

jhemmond commented Mar 9, 2025 via email

@grinco grinco mentioned this pull request Mar 16, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Please provide build instructions
4 participants
@Dts0 @isoos @pmonck @jhemmond