chore: update localai to v3.4.0 #592

sozercan · 2025-08-16T21:56:30Z

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #

Special notes for your reviewer:

sozercan · 2025-08-20T18:37:42Z

pkg/aikit2llb/inference/convert.go

@@ -12,9 +12,10 @@ import (
 )

 const (
-	distrolessBase = "ghcr.io/kaito-project/aikit/base:latest"
+	// distrolessBase = "ghcr.io/kaito-project/aikit/base:latest".
+	distrolessBase = "docker.io/sozercan/base:test"


Copilot

Pull Request Overview

This PR updates LocalAI to v3.4.0 and refactors backend handling to use OCI registry-based backends instead of building from source. The update removes the Mamba backend support and introduces a new backend management system.

Updates LocalAI from v2.26.0 to v3.4.0
Removes Mamba backend support completely
Refactors backend installation to use pre-built OCI images from registry
Updates template syntax for completion prompts across model files

Reviewed Changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/aikit2llb/inference/convert.go	Updates LocalAI version constant and modifies binary download URLs
pkg/aikit2llb/inference/backend.go	Introduces new OCI-based backend installation system with platform-specific logic
pkg/utils/const.go	Removes BackendMamba constant and adds BackendLlamaCpp and BackendOCIRegistry constants
pkg/build/build.go	Updates backend validation logic and adds platform compatibility checks
website/docs/specs-inference.md	Updates backend documentation to remove mamba and add llama-cpp as default
test/ and models/ files	Adds missing template syntax and backend configurations for various test files
Multiple workflow files	Updates security policies from block to audit egress

pkg/aikit2llb/inference/convert.go

pkg/aikit2llb/inference/backend.go

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

Copilot AI review requested due to automatic review settings August 16, 2025 21:56