Skip to content

api: return model capabilities from the show endpoint #10066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Apr 1, 2025
Merged

api: return model capabilities from the show endpoint #10066

merged 6 commits into from
Apr 1, 2025

Conversation

BruceMacD
Copy link
Contributor

With support for multimodal models becoming more varied and common it is important for clients to be able to easily see what capabilities a model has. Retuning these from the show endpoint will allow clients to easily see what a model can do.

This change adds a capabilities field to the show response so that clients can easily see what features a given model supports. It also adds a vision capability based off the presence of a vision.block_count KV.

Sample API response:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 31 Mar 2025 23:26:18 GMT
Connection: close
Transfer-Encoding: chunked

{
  "license": "omitted",
  "modelfile": "omitted",
  "parameters": "omitted",
  "template": "{{- range $i, $_ := .Messages }}\n{{- $last := eq (len (slice $.Messages $i)) 1 }}\n{{- if or (eq .Role \"user\") (eq .Role \"system\") }}\u003cstart_of_turn\u003euser\n{{ .Content }}\u003cend_of_turn\u003e\n{{ if $last }}\u003cstart_of_turn\u003emodel\n{{ end }}\n{{- else if eq .Role \"assistant\" }}\u003cstart_of_turn\u003emodel\n{{ .Content }}{{ if not $last }}\u003cend_of_turn\u003e\n{{ end }}\n{{- end }}\n{{- end }}",
  "details": {
    "parent_model": "",
    "format": "gguf",
    "family": "gemma3",
    "families": [
      "gemma3"
    ],
    "parameter_size": "4.3B",
    "quantization_level": "Q4_K_M"
  },
  "model_info": {
    // ... omitted
  },
  "tensors": [
    // ... omitted
  ],
  "capabilities": [
    "completion",
    "vision"
  ],
  "modified_at": "2025-03-31T16:07:27.293556892-07:00"
}

Sample ollama show CLI output:

❯ ./ollama show llama3.2:1b
  Model
    architecture        llama     
    parameters          1.2B      
    context length      131072    
    embedding length    2048      
    quantization        Q8_0      

  Capabilities
    completion    
    tools         

  License
    LLAMA 3.2 COMMUNITY LICENSE AGREEMENT                 
    Llama 3.2 Version Release Date: September 25, 2024

@BruceMacD
api: return model capabilities from the show endpoint
70a9435 
With support for multimodal models becoming more varied and common it is important for clients to be able to easily see what capabilities a model has. Retuning these from the show endpoint will allow clients to easily see what a model can do.
@BruceMacD BruceMacD mentioned this pull request Mar 31, 2025
Closed
@BruceMacD BruceMacD force-pushed the brucemacd/api-caps branch from a17d78f to 4c4e0aa Compare March 31, 2025 23:37
Copilot
Copilot AI reviewed Mar 31, 2025
View reviewed changes
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the model show endpoint by adding a new field for model capabilities, making it easier for clients to discover what features a model supports. Key changes include:

  • Introducing a new Capability type and constants in types/model/capability.go.
  • Updating server endpoints and CLI commands to utilize the new capabilities information.
  • Extending tests and API documentation to validate and showcase model capabilities.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file
File Description
types/model/capability.go Defines model capability constants and a String method.
server/sched.go Refactors capability calls to use the new model.Capability constants.
server/routes.go Updates route handlers to work with model.Capability and variable renaming.
server/images_test.go Adds tests validating model capabilities extraction from model files.
server/images.go Implements Capabilities and CheckCapabilities methods for models.
docs/api.md Updates API example to include the new capabilities field.
cmd/cmd_test.go Extends tests for CLI output to include capabilities information.
cmd/cmd.go Incorporates capabilities into CLI output and multimodal logic.
api/types.go Updates API response structure to include a Capabilities field.
Comments suppressed due to low confidence (3)

docs/api.md:1220

  • The model name 'llava' in the API example appears inconsistent with other references (e.g. 'llama3.2'); please confirm if this naming is intentional.
  "model": "llava"

server/images.go:137

  • Consider explicitly wrapping all joined errors using errors.Join(errs...) in the error message so that all missing capabilities are clearly reported.
return fmt.Errorf("%w %w", errCapabilities, err)

server/images.go:130

  • For unknown capabilities, consider appending a descriptive error to the errors slice instead of only logging the error, ensuring the caller receives appropriate feedback.
default:

@markjcrane
Copy link

This looks like a good feature. Especially useful for the REST API where you can get details on the model.

pdevine
pdevine reviewed Apr 1, 2025
View reviewed changes
Copy link
Contributor

@pdevine pdevine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Just one suggestion.

@BruceMacD BruceMacD requested a review from pdevine April 1, 2025 21:26
pdevine
pdevine reviewed Apr 1, 2025
View reviewed changes
pdevine
pdevine approved these changes Apr 1, 2025
View reviewed changes
Copy link
Contributor

@pdevine pdevine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm (but you should remove the comment)

BruceMacD
BruceMacD commented Apr 1, 2025
View reviewed changes
@BruceMacD BruceMacD merged commit e172f09 into main Apr 1, 2025
8 checks passed
@BruceMacD BruceMacD deleted the brucemacd/api-caps branch April 1, 2025 22:21
@JHubi1
Copy link
Contributor

JHubi1 commented Apr 2, 2025

Thank you all very much, really appreciate it

@BruceMacD
Copy link
Contributor Author

@JHubi1 thanks for bringing it up

@JHubi1 JHubi1 mentioned this pull request Apr 11, 2025
Closed
halfcrazy pushed a commit to halfcrazy/ollama that referenced this pull request Jun 19, 2025
@BruceMacD @halfcrazy
api: return model capabilities from the show endpoint (ollama#10066)
42f1d85 
With support for multimodal models becoming more varied and common it is important for clients to be able to easily see what capabilities a model has. Retuning these from the show endpoint will allow clients to easily see what a model can do.
@joaomoreno joaomoreno mentioned this pull request Jul 8, 2025
Closed
@taketwo taketwo mentioned this pull request Jul 16, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

Copilot code review Copilot Copilot left review comments

@pdevine pdevine pdevine approved these changes

@jmorganca jmorganca Awaiting requested review from jmorganca

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@BruceMacD @markjcrane @JHubi1 @pdevine