Skip to content

Add support for vision models to HF LLM pipeline #884

@davidmezzetti

Description

@davidmezzetti

Add support for the image-text-to-text task in the HF LLM pipeline. This will enable image models such as the following call.

model = LLM("...")
result = model([{
    "role": "user",
    "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image", "image": "books.jpg"}
    ]}
])

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions