Skip to content

Conversation

zcnrex
Copy link
Owner

@zcnrex zcnrex commented Apr 6, 2025

Motivation

python3 -m sglang.launch_server --model-path \meta-llama/Llama-3.2-11B-Vision-Instruct --chat-template=llama_3_vision --attention-backend fa3 --disable-cuda-graph

python3 sglang/python/sglang/test/send_one.py --image

fa3 disable cuda graph

 A man in a yellow jacket is ironing clothes on the back of a yellow taxi cab. The taxi is driving down a city street. The man is standing on the sidewalk, holding the ironing board and iron, and appears to be ironing a shirt. The taxi has a white sign on the back that says "Taxi" in black letters. The background is a city street with tall buildings and trees. The overall atmosphere suggests a busy urban setting, with the man's unusual activity adding a touch of humor and surprise to the scene. The image captures a moment of everyday life in the city, highlighting the resourcefulness and adaptability of the man in the yellow jacket. The image conveys a sense of urban bustle and activity, with the taxi and other vehicles moving quickly through the scene. The man's ironing board and iron are a humorous addition to the image, adding a touch of whimsy to the otherwise mundane scene. The image suggests that even in the midst of a busy city, people can find ways to make the most of their time and resources. The image conveys a sense of community and connection, as the man is interacting with the taxi and the city around him. The image is a commentary on the human ability to adapt and find creative solutions to everyday problems, even in the midst of a busy and chaotic urban environment. The image is a celebration of the human spirit and the ability to find joy and humor in unexpected places. The image is a commentary on the role of technology in modern life, as the man uses an ironing board and iron in a way that is not typically associated with urban environments. The image is a commentary on the importance of community and connection in urban life, as the man is interacting with the taxi and the city around him. The image is a celebration of the human ability to find beauty and humor in the mundane, as the man's ironing board and iron add a touch of whimsy to the scene. The image is a commentary on the role of creativity and resourcefulness in urban life, as the man finds a way to use an ironing board and iron in a way that is not typically associated with urban environments. The image is a celebration of the human ability to find joy and humor in unexpected places, as the man's ironing board and iron add a touch of whimsy to the scene. The image is a commentary on the importance of community and connection in urban life, as the man is interacting with the taxi and the city around him. The image is a celebration

acc_length=1.00
speed=57.75 token/s

flashinfer disable cuda graph

 A man in a yellow jacket is ironing clothes on the back of a yellow taxi cab. The taxi is driving down a city street. The man is standing on the sidewalk, holding the ironing board and iron, and appears to be ironing a shirt. The taxi has a white license plate with black lettering that reads "TAXI" and a yellow sign on the side that says "TAXI." The background is a city street with tall buildings and trees. The overall atmosphere suggests a busy urban setting, with the man's unusual activity adding a touch of humor and surprise to the scene. The image captures a moment of everyday life in the city, highlighting the resourcefulness and adaptability of the man in the yellow jacket. The image conveys a sense of urban bustle and activity, with the taxi and the man's ironing board adding to the sense of movement and energy. The image also suggests a sense of community and connection, as the man is ironing in a public space, surrounded by other people and vehicles. The image is a commentary on the human experience, highlighting the ways in which people adapt and find ways to make do in unexpected situations. The image is a celebration of the human spirit, showing how people can find joy and purpose in even the most mundane tasks. The image is a commentary on the urban environment, highlighting the ways in which cities can be both chaotic and beautiful. The image is a reflection of the human condition, showing how people can find meaning and purpose in their daily lives, even in the midst of chaos and uncertainty. The image is a celebration of the human experience, highlighting the ways in which people can find beauty and joy in the everyday. The image is a commentary on the human condition, showing how people can find meaning and purpose in their daily lives, even in the midst of chaos and uncertainty. The image is a reflection of the human experience, showing how people can find beauty and joy in the everyday. The image is a celebration of the human experience, highlighting the ways in which people can find meaning and purpose in their daily lives, even in the midst of chaos and uncertainty. The image is a commentary on the human condition, showing how people can find meaning and purpose in their daily lives, even in the midst of chaos and uncertainty. The image is a reflection of the human experience, showing how people can find beauty and joy in the everyday. The image is a celebration of the human experience, highlighting the ways in which people can find meaning and purpose in their daily lives,

acc_length=1.00
speed=43.15 token/s

background

In a VLM, the Query typically comes from one modality (e.g., text embeddings), while the Key and Value come from the other modality (e.g., image features), or vice versa.

The output of cross-attention is a fused representation that combines information from both modalities. This can be fed into further layers for tasks like image captioning, visual question answering, or text-to-image generation.

Modifications

Checklist

@zcnrex zcnrex closed this Apr 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant