Skip to content

Conversation

xiezhq-hermann
Copy link
Collaborator

@xiezhq-hermann xiezhq-hermann commented Mar 16, 2025

Motivation

In a prior PR #4356 the data type for KV cache memory pool indices was changed from int32 to int64, but the change was not complete.

Modifications

This PR amends following type change:
1. Indices in req_to_token_pool
2. Default 0 tensor in Radix and HiRadix tree
3. Host KV cache memory pool

Checklist

@merrymercy merrymercy merged commit f5bbf60 into main Mar 17, 2025
38 of 45 checks passed
@merrymercy merrymercy deleted the xiezhq-int64 branch March 17, 2025 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants