-
Notifications
You must be signed in to change notification settings - Fork 728
Closed
Milestone
Description
I have a reranker (cross encoder) which relies on an absolute positional embedding, limited to 8192 tokens.
In a situation where the length of prompt + document > 8192 tokens, it stops.
Tried using kwargs but didnt find the one passed to the tokenizer
result = self.pipeline([{"text": q, "text_pair": t} for t in texts], top_k=None, function_to_apply="none", num_workers=workers) |
tokenizer = AutoTokenizer.from_pretrained(reranker_path)
model = AutoModelForSequenceClassification.from_pretrained(reranker_path, torch_dtype=torch.float16)
similarity_scorer = Similarity(
(model, tokenizer),
crossencode=True,
)
How to truncate on the right?
Metadata
Metadata
Assignees
Labels
No labels