Skip to content

Allow token mismatch#126

Merged
Saibo-creator merged 2 commits intoepfl-dlab:mainfrom
urroxyz:patch-1
Apr 10, 2025
Merged

Allow token mismatch#126
Saibo-creator merged 2 commits intoepfl-dlab:mainfrom
urroxyz:patch-1

Conversation

@urroxyz
Copy link
Contributor

@urroxyz urroxyz commented Apr 3, 2025

Mismatch between the model logits size and acceptance mask size creates a destructive error.

  • Allow token mismatch as long as it is within reason
  • Automatically realign model logits and acceptance mask

This patch also adds support for Gemma 3 and its quantizations.

@Saibo-creator Saibo-creator merged commit 251947e into epfl-dlab:main Apr 10, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants