mirror of
https://github.com/huggingface/transformers.git
synced 2026-10-29 01:06:24 +08:00
e37e548ce7
* Fix odd head_dim validation for RoPE configurations Validate that rotary dimensions are even in RotaryEmbeddingConfigMixin.validate_rope. RoPE requires an even dimension to compute pairwise 2D rotations; an odd dimension results in dimension mismatch crashes during the forward pass. Fixes #48101 Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com> * Simplify odd head_dim RoPE validation Only reject the case that actually crashes: an odd head_dim that is fully rotated. Odd partial rotary dims are rounded up by `arange(0, dim, 2)` and still fit inside the head, so they work today and shouldn't be rejected. Drop the try/except scaffolding and the per-layer cross product; skip heterogeneous configs instead. Bump a few ConfigTester hidden sizes that produced head_dim=1/3 and would crash in a real forward. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Allow head_dim <= 4 in validate_rope for synthetic test fixtures and Hub checkpoints Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com> --------- Signed-off-by: Soumyajit Ghosh <jobsoumyajit6124@gmail.com> Co-authored-by: Matt <rocketknight1@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>