mirror of
https://github.com/huggingface/transformers.git
synced 2026-10-29 01:06:24 +08:00
823a6ea6d8
* Make longrope compileable
`longrope_frequency_update` picks between the short and long scaling
factors based on the sequence length. `seq_len` is derived from
`position_ids` and is therefore always a tensor, so the check is
data-dependent branching and fullgraph compilation fails on the very
first forward pass.
Unlike the other dynamic RoPE types, the choice genuinely depends on the
sequence length and cannot be resolved at initialization. Compute both
sets of frequencies once and select between them with `torch.where`,
which is traceable and keeps the selected values identical to before.
This also fixes two latent bugs in the same block. The cache lookup used
`f"{layer_type}_long_inv_freq"` instead of the prefix, so for a
non-nested RoPE it looked up `"None_long_inv_freq"`, which never exists
and the long factors were recomputed on every forward pass. And
`long_inv_freq` was never moved to `device`, unlike `original_inv_freq`.
`openbmb/MiniCPM3-4B` is blocked earlier: `yarn_apply_mscale` reads
`rope_parameters["factor"]` unconditionally, but `factor` is optional
and a longrope config does not necessarily carry it, so building the
model raises `KeyError`. Move the read inside the branch that needs it.
That helper is shared through `modular`, so the models deriving from it
are regenerated.
* Make the longrope compile test reach the long branch
Phi3Config copies `original_max_position_embeddings` (default 4096) into
`rope_parameters`, so seq_len=16 never exceeded it and the long factors were
never exercised.
* Don't reassign the original_inv_freq buffer inside forward
On CPU, `.to(device)` returns the same nn.Buffer, so setattr goes through
Module.__setattr__ and reads `.persistent`, which dynamo cannot trace.
The buffer already follows model.to(), so the reassignment is not needed.
* Clarify that longrope swaps the whole frequency set
* Regenerate bailing_hybrid after merging main
* Make dynamic rope compileable
* Drop redundant comment in yarn_apply_mscale
* Move dynamic RoPE compile support to a separate PR
---------
Co-authored-by: Raushan Turganbay <raushan@huggingface.co>
Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>