mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-10-28 09:45:55 +08:00
glm5-next : drop the q_absorbed cont
The meta backend can now split the per-stream view of the permuted q, so the cont is not needed for -sm tensor with a non-unified KV cache. With the cont, the q of flash attention was not contiguous, and when FA ran on the CPU (WebGPU) the meta get_tensor aborted.
This commit is contained in:
1 parent
c72d4223c6
commit
49a581a8d4
1 file changed
+1
-2
@@ -1068,8 +1068,7 @@ ggml_tensor * llama_model_glm5_next::graph::build_dsa_layer(
|
||||
// absorb wk_b so the cache holds only the latent
|
||||
ggml_tensor * q_absorbed = ggml_permute(ctx0, q, 0, 2, 1, 3);
|
||||
q_absorbed = ggml_mul_mat(ctx0, layer.wk_b, q_absorbed);
|
||||
// FIXME: -sm tensor cannot split the per-stream view of a permuted q in build_attn_mha (non-unified KV)
|
||||
q_absorbed = ggml_cont(ctx0, ggml_permute(ctx0, q_absorbed, 0, 2, 1, 3));
|
||||
q_absorbed = ggml_permute(ctx0, q_absorbed, 0, 2, 1, 3);
|
||||
cb(q_absorbed, "q_absorbed", il);
|
||||
|
||||
ggml_tensor * kq_mask = inp_attn->get_kq_mask();
|
||||
|
||||
Reference in new issue
Block a user