mirrors/llama.cpp
0
0 Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-10-28 17:56:13 +08:00
Code Activity

sampling : use greedy selection for eligible temperature-zero chains (#29797)

Browse Source 
* sampling : use greedy selection for eligible temperature-zero chains

Assisted-by: OpenAI Codex

* Apply suggestion from @ggerganov

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* sampling: simplify zero-temperature greedy eligibility

Allow the same greedy selection on CPU and grammar/reasoning-budget paths.
Keep distribution sampling for dynamic temperature and requested probabilities.
Cover the common sampler selection and probability behavior in the existing sampler tests.

Assisted-by: OpenAI Codex

* sampling : use greedy selection after final top-k with k=1

Assisted-by: OpenAI Codex

* cont : clean-up

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Hrishith ThadicherlaandGeorgi Gerganov authored and GitHub committed 2026-10-07 13:34:18 +03:00
1 parent 42b021b4dc
commit d0b490f25e
3 files changed
+155 -48
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

common/sampling.cpp
+5 -2
View File
@@ -399,8 +399,11 @@ struct common_sampler * common_sampler_init(
// only if user explicitly included adaptive-p sampler
samplers.push_back(llama_sampler_init_adaptive_p(params.adaptive_target, params.adaptive_decay, params.seed));
} else {
// default: sample from distribution
samplers.push_back(llama_sampler_init_dist(params.seed));
// Keep distribution sampling when callers request probabilities.
const bool greedy = params.n_probs == 0 && !params.samplers.empty() &&
((params.samplers.back() == COMMON_SAMPLER_TYPE_TEMPERATURE && params.temp == 0.0f && params.dynatemp_range == 0.0f) ||
(params.samplers.back() == COMMON_SAMPLER_TYPE_TOP_K && params.top_k == 1));
samplers.push_back(greedy ? llama_sampler_init_greedy() : llama_sampler_init_dist(params.seed));
}
} else if (params.mirostat == 1) {
samplers.push_back(llama_sampler_init_temp(params.temp));