Peter van VelzenandXuan Son Nguyen d80619397c server : pass id_slot through the Anthropic /v1/messages conversion (#28554) ...

* server : pass id_slot through the Anthropic /v1/messages conversion

server_chat_convert_anthropic_to_oai copies a small whitelist of common
params from the Anthropic request into the internal OAI-style body and
left id_slot out. Every other completion endpoint honours id_slot; a
proxy that pins a session to the slot holding its KV cache needs the
same on /v1/messages, or it silently falls back to automatic slot
selection and, on hybrid/recurrent models, pays a full re-prefill.

Adds a test asserting a request with id_slot set is served by that
slot, observed via GET /slots.

* server : shorten comments

* rm test

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

2026-10-20 13:23:41 +02:00

..

batched-bench

batch: migrate the rest of examples to llama_batch_ext (#29601)

2026-09-30 18:08:43 +02:00

cli

server: add --connect, allow p2p access to server outside LAN (#28277)

2026-10-20 00:23:14 +02:00

completion

server: add --connect, allow p2p access to server outside LAN (#28277)

2026-10-20 00:23:14 +02:00

connect

server: add --connect, allow p2p access to server outside LAN (#28277)

2026-10-20 00:23:14 +02:00

cvector-generator

batch: migrate the rest of examples to llama_batch_ext (#29601)

2026-09-30 18:08:43 +02:00

export-lora

tools : reject truncated GGUF tensor data in split and export tools (#30497)

2026-10-19 09:27:46 +03:00

fit-params

server: use port 9931 by default (#30159)

2026-10-09 10:40:03 +02:00

gguf-split

tools : reject truncated GGUF tensor data in split and export tools (#30497)

2026-10-19 09:27:46 +03:00

imatrix

imatrix : include clocale for std::setlocale (#30079)

2026-10-07 10:58:22 +02:00

llama-bench

llama-bench: respect -fitc if bigger than required benchmark size (#28331)

2026-10-09 19:14:30 +02:00

mtmd

mtmd : match the transformers Qwen-VL resize (#30736)

2026-10-20 01:14:40 +02:00

perplexity

batch: migrate the rest of examples to llama_batch_ext (#29601)

2026-09-30 18:08:43 +02:00

quantize

quantize: cap working memory size to avoid loading big tensors onto RAM (#27795)

2026-08-27 18:31:13 +02:00

results

batch: migrate the rest of examples to llama_batch_ext (#29601)

2026-09-30 18:08:43 +02:00

rpc

rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs (#29544)

2026-10-11 17:43:35 +03:00

server

server : pass id_slot through the Anthropic /v1/messages conversion (#28554)

2026-10-20 13:23:41 +02:00

tokenize

tokenize : drop --stdin mutual-exclusion check (#25672)

2026-07-15 18:41:51 +02:00

tts

args: add --video-* CLI arguments (#24318)

2026-08-27 12:11:12 +02:00

tuning

metal : key the fa-vec tuned table by family instead of SKU (#29075)

2026-09-23 19:23:15 +08:00

ui

ui: hide the native clear button in SearchInput (#30719)

2026-10-20 11:58:38 +08:00

CMakeLists.txt

server: add --connect, allow p2p access to server outside LAN (#28277)

2026-10-20 00:23:14 +02:00