* ui: build the reasoning menu from the template effort levels
When the chat template reports effort levels, the menu offers Default,
Off and those levels, and a pick is sent as reasoning_effort. A
template that reports none keeps the token budget menu.
Picking a model other than the one that produced the last answer resets
the conversation effort to Default, and an effort the current menu
lacks resolves to Default.
* ui: wait for a cold router model before building the request
A router serves the props of a loaded model only, so a request sent to
a cold model was built without its template effort levels and fell
back to a token budget. The request now waits for the model to load,
joining a load already in flight, and a stop during the wait sends
nothing.
Status waiters hold a list per model so a load and the requests that
wait on it each get theirs, and a props request in flight is shared
with a second caller instead of answering null.
* fix: clamp the A prefetch to end_k in the Vulkan coopmat1 int8 matmul, like the B prefetch
* fix: zero the A scale for k steps past end_k in the Vulkan coopmat1 int8 matmul instead of relying on the clamped block
* chat : strip trailing whitespace from parsed reasoning and content
Assisted-by: Claude Fable 5.1
* chat : stop qwen3-coder reasoning and content before template separators
Assisted-by: Claude
* tests : remove the Qwen parser case from the chat session tests
Assisted-by: Claude
* common : handle partial prefix until match at end of generation
---------
Co-authored-by: Zeshawn Zahid <zzshawnn@gmail.com>
Co-authored-by: Alde Rojas <hello@alde.dev>
* ui : show save and reset settings buttons only when relevant
Assisted-by: Claude (Opus 5.5 Medium)
* ui : move reset settings button next to reload app, pad settings caption
Assisted-by: Claude (Opus 5.5 Medium)
* ui: support OGG audio attachments by converting them to WAV
Fixes#24164
Assisted-by: Claude (Opus 5.5 Medium)
* ui: convert all non WAV/MP3 audio uploads to WAV
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep new chat draft when adding a system prompt
Fixes#24338
Assisted-by: Claude (Opus 5.5 Medium)
* ui: remove unused initialMessage from chat form
Assisted-by: Claude (Opus 5.5 Medium)
* ui : replace the sidebar search mode with a search dialog
Search opens a spotlight dialog (Cmd+K or the Search entry under New chat in the sidebar) with keyboard navigation over recent chats and sidebar actions. Phones keep the search page.
Sidebar actions show their shortcuts, with new ones for Settings (Cmd+,) and Models (Shift+Cmd+M).
Assisted-by: Claude (Opus 5.5 Medium)
* ui : ignore search dialog keys while an IME is composing
Enter confirms an IME candidate and the arrow keys move through the candidates, so the search field leaves them to the IME instead of opening the highlighted result or moving the highlight.
Assisted-by: Claude (Opus 5.5 Medium)
* server : add realtime conversation and transcription on /v1/realtime
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5
* examples : default the realtime client to the server port 9931
Assisted-by: Claude Opus 5.5
* server : sample realtime responses right after their own decode
Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs.
Assisted-by: Claude Opus 5.5
* move more stuff to server-realtime, making it self-contained
* examples : fix the type check of the realtime client
Assisted-by: Claude Opus 5.5
* examples : use the model card prompts in the realtime client
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B
Assisted-by: Claude Opus 5.5
* convert : keep the speech task templates out of the d1 GGUFs
Assisted-by: Claude Opus 5.5
* small refactoring: no_batching
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* server: expose the reasoning effort levels of the chat template
The capability probe renders a user turn for each candidate level
(minimal, low, medium, high, xhigh, max) and for an unknown level. A
level is honored when it renders without raising and differently from
the unknown level, which shows the template fallback. Levels sharing a
render are aliases and the highest one is kept.
The caps serialize to JSON, and chat_template_caps in /props reports
the list as reasoning_effort_levels, from lowest to highest.
* jinja: keep the caps free of the json dependency
The caps keep reporting their flags as a map, and the effort levels
stay a plain member. common_chat_templates_get_caps() builds the JSON
the server reports from both, so jinja depends on json only for the
probe inputs.
In --models mode the thread count per model is computed and printed,
but never set in the params, so every context runs on the common
default, the number of physical cores. With several models in
parallel this oversubscribes the CPU, and the busy-wait barriers of
the CPU backend make it collapse.
Set the count in cpuparams and cpuparams_batch.
* model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5
* convert : read the LFM3-Audio config.json as released
A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys).
Assisted-by: Claude Opus 5.5
* mtmd : disable flash attention for the LFM3-Audio depthformer
Assisted-by: Claude Opus 5.5
* tts : add a voice option for LFM3-Audio and LFM3-Omni
The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts.
Assisted-by: Claude Opus 5.5
* clean up template handling
* convert : detect LFM3-Audio and LFM3-Omni by model_type
Assisted-by: Claude Opus 5.5
* move banned tokens --> logits bias --> mtmd_helper
* leave mtmd-cli unchanged
* no more template_asr_system
* left over
* llama : keep the LFM3-Audio encoder convs and codec unquantized
Assisted-by: Claude Opus 5.5
* common, mtmd : shorten comments
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* common : support deferred tools and tool_reference content parts
Assisted-by: Claude Opus 5.5
* common : credit the deferred tools co-author
Co-authored-by: Aldehir Rojas <765227+aldehir@users.noreply.github.com>
Assisted-by: Claude Opus 5.5
* tests : fix the type check of the deferred tools test
Assisted-by: Claude Opus 5.5
* model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5
* tests : set the SWA rope base of the LFM3 test model
Assisted-by: Claude Opus 5.5
* llama : enable recurrent state rollback for LFM3
Assisted-by: Claude Opus 5.5
* Apply batched suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Tarek Dakhran <github@dakhran.com>
* gguf-py : align the lfm3vl mapping comments
Assisted-by: Claude Opus 5.5
* llama : fix CRLF line endings from the review suggestions
Assisted-by: Claude Opus 5.5
* tests : add the LFM3 Metal fusion baseline
Assisted-by: Claude Opus 5.5
* llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
* llama : build the k-pool sparse selection independently of the kq_mask shape
qwen4exp (QSA) and glm5-next pick cache cells by index and no longer read the kq_mask values, but they still shaped the selection like the dense kq_mask. Build the selection as [n_kv, n_tokens] over the cache cells instead, and split it into the streams of the K view right before the attention.
This lets the dense mask change its layout (e.g. per-sequence row lists) without touching the sparse layers.
* improve comment
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* llama : drop the unused build_fused_top_k_mask args, review nits
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* tests : speed up tensor data generation and model saving
Refactor save_models to create only the model, not a llama_context,
which is not needed for saving. Replace the slow std::normal_distribution
draw with a splitmix64 + box-muller normal generator.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
* tests : use a shared pool of normal draws for tensor data
Generate a pool of normal draws once with std::mt19937 and
std::normal_distribution, then have every tensor copy a cyclic slice of
it instead of running a per-tensor box-muller. The name-keyed slice
selection keeps the draw deterministic.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
* tests : log the size of saved models
Report the produced file size in bytes and MB after each save.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
* tests : simplify the model save log
Put the model name, target and size on one line per saved model.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
* vulkan: allow dim 3 broadcast in MUL_MAT
supports_op rejected every MUL_MAT whose src0 and src1 differ in ne3,
although mul_mm, mul_mmq and mul_mat_vec already broadcast src0 over
dim 3 through broadcast3. Such a matmul then ran on the CPU with a
copy of the weight, and the split layout changed with the number of
streams in the ubatch.
Keep the strict check for MUL_MAT_ID only, and route the broadcast
cases away from mul_mat_vec_nc, the one path that requires equal ne3.
* tests : harden MUL_MAT with permuted dim 3 broadcast cases
Harden the MUL_MAT coverage with a weight broadcast over dim 3 against
a permuted per-stream activation, the shape of an MLA output
projection with a non unified KV cache, for a single token, several
heads and several tokens per stream.
The expand button wrapping the system message card defaulted to
inline-block. WebKit takes an inline-block's baseline from its last
line even when that line is clipped by the card's max-height, so the
row grew to the height of the whole prompt and left a large empty gap
below the card. Make the button a block so the row matches the card.
Assisted-by: Claude (Opus 5.5 Medium)
* server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold
estimate with to-be-loaded model size included
use no_alloc to get memory requirements for model load
only set model memory_mb if not previously calculated
use memory margin instead of total size limit, apply to each device separately
add server memory debug logging
move llama_context_device_memory function to llama-ext.h
fix model count exceeded check
improve memory_per_device map naming
improve variable naming, fix style
also strip models memory margin from child processes
cont : clean-up
replace device memory map with buft memory map. Use llama_get_memory_breakdown
extract duplicated check into helper function
move model memory estimation to subprocess
precompute name->buft map, map GPU host types to CPU buft
cleanup unused variable
remove duplicated init calls
* adapt to upstream changes
* wip
* common_memory_reports internface
* SERVER_ROLE_MEASURE
* server-models-sched
* fix build
* server : fix memory accounting of measure children and claim spin
A measure child runs no_alloc, so the free memory it reports does not
include its estimate: subtracting self there erased the memory held by
other processes, down to zero with --fit on, and the next load could
overshoot the margin. Subtract self only for instance reports.
A queued load only checked --models-max before claiming, so when the
memory margin blocked it behind a busy model, load() threw and the
waiter retried in a tight loop. Refuse the claim while the limits are
exceeded and no idle model can be unloaded, so the waiter sleeps on
the condition variable until one goes idle.
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pascal <admin@serveurperso.com>
* model : add MiniCPM-OCR-Jev support
The model is a Qwen3.5-VL trunk with a value head read at the last token: it
scores an OCR text against an image rendered from it, it does not generate.
Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone.
The class scores are the pooled output of the sequence, read with last pooling,
so the conversion turns score.weight into cls.output and sets
embedding_length_out. The checkpoint has no LM head, so text generation is
rejected instead of using the output weights duplicated from the token
embeddings.
The task is fixed, so its prompt is entirely in the systemone template, the
same way h2o-lightning keeps its default there. The processor of the checkpoint
allows up to 16384 merged tokens per image, more than the Qwen-VL default of
mtmd, so the mmproj keeps the image budget of the preprocessor config. The
checkpoint declares MiniCPMOCRForSequenceClassification in its config.json, as
other models do.
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
* fix code
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
* use gguf meta classifier_output_labels
* remove redudant code changes
---------
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* metal : add GGML_OP_TOP_K_MASK
Port the fused top-k mask from the CUDA backend: a grid-stride kernel fills the output with -inf, then one
threadgroup per row copies the mask values of the kept cells over it. Function constants drop the score,
blocks and extra paths that a given graph does not use.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
* metal : clean up top-k mask kernel formatting
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
* ggml : relax scale to non-contiguous views
SCALE requires src0 to be 1D-contiguous. Relax this to dim-0
contiguity with arbitrary row strides (ggml_is_contiguous_rows),
same as the unary ops in the Metal backend.
- ggml: relax the assert in ggml_scale_impl
- cpu: address src0 rows through their byte strides (f32, bf16)
- cuda: stride-aware kernel, fastdiv row decomposition
- metal: support bf16 scale (guarded unary instantiations)
- sycl: keep the gate at contiguous, the kernel is flat f32
Add view variants to test_scale. Verified on CPU, Metal and CUDA.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)
* cuda : skip the softcap fusion on non-contiguous views
The fused softcap kernel reads src and writes dst as dense arrays,
so a view reaching SCALE -> TANH -> SCALE now that SCALE accepts
non-contiguous rows would be read wrongly. Keep such graphs on the
strided kernels until the fusion becomes stride aware.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* tests : fix multi-seq split replay against trained models
The rollback snapshot is captured by the graph that decoded the rolled-back
tail, while the reference context decodes the same prefix with different
batch shapes. The two states differ in the last bits and trained models
amplify the difference with every replayed token. Compare only the first
replayed token of each seq, then restore the rollback state into the
reference context before the seq-1-only tail.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
* tests : require bit-exact seq-1 isolation in split replay
The seq state save is not always bit-lossless - SWA-masked cells are not
saved - so reload the restored state back into the rollback context. That
way both contexts enter the seq-1-only tail with byte-identical memory,
including the cell layout, and the tail logits must match bit-exact. Also
assert that the restored state round-trips bit-identical.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
* tests : drop the n_swa partial-state check from ps
The check keys off llama_model_n_swa(), which returns 0 for dsv4 even
though its raw KV cache has a 128-cell sliding window. The partial state
grows with the tokens there by design, so the flatness check is wrong.
Keep the partial <= full invariant.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
Every caller passes n_pad = 1 and the only two users clamp it to 256.
Drop the parameter, the members and the trivially true kv_size assert.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
The org-less group renders without a header, so when it followed a named
org its models looked like they belonged to that org. Put it ahead of the
named org groups.
Fixes#25227
Assisted-by: Claude (Opus 5.5 Medium)
On desktop, the reasoning effort toggle now gets a muted background on
hover and while its menu is open, matches the model selector's 24px
height, uses a 13px icon and sits 5px from the selector.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: add conversation compaction
Add a compaction action that asks the model to summarize the
conversation so far and stores the summary in a new compaction node
appended to the branch. Later requests slice the branch at the last
compaction node, keeping the system prefix and continuing from the
summary instead of the full history, which frees up context while the
complete history stays visible and navigable in the UI.
The slice is applied once at the request funnel, covering the plain
send, the agentic flow and the KV cache pre-encode, plus the continue
path that bypasses it. The summary request is a one-shot outside the
SSE replay machinery, streamed live into the node with a full rollback
on abort, error or empty result.
The action is exposed as a /compact slash command and a Compact button
in the context gauge popup, with an overridable summarization prompt
in the settings.
* ui: pre-fill KV cache and refresh context gauge after compaction
Once the summary is stored, pre-encode the compacted branch and stamp
the returned prompt timings on the compaction node: re-encoding the
reduced prefix is part of the compaction itself, so the server KV
cache and the context gauge reflect the summary right away.
The summary stream is registered under a prefixed identity
(compact::convId), so the visibility kick and byte-offset resume
protect it through tab switches while the reload probe never
reattaches it as a chat turn. The stamped node becomes the shared
context truth for the gauge and the processing state restore, and the
stale live state that kept the gauge full is cleared.
* chore: format
* ui: allow a dedicated model for conversation compaction
An opt-in setting reveals a model selector whose options come from the
live router model list. The chosen model writes the summary and is
loaded on demand when it is not resident, while the default path keeps
resolving to the conversation's own model.
* ui: align the compaction model field with project conventions
* ui: fix the model option helpers import path
The compaction model selector pulls filterModelOptions and
groupModelOptions from the models component folder, where they no
longer live, so the build fails to resolve the module. Point the
import at the navigation folder, which now owns them.
Missed during the rebase: the helpers moved in a commit that does
not touch any compaction file, so nothing conflicted.
* ui: import the model option helpers from models/utils
* ui: follow the moved models selector in the compaction model field
The models manager moved ModelsSelectorList under ModelsSelector and
the model option helpers to /utils, where groupModelOptions takes
the options alone. Drop the stale copy the rebase kept and read the
helpers from their new home.
* ui: drop the dedicated compaction model
A summary written by another model loses the prefix the conversation
model cached, so the whole history is prefilled again, and the model
then resumes from text it did not write. The conversation model
always writes its own summary.
* ui: keep the last steps of a compacted conversation
A compaction now folds the steps the model reads but the last ones,
a step being a user message or an assistant message with its tool
results, and stores how many it kept on the compaction node. The
model then reads the system prefix, the summary, the kept steps and
what follows.
The summary request is the request the chat sends, with its options,
tools and message shaping, cut before the preserved steps: the server
reuses the prefix it cached and only the instruction is new. The
number of preserved steps is a setting, 10 by default.
* ui: run every compaction summary through one path
Three callers needed the same thing - the automatic pre-encode pass, the
action in the context gauge, and re-running a summary from the block - and
each of them had grown its own copy of the request, the streaming and the
storing.
There is one path now: a summarization owns a node, streams into it, and
reports itself through a ledger the rest of the app reads (`isCompacting`,
which node belongs to which conversation). Re-running a summary is the same
call on an existing node, so the two entry points cannot drift apart again.
* ui: give the compaction row its own summary UI
The summary was a line of text in a tool-call-shaped collapsible, which is
how a compaction reads as a side note rather than as the thing the rest of
the conversation now stands on.
The row is its own block: a divider that names the boundary, and the summary
inside it with the model that wrote it, its statistics, and the same actions
an assistant turn has - copy, edit, fork, regenerate and delete all apply to
the summary, and it collapses to the divider once a summary is in, because
the shrunk context is the headline and the prose is the detail.
The divider is a variant of the shared collapsible block, which also needed
the toggle and the title split into snippets so a caller can lay them out
without copying the markup. The action icons no longer take a role: they
render what a message offers, and a compaction offers the same as any other
stored message.
* ui: report compaction progress on the row
The row owns the run, so the row says how it is going: the prompt progress
in the header while the context is being read, the phase once the summary
starts streaming, and the token count once it is in. Before the first
summary token there is nothing to collapse, so the header stands on its own
instead of offering an empty block.
* ui: keep the summarized turns quiet while a compaction runs
The turns being summarized used to keep their own "Processing..." indicator
and live reading stats while the summary streamed, so the conversation said
the same thing twice in two places. The row reports the run, so the turns it
replaces stay silent until it is done.
* ui: hand the compact action to the context gauge
Compacting is a decision about the context, so the action belongs with the
numbers that show the context filling up rather than inside the form: it sits
at the bottom of both the hover card and the drawer, under the breakdown it
acts on, quiet enough to stay a footnote and disabled - with the row's own
spinner - while a summary is already running.
* ui: follow the bottom, otherwise leave the view alone
The screen scrolled itself with a controller of observers, intervals,
direction heuristics and landing timers, and still managed to move the view
while the reader was reading: any mutation that grew the branch was treated
as a reason to keep following.
The page is the scroller, so it follows when the reader sits at the bottom,
and stays where it is otherwise. A resize observer plus native scroll
anchoring does the rest, and `pointerdown` buys the reader a moment of grace
to be caught mid-drag.
* ui: expand collapsible bodies with a height transition
Expanding a block snapped it open: the body was mounted the moment `open`
flipped, so a tall block moved the conversation around it in one frame.
The body now goes through the same region the models table uses. The height
interpolates and the visibility follows it, so a body is unmounted once the
collapse has finished rather than the moment it starts - a closed block
still costs nothing while tokens stream past it.
* ui: open a compaction summary at the top of the screen
Toggling a summary left the reader wherever they were, so a conversation that
had just compacted showed the divider at the bottom of the screen and read
down from there.
A toggle now scrolls the page once and lands the divider a breath under the
tabs, both ways: the summary is read from its beginning, and closing it returns
to the same boundary instead of wherever the page happens to come to rest. The
scroll waits for the body to reach its height, because before that the page is
too short to scroll the divider up there and the call would clamp.
That wait is the shared collapse duration rather than a transition event: an
engine without `interpolate-size` animates the region's grid row instead of its
height, so a listener on `height` would never fire there and the page would not
move at all.
* ui: spin the compaction icon while the summary runs
The divider reported the phase in its title while the icon still showed a
finished block, so a run in progress and a summary that had landed looked the
same in the one place the eye reads first. The icon spins while the summary is
running and folds back once it is in.
* ui: let the compaction summary speak without the reasoning trimmings
The summary is the thing the conversation now stands on, not a note beside
it, and the conversation model always writes it, so the body drops the
reasoning block's rule and indent (`flushBody` keeps them for the blocks
that borrow them) and the model selector under the summary is gone - there
is nothing left to swap.
* ui: drop the model override from the compaction path
The summary row no longer offers a model selector, so nothing hands
regenerateCompaction a model anymore. Remove the parameter from the
whole path, with the on-demand load of another model it carried: the
summary always comes from the model the request funnel resolves.
* ui: let the compaction summary think as the chat does
The summary request forced thinking off, overriding the reasoning
effort the user chose. The summary is the text the conversation
resumes from, the task where reasoning helps the most, so the request
now carries the very options of the chat.
* ui: pre-encode the compacted branch as the next turn reads it
The post-compaction pre-encode sent the branch without tools, while
the tools render ahead of the messages, so it warmed a prefix the next
turn never reads and that turn prefilled the whole compacted context
again. The pre-encode now sends the tools, message shaping and
reasoning handling of the chat, so the next turn only prefills its new
message.
* ui: pre-encode the branch the next turn actually reads
The per-turn pre-encode sent the messages of the request plus one
assistant message holding the streamed text, without tools. With
tools on, it replaced the slot the next turn extends with a prefix
nobody reads again, and after an agentic turn its history did not
match the stored calls and results. Without agentic, it never ran.
Both pre-encodes now read the stored branch up to its leaf, cut at the
last compaction, and send it with the tools, message shaping and
reasoning handling of the chat. The plain completion path triggers it
too.
* ui: refuse a compaction summary the output limit cut
The stream handler read every chunk but dropped the finish reason, so
a summary cut by max_tokens folded the conversation into half a text.
The handlers now hand the finish reason to onComplete, and a summary
that ends on length rolls the compaction back with an error.
The summary was framed through a replacement string, so $& or 32560 in
it changed the text the model read; it is framed as written now.
* ui: wait for the collapse transition in the tool call tests
The collapsible body now unmounts once its collapse animation ends rather
than as soon as the block closes, so the "folds back" assertions ran before
the body was gone. Wait out COLLAPSIBLE_TRANSITION_MS first.
Assisted-by: Claude (Opus 5.5 Medium)
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
* ui: open model details in a global drawer without leaving the page
The model info actions in the selector used to navigate to the models page
to open its details pane, and a phone opened a separate dialog instead. The
drawer now lives outside the models table, driven by uiStore, so the info
icon opens it over whatever page is showing.
The models dropdown and the phone picker stay open under the drawer's
backdrop, and keep focus while the drawer opens and closes, so the trigger's
tooltip no longer pops over the drawer.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: let the model details drawer reopen after a drag to dismiss
Dragging the drawer down wrote vaul's own open state, which the one-way open prop then kept closed, so the info action could not open it again until a refresh. Bind open both ways through the store instead.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: close the models selector from the details drawer chat actions
Assisted-by: Claude (Opus 5.5 Medium)
Extract the keyboard-shortcut delete confirmation into DialogConversationDelete
and reuse it in the sidebar and mobile search page instead of the native
window.confirm prompt.
Assisted-by: Claude (Opus 5.5 Medium)
* hexagon: add GGML_HEXAGON_VTCM_MB and GGML_HEXAGON_NTHREADS caps
Change-Id: Idffe1613c8db6f96aa842d43d1b2efaed45462a8
* hexagon: rename VTCM cap to GGML_HEXAGON_VTCM_SIZE, drop NTHREADS cap
Change-Id: If4c5b9ce9fd6b725c3561421c2295d40599d012d
* hexagon: make GGML_HEXAGON_VTCM_SIZE an override, accept K/M/G suffixes
Change-Id: Ib8eea84c22f24e6d986b37f215e504b046478c2c
The hyper-connection and DSA graph builders wrote their extra outputs
(post / comb gates, carried top-k selections, inject gates) through
ggml_tensor** out-params. Return them in a small struct next to the
primary output instead.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
Every dst row comes from src0 or src1, or for dim 0 from the row of
src0 followed by the row of src1, so a single kernel covers every
type: a dense source row is one memcpy, quantized types included,
and a strided source is copied element by element with the element
size known at compile time.
Rows are split across threads instead of i2 alone, which is often 1,
and are copied in tiles of 32: for a transposed source the elements
are walked in the outer loop, so the rows of a tile read neighboring
addresses instead of jumping a full row stride per element.
Add transposed b cases to test_concat, the layout of the delta-net
conv input, and a perf case for it.
Pascal