mirror of
https://github.com/huggingface/transformers.git
synced 2026-10-29 01:06:24 +08:00
diffusion-sampler-protocol
35
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Sai-Suraj-27
|
e28fc5dabb |
Show the actual accelerate version for transformers env (#49668)
* Log the accelerate version for transformers env * Few more changes to transformers env * Small tweaks * Use get_torch_version() * Add a test that checks both version and range warning * Add more tests covering possible scenarios * parametrize the tests |
||
|
Wang, Yi
|
8fa11ce8a6 |
is_tracing() now recognizes XPU stream capture, alongside the exist… (#49722)
* `is_tracing()` now recognizes XPU stream capture, alongside the existing CUDA capture check. This ensures code executed during XPU graph capture follows tracing-aware paths. The PR also replaces hard-coded CUDA devices in four continuous-batching serve tests with the selected test accelerator, so those tests can run on XPU. A unit test covers `is_tracing()` when XPU stream capture is active. No new dependencies are required. Signed-off-by: Wang, Yi A <yi.a.wang@intel.com> * make style Signed-off-by: Wang, Yi A <yi.a.wang@intel.com> --------- Signed-off-by: Wang, Yi A <yi.a.wang@intel.com> |
||
|
Arthur
|
1bd535f3c3 | serve: don't crash /v1/models on configs with null architectures (#49509) | ||
|
Marc Sun
|
958dbf24b6 |
v1/systemone endpoint (#49322)
* poc * systemone * update test * docs * test |
||
|
Juhyuk choi
|
e159fcef58 |
Fix backslash handling in generate flag values in transformers chat (#48709)
* Escape generate flag values with json.dumps * Add tests for backslash and list handling in flag values |
||
|
Tarek Ziade
|
f441076206 | QA: applied ruff rule PLW1514 (#48990) | ||
|
Lucain
|
80a79d1bef |
Use huggingface_hub httpx export (#48685)
Use huggingface_hub httpx export |
||
|
Tarek Ziade
|
8fcb52399c |
QA: Added a MemoryCleanupMixin class for tests (#48681)
* initial draft * Address serge review: audit the no-grad default and pin the private hook * You are such a verbose agent. * Review round from @ydshieh, @vasqu and @stevhliu * vasqu round 2: docs cross-reference, a note block, a class name, and test_kernels * @ydshieh round 2: rename the grad knob, name the numbers, explain the rest |
||
|
Tarek ZiadeandClaude Opus 5
|
c694707483 |
Synthetic test assets (#48589)
* Use synthetic CC0 assets instead of third-party licensed media in tests * work file * Fix `_format_py_obj` crashing on values it has no branch for `_format_py_obj` assigns `output` inside a `str`/`int`/`float`/`list`/`tuple`/`dict` chain with no trailing `else`, so any other type raises `UnboundLocalError: local variable 'output' referenced before assignment`. A `torch.Size` hits this: it is a `tuple` subclass, but the branch tests `__class__.__name__`, which is `"Size"`. Any test whose captured assertion argument is a shape then loses its report — the CI value capture dies inside the reporting helper instead of printing the actual value, which is exactly what the failing test needs. Fall back to `repr`, which for `torch.Size` gives the copy-pastable `torch.Size([1, 736, 736])`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Refresh integration expectations recorded against the old third-party assets Values taken from the A10 run-slow run on this branch (https://github.com/huggingface/transformers/actions/runs/34103194667), read out of each job's `captured_info.txt`. Single-GPU and multi-GPU agreed everywhere they overlapped. Refreshed: aya_vision, chameleon, chinese_clip, cohere2_vision, cosmos3_omni, diffusion_gemma (the two minified `generate_with_image_batched*` tests), fast_vlm, higgs_audio_v2, idefics2, instructblip. For instructblip both assertions are updated: the run stopped at the token-id comparison, so the decoded-text expectations come from running the new ids back through each checkpoint's tokenizer. Where every key of an `Expectations` dict was recorded against the old asset, the dict collapses to a single `(None, None)`: only cuda 8.x could be re-measured, and a stale `("xpu", 3)` / `("cuda", 7)` value is worse than falling back to the one value we verified. chmv2 keeps its TODO — the new asset changes the output resolution (448 -> 736), and the depth slice cannot be re-recorded until a run gets past the shape assertion. Left untouched, per the run's own `git bisect` results: edgetam (timm `FeatureListNet.forward()` signature), the four diffusion_gemma tests that fail identically on the base commit, and cohere2_vision's OOM / unchanged-fixture drift / local-path-checkpoint failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix two more `captured_info` corruptions: backslashes and integer-tensor dtype Both silently produced a report that cannot be read back, which is the whole point of the capture: * `_quote_string` escapes quotes but not backslashes, so a value containing one emits a literal that means something else -- and a value *ending* in one swallows the closing quote, leaving an unterminated literal. mllama's decoded output ends in `\`, so its captured value was unusable. * `_format_tensor` strips `tensor(` and `)` but a non-default dtype is repr'd as a trailing kwarg, e.g. `tensor([83, 362], dtype=torch.int16)`. Stripping only the wrapper leaves `dtype=torch.int16` stranded *inside* the literal. Every integer tensor hit this (the pp_ocrv5/v6 detection boxes). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Refresh the remaining expectations from the full-coverage GPU run Round 1 only exercised 16 models -- `get_pr_run_slow_jobs.py` caps its suggestion at `MAX_NUM_JOBS_TO_SUGGEST = 16` and sorts alphabetically, so everything from `internvl` on was never run. This is the second pass, over all 58 model dirs plus `pipelines` and `cli` (https://github.com/huggingface/transformers/actions/runs/34150746428). Of the 173 failures there, the run's own `git bisect` attributes 87 to this branch; 41 fail identically on the base commit (edgetam's timm signature mismatch, four diffusion_gemma tests, cohere2_vision's local-path checkpoints) and 15 are flaky. 83 of the 87 had a captured value; all of those are refreshed here. Two notes on how the values were placed, since both went wrong on the first attempt: * Targets are resolved by enclosing test function *and* the assertion's line. A file usually holds several `EXPECTED_*` blocks and a test that checks several outputs rebinds the same name once per output, so "first match in the file" and "last assignment in the function" both write into the wrong test. This also fixes a bad round-1 edit that landed the multi-speaker value in higgs_audio_v2's `test_single_speaker_smart_voice`. * Which captured side is the expectation is decided structurally, not from the argument name: `assertEqual(EXPECTED, computed)` puts the stale expectation in the slot named `first`. Three tests are written that way. Where every key of an `Expectations` dict was recorded against the old asset the dict collapses to a single `(None, None)`; only cuda 8.x could be re-measured, and a stale xpu/rocm value is worse than the one value we verified. Still marked TODO, awaiting the next run: cosmos3_omni's batched test (OOM), diffusion_gemma's image-token count and qwen3_asr's timestamps (assertions the capture does not instrument), qwen2_5_vl's second batch element and pp_ocrv6_medium_det's boxes (never reached), and instructblip plus prompt_depth_anything, whose assets were just regenerated: * `lavis_confusing_pictures.jpg` was flat geometric shapes standing in for the LAVIS "man ironing on a taxi" photo, and instructblip-flan-t5 answered "What is unusual about this image?" by repeating one sentence until `max_length`. * `promptda_arkit_depth.png` was saved as RGB, but a prompt-depth map is a single-channel metric image -- PromptDepthAnything failed outright with "expected input[1, 3, 27, 36] to have 1 channels". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * qwen3_asr: use real Mandarin speech instead of a synthetic sine tone `audio/mandarin_asr.wav` in the synthetic-assets repo is a 520 Hz sine tone, so Qwen3-ASR transcribed it as nothing: the expected tokens for the second batch element collapsed to `[11528, 2240, 151704, 151645, ...]` -- language tag, then immediate EOS. The test stayed deterministic but stopped exercising transcription at all. Point both references at `mandarin_voxcpm_zh.wav`, mirrored into `hf-internal-testing/dummy-audio-samples` from `adarshxs/voxcpm2-native-generated-audio-user-ref` (`ref_zh.wav`): 4s of apache-2.0 Mandarin TTS, transcript "你好世界，人工智能正在改变我们的生活方式。" per the source's `metadata.json`. Same source repo and licence as the already-GREEN `zs_short.wav`/`zs_medium.wav` there, and recorded in that repo's `provenance.jsonl` with the same fields. Both fixture tests need re-recording against the new clip, so the batched one is marked again; `test_fixture_timestamps_batched` already was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Repoint the qwen2_audio/qwen3_asr docs at the real Mandarin clip too The previous commit only fixed the test and the CI prefetch list, but five doc snippets still pointed at the sine tone. These are copy-paste examples, so a reader running them would get an empty transcription and reasonably conclude the model was broken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Collapse two pixel-identical duplicate assets The migration replaced third-party URLs one-for-one, so several assets are the same picture under a name that records which test it came from. Two pairs are *exactly* identical, so pointing the tests at a single copy cannot change a tensor: images/gemma_golden_gate.png == images/dreamstime_golden_gate_flowers.jpg images/invoice_docquery_b.png == images/invoice_docquery_a.png (byte-identical) Both duplicates are deleted from the assets repo and their entries in the manifest now resolve to the survivor, so the old-URL mapping stays complete. Their prefetch entries go too, since the survivors are already listed. Four further pairs are the same content re-encoded rather than identical (mistral3_view / mistral3_statue_of_liberty / mistral3_golden_gate against their originals, and paddle_ocr_demo2 against paddle_general_ocr_001). Collapsing those would take the count from 40 to 36, but they differ by up to 48/255 per channel, so each one risks another expectation re-record and is better done as one deliberate batch than mixed into this refresh. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Collapse the four re-encoded duplicate assets These are the same picture as an existing asset, re-encoded rather than byte-identical, so unlike the previous pair they do shift the input tensor slightly: mistral3_view.jpg -> llava_view.jpg (max 8/255, 3.1% of pixels) mistral3_statue_of_liberty.jpg -> statue_of_liberty.jpg (max 9/255, 2.5%) mistral3_golden_gate.jpg -> dreamstime_golden_gate_flowers.jpg (max 27/255, 9.8%) paddle_ocr_demo2.jpg -> paddle_general_ocr_001.png (max 48/255, 100%) The six tests that consume them are marked for re-recording: two in mistral3, and all four in `PaddleOCRVLIntegrationTest`, whose URL lives in `setUp`. `paddle_ocr_demo2.jpg` also appeared in a paddleocr_vl docstring, so the change went into `modular_paddleocr_vl.py` and `modeling_paddleocr_vl.py` was regenerated from it. With this the assets repo is 36 files, every one referenced and every reference resolving. Two duplicate groups are deliberately kept: `azure_slide_1..4` are 0.998-1.000 correlated but differ in ~2.5% of pixels, and phi4_multimodal's multi-image test wants four distinct slides -- those want regenerating with more variety, not deduplicating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Refresh expectations from the round 3 GPU run 27 of the 33 round-3 failures that `git bisect` attributed to this branch, across 19 files. Values come from each job's captured_info.txt, not from re-deriving them. Stale `Expectations` dicts collapse to a single `(None, None)` key: every hardware key was recorded against an asset this branch replaces, and only cuda 8.x can be re-measured on this CI, so a stale ("xpu", 3) is worse than the one value we verified. Three pixel_values expectations (qwen2_vl, qwen2_5_omni, qwen3_omni_moe) failed on a bare `assert torch.allclose(...)`, which the capture helper does not instrument. That assertion is CPU-only image-processor preprocessing, so those values are computed off-CI instead; each test's own expected_input_ids reproduces exactly, qwen2_5_omni's slice agrees with qwen2_vl's independently computed one, and each recomputes within the test's own atol=3e-3. Two pre-existing inconsistencies fixed on the way through: - instructblip::test_inference_vicuna_7b had a refreshed token-id expectation next to a stale text expectation, so the two contradicted each other. The ids decode to the new text exactly. - pp_formulanet's expectation held a literal backspace byte, from `\backslash` written in a non-raw string. qwen2_5_vl::test_small_model_integration_test_batch_different_resolutions needed element [1], not element [0] as its TODO claimed: the loop reaches i=1, so [0] already passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Make the last two blocked expectations reportable, and drop a stray file Three of the open TODOs were not waiting on an asset or a measurement -- they were written as assertion forms the capture helper cannot instrument, so every round reports "AssertionError: False is not true" and no value, and the next round would have reported exactly the same thing: - prompt_depth_anything's two tests used `assertTrue(torch.allclose(a, b, atol=1e-3))`. Now `torch.testing.assert_close(a, b, atol=1e-3, rtol=1e-5)` -- rtol spelled out because that is torch.allclose's default, so the tolerance is unchanged. - diffusion_gemma's image-token check used `assertTrue(a == b)`, which hides both operands. Now `assertEqual`, which also reports them. Round 3 incidentally disproves what these TODOs claimed: prompt_depth_anything asserts `predicted_depth.shape == (1, 756, 1008)` *before* the value check, and that passed, so the single-channel depth map was already in place. The stale asset was never the problem there. Also removes src/transformers/utils/hub.py.save, a 40K editor backup committed by accident in |
||
|
|
0a959de1d2 |
Fix GPU memory teardown in CLI serve tests (#48618)
* [fix] Free GPU memory between CLI test classes to prevent meta-device OOM Each tearDownClass now calls cleanup() + sets cls.serve = None to release GPU memory before the next test class loads a model. Also enables expandable_segments to reduce fragmentation from sequential large model loads. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [fix] Set expandable_segments via os.environ before CUDA init in CLI tests set_allocator_settings() is a no-op once the CUDA allocator is first used, which happens at import time. Setting PYTORCH_CUDA_ALLOC_CONF via os.environ at module load time (conftest.py top-level) ensures it takes effect before any CUDA allocation occurs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [fix] Release InferenceThread closure references before blocking on queue.get() While InferenceThread blocks on queue.get() waiting for the next task, the local variable `fn` from the previous iteration still holds the submitted closure. For inference calls this closure captures `model` via `model.generate(**gen_kwargs)`, keeping the model's GPU tensors alive even after delete_model() clears all other references. Explicitly set fn/args/kwargs = None in a finally block so the closure (and the model it captures) is released before the thread blocks again. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [fix] Clean up: remove unneeded expandable_segments env var, add cleanup comment The PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True was added to work around fragmentation, but the InferenceThread fix fully resolves the root cause so it is not needed. Also adds a short comment on cleanup() calls in tearDownClass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Marc Sun
|
0514b65827 |
Fix generate_flags parsing in transformers chat (#48597)
* Fix `generate_flags` parsing in `transformers chat` Three bugs in `parse_generate_flags`, reported in #48588: - a flag without `=` raised `IndexError` from `split("=")[1]`. The interactive `!set` path already validates this; the CLI argument path did not. It now fails with the same message, as a clean CLI error - a value containing `=` was silently truncated at the first one (`bad_key=a=b` -> `a`); `split("=", 1)` keeps it. The `=` -> `:` string replacement below it had to go: unreachable while values were truncated, it turned the recovered value into `a:b` - the `ValueError` for invalid JSON named `generate_flags` and the converted string but had no `f` prefix, so it printed the placeholders Also fixes the argument order in the `generate_flags` help example, which showed `<base_url> <model_id>` while the signature is `MODEL_ID [BASE_URL]`. * Report all flags missing `=` at once |
||
|
Tarek Ziade
|
892044aa2c |
CI: Point test fixtures at hf-internal-testing copies we already host (#48521)
* Point test fixtures at hf-internal-testing copies we already host For 13 of those assets the identical bytes are already stored in hf-internal-testing, so no copy is needed - only the URL changes: - 12 COCO val2017 images -> hf-internal-testing/fixtures-coco - bcn_weather.mp3 -> hf-internal-testing/dummy-audio-samples * Mirror the last two COCO test images into hf-internal-testing * more copies * More mirroring * Mirror audiobox.mp4 out of eustlb/dummy-video-dataset * Load test images through load_test_image instead of fetching them directly * Route chat-template urls and multi-line image fetches through the cache helpers * Drop a dead commented-out image load in the llava test * remove useless if * make style * Wrap the remaining pre-cached urls, keyed on the pre-fetch list |
||
|
Rémi Ouazan
|
b6c0bfe04c |
[CB] Fail faster (#48334)
* Regroup fatal error under the bg thread status * Fail at add request * Fail fast in get outputs * Nit * Actually fail the request before processor is created * No more loud failure * Fixed some issues with is_finished |
||
|
Marc Sun
|
12f2d15519 |
Update serve chat parsing (#46267)
* new chat parsing for serve * fix * update template |
||
|
Dhruv Sharma
|
ffae066da3 |
Route chat management calls to the service root (#47138) (#47303)
* Route chat management calls to the service root (#47138) transformers chat reuses base_url for the serve management endpoints, so with the documented base_url ending in /v1 the health check requests /v1/health and model loading posts to /v1/load_model. The serve app mounts /health, /load_model and /reset at the service root; only the OpenAI-compatible inference API lives under /v1. Derive the management base url by stripping a trailing /v1 segment and use it for check_health and print_model_load, while the AsyncInferenceClient keeps the full base_url. A base_url without /v1 is unchanged, and a /v1 under a reverse-proxy prefix keeps the prefix. Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> * Address review: rename helper to get_service_root_url, drop mocked tests Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> * Build the service root url with urlunparse instead of ParseResult._replace Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> --------- Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> |
||
|
Dhruv SharmaandTarek Ziade
|
326ae42d30 |
Fix CLI compatibility with huggingface_hub 1.22 (#47059) (#47064)
* Fix CLI compatibility with huggingface_hub 1.22 (#47059) Since huggingface_hub 1.22.0, typer_factory returns a click command group (HFCliTyperGroup) instead of a typer.Typer. tests/cli drove the app with typer.testing.CliRunner, whose invoke() calls typer get_command() and reads ._add_completion from the group, raising AttributeError in every CLI test. Drive the app with click.testing.CliRunner instead, which invokes the click group natively. Separately, the serve --reasoning option declared its default as the enum member ReasoningMode.AUTO; click validates defaults against the string choices and rejects it, which breaks transformers serve at runtime under hub 1.22. Use the enum value as the default instead (runtime comparisons are unaffected since ReasoningMode is a str enum). Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> * Use hub-canonical optional enum pattern for serve --reasoning The previous string default failed the ty check (invalid-parameter-default: str not assignable to ReasoningMode). huggingface_hub own CLI never declares enum-member defaults - every enum option there is declared as SomeEnum | None = None, since the hub click converter does not coerce enum defaults. Mirror that pattern: reasoning defaults to None, which is behaviorally identical to AUTO (neither the ON nor OFF branch fires), and the help text states the effective default. Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> * Clean up comments in conftest.py Removed outdated comments regarding `typer_factory` behavior based on reviewer's suggestions. * Keep 'auto' as the serve --reasoning default Apply the review suggestion: the default stays ReasoningMode.AUTO.value so the CLI help keeps showing [default: auto]. The type-ignore is required because ty rejects a str default on a ReasoningMode-annotated parameter, and the hub CLI framework rejects any union annotation except T | None. Also sorts the conftest import block (ruff I001 from the previous push). Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> * Make the cli test fixture compatible with huggingface_hub < 1.22 The fixture drives whatever `typer_factory` returned: with hub >= 1.22 the app is already a click command group and is invoked natively; with older hubs it is a `typer.Typer`, converted via `typer.main.get_command` before invoking. No change to the supported huggingface_hub range. Verified against both huggingface_hub 1.21.0 and 1.22.0 (74 passed each). Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> --------- Signed-off-by: Dhruv Sharma <dhruv7477@gmail.com> Co-authored-by: Tarek Ziade <tarek@ziade.org> |
||
|
|
4ab0479eee |
Streamable chat parsing (#45847)
* scaffolding * Let's do an initial commit * Let's do an initial commit * More cleanup * More cleanup * Add the prefill * One more stash commit * One more stash commit * Renames/docstring cleanup * More renames/docstring cleanup * More renames/docstring cleanup * Docstring cleanup * make fix-repo * Doc cleanup * Doc cleanup * make fix-repo * No more coerce, merge content types * Cleanup: Support tokens/arrays/lists correctly for input and prefix * Add prefix to docs too * Better initial_events handling, better region handling * Allow multiple literal matches * Allow emitting dirty regions * Doc updates * Drop `assemble` to just use `transform` * Lots more doc updates * Lots more doc updates * Remove batched handling, lots of cleanup * Use the new method in test_serve * Small doc update * Update docs/source/en/chat_response_parsing.md Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update docs/source/en/chat_response_parsing.md Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update docs/source/en/chat_response_parsing.md Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Update src/transformers/models/gemma4/convert_gemma4_weights.py Co-authored-by: Pedro Cuenca <pedro@huggingface.co> * Mention the extra * Fix the example stream so it doesn't refer to the previous example * Remove the `strip` arg for numerical/bool types because it was meaningless * Fix convert script bracket bug * Add a regex compatibility section * Update regex compatibility section * jmespath: defeated * jmespath: defeated * Flesh out the Cohere example * Update the docs with proper content_args * Make start_anchor mandatory * Update gemma4 template! * Proper buffering * Corrected prefix handling * Type fixes * make fix-repo * Pipeline updates, batching changes * Post-rebase fixes * Tiny reorder on the version check * Fix licences + no variable abbreviations * Cleanups, remove assertion * Refactor load_response_template * Lots of little doc examples * Fix docs to explain prefix a bit better * Add batching * Add batching * Docs cleanup * Docs cleanup, big block of vasqu comments * Apply suggestions from code review Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * More comment fixes --------- Co-authored-by: Pedro Cuenca <pedro@huggingface.co> Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> |
||
|
Yih-Darandydshieh
|
8e24535c2a |
Fix TestAppRoutes test failures caused by deprecated asyncio.get_event_loop() on Python 3.10+ (#46340)
* Fix unhandled RuntimeError when AsyncTextIteratorStreamer event loop is closed
Guard `on_finalized_text` against `RuntimeError: Event loop is closed` that
occurs when the background `generate()` thread outlives the consumer's event
loop (e.g. early timeout exit or test teardown in pytest-xdist).
* Fix TestAppRoutes failures on Python 3.10+ by replacing deprecated asyncio.get_event_loop()
Replace `asyncio.get_event_loop().run_until_complete()` with `asyncio.run()`
in all four TestAppRoutes test methods. In pytest-xdist worker threads no
event loop is set up, causing `get_event_loop()` to raise RuntimeError.
`asyncio.run()` always creates its own loop and is the recommended approach
since Python 3.10.
* Revert "Fix unhandled RuntimeError when AsyncTextIteratorStreamer event loop is closed"
This reverts commit
|
||
|
Yih-Darandydshieh
|
39603d0e5c |
Make more tests work with read-only cache (#46299)
fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> |
||
|
|
0b25f8c49c |
[serve] Support for reasoning (#45690)
* reasoning * Apply repo consistency fixes * fix gemma reasoning * Apply repo consistency fixes * Update src/transformers/cli/serving/utils.py Co-authored-by: Lysandre Debut <hi@lysand.re> * unbloat response * better reasoning * Apply repo consistency fixes * fix qwen3.5 * style * style * Apply repo consistency fixes * fix --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Lysandre Debut <hi@lysand.re> |
||
|
Marc Sun
|
c641d13ea7 |
[serve] cb error (#45691)
* cb error * simpler tests * fix |
||
|
|
0ea540efff |
Add /v1/completions endpoint (OpenAI legacy completions API) to transformers serve (#44558)
Add /v1/completions endpoint (OpenAI legacy Completions API) to `transformers serve` Adds support for the legacy text completions endpoint, which accepts a freeform text prompt (no chat template) and returns generated text in choices[].text. Supports both streaming and non-streaming modes, suffix for fill-in-the-middle insertion, and proper finish_reason detection. The implementation follows the refactored serving architecture: a new CompletionHandler in serving/completion.py extends BaseHandler, and the route is registered in serving/server.py. Co-authored-by: Antigravity <bot@antigravity.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Quentin Gallouédec
|
243f2d7717 |
Raise 400 on model mismatch when transformers serve is pinned (#45443)
* Raise 400 on model mismatch when `transformers serve` is pinned * test * style |
||
|
Marc SunandLysandre Debut
|
e15297ebc8 |
[serve] Update tool call to switch to parse_response (#45485)
* fix tool call to support parse response * simplify parser * simplification ! * Update src/transformers/cli/serving/response.py Co-authored-by: Lysandre Debut <hi@lysand.re> --------- Co-authored-by: Lysandre Debut <hi@lysand.re> |
||
|
Marc Sun
|
9b1a47c6a5 |
Fix response api support (#45463)
* fix * update tests * fix * better doc |
||
|
|
8426e7e63d |
[serve] Forward tool_calls/tool_call_id in processor inputs (#45418)
* Forward tool-use fields in messages for multi-turn conversations * Apply repo consistency fixes * Update src/transformers/cli/serving/utils.py --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Marc Sun
|
fd45a42772 |
Multimodal serve support (#45220)
* add audio and video * multimodal in chat and response api * fix ! * remove file * style * let's have this here for now * qwen omni works also * more coverage for test * style * add comment * update chat completion docs + redirect * fix * get * fix response * much better doc * update * typing |
||
|
Marc Sun
|
38593c2e83 |
[refactor] Serving into proper modules (#44796)
* new serve file * app * model_manager done * update serve * style * poc done * renaming * fix * new tests * update metrics and processor * hardcode n_batch for now * add response api + compile * more tests * add it for now but we will move it * remove cache impl * add back load_model * fix naming * add transcription * tool calls better ! * vlm support for both response and chat endpoints * update bench * fix vl test * first iteration of cb * cb tests * typing + review * update test * better benchmark * better stream * update bench * fix * serve refactored * merge * update * fix * style * simpler * style * update warmup * remove llamacpp integration for now * styke * styke * style again * remove annoattion * review ! * style * much cleaner * renamed * remove bench for now * batch output * style * type * better tests * update test * queue draining * some logs * readd nathan feature + some minor fixes * fix * guard transcription * better now * fix * adding lock to see if this helps * remove locks * lock again * update bench and remove lock for now |
||
|
Nathan Habibandremi-or
|
a91232af09 |
[serving] Fix continuous batching JSON response serialization (#45057)
* Fix continuous batching JSON response serialization Change model_dump_json() to model_dump() to avoid double JSON encoding. When using continuous batching with stream=false, the response was being double-encoded as a string instead of returning a proper JSON object. * add example script eval-job * fix script * Add test for continuous batching non-streaming JSON response Test verifies that non-streaming responses with continuous batching return proper JSON objects rather than double-encoded JSON strings. This is a regression test for the fix where model_dump_json() was changed to model_dump() in the continuous batching response handler. * fix ci * Update eval script to use official transformers repo main branch Changed dependency from personal fork to official huggingface/transformers@main for production use of the evaluation script. * add kernels and flash attn 2 * Add continuous batching configuration CLI arguments to serve command - Add --cb-block-size, --cb-num-blocks, --cb-max-batch-tokens, --cb-max-memory-percent, and --cb-use-cuda-graph flags - Flags allow users to customize KV cache and performance settings for continuous batching - Update transformers_serve_cb_eval_job.py to support and pass through CB config arguments - Update transformers dependency to use NathanHB/transformers@fix-continuous-batching-json-response branch - All arguments use auto-inference defaults when not specified (backward compatible) * Add thread lock for manager creation to avoid double manager * change transformers dep --------- Co-authored-by: remi-or <remi.pierre_o@orange.fr> |
||
|
Lysandre DebutandMarc Sun
|
e94695e574 |
Propagate the model loading from transformers serve to chat (#44758)
* Propagate the model loading from transformers serve to chat * Docs and tests * Apply suggestions from code review Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> * logigng update * Adjust docs re Marc's comment * Remove model name if too long for current console size * Refactor dual model loading w/ locks --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com> |
||
|
Tarek Ziade
|
0c89522f2a |
fix(flaky): enforce manual seed to reduce flakiness (#43794)
This change aims to reduce flakiness in CI tests. We identified two causes of nondeterministic behavior: - Some tests were not using a fixed RNG seed, which reduced determinism. - The cli tests were occasionally triggering I/O errors due to writes on a closed stdout. This branch was run multiple times and appears to reduce flakiness in all previously unseeded tests. While there’s no deterministic way to prove the improvement, using fixed seeds is still a best practice. |
||
|
Lysandre Debut
|
52b988d83d |
Transformers serve fix (#42570)
* Fix: lacking EOS token + failing AutoProcessor * Tests * Tests |
||
|
Lysandre DebutandLucain
|
52c5c6582b |
Correctly return finish reason length when finished (#42157)
* Correctly return finish reason length when finished * Typos + fixup * Fix a few tests * Update src/transformers/cli/chat.py Co-authored-by: Lucain <lucainp@gmail.com> --------- Co-authored-by: Lucain <lucainp@gmail.com> |
||
|
Lysandre Debut
|
2186f316af |
Transformers serve -> list all generative models from the cache (#42146)
* Scan cache for generative models Co-authored-by: Wauplin <lucainp@gmail.com> * Address test |
||
|
LucainandLysandre
|
af2a66ced9 |
Migrate transformers cli to Typer (#41487)
* Add typer-slim as explicit dependency * Migrate CLI to Typer * code quality * bump release candidate * adapt test_cli.py * Remove ./commands + adapt tests * fix quality * consistency * doctested * do not serve model in chat * style * will it fix them? * fix test * capitalize classes * Rebase * Rebase * tests + fixup tests + fixup * csutom error message * fix ? * should be good * fix caplog globally * inner caplog * last attempt * Retry * Let's try with capsys disabled --------- Co-authored-by: Lysandre <hi@lysand.re> |