0 Fork 0

mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-10-28 17:56:13 +08:00

Files

T

Georgi Gerganov 6b9ba4f2e8 ci : fix data race between runners sharing the model cache (#31046 )

* ci : fix data race between runners sharing the model cache

Two runners per mac mini share HOME, so concurrent jobs corrupt each
other downloads, converted models and venv under the shared MNT cache.

- gg_wget: download in a temp dir, publish with mv, so readers only
  ever see complete files (-N cannot be combined with -O)
- convert, quantize and wikitext unzip now produce under a per-runner
  scratch dir in OUT, published to MNT via hardlink when missing
- gguf-split and quantize tests get a per-runner TMP_DIR
- python venv moved from MNT to OUT
- metal, vulkan and webgpu mac jobs pass a per-runner OUT path

Assisted-by: dsh:llama.cpp/agent-qwen4 (RTX 5090 machine)

* ci : clean up scratch models after tests

Assisted-by: dsh:llama.cpp/agent-qwen4 (RTX 5090 machine)

* ci : fix q4_0 publish when a cached model exists

cp -n skips an existing file with exit code 1, so the ln fallback
aborted the qwen3 job under set -e. Publish through a unique temp
file with an atomic mv: readers never see a partial file and a
stale cached q4_0 gets replaced.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 machine)

* ci : cache-first gg_wget to avoid re-downloads

HF sends no Last-Modified, so wget -N re-downloads the full files on
every run. Skip the download when a non-empty cache file exists: the
HF CI artifacts are static. Downloads still go through a temp dir and
an atomic mv, so concurrent runners never read partial files.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 machine)

2026-10-24 09:21:24 +03:00

README-MUSA.md

musa: build the docker image and CI container from the MUSA SDK images (#29624 )

2026-09-29 10:42:20 +02:00

README.md

ci : migrate ggml ci to self-hosted runners (#16116 )

2025-09-21 16:50:45 +03:00

run.sh

ci : fix data race between runners sharing the model cache (#31046 )

2026-10-24 09:21:24 +03:00

README.md

CI

This CI implements heavy-duty workflows that run on self-hosted runners. Typically the purpose of these workflows is to cover hardware configurations that are not available from Github-hosted runners and/or require more computational resource than normally available.

It is a good practice, before publishing changes to execute the full CI locally on your machine. For example:

mkdir tmp

# CPU-only build
bash ./ci/run.sh ./tmp/results ./tmp/mnt

# with CUDA support
GG_BUILD_CUDA=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt

# with SYCL support
source /opt/intel/oneapi/setvars.sh
GG_BUILD_SYCL=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt

# with MUSA support
GG_BUILD_MUSA=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt

# etc.

Adding self-hosted runners

Add a self-hosted ggml-ci workflow to .github/workflows/build.yml with an appropriate label
Request a runner token from ggml-org (for example, via a comment in the PR or email)
Set-up a machine using the received token (docs)
Optionally update ci/run.sh to build and run on the target platform by gating the implementation with a GG_BUILD_... env