mirrors/pytorch
0
0 Fork 0
mirror of https://github.com/pytorch/pytorch.git synced 2026-10-25 18:53:39 +08:00
Code Activity
113,028 Commits 11,666 Branches 13,439 Tags
main
Commit Graph
281 Commits
Author SHA1 Message Date
TarasandJeff Daily 51177cecf2 [Windows] Re-enable FBGEMM on x64 and fix MSVC AVX flag detection (#198490) 
Fixes:
- #169816.

## Root cause

FBGEMM has been silently disabled on every Windows x64 build since #158210.
`CMAKE_SYSTEM_PROCESSOR` is `AMD64` on Windows, but the gate introduced there only allows `x86_64|aarch64`, so `set(USE_FBGEMM OFF)` fires unconditionally it also overrides `USE_FBGEMM=1` from the environment. Windows is left with `onednn` as the only quantized engine, so torchvision's pretrained quantized models (which pin `backend="fbgemm"` via `weights.meta`) fail with `RuntimeError: Quantized backend not supported`.

## Changes

**`CMakeLists.txt`**: allow `AMD64`, and restore the 32-bit exclusion #158210 dropped. The pointer-size check is required because 32-bit Windows also reports `AMD64`. It additionally excludes 32-bit `-m32`/x32/ILP32 builds. No 64-bit platform changes.

**`cmake/Modules/FindAVX.cmake`**: MSVC exposes AVX intrinsics without any `/arch:` flag, so the no-op `" "` candidate won and cached `CXX_AVX*_FLAGS=' '`. fbgemm reuses those cache variables, so its AVX targets built without `/arch:`, `__AVX2__` went undefined, and the kernels guarded by it failed to compile (`error C3861: 'transpose_kernel_mxn_sse'`). Probe the real flag first on MSVC. The non-MSVC branch is unchanged.

## Behavior change

Default qengine on Windows moves from `onednn` to `x86`, matching Linux.

## Test Plan

```
cmake -U "C_AVX*" -U "CXX_AVX*" -U "C_HAS_AVX*" -U "CXX_HAS_AVX*" -U USE_FBGEMM -S . -B build
pip install -e . -v --no-build-isolation
python -c "import torch; print(torch.backends.quantized.supported_engines)"
```

Before: `['onednn']`.
After: `['onednn', 'x86', 'fbgemm']`.

Issue repro now loads and runs:

```python
import torch, torchvision
m = torchvision.models.quantization.resnet18(weights="DEFAULT", quantize=True).eval()
m(torch.randn(1, 3, 224, 224))
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/198490
Approved by: https://github.com/jeffdaily

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
 2026-10-05 16:27:35 +00:00
PyTorch MergeBot e5329e634e Revert "[Windows] Re-enable FBGEMM on x64 and fix MSVC AVX flag detection (#198490)" 
This reverts commit ac62a17d81.

Reverted https://github.com/pytorch/pytorch/pull/198490 on behalf of https://github.com/jeffdaily due to broke TheRock Windows builds https://github.com/ROCm/rockrel/actions/runs/36959556845/job/110690206683#step:18:3750 ([comment](https://github.com/pytorch/pytorch/pull/198490#issuecomment-5956829957))
 2026-10-02 16:38:27 +00:00
LuFengqingcopilot-swe-agent[bot]jianyizhWang, ChuanqiRobert HardwickAndrey Talman
 		3dbf4ec7d6 Upgrade submodule oneDNN to v3.13 (#196712) 
This PR is to upgrade oneDNN from v3.12 to v3.13.   This upgrade mainly aims at supporting OneDNN GroupedGEMM on Intel GPU.

# Release docs from OneDNN

## Performance improvement
### Intel 64/AMD64 Processors
- Improved performance on future Intel Core Ultra processors with Intel AVX10.2 instruction set support (codename Nova Lake).
- Improved performance of matmul on processors with Intel AMX instruction set support.
- Improved performance of bf16, f16, and f32 matmul with unit M, N or K dimentions (GEMV-like) on processors with Intel AVX2 instruction set support.
- Improved performance of u8/s8 matmul with u4/s4 weights and grouped scales.
- Improved peformance of bf16 and f16 matmul with f8 weights.

### AArch64 Processors
- Improved performance of u8/s8 matmuls with u8/s8, f16, or s32 outputs.
- Improved performance of u8/s8 convolutions with bf16 or f32 outputs.
- Improved u8/s8 layer normalization performance.
- Improved performance of convolution backpropagation and pooling on platforms with 128-bit SVE.
- Improved performance of bf16 inner-product.
- Improved multi-threaded bnorm performance.
- Improved binary primitive and post-op performance.
- Improved performance of eltwise primitive with gelu_erf algorithm and post-op.

### Intel Graphics
- Improved performance for future integrated GPUs based on Xe3p-LPG architecture.
- Improved u8/s8 convolution performance on Intel Arc B-series graphics.
- Improved f16 and u8/s8 matmul performance with u8/s8 and u4/s4 weights in non-transposed layout.

## Functional API
- [experimental] Introduced support for eltwise and binary post-ops in matmul with grouped memory. Optimized implementation is available on Intel GPUs.
- [experimental] Extended grouped matmul with NVFP4 quantization scheme, including support for f4_e2m1 tensors with f8_e4m3 grouped scales and per-group binary post-op to implement global fp32 scale. This is an experimental feature that requires opt-in with [ONEDNN_EXPERIMENTAL_GROUPED_MEMORY=ON]

## Validation results on Xeon CPU

The test results are based on the 3 dynamo benchmark suites with three data types.

| Precision | Shape | Wrapper | Thread | Eager Ratio (v3.13/v3.12) | Inductor Ratio (v3.13/v3.12) |
| -- | -- | -- | -- | -- | -- |
| FP32 | Static | cpp | Multiple | 1.0086 | 1.0086|
| FP32 | Static | cpp | Single | 1.0154 | 1.0156 |
| AMP_BF16 | Static | cpp | Multiple | 0.9992 | 0.9983 |
| AMP_BF16 | Static | cpp | Single | 0.9994| 1.0005|
| AMP_FP16 | Static | cpp | Multiple | 0.9914 | 0.9979 |
| AMP_FP16 | Static | cpp | Single | 0.9933 | 0.9988 |

# Validation results on Intel B60

Shows **NO verified blocker** attributable to the oneDNN v3.12.2 upgrade.

| Area | Total | v3.13 / v3.12 | Verdict |
| --- | ---: | --- | --- |
| UT | 313K test cases| **1.001x** pass rate | No verified regressions|
| Accuracy | 1995 model configs | **1.00x** pass rate | No verified regressions |
| Performance |  1995 model configs | **1.004x** inductor / **1.004x** eager | No verified sustained drops |

Note: All data are based on release/2.14

UT scope owned by https://github.com/intel/torch-xpu-ops
Dynamo benchmark scope
- huggingface (~ 46 models), timm_models (~ 61 models), torchbench (~77 train, ~99 inf)
- float32, float16, bfloat16, amp fp16, amp bf16
- inference, training

# AArch64 Validation Results on Neoverse-V1 and Neoverse-V2

Dynamo pools available matched Hugging Face, timm and TorchBench model/configuration pairs within each compiler, precision and shape, including all fallback-count groups.

| Benchmark | Compiler | Precision | Shape | Tests in geomean | V1 latency geomean (lower is better) | V2 latency geomean (lower is better) | Regression notes |
| --- | --- | --- | --- | ---: | ---: | ---: | --- |
| Dynamo | eager | bf16_amp | dynamic | 41 | 0.99 | 0.99 |  |
| Dynamo | eager | bf16_amp | static | 41 | 0.98 | 0.99 |  |
| Dynamo | eager | fp32 | dynamic | 41 | 1.00 | 0.98 |  |
| Dynamo | eager | fp32 | static | 41 | 1.00 | 0.98 |  |
| Dynamo | inductor | bf16_amp | dynamic | 54 | 0.99 | 1.01 | [R1](#regression-r1) |
| Dynamo | inductor | bf16_amp | static | 54 | 0.98 | 1.01 | [R1](#regression-r1) |
| Dynamo | inductor | fp32 | dynamic | 72 | 0.99 | 0.99 |  |
| Dynamo | inductor | fp32 | static | 72 | 0.98 | 0.99 |  |
| NLP | — | — | — | 11 | 1.00 | 0.99 |  |
| TorchBench | eager | amp_bf16 | — | 75 | 0.97 | 0.97 |  |
| TorchBench | eager | fp32 | — | 78 | 0.98 | 0.98 |  |
| TorchBench | eager | fx_int8 | — | 55 | 2.82 | 2.72 | [R2](#regression-r2) |
| TorchBench | torchscript | amp_bf16 | — | 18 | 0.95 | 0.92 |  |
| TorchBench | torchscript | fp32 | — | 18 | 0.96 | 0.95 |  |

Blank regression-note cells mean no highlighted finding here, not a confirmed absence of regressions.

## Notable regressions

All configurations below use 16 threads. Ratios greater than 1 mean worse latency-equivalent performance; pooled geomeans can hide individual or suite-specific slowdowns.

| Reference | Hardware | Configuration | Finding |
| --- | --- | --- | --- |
| <a id="regression-r1"></a>R1 | V2 | Dynamo / timm / inductor / bf16_amp, dynamic and static | Suite latency geomean **1.10×** for both shapes (about **10% higher latency**), driven by `ghostnet_100`: the candidate takes **6.42 times as long** dynamically (**542% higher latency**) and **6.53 times as long** statically (**553% higher latency**). These are time multipliers, not percentage increases of 6.42% or 6.53%. Each suite/shape group contains 15 matched models. |
| <a id="regression-r2"></a>R2 | V1 / V2 | TorchBench / eager / fx_int8 | Latency-equivalent geomean **2.82× on V1 / 2.72× on V2**: about **182% / 172% higher latency-equivalent ratios**, respectively, across 55 matched model/configuration pairs per hardware family.  Analysis and Fix in https://github.com/pytorch/pytorch/pull/198177|

Pull Request resolved: https://github.com/pytorch/pytorch/pull/196712
Approved by: https://github.com/Skylion007, https://github.com/atalman, https://github.com/EikanWang

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jianyizh <jianyi.zhang@intel.com>
Co-authored-by: Wang, Chuanqi <chuanqi.wang@intel.com>
Co-authored-by: Robert Hardwick <robert.hardwick@arm.com>
Co-authored-by: Andrey Talman <atalman@fb.com>
 2026-10-02 01:40:27 +00:00
Taras ac62a17d81 [Windows] Re-enable FBGEMM on x64 and fix MSVC AVX flag detection (#198490) 
Fixes:
- #169816.

## Root cause

FBGEMM has been silently disabled on every Windows x64 build since #158210.
`CMAKE_SYSTEM_PROCESSOR` is `AMD64` on Windows, but the gate introduced there only allows `x86_64|aarch64`, so `set(USE_FBGEMM OFF)` fires unconditionally it also overrides `USE_FBGEMM=1` from the environment. Windows is left with `onednn` as the only quantized engine, so torchvision's pretrained quantized models (which pin `backend="fbgemm"` via `weights.meta`) fail with `RuntimeError: Quantized backend not supported`.

## Changes

**`CMakeLists.txt`**: allow `AMD64`, and restore the 32-bit exclusion #158210 dropped. The pointer-size check is required because 32-bit Windows also reports `AMD64`. It additionally excludes 32-bit `-m32`/x32/ILP32 builds. No 64-bit platform changes.

**`cmake/Modules/FindAVX.cmake`**: MSVC exposes AVX intrinsics without any `/arch:` flag, so the no-op `" "` candidate won and cached `CXX_AVX*_FLAGS=' '`. fbgemm reuses those cache variables, so its AVX targets built without `/arch:`, `__AVX2__` went undefined, and the kernels guarded by it failed to compile (`error C3861: 'transpose_kernel_mxn_sse'`). Probe the real flag first on MSVC. The non-MSVC branch is unchanged.

## Behavior change

Default qengine on Windows moves from `onednn` to `x86`, matching Linux.

## Test Plan

```
cmake -U "C_AVX*" -U "CXX_AVX*" -U "C_HAS_AVX*" -U "CXX_HAS_AVX*" -U USE_FBGEMM -S . -B build
pip install -e . -v --no-build-isolation
python -c "import torch; print(torch.backends.quantized.supported_engines)"
```

Before: `['onednn']`.
After: `['onednn', 'x86', 'fbgemm']`.

Issue repro now loads and runs:

```python
import torch, torchvision
m = torchvision.models.quantization.resnet18(weights="DEFAULT", quantize=True).eval()
m(torch.randn(1, 3, 224, 224))
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/198490
Approved by: https://github.com/Skylion007
 2026-09-30 16:39:13 +00:00
Egemen TuncarslanandTaras 07070bbd47 Fix silent parallel_for corruption on MSVC: stop mixing vcomp and libomp OpenMP runtimes (#193792) 
Fixes #193784. Also fixes #176091, the SDPA symptom of the same defect.

On MSVC x64 `torch_cpu` imports two OpenMP runtimes at once.
`-openmp:experimental` compiles `#pragma omp parallel` into `_vcomp_*` calls and
links VCOMP140 by itself, while this module's library search finds the
`libomp.lib` shipped with Visual Studio and adds it to the link line. The two
runtimes share no state, so inside a vcomp team every worker asks libomp and is
told `omp_get_num_threads() == 1` and `omp_get_thread_num() == 0`.
`ParallelOpenMP.h` then hands every worker the whole range and
`at::get_thread_num()` returns 0 on all of them, so each one runs the entire
chunk and any non-idempotent kernel body silently accumulates N times.

The fix skips both library searches when the compiler id is MSVC - the generic
one and the `OpenMP_PREFIX` one. There is nothing for them to find: cl.exe links
the runtime matching its `-openmp` flag on its own, so naming a library is
redundant at best, and naming libomp is the defect. The prefix search needs the
guard too because `HINTS` adds to the default search paths rather than replacing
them, so any non-empty `OMP_PREFIX` - even one pointing at a directory that does
not exist - still finds the Visual Studio `libomp.lib`.

The compiler id is checked rather than `MSVC`, because `Platform/Windows-Clang.cmake`
sets `MSVC` for clang-cl too, and clang-cl emits `__kmpc_*` calls that do need
libomp. For the same reason the MSVC ARM64 special case is removed: it was
guarded on bare `MSVC`, so on clang-cl ARM64 it linked vcomp for a compiler that
needs libomp, and on cl.exe ARM64 the new check already covers it.

### Rebuilding

The fix is invisible to an in-place reconfigure. `OpenMP_<LANG>_FLAGS`,
`OpenMP_<LANG>_LIB_NAMES` and `OpenMP_libomp_LIBRARY` are cache entries. This
module skips detection while the first two are cached, and even when detection
reruns, both guarded searches start with `NOT OpenMP_libomp_LIBRARY`, so a cached
libomp path is reused. An existing build directory therefore keeps linking libomp.
Delete the build directory, or clear the entries first:

```
cmake -U "OpenMP_*" <build-dir>
```

### History

I first sent this extending the ARM64 branch to find vcomp explicitly.
@frost-intel pointed out that linking vcomp explicitly is redundant and that
`MSVC` is too broad a guard. #196730 by @taras-janea implemented that suggestion
and also removes the ARM64 block, which is adopted here with co-authorship. @ezyang's review of #196730 caught the unguarded `OpenMP_PREFIX`
search.

### Test Plan

Windows 11, MSVC 14.44.35207, CPU-only, no MKL.

**The module in isolation, each configure in a fresh directory.** A minimal
project that runs `find_package(OpenMP)` under the real MSVC environment and
prints what the module selects:

| case | `OpenMP_CXX_LIB_NAMES` |
|---|---|
| main | `libomp` |
| this PR | empty |
| previous revision of this PR, `OMP_PREFIX=C:\nonexistent` | `libomp` |
| this PR, `OMP_PREFIX=C:\nonexistent` | empty |
| configured with main, then reconfigured in place with this PR | `libomp` (stale cache) |
| same directory, reconfigured with `-U "OpenMP_*"` | empty |

**Full build, 8 intra-op threads.** Integer addition is exactly associative, so a
wrong integer sum cannot come from reassociation:

```
python -c "import torch; y = torch.ones(40_000_000, dtype=torch.int64); print(sorted({y.sum().item() for _ in range(8)}))"
```

```
before:  [600000000, 640000000]      # 15x and 16x the true 40000000
after:   [40000000]
```

```
python test/test_transformers.py TestSDPACpuOnlyCPU TestSDPAFailureModesCPU
```

```
before:  FAILED (failures=3108, skipped=58)
after:   OK (skipped=58)             # 3203 tests
```

```
dumpbin -imports build/bin/torch_cpu.dll
```

```
before:  VCOMP140.DLL and libomp140.x86_64.dll
after:   VCOMP140.DLL only, zero __kmpc_* imports
```

Where these full-build numbers came from: the first revision of this change
(generic-search guard only), on main as of `7744b78e00e`, in an existing build
directory with the `OpenMP_*` cache entries cleared before reconfiguring - the
`cmake -U "OpenMP_*"` equivalent, not a deleted build directory. The later
revisions only remove the ARM64 block and guard the `OMP_PREFIX` search. On x64
without `OMP_PREFIX` they select the same libraries, as the fresh-directory table
above shows, so the link line is unchanged and rebuilding did not relink
`torch_cpu.dll`.

`cmake/public/mkldnn.cmake:20-30` relies on `caffe2::openmp` to give binaries that
pull dnnl objects their OpenMP runtime, because oneDNN adds no link dependency of
its own. With an empty MSVC library list `caffe2::openmp` carries no library, so
those binaries could fail to resolve `omp_*`. All 125 executables under
`build/bin` - the C++ tests plus `protoc` and two benchmarks - were deleted and
linked fresh, with no unresolved externals. `test_jit` is the only one that links
dnnl directly (`test/cpp/jit/CMakeLists.txt:129`). It links, loads and runs: 563 of
590 pass, and none of the 26 failures involves OpenMP - 14 are test backends this
Windows build does not register, 9 are `pinv`/`tensorinv` in a build without
LAPACK, 2 cannot read a model archive, and 1 matches an error message.

The change touches no compiler flag, so nothing recompiles; the full
`pip install -e . -v --no-build-isolation` takes about fifteen minutes.

MSVC ARM64 is covered by CI rather than by me: `windows-arm64-binary-wheel` built
the previous revision of this PR (`e27f297`), which already dropped the ARM64
block, with `USE_MKL` off, `BLAS=apl` and `-openmp:experimental`, and it linked.
That was the real risk of removing the block - no vcomp on the link line and
unresolved `_vcomp_*` symbols - so the autolink holds on ARM64 too. It shows the
link, not runtime behaviour. clang-cl is untested, for lack of the toolchain;
the reasoning is above.

Developed with AI assistance (Claude), which also ran the measurements on my
Windows machine; I reviewed the change and the results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/193792
Approved by: https://github.com/zklaus, https://github.com/atalman

Co-authored-by: Taras <taras@janeasystems.com>
 2026-09-30 09:17:33 +00:00
Lakshay Garg 3fc2dd5af6 Use target_compile_definitions instead of target_compile_options (#198007) 
target_compile_definitions is the preferred way of defining macros. target_compile_options is a general purpose function for specifying arbitrary compile flags.

Assisted-by: gpt-5.6-sol
Pull Request resolved: https://github.com/pytorch/pytorch/pull/198007
Approved by: https://github.com/Skylion007
 2026-09-22 11:46:15 +00:00
Xu HanandCopilot 09ace57daf [BE] Build all XPU wheels on a single runner (#195699) 
Take over #188136.

Builds all Linux XPU manywheel variants in one job, reusing ABI-independent C++/SYCL artifacts across Python versions while rebuilding Python-bound artifacts per interpreter.

## Before vs after

Comparing the eight XPU matrix builds from [#195438 run 33729050328](https://github.com/pytorch/pytorch/actions/runs/33729050328) with the unified build from [this PR, run 34123506914](https://github.com/pytorch/pytorch/actions/runs/34123506914):

### Linux XPU build runner-minutes

| Before (per-Python matrix, 8 jobs) | After (1 unified job) | Delta |
|---|---:|---:|
| 8 x ~78m = **623m** ([matrix jobs](https://github.com/pytorch/pytorch/actions/runs/33729050328)) | **139m** ([unified job](https://github.com/pytorch/pytorch/actions/runs/34123506914/job/101746866663)) | **-78%** |

This saves approximately **484 runner-minutes per XPU manywheel build**.

### Per-Python build breakdown

- First Python (cp310): **~74m** from a clean build.
- Following seven Pythons (cp311 through cp315t): **~7m each** on average.
- The unified job produced all eight XPU wheel artifacts successfully.

The first build establishes the shared XPU backend. Subsequent interpreters reuse the SYCL kernel objects and device link, rebuilding only Python-ABI-bound components.

### Fix for SYCL reuse

The earlier PATH-only tool pin did not prevent scikit-build-core from selecting the importable CMake/Ninja packages in each Python environment. The final implementation explicitly sets `CMAKE_EXECUTABLE` and passes the shared Ninja path through `CMAKE_ARGS`, keeping CMake's generated SYCL depfile commands stable across the loop.

Local Python 3.10 then Python 3.11 XPU builds confirmed **zero SYCL object recompilations** and **zero XPU device relinks** on the second build.

> AI-assisted drafting; CI measurements and implementation reviewed by the PR author.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/195699
Approved by: https://github.com/chuanqi129, https://github.com/atalman

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-09-16 12:57:07 +00:00
MaxXSoft 2f37d7e460 [CMake] Propagate vecLib discovery results to BLAS (#196428) 
Related issue: #196426

Successful vecLib discovery leaves `BLAS_LIBRARIES` unset, so the final BLAS success check fails. This patch forwards `vecLib_LINKER_LIBS` and `vecLib_INCLUDE_DIR` to the generic BLAS outputs.

AI-generated summary:

> ## Test plan
>
> Full CPU builds on macOS 26.6.2 arm64 with Python 3.14.5 succeeded both at baseline `6ee9ad6564b` and at this standalone fix `5127ac095b2`, using `WITH_BLAS=veclib`. BLAS/LAPACK changed from disabled to enabled; the same five runtime checks changed from one failure and four errors to five passes with no skips. They cover float64/complex128, unbatched/batched matrix multiplication, eig, SVD, solve and Cholesky. The fixed libtorch_cpu directly links Apple vecLib/Accelerate. Earlier SDK finder/link probes also passed with the fix, and the Accelerate control remained successful.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/196428
Approved by: https://github.com/Skylion007
 2026-09-09 20:57:33 +00:00
MaxXSoft aaf6649e31 [CMake] Fix ATLAS BLAS propagation and LAPACK detection (#196427) 
Related issue: #196426

ATLAS discovery loses its library outputs, explicit selection reads mismatched variable names, and the link set omits the separate Fortran BLAS interface.

This patch propagates the complete provider outputs and checks LAPACK symbols before enabling ATLAS LAPACK, including a separately linked LAPACK fallback.

AI-generated summary:

> ## Test plan
>
> Prior combined-patch validation: finder checks failed before and passed after in NGC 25.12 and 26.08. A complete baseline CPU build disabled LAPACK; the patched CPU build passed five capability/numerics checks and four existing eig/eigvals tests. Separate controls covered missing LAPACK and the Fortran BLAS link dependency.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/196427
Approved by: https://github.com/malfet, https://github.com/Skylion007
 2026-09-09 18:54:37 +00:00
fengqing.lu f5fd7876a7 Upgrade submodule oneDNN to v3.12.3 (#188785) 
This PR is to upgrade oneDNN from v3.12 to v3.12.2.   This upgrade mainly aims at supporting SYCL Graph for OneDNN XPU primitives.

# Improvements
- Enabled SYCL Graph record/replay mode support on Intel GPUs
- Fixed some correctness and performance issues on x64 and RV64 CPUs.

## Validation results on Xeon CPU
1. Dynamo benchmarks

The test results are based on the 3 dynamo benchmark suites with three data types.

| Precision | Shape | Wrapper | Thread | Eager Ratio (3.12.2/3.12) | Inductor Ratio (3.12.2/3.12) |
| -- | -- | -- | -- | -- | -- |
| FP32 | Static | cpp | Multiple | 1.0094 | 1.0116 |
| FP32 | Static | cpp | Single | 1.0139 | 1.0183 |
| AMP_BF16 | Static | cpp | Multiple | 1.0037 | 1.0035 |
| AMP_BF16 | Static | cpp | Single | 0.9954 | 0.9926 |
| AMP_FP16 | Static | cpp | Multiple | 1.0058 | 0.9991 |
| AMP_FP16 | Static | cpp | Single | 0.9984 | 0.9980 |

## Validation results on Intel B60

Shows **NO verified blocker** attributable to the oneDNN v3.12.2 upgrade.

| Area | Total | v3.12.2 / v3.12 | Verdict |
| --- | ---: | --- | --- |
| UT | 296K | **1.00x** pass rate | No verified regressions |
| Accuracy | 1995 | **1.00x** pass rate | No verified regressions |
| Performance | 1995 | **1.0015x** inductor / **1.0017x** eager | No verified sustained drops |

Note: All data are based on release/2.13

UT scope owned by https://github.com/intel/torch-xpu-ops
Dynamo benchmark scope
- huggingface (~ 46 models), timm_models (~ 61 models), torchbench (~77 train, ~99 inf)
- float32, float16, bfloat16, amp fp16, amp bf16
- inference, training

## Validation results on AArch64

See below for a breakdown of Arm Neoverse-V1 performance for a set of NLP, Torchbench, and Dynamo model configurations. Considering this is a patch release, the performance is similar to `v3.12.0` .


**Arm Neoverse-V1 - Dynamo - 16 Threads:**

Suite | Compiler | Data Type | Shape | Throughput Geomean (lower is better)
-- | -- | -- | -- | --
huggingface | eager | bf16_amp | dynamic | 1.00
huggingface | eager | bf16_amp | static | 1.01
huggingface | eager | fp32 | dynamic | 1.00
huggingface | eager | fp32 | static | 1.00
huggingface | inductor | fp32 | dynamic | 1.00
huggingface | inductor | fp32 | static | 1.00
timm | eager | bf16_amp | dynamic | 1.00
timm | eager | bf16_amp | static | 1.01
timm | eager | fp32 | dynamic | 1.00
timm | eager | fp32 | static | 1.00
timm | inductor | bf16_amp | dynamic | 1.00
timm | inductor | fp32 | dynamic | 1.00
timm | inductor | fp32 | static | 1.01
torchbench | inductor | fp32 | dynamic | 0.99
torchbench | inductor | fp32 | static | 0.99


**Arm Neoverse-V1 - Torchbench - 16 Threads:**

Mode | Precision | Latency Geomean (lower is better)
-- | -- | --
eager | bf16_amp | 1.00
eager | fp32 | 1.00
eager | fx_int8 | 1.00
torchscript | bf16_amp | 0.98
torchscript | fp32 | 1.00


**Arm Neoverse-V1 - NLP - 16 Threads:**

| latency geomean (lower is better) |
| -------------------------------- |
|0.99                      |

Pull Request resolved: https://github.com/pytorch/pytorch/pull/188785
Approved by: https://github.com/Skylion007, https://github.com/guangyey, https://github.com/aditew01, https://github.com/atalman
 2026-08-03 20:54:20 +00:00
Nikita Shulga 9985428cfc [CI] Update CPU x86 builds to clang-21 (#191108) 
Explicitly add vptr sanitizer, as in clang-21 it's no longer default
Skip `cpp/test_jit` under ASAN as it times out, probably because migration broke some stale skips, but JIT is getting deprecated anyway, so why bother)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/191108
Approved by: https://github.com/Skylion007
ghstack dependencies: #191117
 2026-07-27 02:15:42 +00:00
Z LiuandJithun Nair 7e6bc2fdc1 [ROCm] Bump fbgemm to 1.7.0 (#190601) 
Bump fbgemm to 1.7.0 to remove dependency on rocm-smi, needed by https://github.com/pytorch/pytorch/pull/190014

This PR also adds fbgemm as an additional package to build in `inductor-periodic` workflow for test purpose, as fbgemm currently is not consumed anywhere in the CI by ROCm jobs.

It also adds a workaround to build without sccache caching to address an FBGEMM build issue on ROCm when using sccache; and only for gfx950, purely to optimize fbgemm build time in CI.

Bumping `third_party/fbgemm` to 1.7.0 pulls in [FBGEMM #5635](https://github.com/pytorch/FBGEMM/pull/5635) (commit `faef9bf61`), which removed the `&& defined(FBGEMM_FBCODE)` guard around the AVX512-BF16 dequant kernel in `src/QuantUtilsAvx512.cc` (https://github.com/pytorch/FBGEMM/blob/bf6dce360a4fe133bc779e2fd036277678509f95/src/QuantUtilsAvx512.cc#L394). That kernel is now compiled unconditionally on x86, so the `fbgemm_avx512` target requires `-mavx512bf16` to build.

FBGEMM #5635 added the flag to its *own* build (`BUILD.bazel`, `cmake/modules/FindAVX.cmake:111`), but that doesn't reach us: PyTorch builds FBGEMM via `add_subdirectory`, and PyTorch's own `find_package(AVX)` (`CMakeLists.txt:913`) runs first and caches `CXX_AVX512_FLAGS` without bf16. FBGEMM's `FindAVX` then short-circuits on `IF(NOT CXX_AVX512_FOUND)`, so its flag never applies. The `fbgemm_avx512` object compiles without `-mavx512bf16` and the CPU build fails at `QuantUtilsAvx512.cc.o`. This adds the flag to the cached AVX512 flags that the FBGEMM target inherits.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/190601
Approved by: https://github.com/jithunnair-amd, https://github.com/q10, https://github.com/atalman

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
 2026-07-23 15:27:46 +00:00
Xinya ZhangandClaude Opus 4.8 42542475c4 [ROCm] Support USE_ASAN=1 build (#188242) 
Notable New Features:

* ROCm PyTorch now supports ASAN build with TheRock 7.14 SDK
  + Use clang from TheRock 7.14 as the host compiler
  + integration builds cleanly; tests unvalidated due to crashing during the early-initialization phase of the ROCm ASAN runtime.
  + Build of CK SDPA with ASAN is not tested
  + Select ASAN build of AOTriton for SDPA

Notable Bug Fixes:

* Rewrites env var parsers in `cmake/EnvVarForwarding.cmake` with python script
  + The old parser is error-prone and failed to handle complicated `PS1` env var like `${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$` (This is DEFAULT `PS1` from Ubuntu 24.04)
  + An outcome of this buggy parser is: `USE_ASAN` is never set if `PS1` contains `;` and occurs earlier than `USE_ASAN` in env vars.
  + **Breaking** the new parser now honors exported env vars over cmake command-line -D and/or CMakeCache.txt
* Other related ASAN build fixes on TheRock 7.14+Clang Environment

Pull Request resolved: https://github.com/pytorch/pytorch/pull/188242
Approved by: https://github.com/jeffdaily

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-20 19:37:28 +00:00
Frederik Gossen fa79b60282 Fix nightly lint-urls failures on dead and bot-blocked URLs (#189718) 
The nightly Link checks / lint-urls job was failing on three URLs that
the 403/429/503 WAF carve-out (added in #188510) does not cover: two
returned 000 (no HTTP response) and one returned 404.

- cmake/Modules/FindMAGMA.cmake: the MAGMA homepage moved from
  http://icl.cs.utk.edu/magma/ (dead, 000) to https://icl.utk.edu/magma/.
- functorch/dim/README.md: pytorch.org/docs/stable/named_tensor.html now
  404s because named tensors and its doc were removed in #173895. Pin the
  reference to docs.pytorch.org/docs/2.8/named_tensor.html, the last
  release that still serves the page.
- torch.compiler_performance_dashboard.md: hud.pytorch.org is live but its
  WAF drops the linter's connection (000), so annotate the line with
  @lint-ignore, matching how bot-protected hosts are handled.

Test Plan:
lintrunner passed on the three changed files. Verified each replacement
URL returns 200 and the originals fail (000/404) via curl with the same
user agent the linter uses. The nightly lint-urls job exercises the
end-to-end check.

Authored with Claude.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/189718
Approved by: https://github.com/zou3519
 2026-07-13 22:03:04 +00:00
Jeff Daily 2f98635762 [ROCm] Build enablement for ASAN: ODR workaround, HIP linkage, container annotations (#183792) 
ROCm + ASAN builds need several coordinated pieces to compile, link, and not abort at startup. This PR lands them together so reviewers can see the full picture rather than chasing partial fixes.

1. Avoid the ASAN/UBSan ODR-violation abort
   - Add `-mllvm -asan-use-private-alias=1` to ASAN+Clang builds (`cmake/Modules/FindSanitizer.cmake`), forcing private aliases for instrumented globals so the runtime's alignment check doesn't fire on the unaligned `__sanitized_padded_global` references.
   - Skip `Sanitizer::undefined` (and the bare `-fsanitize=undefined` flag in `test/cpp/jit/CMakeLists.txt`) when `USE_ROCM` is set, to avoid re-introducing the alignment mismatch from UBSan.

2. Link Sanitizer::address into torch_hip and extend libstdc++ container annotations to HIP TUs
   - Add `Sanitizer::address` to `Caffe2_HIP_DEPENDENCY_LIBS` so torch_hip links the sanitizer.
   - Extend the existing `_GLIBCXX_SANITIZE_VECTOR` / `_GLIBCXX_SANITIZE_STD_ALLOCATOR` `target_compile_definitions` block to include `COMPILE_LANGUAGE:HIP` so HIP-side TUs update libstdc++ container annotations consistently with cpu-side, removing the largest in-tree class of container-overflow false positives.

3. Disable jiterator under ROCm + ASAN
   - jiterator JITs element-wise kernels through hiprtc, and we haven't set up an ASAN-aware hiprtc runtime. Hard-disable jiterator at compile time when `USE_ROCM AND USE_ASAN`.
   - Implementation: CMake (`cmake/Dependencies.cmake`) exports `-DAT_DISABLE_JITERATOR` (both via `add_definitions` for cpu/cuda TUs and via `HIP_CXX_FLAGS` for HIP-language TUs, since `add_definitions` doesn't reliably propagate to HIP). The `AT_USE_JITERATOR()` macro in `aten/src/ATen/jit_macros.h` checks the flag with `#ifdef AT_DISABLE_JITERATOR` — neutral name keeps the header free of GPU-backend tokens so the `ATEN_CPU` lint rule passes.

Net effect on non-ROCm-ASAN builds: zero. Each gate is on `USE_ROCM`, `USE_ASAN`, or `Clang` detection. Non-ROCm builds and non-ASAN builds behave identically to before.

Note: this does not make `detect_container_overflow=1` viable in all ROCm + ASAN runs. Other third-party libraries linked into the process (e.g. gloo) can re-introduce mismatched annotations. Anyone running the full ASAN test suite under ROCm will still want `detect_container_overflow=0` for now — this set just removes the largest in-tree contributors.

Authored with Claude.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/183792
Approved by: https://github.com/malfet, https://github.com/cyyever
 2026-06-06 01:19:17 +00:00
Jiang, Yanbing 1e8ad2a000 Upgrade submodule oneDNN to v3.12 (#181222) 
This PR is to upgrade oneDNN to v3.12.

## Improvements

- Improved performance on future Intel Core Ultra processors with Intel AVX10.2 instruction set support (code name Nova Lake). These optimizations are now enabled by default on compatible processors.
- Improved performance on future Intel Xeon processors with Intel AVX10.2 and Intel AMX instruction set support (code name Diamond Rapids). These optimizations are now enabled by default on compatible processors.
- Introduced initial performance optimizations for future integrated GPUs based on Xe3p-LPG architecture.
- Introduced initial performance optimizations for future discrete GPUs based on Xe3p-XPC architecture. This is a preview functionality not recommended for production use.
- Improved f16 matmul performance on Intel Arc Graphics for Intel Core Ultra processor Series 3 (formerly Panther Lake).
- Improved matmul and convolution performance on Arm Neoverse V2 cores.

## Validation results on Xeon CPU
1. Dynamo benchmarks

The test results are based on the 3 dynamo benchmark suites with three data types.

Precision | Shape | Wrapper | Thread  | Eager Ratio (3.12/3.11.2) | Inductor Ratio (3.12/3.11.2)
-- | -- | -- | -- | -- | --
FP32 | Static | cpp | Multiple |  0.9938 | 0.9999
FP32 | Static | cpp | Single |  0.9970 | 0.9967
AMP_BF16 | Static | cpp | Multiple |  1.0098 | 1.0046
AMP_BF16 | Static | cpp | Single |  1.0042 | 1.0068
AMP_FP16 | Static | cpp | Multiple |  0.9998 | 0.9990
AMP_FP16 | Static | cpp | Single |  1.0032 | 1.0027

## Validation results on Intel B60

Shows **NO verified blocker** attributable to the oneDNN v3.12 upgrade.

| Area | Total | v3.12 | v3.11.2 | Verdict |
| --- | ---: | --- | --- | --- |
| UT | 296K | **99.7374%** pass rate | 99.7360% pass rate | No verified regressions |
| Accuracy | 1903 | **89.86%** pass rate | 89.70% pass rate | No verified regressions |
| Performance | 1767 | 0.994x inductor / **1.008x** eager | 1.0000x / 1.0000x | No verified sustained drops |

Note: All data are based on release/2.12 with oneDNN change https://github.com/pytorch/pytorch/pull/183248

UT scope owned by https://github.com/intel/torch-xpu-ops
Dynamo benchmark scope
- huggingface (~ 46 models), timm_models (~ 61 models), torchbench (~77 train, ~99 inf)
- float32, float16, bfloat16, amp fp16, amp bf16
- inference, training

## Validation results on AArch64

See below for a breakdown of Arm Neoverse-V1 and Arm Neoverse-V2 performance for a set of NLP, Torchbench, and Dynamo model configurations. We see large improvements for most BF16 + Int8 configurations.

**Arm Neoverse-V1 - NLP - FP32 - 16 Threads:**

| latency geomean (lower is better) |
| --------------------------------- |
|1.000                        |


**Arm Neoverse-V2 - NLP - FP32 - 16 Threads:**

| latency geomean (lower is better) |
| -------------------------------- |
|0.997                      |

**Arm Neoverse-V1 - Torchbench - 16 Threads:**

| Mode        | Precision | Throughput Geomean (Higher is Better) |
|-------------|------------|----------------------------------------|
| Eager       | AMP_BF16  | 1.0058                                 |
| Eager       | FP32      | 1.0017                                 |
| Eager       | INT8      | 1.0519                                 |
| TorchScript | AMP_BF16  | 1.5880                                 |
| TorchScript | FP32      | 1.0027                                 |


**Arm Neoverse-V2 - Torchbench - 16 Threads:**

| Mode        | Precision | Throughput Geomean (Higher is Better) |
|-------------|------------|----------------------------------------|
| Eager       | AMP_BF16  | 1.0102                                 |
| Eager       | FP32      | 1.0022                                 |
| Eager       | INT8      | 1.0177                                 |
| TorchScript | AMP_BF16  | 1.6058                                 |
| TorchScript | FP32      | 1.0449                                 |

**Arm Neoverse-V1 - Dynamo (16 threads):**
| Suite      | Compiler | Data Type | Shape   | Latency Geomean (Lower is Better) |
|-------------|-----------|------------|----------|-----------------------------------|
| huggingface | eager     | AMP_BF16  | dynamic  | 1.0021                            |
| huggingface | eager     | AMP_BF16  | static   | 1.0016                            |
| huggingface | eager     | FP32      | dynamic  | 1.0021                            |
| huggingface | eager     | FP32      | static   | 1.0001                            |
| huggingface | inductor  | AMP_BF16  | dynamic  | 1.0007                            |
| huggingface | inductor  | FP32      | dynamic  | 0.9982                            |
| huggingface | inductor  | FP32      | static   | 1.0005                            |
| timm        | eager     | AMP_BF16  | dynamic  | 0.9947                            |
| timm        | eager     | AMP_BF16  | static   | 0.9954                            |
| timm        | eager     | FP32      | dynamic  | 0.9978                            |
| timm        | eager     | FP32      | static   | 1.0055                            |
| timm        | inductor  | AMP_BF16  | dynamic  | 0.9994                            |
| timm        | inductor  | AMP_BF16  | static   | 0.9937                            |
| timm        | inductor  | FP32      | dynamic  | 1.0001                            |
| timm        | inductor  | FP32      | static   | 1.0002                            |
| torchbench  | eager     | AMP_BF16  | dynamic  | 0.9880                            |
| torchbench  | eager     | AMP_BF16  | static   | 0.9998                            |
| torchbench  | eager     | FP32      | dynamic  | 1.0241                            |
| torchbench  | eager     | FP32      | static   | 1.0072                            |
| torchbench  | inductor  | AMP_BF16  | dynamic  | 0.9935                            |
| torchbench  | inductor  | AMP_BF16  | static   | 0.9758                            |
| torchbench  | inductor  | FP32      | dynamic  | 0.9964                            |
| torchbench  | inductor  | FP32      | static   | 1.0050                            |

**Arm Neoverse-V2 - Dynamo: (16 threads)**

| Suite      | Compiler | Data Type | Shape   | Latency Geomean (Lower is Better) |
|-------------|-----------|------------|----------|-----------------------------------|
| huggingface | eager     | AMP_BF16  | dynamic  | 1.0092                            |
| huggingface | eager     | AMP_BF16  | static   | 1.0024                            |
| huggingface | eager     | FP32      | dynamic  | 1.0013                            |
| huggingface | eager     | FP32      | static   | 1.0004                            |
| huggingface | inductor  | AMP_BF16  | dynamic  | 1.0041                            |
| huggingface | inductor  | FP32      | dynamic  | 1.0017                            |
| huggingface | inductor  | FP32      | static   | 0.9990                            |
| timm        | eager     | AMP_BF16  | dynamic  | 1.0002                            |
| timm        | eager     | AMP_BF16  | static   | 0.9997                            |
| timm        | eager     | FP32      | dynamic  | 0.9897                            |
| timm        | eager     | FP32      | static   | 0.9920                            |
| timm        | inductor  | AMP_BF16  | dynamic  | 0.9854                            |
| timm        | inductor  | AMP_BF16  | static   | 0.9911                            |
| timm        | inductor  | FP32      | dynamic  | 0.9861                            |
| timm        | inductor  | FP32      | static   | 1.0039                            |
| torchbench  | eager     | AMP_BF16  | dynamic  | 1.0025                            |
| torchbench  | eager     | AMP_BF16  | static   | 0.9903                            |
| torchbench  | eager     | FP32      | dynamic  | 0.9958                            |
| torchbench  | eager     | FP32      | static   | 0.9946                            |
| torchbench  | inductor  | AMP_BF16  | dynamic  | 0.9903                            |
| torchbench  | inductor  | AMP_BF16  | static   | 0.9614                            |
| torchbench  | inductor  | FP32      | dynamic  | 0.9953                            |
| torchbench  | inductor  | FP32      | static   | 0.9979                            |

Pull Request resolved: https://github.com/pytorch/pytorch/pull/181222
Approved by: https://github.com/malfet, https://github.com/atalman
 2026-06-04 01:58:13 +00:00
Dmitry RogozhkinandYu, Guangye 34ea1b77ee xpu: match dpclang SYCL_LIBRARY setting with icpx (#185571) 
With `icpx` we set `SYCL_LIBRARY` as a value returned by `find_library()`. This commit aligns `dpclang` path behavior with that which is needed to meet existing expectations when building some pytorch code such as kineto.

This commit builds on top of 00884edba ("xpu: enable dpclang sycl compiler") and additionally allows to build pytorch for XPU based on fully open source stack with enabled kineto profiling.

PTI library required for XPU kineto can be built from the following sources:
* https://github.com/intel/pti-gpu

Validated build command line:
```
XPU_SYCL_COMPILER=dpclang USE_KINETO=1 USE_ONEMKL_XPU=0 \
      python3 setup.py develop
```

Enabled kineto build verified by running `test/profiler/test_xpu_profiler.py`, all tests passing.

CC: @guangyey @EikanWang @chuanqi129 @frenchwr
Pull Request resolved: https://github.com/pytorch/pytorch/pull/185571
Approved by: https://github.com/guangyey, https://github.com/EikanWang

Co-authored-by: Yu, Guangye <guangye.yu@intel.com>
 2026-06-03 04:02:23 +00:00
Yuanyuan Chen 5cdb4b0418 Remove dead THNN/UCC/TH references from CMakeLists (#185062) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/185062
Approved by: https://github.com/jansel, https://github.com/guangyey
 2026-05-25 03:23:26 +00:00
Dmitry Rogozhkin 00884edba1 xpu: enable dpclang sycl compiler (#179763) 
`dpclang` compiler is an open source version of the closed source Intel DPC++ compiler. `dpclang` is being supported by Intel in the https://github.com/intel/llvm public repository and got recently packaged into upcoming Ubuntu 26.04 LTS.

This commit enables minimal support of `dpclang` compiler in PyTorch building towards fully open source PyTorch stack on Linux. Limitations:

* Kineto, XCCL and oneMKL paths were not enabled and might require additional work
* SYCL-TLA, oneDNN and Pytorch itself might require additional changes to properly handle dpclang compiler as it does not support `__INTEL_LLVM_COMPILER` macro

We will address above limitations in the follow up patches. At the moment the following procedure can be used to build PyTorch for XPU with `dpclang` compiler on Ubuntu 26.04:

* Do NOT activate oneAPI environment
* Install open source DPC++ compiler, OpenCL and few other tools with:
```
apt-get install dpclang-6 ocl-icd-opencl-dev pkg-config libze-dev
```
* Build PyTorch as follows:

```
XPU_SYCL_COMPILER=dpclang USE_KINETO=0 USE_ONEMKL_XPU=0 \
  python3 setup.py develop
```

The dependency from https://github.com/intel/torch-xpu-ops/pull/3287 is weak in a sense that the change is transparent to the current production build of the PyTorch for XPU using oneAPI DL Essentials. The change in torch-xpu-ops is only required for the build with the `dpclang`.

Needs: https://github.com/intel/torch-xpu-ops/pull/3287

CC: @guangyey @EikanWang @chuanqi129 @frenchwr

Pull Request resolved: https://github.com/pytorch/pytorch/pull/179763
Approved by: https://github.com/EikanWang, https://github.com/guangyey, https://github.com/atalman
 2026-05-21 14:18:57 +00:00
xinan.lin 617964eee2 FindMKL: respect MKLROOT/CMPLR_ROOT/ONEAPI_ROOT from setvars.sh (#183506) 
Previously FindMKL.cmake hardcoded /opt/intel/oneapi as the default search
path, ignoring environment variables set by oneAPI setvars.sh. Custom oneAPI
installations were not detected even after sourcing setvars.sh.

Detection now follows a clear priority chain:
  1. Environment variables (MKLROOT, CMPLR_ROOT, ONEAPI_ROOT)
  2. Well-known oneAPI directory (/opt/intel/oneapi on Linux)
  3. Legacy Intel Compiler Suite directory (/opt/intel on Linux)

Also adds MKLROOT support on Windows (setvars.bat sets it too).

Authored by Claude.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/183506
Approved by: https://github.com/guangyey, https://github.com/jansel
 2026-05-21 06:25:07 +00:00
Nikita Shulga 4dce2962ca Add SVE128 dispatch (#176256) 
Neoverse V1 supports SVE256, but Neoverse V2 narrows it back down to SVE128 — still useful to dispatch to.

This PR only wires up the dispatch + a small NEON<->SVE bridge. When compiled with `-msve-vector-bits=128`, `svfloat32_t` and `float32x4_t` have identical layout, so the two new helpers `neon_to_sve` / `sve_to_neon` compile down to zero instructions. That lets the SVE128 build call into the existing SLEEF SVE routines from inside what is otherwise the NEON `Vectorized<float>` implementation — currently used by `exp2()` (`Sleef_exp2fx_u10sve`) as a demonstration of the bridge.

Note: unlike the X86 AVX hierarchy (AVX512 ⊇ AVX2 ⊇ DEFAULT), SVE dispatch here is **unversioned** — `SVE128` and `SVE256` are sibling capabilities, not ordered. The dispatcher matches with `==` rather than `>=`, so you cannot dispatch a SVE128 kernel on an SVE256 machine, nor vice versa. Each kernel must be registered for the specific vector length it targets.

Because of the binary size growth, we have to pass an explicit `--stub-group-size=0x2000000` flag to work around R_AARCH64_CALL26 relocation errors.

Also, workaround gcc-13.x bf16 constant-miscompile bug, when it confuses fp16 and bf16 constant which results in vdupq_n_bf16((bfloat16_t)1.0f) being miscomputed into `fmov v?.8h, 1.0e+0`, see https://godbolt.org/z/4sqdxj1GM (gcc-15 uses an interesting numeric trick here)

This PR is provisional and may be reverted if the expected performance gains on SVE128 hardware (e.g. Neoverse V2) do not materialize in benchmarks.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/176256
Approved by: https://github.com/Skylion007, https://github.com/fadara01
 2026-05-14 17:36:32 +00:00
PyTorch MergeBot dc80f55b8b Revert "Add SVE128 dispatch (#176256)" 
This reverts commit 8e63642a0a.

Reverted https://github.com/pytorch/pytorch/pull/176256 on behalf of https://github.com/malfet due to Need to undo submodule updates ([comment](https://github.com/pytorch/pytorch/pull/176256#issuecomment-4453029869))
 2026-05-14 17:24:21 +00:00
Nikita Shulga 8e63642a0a Add SVE128 dispatch (#176256) 
Neoverse V1 supports SVE256, but Neoverse V2 narrows it back down to SVE128 — still useful to dispatch to.

This PR only wires up the dispatch + a small NEON<->SVE bridge. When compiled with `-msve-vector-bits=128`, `svfloat32_t` and `float32x4_t` have identical layout, so the two new helpers `neon_to_sve` / `sve_to_neon` compile down to zero instructions. That lets the SVE128 build call into the existing SLEEF SVE routines from inside what is otherwise the NEON `Vectorized<float>` implementation — currently used by `exp2()` (`Sleef_exp2fx_u10sve`) as a demonstration of the bridge.

Note: unlike the X86 AVX hierarchy (AVX512 ⊇ AVX2 ⊇ DEFAULT), SVE dispatch here is **unversioned** — `SVE128` and `SVE256` are sibling capabilities, not ordered. The dispatcher matches with `==` rather than `>=`, so you cannot dispatch a SVE128 kernel on an SVE256 machine, nor vice versa. Each kernel must be registered for the specific vector length it targets.

Because of the binary size growth, we have to pass an explicit `--stub-group-size=0x2000000` flag to work around R_AARCH64_CALL26 relocation errors.

Also, workaround gcc-13.x bf16 constant-miscompile bug, when it confuses fp16 and bf16 constant which results in vdupq_n_bf16((bfloat16_t)1.0f) being miscomputed into `fmov v?.8h, 1.0e+0`, see https://godbolt.org/z/4sqdxj1GM (gcc-15 uses an interesting numeric trick here)

This PR is provisional and may be reverted if the expected performance gains on SVE128 hardware (e.g. Neoverse V2) do not materialize in benchmarks.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/176256
Approved by: https://github.com/Skylion007
 2026-05-13 22:32:04 +00:00
PyTorch MergeBot 39e69dcff8 Revert "Add SVE128 dispatch (#176256)" 
This reverts commit f230b0bfe3.

Reverted https://github.com/pytorch/pytorch/pull/176256 on behalf of https://github.com/huydhn due to This seems to fail some aarch64 tests ([comment](https://github.com/pytorch/pytorch/pull/176256#issuecomment-4426321955))
 2026-05-12 00:48:11 +00:00
Nikita Shulga f230b0bfe3 Add SVE128 dispatch (#176256) 
Adds SVE128 dispatch path.

Neoverse V1 supports SVE256, but Neoverse V2 narrows it back down to SVE128 — still useful to dispatch to.

This PR only wires up the dispatch + a small NEON↔SVE bridge. When compiled with `-msve-vector-bits=128`, `svfloat32_t` and `float32x4_t` have identical layout, so the two new helpers `neon_to_sve` / `sve_to_neon` compile down to zero instructions. That lets the SVE128 build call into the existing SLEEF SVE routines from inside what is otherwise the NEON `Vectorized<float>` implementation — currently used by `exp2()` (`Sleef_exp2fx_u10sve`) as a demonstration of the bridge.

Note: unlike the AVX hierarchy (AVX512 ⊇ AVX2 ⊇ DEFAULT), SVE dispatch here is **unversioned** — `SVE128` and `SVE256` are sibling capabilities, not ordered. The dispatcher matches with `==` rather than `>=`, so you cannot dispatch a SVE128 kernel on an SVE256 machine, nor vice versa. Each kernel must be registered for the specific vector length it targets.

Because of the binary size growth, we have to pass an explicit `--stub-group-size=0x2000000` flag to work around R_AARCH64_CALL26 relocation errors.

This PR is provisional and may be reverted if the expected performance gains on SVE128 hardware (e.g. Neoverse V2) do not materialize in benchmarks.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/176256
Approved by: https://github.com/Skylion007
 2026-05-11 23:14:04 +00:00
fengqing.lu a885081ea8 [XPU] Disable OneDNN CPU runtime for GPU build (#181831) 
PyTorch XPU uses oneDNN's threadpool API for GPU build, but does not provide it's own threadpool implementation. The verbose behavior in PyTorch GPU builds was impacted by transition of threadpool implementation in OneDNN. Due to oneDNN CPU functionality is not used in XPU builds, we could set CPU runtime as None now.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/181831
Approved by: https://github.com/EikanWang, https://github.com/guangyey
 2026-05-08 08:56:23 +00:00
PyTorch MergeBot 4e0fd9fa5d Revert "Add ASIMD_BF16 Vectorized class specialisation (#177867)" 
This reverts commit 1ff0a3356f.

Reverted https://github.com/pytorch/pytorch/pull/177867 on behalf of https://github.com/malfet due to This broke aarch64 tests, see https://hud.pytorch.org/hud/pytorch/pytorch/c7044a743372255325d43b4757a35a10aa974e50/1?per_page=50&name_filter=ull%20%2F%20linux-jammy-aarch64-py3.10&mergeEphemeralLF=true ([comment](https://github.com/pytorch/pytorch/pull/177867#issuecomment-4398718865))
 2026-05-07 15:58:40 +00:00
Murray Steele 1ff0a3356f Add ASIMD_BF16 Vectorized class specialisation (#177867) 
On AArch64 we compile PyTorch with march=armv8-a. This means the default Vectorized class instance does not support native FP32-to-BF16 conversions, and instead emulates them using several instructions per vector. For smaller operations such as broadcast mul or div, a large part of their runtime is just converting from FP32-to-BF16.

This change adds a new specialization of the Vectorized class which compiles the default Vectorised class compiled with native BF16 support. This allows us to use faster conversions from FP32-to-BF16 when they are available.

See below for speedups for broadcast mul and broadcast div, BF16, on Neoverse-V2 cores:
<img width="1510" height="727" alt="mul" src="https://github.com/user-attachments/assets/d780982a-4f60-455f-a777-0bf9510de86a" />
<img width="1510" height="727" alt="div" src="https://github.com/user-attachments/assets/55d7063e-4cc4-452e-b665-8484ca4818f3" />

Pull Request resolved: https://github.com/pytorch/pytorch/pull/177867
Approved by: https://github.com/Skylion007
 2026-05-07 13:19:50 +00:00
Yu, Guangye 158415c482 Clean up CMake related to XPU (#181355) 
# Motivation
1. oneDNN said until 3.11, oneDNN will not depend on OpenCL in build time, so we removed its dependency
2. Remove the support for the obsolete compiler 2024.*

Pull Request resolved: https://github.com/pytorch/pytorch/pull/181355
Approved by: https://github.com/Skylion007
 2026-04-27 16:12:07 +00:00
Jiang, Yanbingandfengqing.lu c2164200ee Upgrade submodule oneDNN to v3.11.2 (#177607) 
This PR is to upgrade oneDNN to v3.11.2.

## Improvements
- Improved fp32 matmul performance with fp4 compressed weights.
- Improved fp32 matmul performance for cases when one of the tensors has a trivial dimension on processors with Intel AVX-512 instruction set support.
- Improved fp16/bf16 matmul performance for large tensor cases on Intel Graphics for Intel Core Ultra processor Series 3 (formerly Panther Lake).
- Improved matmul performance for cases with 4-byte alignment on Intel GPUs based on Xe2 architecture.
- Improved performance of fp16/bf16 matmul with mxfp4 weights.
- Introduced destination tensor [dynamic quantization](https://uxlfoundation.github.io/oneDNN/v3.11/dev_guide_attributes_quantization.html#dynamic-quantization) in matmul primitive following Open Compute Microscaling (MX) formats specification.
- Introduced support for NVFP4 quantization scheme. The changes include support for fp8_e4m3 grouped scales and dynamic quantization support for destination tensor with NVFP4-specific formula for scales computation.
- Introduced support for dropout as a primitive attribute for matmul, softmax and eltwise primitives.

## Validation results on Xeon CPU
1. Dynamo benchmarks

The test results are based on the 3 dynamo benchmark suites with three data types.

<img width="787" height="231" alt="image" src="https://github.com/user-attachments/assets/5e31dc94-4b36-4354-9aa4-3223888d126d" />

According to the local test results, we found that two torchbench models on AMP_BF16 show over 7% performance regression:

- `torch_multimodal_clip` has 9% e2e regression.
- `BERT_pytorch` has 22% e2e performance regression.

# Validation results on Intel B60

Suite | Precision | Shape | Eager Speedup | Inductor Speedup
-- | -- | -- | -- | --
Huggingface | BF16 | Static | 1.003 | 1.002
Timm | Bf16 | Static | 1.001 | 0.999
Torchbench | BF16 | Static | 1.003 | 1.000

## Validation results on AArch64

### Dynamo Suite: Geomean over latency - prev version vs new version (**Higher is better**)
* for 16 Threads

Highlights for **Neoverse-V2** :
* MobileNetV3  1.15x lower latency with BF16.
* Detectron2  **1.6x** lower latency with BF16.
* AlexNet **6.2x** lower latency with BF16.

| Suite       | Precision | Shape   | Neoverse-V1 Eager Geomean | Neoverse-V1 Inductor Geomean | Neoverse-V2 Eager Geomean | Neoverse-V2 Inductor Geomean |
|------------|-----------|---------|---------------------------|------------------------------|--------------------------|-----------------------------|
| HuggingFace         | f32       | static  | 0.992358837               | 0.995203121                  | 1.0062                   | 1.007                       |
| HuggingFace         | f32       | dynamic | 0.991266938               | 0.993936984                  | 1.0082                   | 1.0081                      |
| HuggingFace         | bf16      | static  | 1.002295256               | 0.987069391                  | 0.997                    | 1.0134                      |
| HuggingFace         | bf16      | dynamic | 0.999110791               | 0.993443274                  | 1.00108                  | 1.0089                      |
| Timm       | f32       | static  | 0.989609104               | 0.997575891                  | 0.9168                   | -                      |
| Timm       | f32       | dynamic | 0.988387827               | 0.998601957                  | 0.92156                  |  -                      |
| Timm       | bf16      | static  | 1.007171058               | 1.01461039                   | 0.9925                   | 0.9949                      |
| Timm       | bf16      | dynamic | 1.008969741               | 1.016053648                  | 0.9928                   | 1.0321                      |
| Torchbench | f32       | static  | 0.997158099               | 1.00775975                   | 0.96                     | -                           |
| Torchbench | f32       | dynamic | 0.998372653               | 1.005439427                  | -                        | -                           |
| Torchbench | bf16      | static  | 1.011766848               | 1.02951623                   | 0.9815                   | 0.98613                     |
| Torchbench | bf16      | dynamic | 1.017915309               | 1.028806584                  | 0.9759                   | 0.9831                      |

### Torchbenchmark - Geomean over speedup (Higher is better)
* on 16 Threads

| Suite       | Precision | Shape   | Neoverse-V1 Eager Geomean | Neoverse-V1 TorchScript Geomean | Neoverse-V2 Eager Geomean | Neoverse-V2 TorchScript Geomean |
|------------|-----------|---------|---------------------------|---------------------------------|--------------------------|---------------------------------|
| Torchbench | amp_bf16  | inference | 1.00043                 | 1.02446                         | 1.023                    | 1.059                           |
| Torchbench | amp_bf16  | training  | 1.03322                 | -                               | -                        | -                               |
| Torchbench | fp32      | inference | 1.00452                 | 1.00715                         | 1.0675                   | 1.1086                          |
| Torchbench | fp32      | training  | 1.00121                 | -                               | -                        | -                               |

### NLP - FP32
* on 16 threads

| Suite | Neoverse-V1 Latency Geomean | Neoverse-V2 Latency Geomean |
|-------|-----------------------------|-----------------------------|
| NLP   | 0.996510961                 | 0.99651                     |

Pull Request resolved: https://github.com/pytorch/pytorch/pull/177607
Approved by: https://github.com/malfet, https://github.com/aditew01, https://github.com/atalman

Co-authored-by: fengqing.lu <fengqing.lu@intel.com>
 2026-04-14 18:44:31 +00:00
Ke WenandAaron Gokaslan 8e7898a6bd Add NCCL comm suspend, resume and memory stats (#176300) 
Added three new APIs:
`backend.suspend()`: free the memory held by the backend/communicator
`backend.resume()`: restore the memory needed by the backend/communicator
`backend.memory_stats()`: return memory usage info of the backend.

```
pytest -vs test/distributed/test_c10d_nccl.py -k test_get_memory_stats
pytest -vs test/distributed/test_c10d_nccl.py -k test_suspend
pytest -vs test/distributed/test_c10d_nccl.py -k test_resume
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/176300
Approved by: https://github.com/Skylion007

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
 2026-03-18 06:16:51 +00:00
Nikita Shulga 313bf1ed32 [Build] Strengthen SVE compile check in FindARM.cmake (#176646) 
Add `vget_low_bf16` call, that exposes gcc-13.1 bug when mixing NEON and SVE
```
/usr/lib/gcc/aarch64-linux-gnu/13/include/arm_neon.h:28347:1: error: inlining failed in call to 'always_inline' 'bfloat16x4_t vget_low_bf16(bfloat16x8_t)': target specific option mismatch
```

Turn missing SVE support for Linux ARM compiler into fatal error, but overridable with `BUILD_IGNORE_SVE_UNAVAILABLE` envvar
Pull Request resolved: https://github.com/pytorch/pytorch/pull/176646
Approved by: https://github.com/Skylion007
ghstack dependencies: #176645
 2026-03-06 03:00:12 +00:00
Nikita Shulga dc58ffcff2 [BE] Remove lots of ARMv7 checks (#176645) 
32-bit ARM systems might or might not support NEON, but PyTorch is only compatible with ARMv8, so delete all of that stale logic

Pull Request resolved: https://github.com/pytorch/pytorch/pull/176645
Approved by: https://github.com/Skylion007
 2026-03-05 23:25:08 +00:00
Nikita Shulga d10e2dfe93 [BE] Simplify SVE256 detection (#176247) 
- Use the same `-march=` combination in both FindARM and Dependencies.cmake
- Remove special casing  for clang compiler ( that introduced in https://github.com/pytorch/pytorch/pull/119571 with no clear explanation why), it fails with internal compiler error using both `-O2` and `-O3` flags
- Do not define `HAVE_ARM_BF16_CPU_DEFINITION` because in order to compile SVE dispatch, both SVE and BF16 capabilities must be present

Pull Request resolved: https://github.com/pytorch/pytorch/pull/176247
Approved by: https://github.com/atalman, https://github.com/Skylion007
 2026-03-03 17:10:55 +00:00
Nikita Shulga 511d3316ab [BE] Remove opentelemetry submodule (#172458) 
It was added by https://github.com/pytorch/pytorch/pull/122999/ but never actually used, to the best of my knowledge.

Put temporary hack into `macos-build.sh` to workaround the issue with checkout action leaving removed submodules in place (to be removed in a month from now), by running following ahead of the build
```shell
if [[ -d third_party/opentelemetry-cpp ]]; then
  rm -rf third_party/opentelemetry-cpp
fi
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/172458
Approved by: https://github.com/seemethere, https://github.com/ezyang
 2026-01-15 23:26:36 +00:00
PyTorch MergeBot 5f360e5301 Revert "[BE] Remove opentelemetry submodule (#172458)" 
This reverts commit 4c78f40245.

Reverted https://github.com/pytorch/pytorch/pull/172458 on behalf of https://github.com/yangw-dev due to sorry but this breaks the trunk due to alueError: could not identify license file for /Users/ec2-user/runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg/ports/sigslot, see https://hud.pytorch.org/pytorch/pytorch/commit/4c78f40245d03722d1880ceb6a09d2f95712ca57#60406597123-box ([comment](https://github.com/pytorch/pytorch/pull/172458#issuecomment-3752093607))
 2026-01-14 23:01:44 +00:00
Nikita Shulga 4c78f40245 [BE] Remove opentelemetry submodule (#172458) 
It was added by https://github.com/pytorch/pytorch/pull/122999/ but never actually used, to the best of my knowledge
Pull Request resolved: https://github.com/pytorch/pytorch/pull/172458
Approved by: https://github.com/seemethere, https://github.com/ezyang
 2026-01-14 21:56:18 +00:00
chenlang df8b6bd5a3 support fp16 shgemm under openblas (#169042) 
# Purpose
This PR is to support fp16 shgemm under openblas. We conducted tests using vLLM on the following platform.
With using this patch, vLLM demonstrates faster inference speed under fp16.

**Platform info:**
Architecture:             riscv64
  Byte Order:             Little Endian
CPU(s):                   64
  On-line CPU(s) list:    0-63
Vendor ID:                0x5b7
  BIOS Vendor ID:         SOPHGO
  Model name:             -
    BIOS Model name:      SG2044 Not Set CPU @ 2.6GHz
    BIOS CPU family:      513
    CPU family:           0x80000000090c0d00
    Model:                0x2047000
    Thread(s) per core:   1
    Core(s) per socket:   64
    Socket(s):            1
    Frequency boost:      disabled
    CPU(s) scaling MHz:   100%
    CPU max MHz:          2600.0000
    CPU min MHz:          1000.0000
Caches (sum of all):
  L1d:                    4 MiB (64 instances)
  L1i:                    4 MiB (64 instances)
  L2:                     32 MiB (16 instances)
  L3:                     64 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:   Not affected
  Itlb multihit:          Not affected
  L1tf:                   Not affected
  Mds:                    Not affected
  Meltdown:               Not affected
  Mmio stale data:        Not affected
  Reg file data sampling: Not affected
  Retbleed:               Not affected

ISA: rv64imafdcv_zicbom_zicboz_zicntr_zicond_zicsr_zifencei_zihintntl_zihintpause_zihpm_zawrs_zfa_zfh_zfhmin_zca_zcb_zcd_zba_zbb_zbc_zbs_zve32f_zve32x_zve64d_zve64f_zve64x_zvfh_zvfhmin_sscofpmf_sstc_svinval_svnapot_svpbmt

**Branch**
openblas: develop
torch: develop
vllm: main

# Test Plan
Base: without this PR
Pytorch use OpenBLAS FP16 GEMM: use this PR

**Base**
export VLLM_CPU_OMP_THREADS_BIND=0-63
export VLLM_CPU_KVCACHE_SPACE=60

vllm bench latency \
  --model /home/models/Qwen2.5-7B-Instruct \
  --tensor-parallel-size 1\
  --dtype float16 \
  --input-len 16 \
  --output-len 16 \
  --enforce-eager \
  --max-model-len 8192 \
  --max-num-batched-tokens 8192 \
  --batch-size 1 \
  --n 1 \
  --num-iters-warmup 5 \
  --num-iters 8 \
  --seed 42 \
  --output-json ./latency_results_fp16_latency_base.json

**Pytorch use OpenBLAS FP16 GEMM**
export VLLM_CPU_OMP_THREADS_BIND=0-63
export VLLM_CPU_KVCACHE_SPACE=60

vllm bench latency \
  --model /home/models/Qwen2.5-7B-Instruct \
  --tensor-parallel-size 1\
  --dtype float16 \
  --input-len 16 \
  --output-len 16 \
  --enforce-eager \
  --max-model-len 8192 \
  --max-num-batched-tokens 8192 \
  --batch-size 1 \
  --n 1 \
  --num-iters-warmup 5 \
  --num-iters 8 \
  --seed 42 \
  --output-json ./latency_results_fp16_latency_with_openblas_support.json

# Result
**Base**
{
    "avg_latency": 62.53946338250171,
    "latencies": [
        58.46783778001554,
        58.230652199999895,
        58.335780619992875,
        59.77051957999356,
        58.587668860011036,
        59.31567866000114,
        58.460076240007766,
        89.14749311999185
    ],
    "percentiles": {
        "10": 58.30424209399498,
        "25": 58.42900233500404,
        "50": 58.52775332001329,
        "75": 59.429388889999245,
        "90": 68.58361164199304,
        "99": 87.09110497219196
    }
}

**Pytorch use OpenBLAS FP16 GEMM**
{
    "avg_latency": 32.42863222499727,
    "latencies": [
        30.742418120033108,
        33.67000828002347,
        29.747197599965148,
        32.11275753995869,
        34.566938299976755,
        30.849812360014766,
        34.46360486000776,
        33.27632073999848
    ],
    "percentiles": {
        "10": 30.44385196401272,
        "25": 30.82296380001935,
        "50": 32.69453913997859,
        "75": 33.86840742501954,
        "90": 34.49460489199846,
        "99": 34.55970495917892
    }
}

Pull Request resolved: https://github.com/pytorch/pytorch/pull/169042
Approved by: https://github.com/aditew01, https://github.com/albanD
 2025-12-11 12:59:48 +00:00
fengqing.lu 5f21d27e71 [xpu][feature] Upgrade XPU OneDNN to v3.10.2 (#169443) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/169443
Approved by: https://github.com/EikanWang
 2025-12-03 10:24:44 +00:00
zhangfei 0ea545b241 Add support to enable the oneDNN backend for RISC-V (#166602) 
# Description
Currently, oneDNN can be successfully compiled and run on RISC-V : [oneDNN riscv](https://github.com/uxlfoundation/oneDNN/pull/2929) , and it has also introduced extensive RVV optimizations: [oneDNN/src/cpu/rv64](https://github.com/uxlfoundation/oneDNN/tree/main/src/cpu/rv64) . Therefore, this PR adds support for enabling the oneDNN backend on the RISC-V architecture. Although the current dependency package ideep uses a relatively lower version of oneDNN. and not yet sufficient for successful compilation, this is only a matter of time—once ideep is updated, compilation will work as expected.

Below are my test results on RISC-V:
```bash
$ uname -m
riscv64
$ cat test.py
import torch
print(torch.backends.mkldnn.is_available())
$ python3 test.py
True
```

Pull Request resolved: https://github.com/pytorch/pytorch/pull/166602
Approved by: https://github.com/malfet
 2025-11-20 22:46:22 +00:00
Luo, Yu 4de24bcc56 [Fix XPU typo] Fix a comment typo of FindSYCLToolkit.cmake (#165884) 
The character U+ff1a "：" could be confused with the ASCII character U+003a ":", which is more common in source code.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/165884
Approved by: https://github.com/cyyever, https://github.com/guangyey, https://github.com/EikanWang
 2025-11-13 12:32:48 +00:00
Sv. Lockal 066c5c57a9 Fix typo in gloo_hip library name (#166502) 
The typo was never noticed; conditions to enable it require system gloo: `-DUSE_SYSTEM_GLOO=ON -DUSE_GLOO=ON -DUSE_DISTRIBUTED=ON -DUSE_ROCM=ON`.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/166502
Approved by: https://github.com/jerryzh168, https://github.com/cyyever
 2025-11-05 04:14:01 +00:00
Joseph Macaranas 92381a5aa7 [ROCm] Custom OpenBLAS library name (#166333) 
- TheRock build system for ROCm builds OpenBLAS from source and uses a custom name for the library.
- Following existing conventions in `FindOpenBLAS.cmake` to support finding a custom named version of OpenBLAS.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/166333
Approved by: https://github.com/jeffdaily
 2025-10-27 22:13:05 +00:00
Yuanyuan Chen 3dfd0c7584 Improve PATH hints in FindvecLib.cmake (#165881) 
Change  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk to /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk in `cmake/Modules/FindvecLib.cmake` which is more general (and MacOSX10.9 is not supported now). Otherwise, vecLib can't be found on MacOS 26.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/165881
Approved by: https://github.com/ezyang
 2025-10-21 16:44:12 +00:00
fengqing.lu acece97c3a [Intel GPU] Upgrade OneDNN XPU Tag to v3.9.1 (#161932) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/161932
Approved by: https://github.com/EikanWang, https://github.com/Skylion007, https://github.com/guangyey
 2025-09-04 11:05:10 +00:00
Nathan Brown 93da9952a7 gloo: fix building system gloo with CUDA/HIP (#146637) 
Fix incorrect linking of Gloo's libraries when building with system Gloo. Previously, either Gloo's native library or Gloo's CUDA library were linked. However, Gloo had changed such that all users of Gloo must link the native library, and can optionally link the CUDA or HIP library for Gloo + CUDA/HIP support.
This had been updated when building/linking with vendored Gloo, but not when using system Gloo.

Fixes: #146239

Reported-by: Adam J Stewart <ajstewart426@gmail.com>

Pull Request resolved: https://github.com/pytorch/pytorch/pull/146637
Approved by: https://github.com/malfet
 2025-08-06 22:56:31 +00:00
cyy 3ee8828c87 [1/N] Don't use CUDA.cmake module (#157188) 
Small changes before removing CUDA.cmake.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/157188
Approved by: https://github.com/ezyang
 2025-07-08 03:05:35 +00:00
Isuru Fernando 8f0998aafe Check F2C BLAS for OpenBLAS and other vendors (#143846) 
This issue came from https://github.com/conda-forge/pytorch-cpu-feedstock/issues/180. MKL follows the F2C convention for returning single precision floats as doubles and uses the G77 convention for returning complex valued scalars. OpenBLAS does the opposite. There is a check for this already, but it's done only when the Generic BLAS vendor code path is used and this PR moves that code to `Dependencies.cmake` to make it work when the BLAS vendor is OpenBLAS and others

Pull Request resolved: https://github.com/pytorch/pytorch/pull/143846
Approved by: https://github.com/rgommers, https://github.com/atalman
 2025-07-01 05:56:24 +00:00
Avanish Tiwari 5e18bc3331 [PowerPC] Fixed build issue for vsx vec256 complexfloat and scaled_mm_out_cpu (#155255) 
Pytorch build is failing on power system from this commit ec24f8f58a

***Build Failure Logs***

**Error related to mkldnn**
```
pytorch/aten/src/ATen/native/Blas.cpp:302:26: error: ‘cpuinfo_has_x86_amx_int8’ was not declared in this scope
  302 |     if ((!mixed_dtype && cpuinfo_has_x86_amx_int8()) ||
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~
pytorch/aten/src/ATen/native/Blas.cpp:303:25: error: ‘cpuinfo_has_x86_amx_fp16’ was not declared in this scope
  303 |         (mixed_dtype && cpuinfo_has_x86_amx_fp16())) {
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~

```

**Error related to vec256 complex float redefinition**
```
aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:19:7: error: specialization of ‘at::vec::DEFAULT::Vectorized<c10::complex<float> >’ after instantiation
   19 | class Vectorized<ComplexFlt> {
      |       ^~~~~~~~~~~~~~~~~~~~~~
aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:19:7: error: redefinition of ‘class at::vec::DEFAULT::Vectorized<c10::complex<float> >’

aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:633:18: error: ‘const class at::vec::DEFAULT::Vectorized<c10::complex<float> >’ has no member named ‘abs_2_’
  633 |   auto abs_a = a.abs_2_();
      |                  ^~~~~~
aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:634:18: error: ‘const class at::vec::DEFAULT::Vectorized<c10::complex<float> >’ has no member named ‘abs_2_’
  634 |   auto abs_b = b.abs_2_();
      |                  ^~~~~~

/aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:666:17: error: ‘const class at::vec::DEFAULT::Vectorized<c10::complex<float> >’ has no member named ‘vec0’
  666 |       vec_add(a.vec0(), b.vec0()), vec_add(a.vec1(), b.vec1())};
aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:673:17: error: ‘const class at::vec::DEFAULT::Vectorized<c10::complex<float> >’ has no member named ‘vec0’
  673 |       vec_sub(a.vec0(), b.vec0()), vec_sub(a.vec1(), b.vec1())};
      |                 ^~~~
aten/src/ATen/cpu/vec/vec256/vsx/vec256_complex_float_vsx.h:680:27: error: ‘const class at::vec::DEFAULT::Vectorized<c10::complex<float> >’ has no member named ‘vec0’
  680 |       vec_and(a.vec0(), b.vec0()), vec_and(a.vec1(), b.vec1())};
```

***With  this changes build logs***
```
Building wheel torch-2.8.0a0+gita3098a7
-- Building version 2.8.0a0+gita3098a7
-- Checkout nccl release tag: v2.26.5-1
cmake -GNinja -DBLAS=OpenBLAS -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/avanish/OfficeWork2025/JuneWork/pytorch_5Jun/pack/torch_night_5Jun/pytorch/torch -DCMAKE_PREFIX_PATH=/home/avanish/OfficeWork2025/JuneWork/pyenv/pytorch_5Jun/lib/python3.12/site-packages -DPython_EXECUTABLE=/home/avanish/OfficeWork2025/JuneWork/pyenv/pytorch_5Jun/bin/python -DTORCH_BUILD_VERSION=2.8.0a0+gita3098a7 -DUSE_MKLDNN=ON -DUSE_MKLDNN_CBLAS=ON -DUSE_NUMPY=True -DUSE_OPENMP=ON /home/avanish/OfficeWork2025/JuneWork/pytorch_5Jun/pack/torch_night_5Jun/pytorch
cmake --build . --target install --config Release
running build_ext
-- Building with NumPy bindings
-- Not using cuDNN
-- Not using CUDA
-- Not using XPU
-- Using MKLDNN
-- Not using Compute Library for the Arm architecture with MKLDNN
-- Using CBLAS in MKLDNN
-- Not using NCCL
-- Building with distributed package:
  -- USE_TENSORPIPE=True
  -- USE_GLOO=True
  -- USE_MPI=False
-- Building Executorch
-- Not using ITT
Copying functorch._C from functorch/functorch.so to /home/avanish/OfficeWork2025/JuneWork/pytorch_5Jun/pack/torch_night_5Jun/pytorch/build/lib.linux-ppc64le-cpython-312/functorch/_C.cpython-312-powerpc64le-linux-gnu.so
copying functorch/functorch.so -> /home/avanish/OfficeWork2025/JuneWork/pytorch_5Jun/pack/torch_night_5Jun/pytorch/build/lib.linux-ppc64le-cpython-312/functorch/_C.cpython-312-powerpc64le-linux-gnu.so
building 'torch._C' extension
creating build/temp.linux-ppc64le-cpython-312/torch/csrc

```

This patch will fix the pytorch build issue on power, and i am able to build successfully.

Hi @malfet  @albanD

Please review this PR for pytorch build issue that we are observing on power.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/155255
Approved by: https://github.com/albanD, https://github.com/malfet
 2025-06-30 17:54:37 +00:00
PyTorch MergeBot b1d62febd0 Revert "Use official CUDAToolkit module in CMake (#154595)" 
This reverts commit 08dae945ae.

Reverted https://github.com/pytorch/pytorch/pull/154595 on behalf of https://github.com/malfet due to It breaks on some local setup with no clear diagnostic, but looks like it fails to find cuFile ([comment](https://github.com/pytorch/pytorch/pull/154595#issuecomment-2997959344))
 2025-06-23 21:15:31 +00:00