Adds `torch::stable::from_file` to the stable ABI. Needed for torchvision stable-ABI, which currently carries a local [stable_from_file helper](https://github.com/pytorch/vision/blob/7b0e250acf82aac5a2389f54c6855da17bfeace9/torchvision/csrc/io/image/cpu/read_write_file.cpp#L21-L40) for image decoding.
## Summary
- Introduces `torch::stable::from_file` and it returns a 1-D tensor backed by a memory-map of the file.
- Functional differences between the ATen and stable signatures include `filename` being `const std::string&` rather than `std::string_view`.
- Adds my_from_file to the libtorch_agn_2_10 extension and tests mapping the entire file with shared=False and dtype as well as checking the returned bytes against the original data.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/198263
Approved by: https://github.com/janeyx99
Moves the accumulate-type traits into `torch::headeronly`. Needed for torchvision stable-ABI, which currently carries a local [AccType helper](https://github.com/pytorch/vision/blob/9769007f09193b307a90d2b833aac31d364f03e3/torchvision/csrc/ops/cuda/nms_kernel.cu#L27-L38) that only covers the types it dispatches.
## Summary
- Introduces `acc_type`, `acc_type_device`, `AccumulateType` and `AccumulateTypeDevice` in the `torch::headeronly` namespace.
## BC-breaking note
### `at::AccumulateTypeDevice` can no longer be specialized inside namespace `at`
Moves the trait definition to `torch::headeronly` so `at::AccumulateTypeDevice` is now a using declaration. Code that specializes it inside `namespace at` no longer compiles.
Previously a specialization compiled and `at::acc_type` returned the type it named. To get the previous behavior, specialize in `torch::headeronly` instead.
Example (the same applies to `at::AccumulateType`) :
Version 2.14.0:
```cpp
namespace at {
template <>
struct AccumulateTypeDevice<MyType, c10::DeviceType::CPU> {
using type = double;
};
}
```
Version 2.15.0:
```cpp
// The code above fails with:
// error: specialization of 'template<class T, c10::DeviceType D> struct torch::headeronly::AccumulateTypeDevice' in different namespace
// Specialize it by its torch::headeronly name:
template <>
struct torch::headeronly::AccumulateTypeDevice<MyType, c10::DeviceType::CPU> {
using type = double;
};
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/199203
Approved by: https://github.com/janeyx99
authored with codex
as discussed w/ @eellison , @ngimel ,~~~ just stashing this prototype here as performance doesn't look great on the hot path:~~~
```
Latest benchmark results:
Backend Shape (M,N,K) Cached Eager Delta
━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━
cuBLAS (64,64,64) 14.807 µs 15.092 µs +0.284 µs / +1.92%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (256,256,256) 14.904 µs 15.301 µs +0.397 µs / +2.66%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (1024,1024,1024) 14.850 µs 15.363 µs +0.513 µs / +3.46%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (32,2048,10944) 16.777 µs 17.626 µs +0.849 µs / +5.06%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (64,64,64) 14.781 µs 15.181 µs +0.399 µs / +2.70%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (256,256,256) 14.850 µs 15.099 µs +0.249 µs / +1.68%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (1024,1024,1024) 14.718 µs 14.792 µs +0.074 µs / +0.50%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (32,2048,10944) 16.715 µs 17.247 µs +0.532 µs / +3.18%
Paste-ready discussion summary—if this goes on GitHub, please add your own commentary around the quoted AI-generated material:
> AI-generated benchmark summary, provided for human review
>
> Benchmarked cached versus operation-scoped eager cuBLAS workspaces on an NVIDIA GB300, CC 10.3, CUDA 13.4. PyTorch was built for CC 10.0. Measurements are medians across five alternating cached/eager process trials using synchronized
> torch.utils.benchmark.Timer measurements of BF16 torch.mm.
>
> Eager workspace allocation added approximately 0.07–0.85 µs per operation. Relative overhead ranged from 0.5% to 5.1%, with the largest measured overhead on the legacy cuBLAS (M,N,K)=(32,2048,10944) regression shape.
>
> The cached implementation reproduced CUDA_ERROR_ILLEGAL_ADDRESS when separately captured graphs reused a workspace on the same stream. The eager implementation passed that reproducer.
>
> Given that most of the eager overhead appears to come from caching-allocator allocation/free bookkeeping, a simpler alternative under consideration is to retain the existing workspace cache but explicitly clear or isolate cached workspaces at
> graph-capture boundaries.
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/194311
Approved by: https://github.com/ngimel, https://github.com/eellison
authored with codex
as discussed w/ @eellison , @ngimel ,~~~ just stashing this prototype here as performance doesn't look great on the hot path:~~~
```
Latest benchmark results:
Backend Shape (M,N,K) Cached Eager Delta
━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━
cuBLAS (64,64,64) 14.807 µs 15.092 µs +0.284 µs / +1.92%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (256,256,256) 14.904 µs 15.301 µs +0.397 µs / +2.66%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (1024,1024,1024) 14.850 µs 15.363 µs +0.513 µs / +3.46%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLAS (32,2048,10944) 16.777 µs 17.626 µs +0.849 µs / +5.06%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (64,64,64) 14.781 µs 15.181 µs +0.399 µs / +2.70%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (256,256,256) 14.850 µs 15.099 µs +0.249 µs / +1.68%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (1024,1024,1024) 14.718 µs 14.792 µs +0.074 µs / +0.50%
────────── ────────────────── ─────────── ─────────── ────────────────────
cuBLASLt (32,2048,10944) 16.715 µs 17.247 µs +0.532 µs / +3.18%
Paste-ready discussion summary—if this goes on GitHub, please add your own commentary around the quoted AI-generated material:
> AI-generated benchmark summary, provided for human review
>
> Benchmarked cached versus operation-scoped eager cuBLAS workspaces on an NVIDIA GB300, CC 10.3, CUDA 13.4. PyTorch was built for CC 10.0. Measurements are medians across five alternating cached/eager process trials using synchronized
> torch.utils.benchmark.Timer measurements of BF16 torch.mm.
>
> Eager workspace allocation added approximately 0.07–0.85 µs per operation. Relative overhead ranged from 0.5% to 5.1%, with the largest measured overhead on the legacy cuBLAS (M,N,K)=(32,2048,10944) regression shape.
>
> The cached implementation reproduced CUDA_ERROR_ILLEGAL_ADDRESS when separately captured graphs reused a workspace on the same stream. The eager implementation passed that reproducer.
>
> Given that most of the eager overhead appears to come from caching-allocator allocation/free bookkeeping, a simpler alternative under consideration is to retain the existing workspace cache but explicitly clear or isolate cached workspaces at
> graph-capture boundaries.
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/194311
Approved by: https://github.com/ngimel, https://github.com/eellison
This is needed so that downstream targets inherits the c++20 requirement.
This change propagates the required standard only to CXX translation units; downstream consumers compiling CUDA sources must explicitly set CMAKE_CUDA_STANDARD (for example, to 20), because the Torch target does not propagate a CUDA standard.
Without it, downstream C++ code fails with for example:
```
pytorch/torch/include/ATen/core/TensorBase.h:1041:5: note: ‘requires’ only available with ‘-std=c++20’ or ‘-fconcepts’ pytorch/torch/include/ATen/core/TensorBase.h:1044:5: error: ‘requires’ does not name a type
1044 | requires std::is_same_v<T, int64_t>
| ^~~~~~~~
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/190784
Approved by: https://github.com/Skylion007
## Summary
- Add `torch::stable::{permute,view_dtype}` wrappers (gated at 2.14) via `torch_call_dispatcher`
- Document the new APIs in the stable operators docs
- Cover them in the 2.14 libtorch-agnostic extension tests
~~Stacked on #191973. (testing new gh stack)~~
stack seemed to be disabled. Will rebase this when #191973 lands
## Test plan
- [x] `python -m pytest test/cpp_extensions/test_libtorch_agnostic.py -k 'test_my_permute or test_my_view_dtype'`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/192083
Approved by: https://github.com/janeyx99
## Summary
- Add `torch::stable::{bitwise_and,bitwise_or,bitwise_left_shift,bitwise_right_shift}` Tensor–Tensor wrappers (gated at 2.14) via `torch_call_dispatcher`
- Document the new APIs in the stable operators docs
- Cover them in the 2.14 libtorch-agnostic extension tests (same-shape + broadcasting)
## Test plan
- [x] `python -m pytest test/cpp_extensions/test_libtorch_agnostic.py -k test_my_bitwise_ops`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/191973
Approved by: https://github.com/janeyx99
Co-authored-by: Jane (Yuan) Xu <31798555+janeyx99@users.noreply.github.com>
Addresses the 404 link-rot portion of [#189557](https://github.com/pytorch/pytorch/issues/189557).
This PR updates or removes stale URLs reported by the nightly lint-urls job:
- Updates moved PyTorch documentation links to current working pages.
- Removes hyperlinks where the referenced page no longer has a reliable replacement.
- Updates the ROCm PR reference from `ROCm/clr` to `ROCm/rocm-systems`.
- Leaves reachable links unchanged, including the `pytorch/data` issue link and the `hud.pytorch.org` URLs.
The `hud.pytorch.org` 000 failures are left unchanged because the issue notes they may be transient/rate-limit related rather than dead links. I did not add `@lint-ignore` for those URLs in this PR; if maintainers prefer suppressing those flaky checks, I can follow up separately.
Validation:
- Manually rechecked the originally reported URLs.
- Verified the stale 404 URLs are present on `upstream/main`.
- Verified the same stale URL patterns are absent on `fix-stale-doc-links`.
- Ran `git diff --check upstream/main...HEAD`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/189619
Approved by: https://github.com/albanD
Adds aoti_torch_stream_native_handle and torch::stable::accelerator::Stream::nativeHandle() behind TORCH_FEATURE_VERSION >= 2.13, wires stable/version.h into the AOTI shim header for correct guards, and covers the behavior with a libtorch_agn_2_13 extension test plus a CUDA test_libtorch_agnostic case.
@janeyx99
Pull Request resolved: https://github.com/pytorch/pytorch/pull/183930
Approved by: https://github.com/janeyx99
## Summary
Fixes#167467.
The C++ `TensorOptions` example in `docs/cpp/source/api/aten/creation.md` explicitly sets `.dtype(torch::kFloat32)`, which is exactly the documented default listed in the immediately-following **Defaults** section of the same page. As a result the example doesn't actually demonstrate the point of `TensorOptions` — overriding the defaults — for the `dtype` axis.
This PR switches the example (and its matching assertion) to `kFloat64` so it shows a real override of the `kFloat32` default, while leaving `layout` / `device` / `requires_grad` overrides untouched.
## Problem
`docs/cpp/source/api/aten/creation.md` currently has:
```cpp
auto options =
torch::TensorOptions()
.dtype(torch::kFloat32) // <- same as the documented default
.layout(torch::kStrided)
.device(torch::kCUDA, 1)
.requires_grad(true);
torch::Tensor tensor = torch::full({3, 4}, /*value=*/123, options);
assert(tensor.dtype() == torch::kFloat32);
```
…and 7 lines later the same page says:
> **Defaults:** Any axis you omit takes its default value: `kFloat32` for dtype, `kStrided` for layout, `kCPU` for device, and `false` for `requires_grad`.
So the only axes the example actually *overrides* are `device` and `requires_grad`; the `.dtype(...)` call is a no-op vs. the documented default.
## Solution
Change both the example call and the assertion from `kFloat32` to `kFloat64`. That makes `.dtype(...)` an actual override, matching the spirit of the surrounding prose ("creating a `TensorOptions` object" by configuring its axes). `kFloat64` is the same dtype suggested in @malfet's review of PR #167474 ("better example would be to change value of `dtype` to `kFloat64`") and is what PR #171406 implemented before going stale.
## Prior art
Two earlier attempts on this issue were approved on substance but couldn't merge:
- #167474 — first attempt, flipped the surrounding prose ("64-bit float" → "32-bit float") instead of the code. Approved by @malfet who suggested the inverse: change the example dtype to `kFloat64`. Went stale.
- #171406 — implemented exactly the `kFloat32` → `kFloat64` change here. Approved by @jbschlosser, who tried to merge it (`@pytorchbot merge`), but the merge failed on EasyCLA: `The commit ... is not authorized under a signed CLA`. PR then went stale.
The original prose "64-bit float, strided tensor that requires a gradient, and lives on CUDA device 1" that #167467 originally cited was rewritten by the C++ docs restructure (#174096), so the wording mismatch is already gone — but the example still uses the default `dtype`, which is the substantive thing the issue and the maintainer feedback were after.
## Out of scope
- The **Defaults** section a few lines below still correctly documents `kFloat32` as the dtype default — left untouched.
- The "shorthand" examples further down the page (`torch::ones(10, torch::kFloat32)`, etc.) intentionally use `kFloat32` to show the shorthand syntax against the default and are unrelated to the "override the default" point. Left untouched.
## Test plan
- [ ] Docs-only change to a single Markdown code block — no Python/C++ behavior changes, no test changes needed.
- [ ] CI docs build / link-check to confirm the C++ docs page still renders.
## Related
- #167467 — original issue (still open)
- #167474 — prior attempt, approved by @malfet with the kFloat64 suggestion
- #171406 — prior attempt, approved by @jbschlosser, merge blocked on EasyCLA, went stale
- #174096 — C++ docs restructure that moved this content into `docs/cpp/source/api/aten/creation.md` and rewrote the surrounding prose
Pull Request resolved: https://github.com/pytorch/pytorch/pull/184457
Approved by: https://github.com/malfet
Fixes#180492.
The stable ABI docs pointed users at `torch/csrc/stable/tensor_struct.h` and `device_struct.h`, which only contain forward declarations for constructors that are defined `inline` in the matching `*_inl.h` files. Following the docs led to a linker error such as:
```
undefined reference to `torch::stable::Device::Device(std::string const&)'
```
Update the docs to point at the umbrella headers `tensor.h` and `device.h`, which include both the struct and the inl headers and match what `test/cpp_extensions` already uses. Also add an entry for `device.h` in the header list in `libtorch_stable_abi.md`, which was previously missing.
## Test plan
Pure docs change; relying on the docs-preview build in CI to confirm rendering.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/181393
Approved by: https://github.com/janeyx99
The fix-cpp-build branch is a major restructuring that:
- Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives
- Reorganizes notes into a hierarchical API reference with proper categories
- Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs
- Always runs Doxygen instead of gating on an env var
Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096
Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD
This PR adds two main parts:
- shim.h stable C APIs into torch::Library APIs
- a higher level API in torch/csrc/stable/library.h that calls into this shim.h + otherwise is self contained
Goal: custom kernel writers should be able to call the apis in the directories above in order to register their library in a way that allows their custom extension to run with a different libtorch version than it was built with.
Subplots resolved:
- Do we want a whole separate StableLibrary or do we want to freeze torch::Library and add `m.stable_impl(cstring, void (*fn)(void **, int64_t, int64_t)` into it
- Yes, we want a separate StableLibrary. We cannot freeze Library and it is NOT header only.
- Should I use unint64_t as the common denominator instead of void* to support 32bit architectures better?
- Yes, and done
- Should I add a stable `def` and `fragment` when those can be done in python?
- I think we do want these --- and now they're done
- Where should library_stable_impl.cpp live? -- no longer relevant
- I need some solid test cases to make sure everything's going ok. I've intentionally thrown in a bunch of random dtypes into the signature, but I still haven't tested returning multiple things, returning nothing, complex dtypes, etc.
- Have since tested all the torch library endpoints. the others can be tested in a followup to separate components that need to be in shim.h vs can be added later
Pull Request resolved: https://github.com/pytorch/pytorch/pull/148124
Approved by: https://github.com/albanD, https://github.com/zou3519, https://github.com/atalman
This PR adds two main parts:
- shim.h stable C APIs into torch::Library APIs
- a higher level API in torch/csrc/stable/library.h that calls into this shim.h + otherwise is self contained
Goal: custom kernel writers should be able to call the apis in the directories above in order to register their library in a way that allows their custom extension to run with a different libtorch version than it was built with.
Subplots resolved:
- Do we want a whole separate StableLibrary or do we want to freeze torch::Library and add `m.stable_impl(cstring, void (*fn)(void **, int64_t, int64_t)` into it
- Yes, we want a separate StableLibrary. We cannot freeze Library and it is NOT header only.
- Should I use unint64_t as the common denominator instead of void* to support 32bit architectures better?
- Yes, and done
- Should I add a stable `def` and `fragment` when those can be done in python?
- I think we do want these --- and now they're done
- Where should library_stable_impl.cpp live? -- no longer relevant
- I need some solid test cases to make sure everything's going ok. I've intentionally thrown in a bunch of random dtypes into the signature, but I still haven't tested returning multiple things, returning nothing, complex dtypes, etc.
- Have since tested all the torch library endpoints. the others can be tested in a followup to separate components that need to be in shim.h vs can be added later
Pull Request resolved: https://github.com/pytorch/pytorch/pull/148124
Approved by: https://github.com/albanD, https://github.com/zou3519
Fixes#113124.
## Description
I modified the installing.rst file to address the system requirements and troubleshooting steps for using LibTorch with different GLIBC versions.
### Summary of Changes
- Added system requirements specifying the GLIBC version needed for both the cxx11 ABI version and the pre-cxx11 ABI version of LibTorch.
- Included a troubleshooting section with instructions on how to check the dependencies of the LibTorch libraries and identify the required GLIBC version using the `ldd lib/libtorch.so` command.
## Checklist
- [X] The issue that is being fixed is referred in the description
- [X] Only one issue is addressed in this pull request
- [X] Labels from the issue that this PR is fixing are added to this pull request
- [X] No unnecesary issues are included into this pull request
Pull Request resolved: https://github.com/pytorch/pytorch/pull/128135
Approved by: https://github.com/jbschlosser
Adrian Abeyta