## Author Notes
Pretty mechanical, nothing special here.
## Agent Notes
Written by Claude Code:
> The two test directories are 48 findings, and they come apart cleanly: 43 are
> stubs and error checks that convert mechanically, and 5 are throws whose type is
> the point of the test they sit in. Getting those 5 wrong would leave the suite
> green while testing the wrong thing, which is the only real risk in this PR.
>
> The 43: `rng_extension.cpp` and `cpp_c10d_extension.cpp` are "not implemented" /
> "does not support" overrides (8 and 10); `aoti_inference/test.cpp` is CUDA API
> error checks (16); the rest are `aoti_custom_class.cpp`,
> `test_custom_class_registrations.cpp` (6, a toy stack machine's error paths) and
> the two `my_from_blob_*` extensions. None of their messages is asserted on
> anywhere - I grepped the tree for each, including the Python suites.
>
> The two `my_from_blob_*` sites get STD_TORCH_CHECK rather than TORCH_CHECK.
> Those TUs include only `torch/csrc/stable/*`, which reaches
> torch/headeronly/util/Exception.h but never c10/util/Exception.h, so TORCH_CHECK
> is not declared there at all. STD_TORCH_CHECK preserves the exception type
> exactly - it throws std::runtime_error, same as the raw throw - but not the
> message: like the serialization-codegen PR earlier in this stack, it prepends
> __func__, __FILE__ and __LINE__, so
>
> before: cudaMalloc failed
> after: my_from_blob_with_cuda_deleter, <path>/my_from_blob_with_deleter.cpp:73,
> cudaMalloc failed
>
> These are the only two sites in this PR where anything observable moves. Nothing
> asserts on either string, both are behind #ifdef LAE_USE_CUDA in a test
> extension, and STD_TORCH_CHECK is already the house style in the sibling
> libtorch_agn_2_* extensions (17 uses), so their messages are decorated the same
> way. The alternative was to leave these two raw and annotated; taking the
> decoration keeps the file consistent with its neighbours.
>
> The 5 that stay raw, with the reason on each site:
>
> ProcessGroupNCCLErrorsTest.cpp TestHeartbeatMonitor::runLoop, 16 lines
> above, catches std::runtime_error by name
> to set hasMonitorThreadCaughtError_. A
> c10::Error is not caught by that, and the
> throw happens on the monitor thread.
> dataloader.cpp The test rethrows e.original_exception and
> asserts it is still a std::invalid_argument
> - the point being that WorkerException
> round-trips the original type.
> parallel.cpp ParallelApplyRethrowsException is about a
> user exception escaping forward().
> FileStoreTest.cpp std::system_error carrying errno as a
> std::error_code.
> OpenRegException.cpp Reports a caller-supplied SourceLocation;
> a TORCH_CHECK there would stamp this
> file's own location instead.
>
> Two of those five are load-bearing in a way I could measure rather than argue,
> and the test plan shows the failures. `parallel.cpp` is the one that is not:
> its assertion is `ASSERT_THROWS_WITH(..., "Badness!")`, a substring match that
> would still pass after conversion. It is annotated for intent, not because it
> would break, and it seemed worth being explicit about that difference rather
> than implying all five were equally forced.
>
> The exclude list goes from 11 entries to 9.
>
> Test Plan:
>
> Half of this is CUDA or distributed and does not build here, so the two CUDA
> files - parallel.cpp and ProcessGroupNCCLErrorsTest.cpp - deliberately received
> comment-only edits, which is the safest possible change to code a configuration
> cannot compile.
>
> For the rest, what matters is whether the *changed lines* were compiled, not the
> files: three of them sit behind preprocessor guards that a plain CPU build skips
> straight over. `aoti_inference/test.cpp`'s 16 conversions are inside
> `#if defined(USE_CUDA) || defined(USE_ROCM)`, and the two `my_from_blob_*` sites
> are inside `#ifdef LAE_USE_CUDA`, which setup.py only defines when CUDA is
> available. The two my_from_blob sites were compiled with the guard forced on,
> against a stub cuda_runtime.h. There is no CUDA toolkit here, so the
> aoti_inference block was not compiled; what I checked instead is that
> `<torch/torch.h>` is included unconditionally above it and does provide
> TORCH_CHECK, and that the block contained no TORCH_CHECK before this change, so
> the macro's availability there is an argument rather than a measurement. Every
> other converted line is unguarded.
>
> BUILD_TEST=0 does not prevent compiling these directly:
>
> aoti_inference/test.cpp syntax-only (needs gmock headers)
> aoti_custom_class.cpp syntax-only
> dataloader.cpp 87 tests passed
> test_custom_class_registrations.cpp compiles and links (no TESTs of its own)
> FileStoreTest.cpp syntax-only
> cpp_c10d_extension.cpp syntax-only (needs Python.h, pybind11)
> rng_extension.cpp syntax-only (same)
> my_from_blob_with_deleter.cpp syntax-only
> my_from_blob_with_lambda_deleter.cpp syntax-only
> OpenRegException.cpp syntax-only (needs the openreg headers)
>
> ```
> g++ -std=c++20 <includes> test/cpp/api/dataloader.cpp gtest-all.o gtest_main.o \
> -Lbuild/lib -lc10 -ltorch_cpu -lpthread && ./a.out
> [ PASSED ] 87 tests.
> ```
>
> Positive control on the dataloader annotation - the same file with the throw
> converted:
>
> ```
> [ FAILED ] DataLoaderTest.TestExceptionsArePropagatedFromWorkers
> ```
>
> So that annotation is required, not preference.
>
> ```
> lintrunner --take RAWTHROW --all-files
> spin lint
> ```
>
> Both clean; test/cpp and test/cpp_extensions go from 48 findings to 0.
> CLANGFORMAT rewrapped three of the new comments, applied via `spin fixlint`.
>
> One scope caveat worth stating plainly: "48 findings to 0" is about the linter's
> view, not the directories. RAWTHROW's include_patterns are `**/*.cpp` and
> `**/*.h`, so four raw throws here stay invisible - three std::system_error sites
> in `test/cpp/c10d/TestUtils.hpp` (two of which are literal copies of the throw
> this PR annotates in FileStoreTest.cpp) and a CUDA error-check macro in
> `test/cpp/aoti_abi_check/cuda/test_kernel_utils.cu`. A planned later phase
> widens include_patterns to .hpp/.cu and picks them up.
>
> One coupling to carry forward, in a file this PR touches:
> `aoti_inference/test.cpp:301` catches `const std::runtime_error&` around
> `update_constant_buffer` and matches "API call failed at" at :302. That throw comes from
> `torch/headeronly/util/shim_utils.h`, which is annotated and unconverted today -
> if a later phase converts it to TORCH_CHECK rather than STD_TORCH_CHECK, this
> catch stops matching.
>
> Not verified locally: the CUDA and distributed builds, and CLANGTIDY.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/194773
Approved by: https://github.com/ezyang
ghstack dependencies: #194379, #194772
## Author Notes
Pretty mechanical, nothing special here.
## Agent Notes
Written by Claude Code:
> The two test directories are 48 findings, and they come apart cleanly: 43 are
> stubs and error checks that convert mechanically, and 5 are throws whose type is
> the point of the test they sit in. Getting those 5 wrong would leave the suite
> green while testing the wrong thing, which is the only real risk in this PR.
>
> The 43: `rng_extension.cpp` and `cpp_c10d_extension.cpp` are "not implemented" /
> "does not support" overrides (8 and 10); `aoti_inference/test.cpp` is CUDA API
> error checks (16); the rest are `aoti_custom_class.cpp`,
> `test_custom_class_registrations.cpp` (6, a toy stack machine's error paths) and
> the two `my_from_blob_*` extensions. None of their messages is asserted on
> anywhere - I grepped the tree for each, including the Python suites.
>
> The two `my_from_blob_*` sites get STD_TORCH_CHECK rather than TORCH_CHECK.
> Those TUs include only `torch/csrc/stable/*`, which reaches
> torch/headeronly/util/Exception.h but never c10/util/Exception.h, so TORCH_CHECK
> is not declared there at all. STD_TORCH_CHECK preserves the exception type
> exactly - it throws std::runtime_error, same as the raw throw - but not the
> message: like the serialization-codegen PR earlier in this stack, it prepends
> __func__, __FILE__ and __LINE__, so
>
> before: cudaMalloc failed
> after: my_from_blob_with_cuda_deleter, <path>/my_from_blob_with_deleter.cpp:73,
> cudaMalloc failed
>
> These are the only two sites in this PR where anything observable moves. Nothing
> asserts on either string, both are behind #ifdef LAE_USE_CUDA in a test
> extension, and STD_TORCH_CHECK is already the house style in the sibling
> libtorch_agn_2_* extensions (17 uses), so their messages are decorated the same
> way. The alternative was to leave these two raw and annotated; taking the
> decoration keeps the file consistent with its neighbours.
>
> The 5 that stay raw, with the reason on each site:
>
> ProcessGroupNCCLErrorsTest.cpp TestHeartbeatMonitor::runLoop, 16 lines
> above, catches std::runtime_error by name
> to set hasMonitorThreadCaughtError_. A
> c10::Error is not caught by that, and the
> throw happens on the monitor thread.
> dataloader.cpp The test rethrows e.original_exception and
> asserts it is still a std::invalid_argument
> - the point being that WorkerException
> round-trips the original type.
> parallel.cpp ParallelApplyRethrowsException is about a
> user exception escaping forward().
> FileStoreTest.cpp std::system_error carrying errno as a
> std::error_code.
> OpenRegException.cpp Reports a caller-supplied SourceLocation;
> a TORCH_CHECK there would stamp this
> file's own location instead.
>
> Two of those five are load-bearing in a way I could measure rather than argue,
> and the test plan shows the failures. `parallel.cpp` is the one that is not:
> its assertion is `ASSERT_THROWS_WITH(..., "Badness!")`, a substring match that
> would still pass after conversion. It is annotated for intent, not because it
> would break, and it seemed worth being explicit about that difference rather
> than implying all five were equally forced.
>
> The exclude list goes from 11 entries to 9.
>
> Test Plan:
>
> Half of this is CUDA or distributed and does not build here, so the two CUDA
> files - parallel.cpp and ProcessGroupNCCLErrorsTest.cpp - deliberately received
> comment-only edits, which is the safest possible change to code a configuration
> cannot compile.
>
> For the rest, what matters is whether the *changed lines* were compiled, not the
> files: three of them sit behind preprocessor guards that a plain CPU build skips
> straight over. `aoti_inference/test.cpp`'s 16 conversions are inside
> `#if defined(USE_CUDA) || defined(USE_ROCM)`, and the two `my_from_blob_*` sites
> are inside `#ifdef LAE_USE_CUDA`, which setup.py only defines when CUDA is
> available. The two my_from_blob sites were compiled with the guard forced on,
> against a stub cuda_runtime.h. There is no CUDA toolkit here, so the
> aoti_inference block was not compiled; what I checked instead is that
> `<torch/torch.h>` is included unconditionally above it and does provide
> TORCH_CHECK, and that the block contained no TORCH_CHECK before this change, so
> the macro's availability there is an argument rather than a measurement. Every
> other converted line is unguarded.
>
> BUILD_TEST=0 does not prevent compiling these directly:
>
> aoti_inference/test.cpp syntax-only (needs gmock headers)
> aoti_custom_class.cpp syntax-only
> dataloader.cpp 87 tests passed
> test_custom_class_registrations.cpp compiles and links (no TESTs of its own)
> FileStoreTest.cpp syntax-only
> cpp_c10d_extension.cpp syntax-only (needs Python.h, pybind11)
> rng_extension.cpp syntax-only (same)
> my_from_blob_with_deleter.cpp syntax-only
> my_from_blob_with_lambda_deleter.cpp syntax-only
> OpenRegException.cpp syntax-only (needs the openreg headers)
>
> ```
> g++ -std=c++20 <includes> test/cpp/api/dataloader.cpp gtest-all.o gtest_main.o \
> -Lbuild/lib -lc10 -ltorch_cpu -lpthread && ./a.out
> [ PASSED ] 87 tests.
> ```
>
> Positive control on the dataloader annotation - the same file with the throw
> converted:
>
> ```
> [ FAILED ] DataLoaderTest.TestExceptionsArePropagatedFromWorkers
> ```
>
> So that annotation is required, not preference.
>
> ```
> lintrunner --take RAWTHROW --all-files
> spin lint
> ```
>
> Both clean; test/cpp and test/cpp_extensions go from 48 findings to 0.
> CLANGFORMAT rewrapped three of the new comments, applied via `spin fixlint`.
>
> One scope caveat worth stating plainly: "48 findings to 0" is about the linter's
> view, not the directories. RAWTHROW's include_patterns are `**/*.cpp` and
> `**/*.h`, so four raw throws here stay invisible - three std::system_error sites
> in `test/cpp/c10d/TestUtils.hpp` (two of which are literal copies of the throw
> this PR annotates in FileStoreTest.cpp) and a CUDA error-check macro in
> `test/cpp/aoti_abi_check/cuda/test_kernel_utils.cu`. A planned later phase
> widens include_patterns to .hpp/.cu and picks them up.
>
> One coupling to carry forward, in a file this PR touches:
> `aoti_inference/test.cpp:301` catches `const std::runtime_error&` around
> `update_constant_buffer` and matches "API call failed at" at :302. That throw comes from
> `torch/headeronly/util/shim_utils.h`, which is annotated and unconverted today -
> if a later phase converts it to TORCH_CHECK rather than STD_TORCH_CHECK, this
> catch stops matching.
>
> Not verified locally: the CUDA and distributed builds, and CLANGTIDY.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/194773
Approved by: https://github.com/pytorchgreenlight
ghstack dependencies: #194378, #194379, #194772
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66744
Modified loops in files under fbsource/fbcode/caffe2/ from the format
`for(TYPE var=x0;var<x_max;x++)`
to the format
`for(const auto var: irange(xmax))`
This was achieved by running r-barnes's loop upgrader script (D28874212) with some modification to exclude all files under /torch/jit and a number of reversions or unused variable suppression warnings added by hand.
Test Plan: Sandcastle
Reviewed By: ngimel
Differential Revision: D31705358
fbshipit-source-id: d6ea350cbaa8f452fc78f238160e5374be637a48
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66234
Modified loops in files under fbsource/fbcode/caffe2/ from the format
`for(TYPE var=x0;var<x_max;x++)`
to the format
`for(const auto var: irange(xmax))`
This was achieved by running r-barnes's loop upgrader script (D28874212) with some modification to exclude all files under /torch/jit and a number of reversions or unused variable suppression warnings added by hand.
bypass_size_limit
allow-large-files
Test Plan: Sandcastle
Reviewed By: ngimel
Differential Revision: D30652629
fbshipit-source-id: 0ae6c4bbbb554bad42e372792a6430e1acf15e3e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/37704
If input tensor can not be chunked, run `parallel_apply` on fewer devices
Modfy input tensor dimention in `DataParallelUsesAllAvailableCUDADevices_CUDA` to be chunkable by any number of available CUDA devices
Test Plan: Run `test/cpp/api/parallel` on machine with 6 GPUs
Differential Revision: D21365416
fbshipit-source-id: 60fdfed4a0e6256b2c966c2ea3e8d0bfb298d9a8
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28523
New features:
1. Previously, `torch::tensor({true, false, true})` throws `"tensor_cpu" not implemented for 'Bool'`. After this PR, it produces the correct bool tensor, matching the Python API behavior.
2. Tensors with zero-size dimensions are now supported, e.g. `torch::tensor({{}, {}})` produces a tensor with sizes `{2, 0}`, matching the Python API behavior.
BC-breaking bug fixes:
1. Previously, `torch::tensor({{1}, {2}})` produces a tensor of sizes `{2}`. After this PR, it produces a tensor of sizes `{2, 1}`, matching the Python API behavior.
2. Fixed semantics of `torch::tensor(1.1)`: it now returns a 0-dim tensor instead of a 1-dim tensor, matching the Python API behavior.
3. Previously, when passed a non-dtype `TensorOptions` to the `torch::tensor` constructor, it always produces a tensor of dtype `float`. After this PR, it produces tensor of different dtypes based on the dtype of the braced-init-list, matching the behavior of the no-options case.
```cpp
// Previously:
torch::tensor({1, 2, 3}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> float
torch::tensor({{1, 2, 3}}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> float
torch::tensor({1., 2., 3.}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> float
torch::tensor({{1., 2., 3.}}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> float
// Now:
torch::tensor({1, 2, 3}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> int
torch::tensor({{1, 2, 3}}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> int
torch::tensor({1., 2., 3.}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> double
torch::tensor({{1., 2., 3.}}, torch::TensorOptions(/*non-dtype-options*/)).dtype() -> double
// As comparison, currently:
torch::tensor({1, 2, 3}).dtype() -> int
torch::tensor({{1, 2, 3}}).dtype() -> int
torch::tensor({1., 2., 3.}).dtype() -> double
torch::tensor({{1., 2., 3.}}).dtype() -> double
```
Notes:
1. From now on, the behavior of `at::tensor(scalar_value)` (which produces a 1-dim tensor) would be different from `torch::tensor(scalar_value)` (which produces a 0-dim tensor). I will fix the behavior of `at::tensor(scalar_value)` in a follow-up PR.
2. From now on, the behavior of `at::tensor({1, 2, 3}, torch::TensorOptions(/*non-dtype-options*/))` (which produces a `float` tensor) would be different from `torch::tensor({1, 2, 3}, torch::TensorOptions(/*non-dtype-options*/))` (which produces a an `int` tensor). I will fix this behavior of `at::tensor` constructor in a follow-up PR.
Context for the changes in this PR:
The motivation comes from fixing the "`torch::tensor({{1}, {2}})` gives tensor of wrong sizes" bug - in order to fix it, I have to move the handling of `at::ArrayRef` and `std::vector` into `InitListTensor` (see below on why we need to do this) and renamed `InitListTensor` to `TensorDataContainer`. After such changes, support for bool values comes out of the box without extra effort, and support for tensors with zero-size dimensions only requires adding a default constructor for `TensorDataContainer`, so I added those two in this PR.
For the semantic change of `torch::tensor(1.1)`, it's actually more effort to preserve the original wrong behavior (i.e. we need to check the sizes of the tensor converted from `TensorDataContainer` and reshape any scalar tensor to a 1-D tensor). I think preserving the original wrong behavior doesn't give us much value, and since the above changes naturally fix the problem, we should just start using the right behavior instead.
For the "constructor with non-dtype options behavior" fix, the code looks simpler and easier to reason about with the fix, so I included it in this PR.
--------
Why we need to move the handling of `at::ArrayRef` and `std::vector` into `TensorDataContainer`:
`torch::tensor({{1}, {2}})` can match this function overload:
`torch::tensor(at::ArrayRef<int> values)`, because `{1}` and `{2}` can be treated as
a list-initialization of an `int` value. However, this will produce a Tensor with sizes `{2}`,
but we actually want a Tensor with sizes `{2, 1}`. In order to avoid matching this function overload,
we removed the function overload and moved the ability to convert `at::ArrayRef<T>`
(and similarly `std::vector<T>`) into `TensorDataContainer`, and since for braced-init-list the
`TensorDataContainer(std::initializer_list<TensorDataContainer>)` constructor is always preferred over all other constructors, it will take the `std::initializer_list` path, and all is good.
Test Plan: Imported from OSS
Differential Revision: D18234625
Pulled By: yf225
fbshipit-source-id: 0f3f6912e82e2117d2103e31b74e7e97baaa8693
Summary:
Fixes#19540
CC nmerrill67
C++ data parallel was using Module.clone() to create module replicas on every destination device. However, clone() does not set up gradient edges to point from replicas to the original module. As a result, the gradient will not be aggregated into the original module. This commit fixes the the problem by manually setting gradient edges from every parameter X in every replica to the same parameter X in the original module.
## Failed Attempt
Initially I tried implementing what we did in `replicate.py`, which
1. create module replicas
2. use Python `Broadcast` autograd function to broadcast every parameter in the original module to all destination devices.
3. assign the broadcast result params to module replicas' `_parameters` dict.
This works in Python because derived module member field params (e.g., `Linear.weight`) and base module `_parameters` (e.g., `Linear._parameters['weight']`) are referencing the same parameter instance. Assigning one of them will apply to both. However, in C++, even though I can modify Module's `parameters_ `values and gradient edges to point to the broadcast source, I cannot touch the weight and bias member fields in Linear, because replicate cannot (and should not) add special-case handlers to every different module. (See `Linear` [.h](https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/include/torch/nn/modules/linear.h), [.cpp](https://github.com/pytorch/pytorch/blob/master/torch/csrc/api/src/nn/modules/linear.cpp)) Although they initially point to the same `TensorImpl` instance, after assigning to `Module.parameters_['weight']`, it will be different from `Linear.weight`.
## Solution Options
gchanan and I had several discussions on this issue and figured two solutions to this problem.
### Option One [implemented in this PR]
Replicate the module in two steps:
1. call `Module.clone()` to create a module replica on every destination device.
2. manually setting gradient edges from every parameter in every replica to the same parameter in the original module.
* Pro: Does not need to change any existing module, and relatively easier to implement
* Con: It is a little hackish.
### Options Two
Implement a `Replicatable` class (similar to `Cloneable`), and make it a friend class of `Module`. For more details see `Note [Replicating Modules]` in the code change.
* Pro: Maybe this aligns more with our existing approach implemented in `Cloneable`?
* Con: Require changes to every existing module.
I am inclined to go with option one, because `replicate` will only be used on data parallel. I feel it is too big an overkill if we have to change all existing module implementations due to a data parallel requirement.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/20910
Differential Revision: D15556426
Pulled By: mrshenli
fbshipit-source-id: aa836290ec657b32742e2bea80bd0ac2404ef3b0
Summary:
This reverts commit 37cb357d8d.
Try to see if it unbreaks master
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14082
Differential Revision: D13095888
Pulled By: bddppq
fbshipit-source-id: c728f80f233b4d9daaf65f43202d8104651029a9
Summary:
Deletes the `OptionsGuard` from ATen. This works towards the goal of reworking `DefaultTensorOptions`. `OptionsGuard` is troublesome because it relies on mutating thread local state. This PR fixes those code locations and then deletes the `OptionsGuard`.
ezyang gchanan
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13738
Differential Revision: D13000962
Pulled By: goldsborough
fbshipit-source-id: c8143ee75070c2280f5fd1d9af86f8ce14279b72
Summary:
This is a pre-cursor diff to Python <-> C++ frontend integration -- I have a follow-up PR coming for that. This PR changes the C++ frontend module interface to replace the custom "cursor"s I introduced some time ago with `OrderedDict`. I introduced cursors at the time as a convenient way of applying functions and query operations on a modules' parameters, buffers and modules, allowing things like `module.parameters().map(my_func)`. However, I noticed that (1) this functionality is easily implement-able on top of a regular data structure and (2) more importantly, using OrderedDicts is much, much easier for Python integration. This is especially true given that ScriptModule today also uses OrderedDict. Since C++ frontend modules and ScriptModules will soon too share as many implementation details as possible, it is overall the best move to ditch the custom cursor datastructure and pervasively use OrderedDict everywhere.
For this I did:
1. Changed the C++ frontend module interface to more closely match the Python one by providing `parameters()`, `named_parameters()` and other methods Python provides. This is very important for the following diff which binds these into Python for inter-op with Python modules.
2. In lieu of the `Cursor::apply()` method I added `nn::Module::apply`. This again is one more unifying step between Python and C++, since Python modules have an apply function too.
3. Deleted all uses of Cursor.
4. Tidied and beefed up the `OrderedDict` class. In particular, I made `OrderedDict::Item` store an `std::pair` under the hood, because that is trivial to bind into Python and saved me a lot of headaches. `key` and `value` become methods instead of fields, which they should have been from the very start anyway because it allows exactly these kinds of changes, as per usual good software engineering principle of encapsulation.
5. Added many tests for the OrderedDict use in `nn::Module`.
ebetica ezyang
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13427
Differential Revision: D12894092
Pulled By: goldsborough
fbshipit-source-id: 715770c95a9643753a1db26d7f9da9a78619a15d
Summary:
In TorchScript and C++ extensions we currently advocate a mix of `torch::` and `at::` namespace usage. In the C++ frontend I had instead exported all symbols from `at::` and some from `c10::` into the `torch::` namespace. This is far, far easier for users to understand, and also avoid bugs around creating tensors vs. variables. The same should from now on be true for the TorchScript C++ API (for running and loading models) and all C++ extensions.
Note that since we're just talking about typedefs, this change does not break any existing code.
Once this lands I will update stuff in `pytorch/tutorials` too.
zdevito ezyang gchanan
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13523
Differential Revision: D12942787
Pulled By: goldsborough
fbshipit-source-id: 76058936bd8707b33d9e5bbc2d0705fc3d820763
Summary:
Does
```cpp
namespace torch {
using c10::optional;
using c10::nullopt;
}
```
So that users can be oblivious of our changes with ATen/c10 happening in the background, and also don't have to deal with multiple namespaces (which is very confusing).
ezyang
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12927
Differential Revision: D10510630
Pulled By: goldsborough
fbshipit-source-id: e456264f2fbca3eda277712de11cdd8acc77fbd4
Summary:
This PR adds optional type to ATen native, autograd, JIT schema and Python Arg parser, closes#9513. It allows us to use optional default values (including None) for function signature and implementations like clamp, etc., and also let us remove the python_default_init hack.
Follow up:
remove python_default_init completely.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12582
Differential Revision: D10417423
Pulled By: wanchaol
fbshipit-source-id: 1c80f0727bb528188b47c595629e2996be269b89
Summary:
There are still a few work to be done:
- Move logging and unify AT_WARN with LOG(ERROR).
- A few header files are still being plumbed through, need cleaning.
- caffe2::EnforceNotMet aliasing is not done yet.
- need to unify the macros. See c10/util/Exception.h
This is mainly a codemod and not causing functional changes. If you find your job failing and trace back to this diff, usually it can be fixed by the following approaches:
(1) add //caffe2/c10:c10 to your dependency (or transitive dependency).
(2) change objects such as at::Error, at::Optional to the c10 namespace.
(3) change functions to the c10 namespace. Especially, caffe2::MakeString is not overridden by the unified c10::str function. Nothing else changes.
Please kindly consider not reverting this diff - it involves multiple rounds of rebasing and the fix is usually simple. Contact jiayq@ or AI Platform Dev for details.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12354
Reviewed By: orionr
Differential Revision: D10238910
Pulled By: Yangqing
fbshipit-source-id: 7794d5bf2797ab0ca6ebaccaa2f7ebbd50ff8f32
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12103
This defers lookup of defaults to the site where we read
out of TensorOptions. THIS IS A BC-BREAKING BEHAVIOR CHANGE,
but we expect the bulk of uses of OptionsGuard don't allocate TensorOptions
inside the OptionsGuard region, and then use it outside of the region
(the situation where behavior could change.)
I also optimize the size of TensorOptions by rearranging fields, so that we
always fit in two 64-bit words.
Reviewed By: goldsborough
Differential Revision: D10052523
fbshipit-source-id: f454a15b4dbf8cd17bc902ab7d2016f2f689ed13
Summary:
This PR is a large codemod to rewrite all C++ API tests with GoogleTest (gtest) instead of Catch.
You can largely trust me to have correctly code-modded the tests, so it's not required to review every of the 2000+ changed lines. However, additional things I changed were:
1. Moved the cmake parts for these tests into their own `CMakeLists.txt` under `test/cpp/api` and calling `add_subdirectory` from `torch/CMakeLists.txt`
2. Fixing DataParallel tests which weren't being compiled because `USE_CUDA` wasn't correctly being set at all.
3. Updated README
ezyang ebetica
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11953
Differential Revision: D9998883
Pulled By: goldsborough
fbshipit-source-id: affe3f320b0ca63e7e0019926a59076bb943db80
Summary:
This PR removes the `using Tensor = autograd::Variable;` alias from `torch/tensor.h`, which means `torch::Tensor` is now `at::Tensor`. This PR fixes up some last uses of `.data()` and tidies up the resulting code. For example, I was able to remove `TensorListView` such that code like
```
auto loss = torch::stack(torch::TensorListView(policy_loss)).sum() +
torch::stack(torch::TensorListView(value_loss)).sum();
```
is now
```
auto loss = torch::stack(policy_loss).sum() + torch::stack(value_loss).sum();
```
CC jgehring
ebetica
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10516
Differential Revision: D9324691
Pulled By: goldsborough
fbshipit-source-id: a7c1cb779c9c829f89cea55f07ac539b00c78449
Summary:
This PR adds the functional version of `DataParallel` (i.e. `data_parallel`) to the C++ frontend.
For this, I had to:
1. Add "differentiable" versions of scatter and gather, which perform their inverse operation in the backward pass, to C++. I've added them under `torch/csrc/autograd/functions/comm.{h,cpp}`. I had to move some utilities from `VariableType.cpp` into `torch/csrc/autograd/functions/utils.h`, and changed them a bit to fix the `const_cast`s for which there were `TODO`s,
2. Implement the `replicate`, `parallel_apply` and the combining `data_parallel` functions in C++.
`replicate` is implemented based on our existing `clone()` interface, along with the ability to set the current device via `at::OptionsGuard` (so nice).
`parallel_apply` is implemented using `at::parallel_for` (CC cpuhrsch) and [follows the code from PyTorch](https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/parallel_apply.py).
Added lots of tests for these things.
apaszke ezyang ebetica colesbury
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9234
Differential Revision: D8865182
Pulled By: goldsborough
fbshipit-source-id: 4f1fecf2b3f3bc1540c071dfb2d23dd45de433e4
albanD