0 Fork 0

mirror of https://github.com/pytorch/pytorch.git synced 2026-10-28 18:56:51 +08:00

Files

T

Svetlana Karslioglu 19da3da3d3 Restructure C++ docs (#174096 )

The fix-cpp-build branch is a major restructuring that:

- Drops Exhale (which auto-generated API docs but caused OOM in CI) in favor of hand-written RST pages using Breathe directives
- Reorganizes notes into a hierarchical API reference with proper categories
- Adds new content: XPU docs, stable ABI docs, DeviceGuard/Stream docs
- Always runs Doxygen instead of gating on an env var

Pull Request resolved: https://github.com/pytorch/pytorch/pull/174096
Approved by: https://github.com/janeyx99, https://github.com/seemethere, https://github.com/albanD

2026-04-07 17:24:38 +00:00

2.3 KiB

Raw Permalink Blame History

myst

html_meta

description	keywords
DataLoader in PyTorch C++ — parallel data loading with batching, sampling, and multi-worker support.	PyTorch, C++, DataLoader, data loading, batching, workers, make_data_loader

DataLoader

The DataLoader batches samples from a dataset and optionally shuffles and parallelizes the loading process. It is the main interface for iterating over training data.

DataLoader Classes

:members:
:undoc-members:

DataLoaderOptions

:members:
:undoc-members:

StatefulDataLoader

A DataLoader for StatefulDataset types that manage their own batching logic internally.

:members:
:undoc-members:

StatelessDataLoader

A DataLoader for Dataset types that use external samplers for batching.

:members:
:undoc-members:

Iterator

:members:
:undoc-members:

Creating a DataLoader

Use make_data_loader to create a DataLoader from a dataset:

auto data_loader = torch::data::make_data_loader(
    std::move(dataset),
    torch::data::DataLoaderOptions()
        .batch_size(64)
        .workers(4));

for (auto& batch : *data_loader) {
    auto data = batch.data;
    auto target = batch.target;
    // Train on batch
}

Complete Training Example

#include <torch/torch.h>

int main() {
  // Load dataset
  auto dataset = torch::data::datasets::MNIST("./data")
      .map(torch::data::transforms::Normalize<>(0.1307, 0.3081))
      .map(torch::data::transforms::Stack<>());

  // Create data loader
  auto data_loader = torch::data::make_data_loader(
      std::move(dataset),
      torch::data::DataLoaderOptions().batch_size(64).workers(2));

  // Create model and optimizer
  auto model = std::make_shared<Net>();
  auto optimizer = torch::optim::Adam(model->parameters(), 0.001);

  // Training loop
  for (size_t epoch = 1; epoch <= 10; ++epoch) {
    for (auto& batch : *data_loader) {
      optimizer.zero_grad();
      auto output = model->forward(batch.data);
      auto loss = torch::nll_loss(output, batch.target);
      loss.backward();
      optimizer.step();
    }
  }
}

2.3 KiB Raw Permalink Blame History