Improve how scripts/release_notes assigns `release notes:` labels to worksheet
files in three related ways:
- Distributed: previously every `distributed (...)` sub-area label was collapsed
into a single `distributed` category, so the generator produced one combined
result_distributed.md. Keep each sub-area label as its own category so a
separate `result_distributed (<subarea>).md` is emitted per area (c10d,
dtensor, fsdp, fsdp2, checkpoint, pipeline, torchelastic, symm_mem, ddp,
composable, rpc, sharded). Done by appending `distributed.categories` to the
master category list and dropping the collapse branch in `category_remapper`.
- Optimizer: `optim` and `optimizer` are duplicate labels for the same area.
Add an `optim` CategoryGroup mapping both to `optim`, remove the stale
`optimizer` frontend category, and point the `[optimizer]` title heuristic at
`optim`.
- AOTDispatcher: `aot autograd` and `aotdispatcher` name the same subsystem; add
a CategoryGroup mapping both to `aotdispatcher`.
The resulting `common.categories` contains no duplicates and drops no
previously-handled category; other areas are unaffected.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/188453
Approved by: https://github.com/atalman
Summary:
simple progress printing for initial run of `commitlist.py`:
```bash
$ time with-proxy python commitlist.py --create-new v2.10.0
origin/release/2.11 --verbose
...
[18:17:48] Processing 2621 out of 2699 commits
[18:17:53] Processing 2631 out of 2699 commits
[18:17:58] Processing 2641 out of 2699 commits
[18:18:03] Processing 2651 out of 2699 commits
[18:18:08] Processing 2661 out of 2699 commits
[18:18:13] Processing 2671 out of 2699 commits
[18:18:19] Processing 2681 out of 2699 commits
[af5c4ec030: [RELEASE 2.11] Release only changes (#175091)] Could not
parse PR number, ignoring PR
[18:18:24] Processing 2691 out of 2699 commits
[a2813af656: [release-only] Remove +ptx from cuda 13.0 builds
(#175567)] Could not parse PR number, ignoring PR
Finished creating new commit list. Results have been saved to
results/commitlist.csv
```
dead simple, no dependencies, no fancy features. This way I can know that the script is chugging along without waiting 30 mins with no input.
Test Plan:
```bash
time with-proxy python commitlist.py --create-new v2.10.0 origin/release/2.11 --verbose
```
Reviewers:
Subscribers:
Tasks:
Tags:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/175704
Approved by: https://github.com/janeyx99
This PR should be merged ~1 week after the copy-scripts PR lands, to give in-flight PRs time to pick up the new script locations.
The previous PR copied the binary build scripts to .ci/pytorch/. This PR updates all workflow references (`.circleci/scripts/` → `.ci/pytorch/`), deletes the old copies, and removes the now-empty `.circleci/` directory.
**Workflows that need the bake time:**
- `build-triton-wheel.yml` — triggers on `pull_request` (with path filters) and uses `checkout-pytorch@main` (head-sha checkout)
- `build-vllm-wheel.yml` — same
These create a workflow-YAML-vs-checkout version mismatch for pre-move PRs.
**Workflows that don't need it** (only trigger on schedule/workflow_dispatch/workflow_call):
- `_binary-build-linux.yml`
- `_binary-upload.yml`
- `_binary-upload-flash-attention-wheel.yml`
- `test-pytorch-binary` composite action
- Generated nightly workflows (windows/macos binary builds)
See PR #175987 for details on the mismatch mechanism.
Authored with Claude.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/175917
Approved by: https://github.com/atalman, https://github.com/malfet
ghstack dependencies: #175915, #175930, #175933
Update ruff to 0.4.1 .
This version fixes a lot false negatives/false positives, is 20-40% faster, and has various other bug fixes.
Below is a before and after table showing the execution time of ruff lint and ruff format in milliseconds courtesy of https://astral.sh/blog/ruff-v0.4.0
| Repository | Linter (v0.3) | Linter (v0.4) | Formatter (v0.3) | Formatter (v0.4) |
|----------------------------------------------------|---------------|---------------|------------------|------------------|
| [pytorch/pytorch](https://github.com/pytorch/pytorch) | 328.7 | 251.8 | 351.1 | 274.9 |
Pull Request resolved: https://github.com/pytorch/pytorch/pull/124549
Approved by: https://github.com/ezyang
# Summary
This PR made some significant changes to the scripts around Release Scripts. At a high level:
- Turned the quips into docs and updated links
- Update the common.categorizes list in the hopes to make this the source of truth for releases- This is hard since the release_notes labels can be changed at will. An alternative would be to poll from github api. However, I think that is overkill. The notebook does a set compare and will show you knew categories. I think we want this to be manual so that the release note engineer will decided how to categorize.
- Create cateogry group from speaking with folks on distributed and AO that told me these different release categories can be merged.
- I am the newest person to Core and don't use ghstack soo made token getting a lil more generic.
- Added a classifier.py file. This file will train a commit categorizer for you, hopefully with decent accuracy. I was able to achieve 75% accuracy. I drop the highest frequency class - "skip" since this creates a more useful cateogrizer.
- I updated the categorize.py script so that the prompt will be what the classifier thinks, gated by a flag.
- Added a readme that will hopefully help future release notes engineers.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94560
Approved by: https://github.com/albanD
Merges startswith, endswith calls to into a single call that feeds in a tuple. Not only are these calls more readable, but it will be more efficient as it iterates through each string only once.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/96754
Approved by: https://github.com/ezyang
Preferring dash over underscore in command-line options. Add `--command-arg-name` to the argument parser. The old arguments with underscores `--command_arg_name` are kept for backward compatibility.
Both dashes and underscores are used in the PyTorch codebase. Some argument parsers only have dashes or only have underscores in arguments. For example, the `torchrun` utility for distributed training only accepts underscore arguments (e.g., `--master_port`). The dashes are more common in other command-line tools. And it looks to be the default choice in the Python standard library:
`argparse.BooleanOptionalAction`: https://github.com/python/cpython/blob/4a9dff0e5adc91cbb1ed68c495dac64ccfe608bd/Lib/argparse.py#L893-L895
```python
class BooleanOptionalAction(Action):
def __init__(...):
if option_string.startswith('--'):
option_string = '--no-' + option_string[2:]
_option_strings.append(option_string)
```
It adds `--no-argname`, not `--no_argname`. Also typing `_` need to press the shift or the caps-lock key than `-`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94505
Approved by: https://github.com/ezyang, https://github.com/seemethere
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64929
Auto categorized 63% of the commits for PyTorch 1.10 release (2.2k out of 3.4k commits)
Test Plan: Imported from OSS
Reviewed By: malfet
Differential Revision: D33768760
Pulled By: anjali411
fbshipit-source-id: 0655090af83e923f8c26fa1ce9f190edc542b97e
(cherry picked from commit 2fe30f77b8)
Summary:
This adds:
- new categories
- global commit counter
- support for new "Reverted" label on PRs
- new export system to multiple files
Pull Request resolved: https://github.com/pytorch/pytorch/pull/54594
Reviewed By: H-Huang
Differential Revision: D27396011
Pulled By: albanD
fbshipit-source-id: ca1ec3a1b90221ba26fd8b053dfb10f614f05909
Summary:
First commit contains the initial code from Richard's branch.
Second commit are the changes that I made during the writing process
Third commit is the update to support category/topic pair for each commit
Pull Request resolved: https://github.com/pytorch/pytorch/pull/47360
Reviewed By: ejguan
Differential Revision: D24741003
Pulled By: albanD
fbshipit-source-id: d0fcc6765968dc1732d8a515688d11372c7e653d
angelayi