Retire torch.distributed.pipeline #127354

kwen2501 · 2024-05-28T23:07:39Z

Stack from ghstack (oldest at bottom):

-> Retire torch.distributed.pipeline #127354

Actually retiring module after deprecation warning for a while.
The new supported module is: torch.distributed.pipelining.
Please migrate.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k @ezyang @gchanan @albanD

[ghstack-poisoned]

pytorch-bot · 2024-05-28T23:07:43Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/127354

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 0f84020 with merge base 70724bd ():

NEW FAILURE - The following job has failed:

pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 3, 5, linux.g5.4xlarge.nvidia.gpu) (gh)
test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_inplace_cuda_bool

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: f7a211e Pull Request resolved: #127354

wconstab

oops i submitted a parallel PR to remove old pipeline.rst. ill cancel mine.

albanD · 2024-05-30T20:42:22Z

docs/source/conf.py

@@ -606,29 +606,22 @@
    # torch.distributed.optim.utils
    "as_functional_optim",
    "register_functional_optim",
-    # torch.distributed.pipeline.sync.checkpoint


Please delete the corresponding entries! Not just the comment saying which module they come from!

hahahaha, sorry my bad, I thought those are real entries, but just commented out :) Thanks a lot!

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k ezyang gchanan albanD [ghstack-poisoned]

ghstack-source-id: 851179d Pull Request resolved: #127354

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k ezyang gchanan albanD [ghstack-poisoned]

ghstack-source-id: 1405ee5 Pull Request resolved: #127354

kwen2501 · 2024-06-04T07:01:39Z

@pytorchbot merge -f "Minor change to fix conflict. All tests have passed previously"

pytorchmergebot · 2024-06-04T07:03:13Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

huydhn · 2024-06-04T18:17:41Z

@pytorchbot revert -m 'Sorry for reverting your change but the doc build failure looks legit https://hud.pytorch.org/pytorch/pytorch/commit/b9c058c203ee38032594f898f27cd8404f113a63' -c ignoredsignal

pytorchmergebot · 2024-06-04T18:19:22Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

This reverts commit b9c058c. Reverted #127354 on behalf of https://github.com/huydhn due to Sorry for reverting your change but the doc build failure looks legit https://hud.pytorch.org/pytorch/pytorch/commit/b9c058c203ee38032594f898f27cd8404f113a63 ([comment](#127354 (comment)))

pytorchmergebot · 2024-06-04T18:19:34Z

@kwen2501 your PR has been successfully reverted.

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. Pull Request resolved: #127354 Approved by: https://github.com/wconstab (cherry picked from commit b9c058c)

This reverts commit b9c058c. Reverted #127354 on behalf of https://github.com/huydhn due to Sorry for reverting your change but the doc build failure looks legit https://hud.pytorch.org/pytorch/pytorch/commit/b9c058c203ee38032594f898f27cd8404f113a63 ([comment](#127354 (comment))) (cherry picked from commit 0ff6023)

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. Pull Request resolved: pytorch#127354 Approved by: https://github.com/wconstab

This reverts commit b9c058c. Reverted pytorch#127354 on behalf of https://github.com/huydhn due to Sorry for reverting your change but the doc build failure looks legit https://hud.pytorch.org/pytorch/pytorch/commit/b9c058c203ee38032594f898f27cd8404f113a63 ([comment](pytorch#127354 (comment)))

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k ezyang gchanan albanD [ghstack-poisoned]

kwen2501 · 2024-06-07T07:50:26Z

@pytorchbot merge -f "Minor doc fix; CI should pass now"

pytorchmergebot · 2024-06-07T07:52:24Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-06-07T07:52:40Z

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x 37c5c4d93d2e16fa11d0ad070b8a00c77d5b05b6 returned non-zero exit code 1

Auto-merging .lintrunner.toml
Auto-merging docs/source/conf.py
CONFLICT (content): Merge conflict in docs/source/conf.py
Auto-merging docs/source/distributed.pipelining.rst
CONFLICT (content): Merge conflict in docs/source/distributed.pipelining.rst
Auto-merging test/allowlist_for_publicAPI.json
Auto-merging torch/distributed/pipelining/PipelineSchedule.py
error: could not apply 37c5c4d93d2... Retire torch.distributed.pipeline
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k ezyang gchanan albanD [ghstack-poisoned]

ghstack-source-id: b354d98 Pull Request resolved: #127354

kwen2501 · 2024-06-07T08:09:00Z

@pytorchbot merge -f "rebase"

pytorch-bot · 2024-06-07T08:09:05Z

You need to provide a reason for using force merge, in the format @pytorchbot merge -f 'Explanation'.
The explanation needs to be clear on why this is needed. Here are some good examples:

Bypass checks due to unrelated upstream failures from ...
This is a minor fix to ..., which shouldn't break anything
This is pre-tested in a previous CI run
Bypass flaky ... check

kwen2501 · 2024-06-07T08:10:02Z

@pytorchbot merge -f "this is a rebase to resolve merge conflict in doc"

pytorchmergebot · 2024-06-07T08:11:44Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Actually retiring module after deprecation warning for a while. The new supported module is: torch.distributed.pipelining. Please migrate. Pull Request resolved: pytorch#127354 Approved by: https://github.com/wconstab

Retire torch.distributed.pipeline

de177d7

[ghstack-poisoned]

kwen2501 requested a review from albanD as a code owner May 28, 2024 23:07

pytorch-bot bot added oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (pipeline) release notes category labels May 28, 2024

kwen2501 added a commit that referenced this pull request May 28, 2024

Retire torch.distributed.pipeline

05fb836

ghstack-source-id: f7a211e Pull Request resolved: #127354

kwen2501 requested review from wconstab and H-Huang May 28, 2024 23:11

wconstab approved these changes May 29, 2024

View reviewed changes

albanD requested changes May 30, 2024

View reviewed changes

kwen2501 added the skip-pr-sanity-checks label Jun 4, 2024

kwen2501 added a commit that referenced this pull request Jun 4, 2024

Retire torch.distributed.pipeline

3b254bd

ghstack-source-id: 851179d Pull Request resolved: #127354

kwen2501 added a commit that referenced this pull request Jun 4, 2024

Retire torch.distributed.pipeline

fd2ceb8

ghstack-source-id: 1405ee5 Pull Request resolved: #127354

pytorchmergebot added the merging label Jun 4, 2024

pytorchmergebot closed this in b9c058c Jun 4, 2024

pytorchmergebot added Merged and removed merging labels Jun 4, 2024

pytorchmergebot added the Reverted label Jun 4, 2024

pytorchmergebot reopened this Jun 4, 2024

pytorchmergebot added the merging label Jun 7, 2024

pytorchmergebot removed the merging label Jun 7, 2024

kwen2501 added a commit that referenced this pull request Jun 7, 2024

Retire torch.distributed.pipeline

e623fbc

ghstack-source-id: b354d98 Pull Request resolved: #127354

pytorchmergebot added the merging label Jun 7, 2024

pytorchmergebot closed this in 01601eb Jun 7, 2024

pytorchmergebot removed the merging label Jun 7, 2024

github-actions bot deleted the gh/kwen2501/40/head branch July 8, 2024 01:58

Retire torch.distributed.pipeline #127354

Retire torch.distributed.pipeline #127354

Uh oh!

Conversation

kwen2501 commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/127354

❌ 1 New Failure

Uh oh!

wconstab left a comment

Choose a reason for hiding this comment

Uh oh!

albanD May 30, 2024

Choose a reason for hiding this comment

Uh oh!

kwen2501 May 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kwen2501 commented Jun 4, 2024

Uh oh!

pytorchmergebot commented Jun 4, 2024

Merge started

Uh oh!

huydhn commented Jun 4, 2024

Uh oh!

pytorchmergebot commented Jun 4, 2024

Uh oh!

pytorchmergebot commented Jun 4, 2024

Uh oh!

kwen2501 commented Jun 7, 2024

Uh oh!

pytorchmergebot commented Jun 7, 2024

Merge started

Uh oh!

pytorchmergebot commented Jun 7, 2024

Merge failed

Uh oh!

kwen2501 commented Jun 7, 2024

Uh oh!

pytorch-bot bot commented Jun 7, 2024

Uh oh!

kwen2501 commented Jun 7, 2024

Uh oh!

pytorchmergebot commented Jun 7, 2024

Merge started

Uh oh!

Uh oh!

kwen2501 commented May 28, 2024 •

edited

Loading

pytorch-bot bot commented May 28, 2024 •

edited

Loading

kwen2501 May 31, 2024 •

edited

Loading