[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169

z-a-f · 2021-01-27T01:05:29Z

Stack from ghstack:

[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169 [quant][refactor] Factor out MHA code dup from nn and nn.quantizable
[quant] Factoring out the list of no_observers #50459 [quant] Factoring out the list of no_observers
[quant] Quantizable MultiheadAttention #49866 [quant] Quantizable MultiheadAttention

Quantizable MHA shares some code with its FP equivalent.
This factors out common parts.

Test Plan:

Explicit testing is not available, as there is no functional tests for the nn.MHA.
However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart.

Differential Revision: D26092530

…antizable Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. [ghstack-poisoned]

facebook-github-bot · 2021-01-27T01:05:45Z

💊 CI failures summary and remediations

As of commit 1416aab (more details on the Dr. CI page):

6/10 failures possibly* introduced in this PR
- 2/6 non-CircleCI failure(s)
4/10 broken upstream at merge base 31e2a15 on Jan 26 from 12:49pm to 6:17pm

🕵️ 4 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_macos_10_13_py3_test (1/4)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Feb 17 00:42:33 RuntimeError: test_jit failed!

Feb 17 00:42:33 Generated XML report: test-reports/dist-gloo/TEST-jit.test_warn.TestWarn-20210217003923.xml
Feb 17 00:42:33 Generated XML report: test-reports/dist-gloo/TEST-jit.test_with.TestWith-20210217003923.xml
Feb 17 00:42:33 Generated XML report: test-reports/dist-gloo/TEST-jit.test_backends.TestBackends-20210217003923.xml
Feb 17 00:42:33 Generated XML report: test-reports/dist-gloo/TEST-jit.test_data_parallel.TestDataParallel-20210217003923.xml
Feb 17 00:42:33 Generated XML report: test-reports/dist-gloo/TEST-jit.test_torchbind.TestTorchbind-20210217003923.xml
Feb 17 00:42:33 Traceback (most recent call last):
Feb 17 00:42:33   File "test/run_test.py", line 925, in <module>
Feb 17 00:42:33     main()
Feb 17 00:42:33   File "test/run_test.py", line 904, in main
Feb 17 00:42:33     raise RuntimeError(err_message)
Feb 17 00:42:33 RuntimeError: test_jit failed!
Feb 17 00:42:33 + cleanup
Feb 17 00:42:33 + retcode=1
Feb 17 00:42:33 + set +x


Exited with code exit status 1

pytorch_windows_vs2019_py36_cuda10.1_test1 (2/4)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_jit failed!

Generated XML report: test-reports\python-unittest\TEST-jit.test_with.TestWith-20210217002826.xml
Generated XML report: test-reports\python-unittest\TEST-jit.test_backends.TestBackends-20210217002826.xml
Generated XML report: test-reports\python-unittest\TEST-jit.test_cuda.TestCUDA-20210217002826.xml
Generated XML report: test-reports\python-unittest\TEST-jit.test_data_parallel.TestDataParallel-20210217002826.xml
Generated XML report: test-reports\python-unittest\TEST-jit.test_torchbind.TestTorchbind-20210217002826.xml
Traceback (most recent call last):
  File "run_test.py", line 925, in <module>
    main()
  File "run_test.py", line 904, in main
    raise RuntimeError(err_message)
RuntimeError: test_jit failed!

(base) C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1 
+ cleanup
+ retcode=1
+ set +x


Exited with code exit status 1

pytorch_linux_xenial_cuda10_2_cudnn7_py3_jit_legacy_test (3/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Feb 17 00:42:37 RuntimeError: test_jit_legacy failed!

Feb 17 00:42:36 Generated XML report: test-reports/python-unittest/TEST-jit.test_tracer.TestTracer-20210217003843.xml
Feb 17 00:42:36 Generated XML report: test-reports/python-unittest/TEST-jit.test_type_sharing.TestTypeSharing-20210217003843.xml
Feb 17 00:42:36 Generated XML report: test-reports/python-unittest/TEST-jit.test_unsupported_ops.TestUnsupportedOps-20210217003843.xml
Feb 17 00:42:36 Generated XML report: test-reports/python-unittest/TEST-jit.test_warn.TestWarn-20210217003843.xml
Feb 17 00:42:36 Generated XML report: test-reports/python-unittest/TEST-jit.test_with.TestWith-20210217003843.xml
Feb 17 00:42:37 Traceback (most recent call last):
Feb 17 00:42:37   File "test/run_test.py", line 925, in <module>
Feb 17 00:42:37     main()
Feb 17 00:42:37   File "test/run_test.py", line 904, in main
Feb 17 00:42:37     raise RuntimeError(err_message)
Feb 17 00:42:37 RuntimeError: test_jit_legacy failed!
Feb 17 00:42:38 + cleanup
Feb 17 00:42:38 + retcode=1
Feb 17 00:42:38 + set +x
Feb 17 00:42:38 =================== sccache compilation log ===================
Feb 17 00:42:38 =========== If your build fails, please take a look at the log above for possible reasons ===========
Feb 17 00:42:38 Compile requests                    144
Feb 17 00:42:38 Compile requests executed             0
Feb 17 00:42:38 Cache hits                            0
Feb 17 00:42:38 Cache misses                          0
Feb 17 00:42:38 Cache timeouts                        0

pytorch_linux_xenial_py3_6_gcc5_4_jit_legacy_test (4/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Feb 17 00:21:12 RuntimeError: test_jit_legacy failed!

Feb 17 00:21:12 Generated XML report: test-reports/python-unittest/TEST-jit.test_type_sharing.TestTypeSharing-20210217001818.xml
Feb 17 00:21:12 Generated XML report: test-reports/python-unittest/TEST-jit.test_unsupported_ops.TestUnsupportedOps-20210217001818.xml
Feb 17 00:21:12 Generated XML report: test-reports/python-unittest/TEST-jit.test_warn.TestWarn-20210217001818.xml
Feb 17 00:21:12 Generated XML report: test-reports/python-unittest/TEST-jit.test_with.TestWith-20210217001818.xml
Feb 17 00:21:12 Generated XML report: test-reports/python-unittest/TEST-jit.test_data_parallel.TestDataParallel-20210217001818.xml
Feb 17 00:21:12 Traceback (most recent call last):
Feb 17 00:21:12   File "test/run_test.py", line 925, in <module>
Feb 17 00:21:12     main()
Feb 17 00:21:12   File "test/run_test.py", line 904, in main
Feb 17 00:21:12     raise RuntimeError(err_message)
Feb 17 00:21:12 RuntimeError: test_jit_legacy failed!
Feb 17 00:21:12 + cleanup
Feb 17 00:21:12 + retcode=1
Feb 17 00:21:12 + set +x
Feb 17 00:21:12 =================== sccache compilation log ===================
Feb 17 00:21:12 =========== If your build fails, please take a look at the log above for possible reasons ===========
Feb 17 00:21:12 Compile requests                    146
Feb 17 00:21:12 Compile requests executed             0
Feb 17 00:21:12 Cache hits                            0
Feb 17 00:21:12 Cache misses                          0
Feb 17 00:21:12 Cache timeouts                        0

🚧 4 fixed upstream failures:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

pytorch_linux_xenial_py3_6_gcc5_4_test on Jan 26 from 1:51pm to 5:48pm (f08464f - 5748410)
- 🔁 rerun
pytorch_linux_xenial_py3_clang5_asan_test1 on Jan 26 from 12:49pm to 5:48pm (a949d7b - 5748410)
- 🔁 rerun
pytorch_linux_bionic_py3_6_clang9_test on Jan 26 from 1:29pm to 5:39pm (6f3aa58 - 5748410)
- 🔁 rerun
pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test1 on Jan 26 from 1:29pm to 6:17pm (6f3aa58 - 5748410)
- 🔁 rerun

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.0.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

…n and nn.quantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. [ghstack-poisoned]

…antizable Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: 33af6f8 Pull Request resolved: #51169

vkuzo · 2021-01-27T01:16:08Z

thanks for doing this, high level LGTM if the numerics did not change and it's just copy-pasta. Do we need the fp32 MHA folks to approve?

z-a-f · 2021-01-27T01:22:34Z

thanks for doing this, high level LGTM if the numerics did not change and it's just copy-pasta. Do we need the fp32 MHA folks to approve?

The more I think about this PR, the more I think this is a bad idea :) We can discuss it offline

…n and nn.quantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

…antizable Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: 63fdad7 Pull Request resolved: #51169

…n and nn.quantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

…uantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: 5f49abe Pull Request resolved: #51169

z-a-f · 2021-02-03T20:03:01Z

The main con of this refactor is that once we implement the quantized BMM and Softmax, we will have to refactor again.

…uantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: e2e150e Pull Request resolved: #51169

…uantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: 065da6e Pull Request resolved: #51169

…uantizable" Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. Differential Revision: [D26092530](https://our.internmc.facebook.com/intern/diff/D26092530) [ghstack-poisoned]

Quantizable MHA shares some code with its FP equivalent. This factors out common parts. Test Plan: Explicit testing is not available, as there is no functional tests for the nn.MHA. However, the functional equivalence of this PR was tested across commits by comparing the numerics of the pre-change and post-change MHA. Similar was done for the quantizable counterpart. ghstack-source-id: d6f4c3e Pull Request resolved: #51169

vkuzo · 2021-02-24T15:57:14Z

what are good next steps here? Are we planning to land this PR, or do something else?

github-actions · 2022-04-13T02:57:22Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

This was referenced Jan 27, 2021

[quant] Quantizable MultiheadAttention #49866

Closed

[quant] Factoring out the list of no_observers #50459

Closed

facebook-github-bot added the cla signed label Jan 27, 2021

z-a-f requested review from vkuzo, driazati and ezyang January 27, 2021 01:05

raghuramank100 requested a review from cpuhrsch January 27, 2021 02:06

z-a-f changed the title ~~[quant][sparsity][refactor] Factor out MHA code dup from nn and nn.quantizable~~ [quant][refactor] Factor out MHA code dup from nn and nn.quantizable Feb 3, 2021

z-a-f requested review from dzhulgakov and removed request for driazati February 22, 2021 18:15

github-actions bot added the Stale label Apr 13, 2022

github-actions bot closed this May 13, 2022

facebook-github-bot deleted the gh/z-a-f/100/head branch June 12, 2022 14:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169

[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169

Uh oh!

z-a-f commented Jan 27, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented Jan 27, 2021 •

edited

Loading

Uh oh!

vkuzo commented Jan 27, 2021

Uh oh!

z-a-f commented Jan 27, 2021

Uh oh!

z-a-f commented Feb 3, 2021

Uh oh!

vkuzo commented Feb 24, 2021

Uh oh!

github-actions bot commented Apr 13, 2022

Uh oh!

Uh oh!

[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169

[quant][refactor] Factor out MHA code dup from nn and nn.quantizable #51169

Uh oh!

Conversation

z-a-f commented Jan 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jan 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 4 new failures recognized by patterns

pytorch_macos_10_13_py3_test (1/4)

pytorch_windows_vs2019_py36_cuda10.1_test1 (2/4)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_jit_legacy_test (3/4)

pytorch_linux_xenial_py3_6_gcc5_4_jit_legacy_test (4/4)

🚧 4 fixed upstream failures:

ci.pytorch.org: 1 failed

Uh oh!

vkuzo commented Jan 27, 2021

Uh oh!

z-a-f commented Jan 27, 2021

Uh oh!

z-a-f commented Feb 3, 2021

Uh oh!

vkuzo commented Feb 24, 2021

Uh oh!

github-actions bot commented Apr 13, 2022

Uh oh!

Uh oh!

z-a-f commented Jan 27, 2021 •

edited

Loading

facebook-github-bot commented Jan 27, 2021 •

edited

Loading