[pytorch] CMakeLists changes to enable libtorch for Android #19762

ljk53 · 2019-04-25T18:49:16Z

Stack from ghstack:

[pytorch] exclude some caffe2 modules from libtorch mobile build #20000 [pytorch] exclude some caffe2 modules from libtorch mobile build
[pytorch] CMakeLists changes to enable libtorch for Android #19762 [pytorch] CMakeLists changes to enable libtorch for Android

Summary:
Stole from #16242
Only include CMakeLists changes to make it easer for reviewers.

Change pytorch android CI job to build libtorch instead of caffe2 -
turns out it's as simple as adding a few cmake macros. Will properly
update tools/build_pytorch_libs.py in subsequent diffs.

Test Plan:

Make sure pytorch android CI job succeeds;
Test libtorch.so on Android device;

Differential Revision: D15087653

Summary: Stole from #16242 Only include CMakeLists changes to make it easer for reviewers. Change pytorch android CI job to build libtorch instead of caffe2 - turns out it's as simple as adding a few cmake macros. Will properly update tools/build_pytorch_libs.py in subsequent diffs. Test Plan: - Make sure pytorch android CI job succeeds; - Test libtorch.so on Android device;

dzhulgakov

I wonder whether it'd be easier to understand the details if we had separate vars for platform (e.g. what current BUILD_ATEN_MOBILE is) and have derived var BUILD_ATEN or BUILD_JIT determined based on it. That way we'd have mostly single conditionals around instead of BUILD_ATEN_MOBILE AND NOT BUILD_TORCH_MOBILE. But it's a soft suggestion, not insisting

CMakeLists.txt

cmake/TorchConfig.cmake.in

orionr

Exciting stuff! Biggest comment is that we should be able to go with just BUILD_MOBILE and remove the separate BUILD_TORCH_MOBILE and BUILD_ATEN_MOBILE options. I think @smessmer and @ezyang might have comments on whether ATen core will be needed then or not.

CMakeLists.txt

aten/CMakeLists.txt

caffe2/CMakeLists.txt

cmake/Codegen.cmake

cmake/Dependencies.cmake

cmake/TorchConfig.cmake.in

t-vi · 2019-04-26T23:33:27Z

The TorchConfig change is needed to keep CMake from finding host libs when building C++ extensions / custom ops for Android.

[pytorch] CMakeLists changes to enable libtorch for Android Summary: Stole from #16242 Only include CMakeLists changes to make it easer for reviewers. Change pytorch android CI job to build libtorch instead of caffe2 - turns out it's as simple as adding a few cmake macros. Will properly update tools/build_pytorch_libs.py in subsequent diffs. Test Plan: - Make sure pytorch android CI job succeeds; - Test libtorch.so on Android device; gh-metadata: pytorch pytorch 19762 gh/ljk53/14/head

dzhulgakov

The new setup looks good. I'll let other folks take a final look and accept

.jenkins/pytorch/build.sh

cmake/TorchConfig.cmake.in

[pytorch] CMakeLists changes to enable libtorch for Android Summary: Stole from #16242 Only include CMakeLists changes to make it easer for reviewers. Change pytorch android CI job to build libtorch instead of caffe2 - turns out it's as simple as adding a few cmake macros. Will properly update tools/build_pytorch_libs.py in subsequent diffs. Test Plan: - Make sure pytorch android CI job succeeds; - Test libtorch.so on Android device; gh-metadata: pytorch pytorch 19762 gh/ljk53/14/head

cmake/Summary.cmake

[pytorch] CMakeLists changes to enable libtorch for Android Summary: Stole from #16242 Only include CMakeLists changes to make it easer for reviewers. Change pytorch android CI job to build libtorch instead of caffe2 - turns out it's as simple as adding a few cmake macros. Will properly update tools/build_pytorch_libs.py in subsequent diffs. Test Plan: - Make sure pytorch android CI job succeeds; - Test libtorch.so on Android device; gh-metadata: pytorch pytorch 19762 gh/ljk53/14/head

Summary: Pull Request resolved: pytorch/pytorch#19762 ghimport-source-id: 287aa7fea4efd38994e14d794123eb2046b91fc0 Differential Revision: D15087653 Pulled By: ljk53 fbshipit-source-id: 4498ff9f7f7903c3e25541184302b811267958e9

t-vi · 2019-05-03T17:23:37Z

I'm building PyTorch for Android now. Thank you @ljk53 and everyone involved. You are awesome!

facebook-github-bot · 2019-05-03T19:12:16Z

@ljk53 merged this pull request in c7c0272.

ljk53 · 2019-05-03T19:13:54Z

@t-vi thanks for initiating this effort! It's all based on your work :)

kindlychung · 2019-07-03T13:39:16Z

Is there a documentation on how to use this on android? Thanks.

ezyang · 2019-07-03T19:54:08Z

Check the CI scripts for our Android build; they're the most canonical reference at the moment. Folder is .circleci/pytorch also cc @ljk53 Edward Excerpts from Kaiyin Zhong's message of 2019-07-03 06:40:17 -0700:

…

Is there a documentation on how to use this on android? Thanks.

ljk53 · 2019-07-06T23:59:23Z

What @ezyang said, you can build the current version with scripts/build_android.sh + "BUILD_CAFFE2_MOBILE=OFF" and pytorch environment for codegen:

./scripts/build_android.sh \
-DBUILD_CAFFE2_MOBILE=OFF \
-DCMAKE_PREFIX_PATH=$(python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())') \
-DPYTHON_EXECUTABLE=$(python -c 'import sys; print(sys.executable)')

The work is not done - we are working on improvements in terms of build size, perf and documentation. Stay tuned!

pytorchbot added caffe2 module: build Build system issues module: ci Related to continuous integration module: internals Related to internal abstractions in c10 and ATen labels Apr 25, 2019

This was referenced Apr 25, 2019

[pytorch] remove C10_MOBILE from LegacyTypeDispatch.cpp #19758

Closed

[pytorch] fix THAllocator.cpp #19759

Closed

[pytorch] fix labs warning in THTensorMoreMath.cpp #19760

Closed

[pytorch] add new macro TORCH_MOBILE for libtorch mobile build #19761

Closed

Update on "[pytorch] CMakeLists changes to enable libtorch for Android"

05ec376

dzhulgakov requested review from orionr and kostmo April 26, 2019 07:00

dzhulgakov reviewed Apr 26, 2019

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

CMakeLists.txt Outdated Show resolved Hide resolved

cmake/TorchConfig.cmake.in Show resolved Hide resolved

orionr reviewed Apr 26, 2019

View reviewed changes

ljk53 mentioned this pull request Apr 30, 2019

[pytorch] rename BUILD_ATEN_MOBILE to INTERN_BUILD_MOBILE and make it private #19942

Closed

ljk53 added 2 commits April 29, 2019 19:13

ljk53 requested review from soumith, ezyang, orionr, dzhulgakov and pjh5 April 30, 2019 02:51

dzhulgakov reviewed Apr 30, 2019

View reviewed changes

pjh5 reviewed Apr 30, 2019

View reviewed changes

.jenkins/pytorch/build.sh Outdated Show resolved Hide resolved

cmake/TorchConfig.cmake.in Show resolved Hide resolved

ljk53 added 3 commits April 30, 2019 15:24

ljk53 mentioned this pull request May 1, 2019

[pytorch] fix labs warning in THVectorDefault.cpp #19999

Closed

ljk53 mentioned this pull request May 1, 2019

[pytorch] exclude some caffe2 modules from libtorch mobile build #20000

Closed

t-vi mentioned this pull request May 2, 2019

Libtorch android #16242

Closed

ljk53 added 2 commits May 2, 2019 12:27

pjh5 reviewed May 2, 2019

View reviewed changes

cmake/Summary.cmake Show resolved Hide resolved

orionr approved these changes May 2, 2019

View reviewed changes

facebook-github-bot closed this in c7c0272 May 3, 2019

zou3519 deleted the gh/ljk53/14/head branch May 3, 2019 16:31

facebook-github-bot added the merged label May 3, 2019

[pytorch] CMakeLists changes to enable libtorch for Android #19762

[pytorch] CMakeLists changes to enable libtorch for Android #19762

Uh oh!

Conversation

ljk53 commented Apr 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

orionr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-vi commented Apr 26, 2019 via email

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-vi commented May 3, 2019

Uh oh!

facebook-github-bot commented May 3, 2019

Uh oh!

ljk53 commented May 3, 2019

Uh oh!

kindlychung commented Jul 3, 2019

Uh oh!

ezyang commented Jul 3, 2019 via email

Uh oh!

ljk53 commented Jul 6, 2019

Uh oh!

Uh oh!

ljk53 commented Apr 25, 2019 •

edited

Loading