Fix ModuleNotFoundError: No module named 'cmake' #5195

guoyuhong · 2023-05-03T10:09:30Z

Description

Described by #5194

Motivation and Context

Since python setup.py build works fine, I compared the os.environ and found that pip install -e . add a strange PYTHONPATH to os.environ. Deleting it make the installation working fine.

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

setup.py

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

justinchuby · 2023-05-04T04:00:09Z

Is this tested in all different environments: venv, conda and w/o venv?

guoyuhong · 2023-05-04T15:27:46Z

@justinchuby I have tested this change in following envs:

conda env in Linux, MacOS, Windows
virtualenv in Linux and MacOS
w/o venv(python installed in system) in Linux and MacOS

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com> Co-authored-by: Yuhong Guo <yuhong.gyh@antgroup.com> Co-authored-by: Chun-Wei Chen <jacky82226@gmail.com> Signed-off-by: Aditya Goel <agoel4512@gmail.com>

### Description * Fix source and pip tar.gz builds on s390x systems * Revert (#5195) and replace with fix from [this comment](#5194 (comment)) from the original issue (#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing from pypi for environments where we don't have pre-built wheels * Failures include not being able to find pacakges dispite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (#5195) which was to fix (#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH is set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix

* Fix builds from source and pip tar.gz on s390x systems * Revert (#5195) and replace with fix from [this comment](#5194 (comment)) from the original issue (#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (#5195) which was to fix (#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com>

### Description * Fix builds from source and pip tar.gz on s390x systems * Revert (#5195) and replace with fix from [this comment](#5194 (comment)) from the original issue (#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (#5195) which was to fix (#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com>

### Description * Cherry-pick (#5984) - Fix builds from source and pip tar.gz on s390x systems * Revert (#5195) and replace with fix from [this comment](#5194 (comment)) from the original issue (#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (#5195) which was to fix (#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com>

…5986) ### Description * Cherry-pick (#5984) - Fix builds from source and pip tar.gz on s390x systems * Revert (#5195) and replace with fix from [this comment](#5194 (comment)) from the original issue (#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (#5195) which was to fix (#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com>

### Description * Fix builds from source and pip tar.gz on s390x systems * Revert (onnx#5195) and replace with fix from [this comment](onnx#5194 (comment)) from the original issue (onnx#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (onnx#5195) which was to fix (onnx#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](onnx#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: isdanni <leedanni@gmail.com>

### Description * Fix builds from source and pip tar.gz on s390x systems * Revert (onnx#5195) and replace with fix from [this comment](onnx#5194 (comment)) from the original issue (onnx#5194) ### Motivation and Context * Building on s390x (and other environments depending on python setup) fails * The issue also can also appear when installing `1.16.0rc1` from TestPyPI for environments where we don't have pre-built wheels * Failures include not being able to find pacakges despite them being installed ``` pip3 install -e . ... -- Could NOT find pybind11 (missing: pybind11_DIR) ... Traceback (most recent call last): File "/onnx/.setuptools-cmake-build/tools/protoc-gen-mypy.py", line 28, in <module> import google.protobuf.descriptor_pb2 as d_typed ``` * The issue is cased by (onnx#5195) which was to fix (onnx#5194) * The original fix bypasses the pip venv PYTHONPATH and attempts to rely on sys.path. However without the pip venv PYTHONPATH, some modules may not be found even if they are installed. * For reference, the pip venv PYTHONPATH gets set to `/tmp/pip-build-env-bn_o_cjc/site` which contains `sitecustomize.py` which comes from https://github.com/pypa/pip/blob/main/tests/lib/venv.py#L171-L189 * Instead of bypassing the pip venv PYTHONPATH, [this comment](onnx#5194 (comment)) from the original issue shows how to address the issue using pyproject.toml's `[build-system]` * I replicated the origial issue and confirmed this fix resolves it. * I confirmed the builds work on s390x with this fix Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Linsho Kaku <linsho@preferred.jp>

guoyuhong requested a review from a team as a code owner May 3, 2023 10:09

Fix ModuleNotFoundError: No module named 'cmake'

c1658d4

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

guoyuhong force-pushed the fix_pip_editable branch from f6f04da to c1658d4 Compare May 3, 2023 10:11

xadupre reviewed May 3, 2023

View reviewed changes

setup.py Outdated Show resolved Hide resolved

guoyuhong added 3 commits May 3, 2023 19:13

Exclude PYTHONPATH env var check in Windows

fcc9c7f

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

Refine setup.py with detail comments

4761836

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

Python lint

63b4af2

Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>

jcwchen added the topic: build Issues related to ONNX builds and packages label May 4, 2023

jcwchen linked an issue May 4, 2023 that may be closed by this pull request

ModuleNotFoundError: No module named 'cmake' during pip install -e . #5194

Closed

jcwchen added the run release CIs Use this label to trigger release tests in CI label May 4, 2023

Merge branch 'main' into fix_pip_editable

e953c2a

xadupre approved these changes May 5, 2023

View reviewed changes

Merge branch 'main' into fix_pip_editable

c1bccdb

jcwchen merged commit 8af2a3d into onnx:main May 11, 2023

guoyuhong deleted the fix_pip_editable branch May 11, 2023 05:08

cjvolzka mentioned this pull request Mar 1, 2024

Fix source and pip tar.gz builds on s390x systems #5984

Merged

cjvolzka mentioned this pull request Mar 2, 2024

[Cherry-pick] Fix builds from source and pip tar.gz on s390x systems #5986

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix ModuleNotFoundError: No module named 'cmake' #5195

Fix ModuleNotFoundError: No module named 'cmake' #5195

Uh oh!

guoyuhong commented May 3, 2023 •

edited

Loading

Uh oh!

Uh oh!

justinchuby commented May 4, 2023

Uh oh!

guoyuhong commented May 4, 2023

Uh oh!

Uh oh!

Fix ModuleNotFoundError: No module named 'cmake' #5195

Fix ModuleNotFoundError: No module named 'cmake' #5195

Uh oh!

Conversation

guoyuhong commented May 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

Uh oh!

justinchuby commented May 4, 2023

Uh oh!

guoyuhong commented May 4, 2023

Uh oh!

Uh oh!

guoyuhong commented May 3, 2023 •

edited

Loading