Feature dtcenter/METbaseimage#30 Use Python 3.12.0 #3105

georgemccabe · 2025-03-12T21:29:23Z

This PR will switch MET to using the v3.4 METbaseimage which contains Python 3.12.0, so MET will use Python 3.12.0 for Python Embedding.

Expected Differences

Do these changes introduce new tools, command line arguments, or configuration file options? [No]

If yes, please describe:
Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

If yes, please describe:

Pull Request Testing

Describe testing already performed for these changes:

Ran unit tests in branch confirming (see this GHA workflow run)
Ran METplus use case tests with MET branch (only minimal diffs with lat/lon values were reported)

Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

Confirm all tests pass

Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

Note: I did NOT update the python requirements in the documentation because the changes to support Python 3.12.0 were backwards compatible with Python 3.10.4. The minimum requirements didn't change.

Do these changes include sufficient testing updates? [Yes]
Will this PR result in changes to the MET test suite? [No]

If yes, describe the new output and/or changes to the existing output:
Will this PR result in changes to existing METplus Use Cases? [Yes]

If yes, create a new Update Truth METplus issue to describe them.

dtcenter/METplus#2925

Do these changes introduce new SonarQube findings? [No]

If yes, please describe:
Please complete this pull request review by 3/13/2025.

Pull Request Checklist

See the METplus Workflow for details.

Review the source issue metadata (required labels, projects, and milestone).
Complete the PR definition above.
Ensure the PR title matches the feature or bugfix branch name.
Define the PR metadata, as permissions allow.
Select: Reviewer(s) and Development issue
Select: Milestone as the version that will include these changes
Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
Close the linked issue and delete your feature or bugfix branch from GitHub.
… python 3.10 to 3.12.0… python 3.10 to 3.12.0

… python 3.10 to 3.12.0

…json to prevent error handling float32 values

…s://gist.github.com/jonathanlurie/1b8d12f938b400e54c1ed8de21269b65

…2 calls to json.dump

…ed in fire weather use case moving to numpy 2.X

JohnHalleyGotway · 2025-03-13T17:25:10Z

Recompiled myself on seneca using the updated development.seneca environment. It compiles fine, although there's a new warning discussed below.
Confirmed that all the GHA runs pass without error.
Searched through the code looking for reference to Python 3.10 or METbaseimage version 3.3 you may have missed and found the following:
1. docs/Users_Guide/installation.rst references Python 3.10 -- SHOULD THIS BE UPDATED?
2. internal/scripts/environment/development.seneca_intel_oneapi references Python 3.10 -- SHOULD THIS BE UPDATED?
3. Several install files in internal/scripts/installation/config/install_met_env* reference Python 3.10, but presumably that's fine and @jprestop will update those env files only after updating the build environment there.
4. We're still using METbaseimage v3.3 in many spots. Is there any good reason to retain these rather than switching them all to v3.4?

./internal/scripts/docker/Dockerfile.sonarqube:ARG MET_BASE_TAG=v3.3
./internal/scripts/docker/Dockerfile.copy:ARG MET_BASE_TAG=v3.3
./internal/scripts/docker/Dockerfile:ARG MET_BASE_TAG=v3.3
./.github/workflows/sonarqube.yml:          MET_BASE_TAG: v3.3
./.github/workflows/build_docker_and_trigger_metplus.yml:          MET_BASE_TAG: v3.3

Compiling on seneca using Python 3.12 results in 2 new categories of warnings.

warning: ‘void Py_SetProgramName(const wchar_t*)’ is deprecated [-Wdeprecated-declarations] appears 20 times.
warning: ‘void PySys_SetArgv(int, wchar_t**)’ is deprecated [-Wdeprecated-declarations] appears 6 times.

Some options about the first one are discussed in Stack Overflow. It will apparently be fully removed from Python 3.15.

Rather than holding up this PR, I will write up a separate issue to deal in MET to deal with those deprecation warnings that apparently first appeared in Python 3.11.

…se image 3.4 and Python 3.12

georgemccabe · 2025-03-13T17:35:28Z

@JohnHalleyGotway, I updated the references to METbaseimage v3.3 to v.3.4 and Python 3.10 to Python 3.12 in the files you listed. In the PR text, I mentioned that I did not update the version in the documentation because we have not increased the minimum python requirement from 3.10 since we confirmed that these changes still work using Python 3.10.

Thanks for creating an issue to address the deprecated Python/C++ code.

JohnHalleyGotway

I approve of these changes.

Thanks for the updates and confirming the detail about the documentation. As we move forward with future versions, seems like we really should test with Python 3.10 at some frequency to confirm that the minimum version continues to work.

…nter/MET#3105

georgemccabe added 14 commits February 4, 2025 13:49

per dtcenter/METbaseimage#30, update variables to reflect change from…

d24a84d

… python 3.10 to 3.12.0

set met base tag to test

77879df

remove warning that $ is invalid

79b093d

print exception error so it is easier to debug failures in script

82b4de7

change np.NaN to np.nan because the former was deprecated in numpy 2.0

834b94c

convert python embedding attributes to a string before converting to …

07ac764

…json to prevent error handling float32 values

define json encoder to convert numpy types to python -- see gist http…

9545467

…s://gist.github.com/jonathanlurie/1b8d12f938b400e54c1ed8de21269b65

move numpy json encoder to common area where it can be referenced by …

4ea7407

…2 calls to json.dump

add encoder to python embedding script (to test, should move later)

f82db67

update /nrit/ral python to use 3.12

a726f02

Merge branch 'develop' into feature_baseimage_30_py3.12.0

83acd35

convert numpy types to python in the attributes to fix issue introduc…

f5c488e

…ed in fire weather use case moving to numpy 2.X

merge develop and resolved conflicts

0d2d444

use v3.4 METbaseimage that uses Python 3.12.0

7121ba7

georgemccabe added this to the MET-12.1.0 milestone Mar 12, 2025

georgemccabe added this to METplus-6.1 Development Mar 12, 2025

github-project-automation bot moved this to 🩺 Needs Triage in METplus-6.1 Development Mar 12, 2025

georgemccabe moved this from 🩺 Needs Triage to 🔎 In review in METplus-6.1 Development Mar 12, 2025

georgemccabe mentioned this pull request Mar 12, 2025

Update Truth: For #2930, #2926 and dtcenter/MET#3105 dtcenter/METplus#2925

Closed

11 tasks

georgemccabe requested a review from JohnHalleyGotway March 12, 2025 22:32

georgemccabe marked this pull request as ready for review March 12, 2025 22:32

JohnHalleyGotway mentioned this pull request Mar 13, 2025

Resolve Python deprecation warnings introduced during the switch to Python 3.12 #3106

Closed

21 tasks

per #3105 PR review from @JohnHalleyGotway, update versions to MET ba…

4d90ae0

…se image 3.4 and Python 3.12

JohnHalleyGotway approved these changes Mar 14, 2025

View reviewed changes

georgemccabe merged commit 42e6be4 into develop Mar 14, 2025
40 checks passed

github-project-automation bot moved this from 🔎 In review to 🏁 Done in METplus-6.1 Development Mar 14, 2025

georgemccabe deleted the feature_baseimage_30_py3.12.0 branch March 14, 2025 16:21

metplus-bot added a commit to dtcenter/METplus that referenced this pull request Mar 14, 2025

added entry to update truth change log: develop dtcenter/MET#3105

9f51317

github-actions bot mentioned this pull request Mar 14, 2025

Update develop-ref after dtcenter/MET#3105 dtcenter/METplus#2929

Merged

georgemccabe pushed a commit to dtcenter/METplus that referenced this pull request Mar 14, 2025

Update develop-ref after dtcenter/MET#3105 (#2929)

90c5eff

metplus-bot added a commit to dtcenter/METplus that referenced this pull request Mar 19, 2025

added entry to update truth change log: develop #2930, #2926 and dtce…

8124a84

…nter/MET#3105

georgemccabe pushed a commit to dtcenter/METplus that referenced this pull request Mar 19, 2025

Update develop-ref after #2930, #2926 and dtcenter/MET#3105 (#2934)

c1dca9e

hsoh-u pushed a commit that referenced this pull request May 15, 2025

#3105 Save the exception to the log message

3e4f0f9

hsoh-u pushed a commit that referenced this pull request May 15, 2025

#3105 log message to the log file if MET_PYTHON_DEBUG is set

563220e

hsoh-u pushed a commit that referenced this pull request May 15, 2025

#3105 Changed Py_SetProgramName to PyConfig_SetString

ccbe38e

hsoh-u pushed a commit that referenced this pull request May 15, 2025

#3105 Changed the first argumenmt type of python_point_data

b44e33f

hsoh-u pushed a commit that referenced this pull request May 15, 2025

#3105 Changed PySys_SetArgv to PyConfig_SetArgv

ed01ebc

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 Used f-string

08346da

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 Corrected not operator

38e0fa0

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 SonarQube finding: use auto

23146ea

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 Moved include directives to top

987e065

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 Removed unused variables

9cd57a0

hsoh-u pushed a commit that referenced this pull request May 16, 2025

#3105 Fix SonarQube findings

2c8ff8a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature dtcenter/METbaseimage#30 Use Python 3.12.0 #3105

Feature dtcenter/METbaseimage#30 Use Python 3.12.0 #3105

Uh oh!

georgemccabe commented Mar 12, 2025 •

edited

Loading

Uh oh!

JohnHalleyGotway commented Mar 13, 2025 •

edited

Loading

Uh oh!

georgemccabe commented Mar 13, 2025

Uh oh!

JohnHalleyGotway left a comment

Uh oh!

Uh oh!

Uh oh!

Feature dtcenter/METbaseimage#30 Use Python 3.12.0 #3105

Feature dtcenter/METbaseimage#30 Use Python 3.12.0 #3105

Uh oh!

Conversation

georgemccabe commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Expected Differences

Pull Request Testing

Pull Request Checklist

Uh oh!

JohnHalleyGotway commented Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

georgemccabe commented Mar 13, 2025

Uh oh!

JohnHalleyGotway left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

georgemccabe commented Mar 12, 2025 •

edited

Loading

JohnHalleyGotway commented Mar 13, 2025 •

edited

Loading