Bugfix #3210 main_v12.1 WDIR #3211
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… the value for cnt_logic instead.
…alysis aggregates MPR lines to compute WDIR output. In main_v11.0, that job automatically removes 0 vectors when computing ROW_MEAN_WDIR output.
23 tasks
added 3 commits
July 23, 2025 20:44
…ot just the non-zero MPR vectors.
This was referenced Jul 23, 2025
michelleharrold
previously approved these changes
Jul 23, 2025
|
@michelleharrold, FYI, one more piece of good news, I did do a small comparison between the WDIR and VCNT output statistics and found them to be consistent:
To get:
To get: Note that the ME values of |
…ince the local v_info object was declared outside the loop, when d_diff is bad data, the values from the previous iteration were being using. This cause the TOTAL_DIR value to be reported as the TOTAL value and not properly skip over the zero length vectors as it should... and as Point-Stat does.
|
@michelleharrold I need to request a re-review of this PR after finding/fixing one additional bug in the aggregation of MPR inputs and writing VCNT outputs. I found that aggregating MPR inputs DID NOT produce the exact same VCNT output that is computed by Point-Stat. This problem is fixed by this commit. Note that I did add this detail to the #3210 issue: There is logic to check that the direction difference ( The fix is just moving the declaration of |
michelleharrold
approved these changes
Jul 24, 2025
JohnHalleyGotway
added a commit
that referenced
this pull request
Jul 24, 2025
* Bugfix #3165 main v12.1 intel compilation (#3167) * Per #3165, correct two poorly formed OpenMP pragma statements. * Per #3165, add missing array include to ensemble_stat, point_stat, and plot_point_obs that's appearently needed when compiling with the Intel OneAPI compilers and with the --disable-opemp configuration option. * Per #3165, move 'using namespace std' after the includes in the all the test code and enum_parser.yy to avoid Intel OneAPI compilation warnings about it being implicitly defined. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, move 'using namespace std' after the includes in the all the test code. * Per #3165, add DISABLE_OPENMP option to compilation script * Per #3165, updating beta2 reference to rc1 * Per #3165, fix typo in internal/test_util/libcode/vx_grid/Makefile.am * Per #3165, removing typo in acorn config file * Per #3165, Test whether disabling custom reductions from the vx_series_data library enables MET to compile on WCOSS2 with Intel classic compilers. --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Hotfix to main_v12.1 to update the rc1 release note contents and date. * Updating versions for atlas and eckit * per #3175, replace bits/stdc++.h include that is GCC-specific with regex which works with other compilers (#3177) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. (#3180) * Bugfix #3179 main_v12.1 climo_file_type TAKE 2 (#3182) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. * Per #3179, modify the climo_name file_type logic to parse it from the top-level climo dictionary rather than parsing it separately for each field array entry. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Update install_met_env.wcoss2 Removing "-rc1" * Bugfix #3164 main_v12.1 slice range (#3184) * #3164 The range selection is inclusive like GRIB (the upper offset was not included before) * #3164 CHnaged warning to error for bad offset for Z * #3164 Changed index offsets to value notations * #3164 Support @level1-@lavel2 --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per dtcenter/METbaseimage#32, hotfix to the MET main_v12.1 branch to make the docker build enviornment more portable. * Adding modulefile for new machine ursa * Adding config file for new machine ursa * Feature dtcenter/METbaseimage#32 main_v12.1 release docker (#3195) * Per dtcenter/METbaseimage#32, port over changes from main_v12.0 to main_v12.1, but update the schedule to rebuild images weekly rather than daily. * Better comment * Feature #3198 main_v12.1 scan_for_CVEs (#3201) * Per #3198, port over CVE scanning logic from METbaseimage to MET. * Per #3198, use SOURCE_BRANCH instead of GITHUB_NAME for the scan_docker_image.sh script. * Fix typo * Update CVE log message * Per #3198, update the versions listed in the release-docker-images.yml workflow for the develop branch. * Bugfix #3206 main_v12.1 AW_MEAN (#3207) * Unrelated to #3206, increase the METbaseimage version number from v3.4 to v3.4.1 so that the resulting Docker images contain 0 critical vulnerabilities. * Per #3206, update met_regrid_area_weighted() function to fix bug when OMP_NUM_THREADS is set very large. Need to use a reduction for vectors of doubles to prevent two threads from updating the same to_data value at the same time. * Per #3206, add comment. * Bugfix #3210 main_v12.1 WDIR (#3211) * Fix jobstring to write -out_wind_logic correctly instead of reporting the value for cnt_logic instead. * Restore logic in aggr_mpr_wind_lines() removed for #2395 when STAT-Analysis aggregates MPR lines to compute WDIR output. In main_v11.0, that job automatically removes 0 vectors when computing ROW_MEAN_WDIR output. * Per #3210, update the revision history * Per #3210, update the WDIR documentation * Per #3210, add one more WDIR job to STAT-Analysis using all inputs, not just the non-zero MPR vectors. * Unrelated to #3210, but noted by @CPKalb, note that MET can read raw WRF output. * Per #3210, solve one additional bug in the MPR -> VCNT aggregation. Since the local v_info object was declared outside the loop, when d_diff is bad data, the values from the previous iteration were being using. This cause the TOTAL_DIR value to be reported as the TOTAL value and not properly skip over the zero length vectors as it should... and as Point-Stat does. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 12, 2025
* Bugfix #3165 main v12.1 intel compilation (#3167) * Per #3165, correct two poorly formed OpenMP pragma statements. * Per #3165, add missing array include to ensemble_stat, point_stat, and plot_point_obs that's appearently needed when compiling with the Intel OneAPI compilers and with the --disable-opemp configuration option. * Per #3165, move 'using namespace std' after the includes in the all the test code and enum_parser.yy to avoid Intel OneAPI compilation warnings about it being implicitly defined. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, move 'using namespace std' after the includes in the all the test code. * Per #3165, add DISABLE_OPENMP option to compilation script * Per #3165, updating beta2 reference to rc1 * Per #3165, fix typo in internal/test_util/libcode/vx_grid/Makefile.am * Per #3165, removing typo in acorn config file * Per #3165, Test whether disabling custom reductions from the vx_series_data library enables MET to compile on WCOSS2 with Intel classic compilers. --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Hotfix to main_v12.1 to update the rc1 release note contents and date. * Updating versions for atlas and eckit * per #3175, replace bits/stdc++.h include that is GCC-specific with regex which works with other compilers (#3177) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. (#3180) * Bugfix #3179 main_v12.1 climo_file_type TAKE 2 (#3182) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. * Per #3179, modify the climo_name file_type logic to parse it from the top-level climo dictionary rather than parsing it separately for each field array entry. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Update install_met_env.wcoss2 Removing "-rc1" * Bugfix #3164 main_v12.1 slice range (#3184) * #3164 The range selection is inclusive like GRIB (the upper offset was not included before) * #3164 CHnaged warning to error for bad offset for Z * #3164 Changed index offsets to value notations * #3164 Support @level1-@lavel2 --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per dtcenter/METbaseimage#32, hotfix to the MET main_v12.1 branch to make the docker build enviornment more portable. * Adding modulefile for new machine ursa * Adding config file for new machine ursa * Feature dtcenter/METbaseimage#32 main_v12.1 release docker (#3195) * Per dtcenter/METbaseimage#32, port over changes from main_v12.0 to main_v12.1, but update the schedule to rebuild images weekly rather than daily. * Better comment * Feature #3198 main_v12.1 scan_for_CVEs (#3201) * Per #3198, port over CVE scanning logic from METbaseimage to MET. * Per #3198, use SOURCE_BRANCH instead of GITHUB_NAME for the scan_docker_image.sh script. * Fix typo * Update CVE log message * Per #3198, update the versions listed in the release-docker-images.yml workflow for the develop branch. * Bugfix #3206 main_v12.1 AW_MEAN (#3207) * Unrelated to #3206, increase the METbaseimage version number from v3.4 to v3.4.1 so that the resulting Docker images contain 0 critical vulnerabilities. * Per #3206, update met_regrid_area_weighted() function to fix bug when OMP_NUM_THREADS is set very large. Need to use a reduction for vectors of doubles to prevent two threads from updating the same to_data value at the same time. * Per #3206, add comment. * Bugfix #3210 main_v12.1 WDIR (#3211) * Fix jobstring to write -out_wind_logic correctly instead of reporting the value for cnt_logic instead. * Restore logic in aggr_mpr_wind_lines() removed for #2395 when STAT-Analysis aggregates MPR lines to compute WDIR output. In main_v11.0, that job automatically removes 0 vectors when computing ROW_MEAN_WDIR output. * Per #3210, update the revision history * Per #3210, update the WDIR documentation * Per #3210, add one more WDIR job to STAT-Analysis using all inputs, not just the non-zero MPR vectors. * Unrelated to #3210, but noted by @CPKalb, note that MET can read raw WRF output. * Per #3210, solve one additional bug in the MPR -> VCNT aggregation. Since the local v_info object was declared outside the loop, when d_diff is bad data, the values from the previous iteration were being using. This cause the TOTAL_DIR value to be reported as the TOTAL value and not properly skip over the zero length vectors as it should... and as Point-Stat does. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #3191 v12.1.0 (#3217) * Per #3191, update GitHub actions logic prior to the MET-12.1.0 release to use consistent indentation, switch to using , update scan_docker_image.sh to fail on critical CVEs, update the cve scanning function to print a summary, and update the versions of MET to be rebuilt on schedule. * Per #3191, update the METbaseimage version from 'v3.4.1' to '3.4-latest' to enable security updates in METbaseimage to be incorparated into future rebuilds of the MET image. * Per #3191, consolidate MET-12.1.0 release notes. * Per #3191, more updates to docs for the 12.1.0 release. * Per #3191, add back in the MET-12.0.Z release notes since we only remove old release notes when we increase the major version number. * Per #3191, refine GHA job logic and log formatting. * Per #3191, switch to using ENV a=b instead of deprecated ENV a b * In installation.rst change python 3.10 to 3.12 * change release date --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 28, 2025
* Bugfix #3165 main v12.1 intel compilation (#3167) * Per #3165, correct two poorly formed OpenMP pragma statements. * Per #3165, add missing array include to ensemble_stat, point_stat, and plot_point_obs that's appearently needed when compiling with the Intel OneAPI compilers and with the --disable-opemp configuration option. * Per #3165, move 'using namespace std' after the includes in the all the test code and enum_parser.yy to avoid Intel OneAPI compilation warnings about it being implicitly defined. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, move 'using namespace std' after the includes in the all the test code. * Per #3165, add DISABLE_OPENMP option to compilation script * Per #3165, updating beta2 reference to rc1 * Per #3165, fix typo in internal/test_util/libcode/vx_grid/Makefile.am * Per #3165, removing typo in acorn config file * Per #3165, Test whether disabling custom reductions from the vx_series_data library enables MET to compile on WCOSS2 with Intel classic compilers. --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Hotfix to main_v12.1 to update the rc1 release note contents and date. * Updating versions for atlas and eckit * per #3175, replace bits/stdc++.h include that is GCC-specific with regex which works with other compilers (#3177) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. (#3180) * Bugfix #3179 main_v12.1 climo_file_type TAKE 2 (#3182) * Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. * Per #3179, modify the climo_name file_type logic to parse it from the top-level climo dictionary rather than parsing it separately for each field array entry. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Update install_met_env.wcoss2 Removing "-rc1" * Bugfix #3164 main_v12.1 slice range (#3184) * #3164 The range selection is inclusive like GRIB (the upper offset was not included before) * #3164 CHnaged warning to error for bad offset for Z * #3164 Changed index offsets to value notations * #3164 Support @level1-@lavel2 --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per dtcenter/METbaseimage#32, hotfix to the MET main_v12.1 branch to make the docker build enviornment more portable. * Adding modulefile for new machine ursa * Adding config file for new machine ursa * Feature dtcenter/METbaseimage#32 main_v12.1 release docker (#3195) * Per dtcenter/METbaseimage#32, port over changes from main_v12.0 to main_v12.1, but update the schedule to rebuild images weekly rather than daily. * Better comment * Feature #3198 main_v12.1 scan_for_CVEs (#3201) * Per #3198, port over CVE scanning logic from METbaseimage to MET. * Per #3198, use SOURCE_BRANCH instead of GITHUB_NAME for the scan_docker_image.sh script. * Fix typo * Update CVE log message * Per #3198, update the versions listed in the release-docker-images.yml workflow for the develop branch. * Bugfix #3206 main_v12.1 AW_MEAN (#3207) * Unrelated to #3206, increase the METbaseimage version number from v3.4 to v3.4.1 so that the resulting Docker images contain 0 critical vulnerabilities. * Per #3206, update met_regrid_area_weighted() function to fix bug when OMP_NUM_THREADS is set very large. Need to use a reduction for vectors of doubles to prevent two threads from updating the same to_data value at the same time. * Per #3206, add comment. * Bugfix #3210 main_v12.1 WDIR (#3211) * Fix jobstring to write -out_wind_logic correctly instead of reporting the value for cnt_logic instead. * Restore logic in aggr_mpr_wind_lines() removed for #2395 when STAT-Analysis aggregates MPR lines to compute WDIR output. In main_v11.0, that job automatically removes 0 vectors when computing ROW_MEAN_WDIR output. * Per #3210, update the revision history * Per #3210, update the WDIR documentation * Per #3210, add one more WDIR job to STAT-Analysis using all inputs, not just the non-zero MPR vectors. * Unrelated to #3210, but noted by @CPKalb, note that MET can read raw WRF output. * Per #3210, solve one additional bug in the MPR -> VCNT aggregation. Since the local v_info object was declared outside the loop, when d_diff is bad data, the values from the previous iteration were being using. This cause the TOTAL_DIR value to be reported as the TOTAL value and not properly skip over the zero length vectors as it should... and as Point-Stat does. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #3191 v12.1.0 (#3217) * Per #3191, update GitHub actions logic prior to the MET-12.1.0 release to use consistent indentation, switch to using , update scan_docker_image.sh to fail on critical CVEs, update the cve scanning function to print a summary, and update the versions of MET to be rebuilt on schedule. * Per #3191, update the METbaseimage version from 'v3.4.1' to '3.4-latest' to enable security updates in METbaseimage to be incorparated into future rebuilds of the MET image. * Per #3191, consolidate MET-12.1.0 release notes. * Per #3191, more updates to docs for the 12.1.0 release. * Per #3191, add back in the MET-12.0.Z release notes since we only remove old release notes when we increase the major version number. * Per #3191, refine GHA job logic and log formatting. * Per #3191, switch to using ENV a=b instead of deprecated ENV a b * In installation.rst change python 3.10 to 3.12 * change release date --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #3219 main_v12.1 python (#3220) * Per #3219, add GlobalPython:set_args() function to store the Python path prior to calling Py_InitializeFromConfig() and then restore the Python path afterwards. For Python 3.10, the call to Py_InitializeFromConfig() resets the Python path while in Python 3.12 it does not. * Per #3219, update logic to eliminate duplicated log messages. * Hotfix to `release-docker-images.yml` to disable fail-fast. * Per #3228, updates for the METv12.1.1 bugfix release, including release notes, doc version, code version, and the release docker images workflow. (#3229) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Expected Differences
Two files are modified:
stat_job.cchas two changes:out_cnt_logictoout_wind_logictypoWDIRoutput. Theaggr_mpr_wind_lines()function is called for WDIR, VCNT, and VL1L2 output line types. And-out_wind_logicshould be reported for all. Rather than checking 3 possible output line types, just remove the check, since checking thatout_fcst_wind_threshorout_obs_wind_threshis actually set is sufficient and much simpler. If the user has specified those wind thresholds, then we should report the logic being applied to them.aggr_stat_line.ccadds back in the check for zero length vectors but only applies it when WDIR output is requested.Do these changes introduce new tools, command line arguments, or configuration file options? [No]
If yes, please describe:
Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [Yes or No]
If yes, please describe:
Pull Request Testing
Describe testing already performed for these changes:
main_v12.1branch:To produce this output where
TOTAL = 934:To produce this output where
TOTAL = 934forAGGR_WDIRandTOTAL = 685forROW_MEAN_WDIR:WDIRoutput is NOT requested, all pairs are used (e.g.TOTAL = 934).To produce this output where
TOTAL = 685for both.Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
Review code changes. Review my small doc update. Run any additional tests you'd like.
Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
Yes, I added more detail in the STAT-Analysis WDIR section.
Do these changes include sufficient testing updates? [Yes]
Yes, added one new STAT-Analysis job to an existing config file to compute WDIR on all inputs... immediately before computing it only on the non-zero ones.
Will this PR result in changes to the MET test suite? [Yes]
If yes, describe the new output and/or changes to the existing output:
The modified STAT-Analysis output will be flagged as a difference.
Will this PR result in changes to existing METplus Use Cases? [No]
If yes, create a new Update Truth METplus issue to describe them.
Do these changes introduce new SonarQube findings? [No]
If yes, please describe:
One code smell is flagged in "New Code" but the overall number of code smells for main_v12.1 remain unchanged at 15,585.
Please complete this pull request review by [Thurs 7/24].
Pull Request Checklist
See the METplus Workflow for details.
Select: Reviewer(s) and Development issue
Select: Milestone as the version that will include these changes
Select: METplus-X.Y Support project for bugfix releases or MET-X.Y Development project for the next coordinated release