Bugfix #3020 main_v12.0 grid_stat_seeps #3022
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file.
23 tasks
j-opatz
approved these changes
Nov 18, 2024
JohnHalleyGotway
added a commit
that referenced
this pull request
Nov 22, 2024
… GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests.
JohnHalleyGotway
added a commit
that referenced
this pull request
Dec 19, 2024
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Feb 1, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * Fix typo, missing one * to make SciPy bold in appendixF.rst * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Bugfix #3054 main_v12.0 parusr (#3068) * Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that I checked all other calls to c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, revert emplace_back() to its original push_back() to make the bugfix diffs as limited as possible. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Per #3070, updates for the 12.0.1 bugfix release. (#3071) --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
JohnHalleyGotway
added a commit
that referenced
this pull request
Feb 13, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * Fix typo, missing one * to make SciPy bold in appendixF.rst * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Bugfix #3054 main_v12.0 parusr (#3068) * Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that I checked all other calls to c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, revert emplace_back() to its original push_back() to make the bugfix diffs as limited as possible. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Per #3070, updates for the 12.0.1 bugfix release. (#3071) * Updating file for 12.0.1 installation for NCO * Updating to 12.0.1 for NCO * Update and rename 12.0.0_acorn to 12.0.1_acorn for NCO * Rename 12.0.0.lua_wcoss2 to 12.0.1.lua_wcoss2 for NCO * Update 12.0.0_hercules * Update install_met_env.hercules * Update compiler and MET version in install_met_env.orion * Update compiler and MET version in 12.0.0_orion * Bugfix #3075 main_v12.0 optimization (#3076) * Per #3075, update get_att_value_chars() utility function to store the attribute value in a string rather than a fixed-length character array for which overflow may occur. * Per #3075, switch from compiling MET in Docker using the -g debug flag to using -O2 optimization since that's what how we configure installations on supported platforms. This makes the testing environment more simliar to the deployed versions. And we've found some bugs due to unexpected behavior when compiled with -O2 optimization. * Per #3075, remove accidentally committed log file * Per #3075, update TrackInfo::diag_name() to return a string rather than a pointer to temporary memory to solve the problem with diagnostic names in unit_tc_pairs.xml when compiled with optimization enabled. * Per #3075, update read_netcdf_logic() to store pointers to class members rather than local variables which go out of scope. * Per #3075, don't need to use local variables at all. * Per #3075, switch to using STL vectors for memory management * Per #3075, reimplement month_name_to_m() with stl strings to avoid variable length arrays. * Per #3075, update MetNcFile::readFile() to use stl vectors instead of variable length arrays * Per #3075, update NcCfFile member functions to use stl vectors instead of variable length arrays * Per #3075, update is_netcdf_file() to use stl vectors instead of variable length arrays * Per #3075, update 3d_conv.cc to use stl vectors instead of variable length arrays * Per #3075, update the vx_util library to use stl vectors instead of variable length arrays * Per #3075, update ensemble_stat to use stl vectors instead of variable length arrays * Per #3075, update decode_lat_lon() to use stl vectors instead of variable length arrays * Per #3075, update grid_diag to use stl vectors instead of variable length arrays * Per #3075, update ioda2nc to use stl vectors instead of variable length arrays * Per #3075, update madis2nc to use stl vectors instead of variable length arrays * Per #3075, update mode_graphics to use stl vectors instead of variable length arrays * Per #3075, update the vx_nc_obs library to use stl vectors instead of variable length arrays * Per #3075, update plot_point_obs to use stl vectors instead of variable length arrays * Per #3075, update point_stat to use stl vectors instead of variable length arrays * Per #3075, update wavelet_stat to use stl vectors instead of variable length arrays * Per #3075, no real code change, just whitespace * Per #3075, removing commented out code * Per #3075, need to add 2 to account for time_count being initialized to -1. An array of length 0 is different from a vector of length 0. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, update looping logic * Per #3075, eliminate all instances of vector<vector<type>> since it's not stored in contiguous memory and therefore not useful for reading data from the NetCDF files. * Per #303075, bit more madis2nc changes. * Per #3075, fix Nx typo * Per #3075, fix chaNetCDF attribute character type * Per #3075, minor changes to satisfy SonarQube findings. * Per #3075, when sizing vectors of type <char> add one for the trailing null. * Per #3075, remove debugging code. * Per #3075, unit_ioda2nc.xml fails when compiled with Intel since there are issues parsing NC_STRING attribute types. Reverting back to the previous logic from main_v12.0 since that works. * Per #3750, back out the change to using -O2 in development.docker. With it, differences on flagged by GHA. I'd like to make sure all the changes on this branch cause NO differences before switching to using -O2... most likely in the develop branch rather than main_v12.0. * Per #3075, getting segfault from point2grid. Null terminating character vectors after reading NetCDF attributes just to be safe. * Unrelated to #3075, only whitespace changes. * Per #3075, fix logic of the write_nc(...) function so that all variable attributes and added and defined prior to writing the data for that variable. Writing attributes AFTER the data, as we had been doing, causes unexpected failures, as found when compiled with Intel. * Per #3075, update args to write_nc(...) to minimize regression test diffs. * Per #3075, fix madis2nc i_buf definition problem. * Per #3075, more refinement of i_buf definition in madis2nc for acars and raob inputs. * Per #3075, remove FFLAGS from development.docker becuase there's no good reason to add it. * replace raw array with std array * replace raw array with std array in 2 other places * Per #3075, fix clear bug in vx_bool_calc/tokenizer.cc where the = assignment operator is used when the == comparison operator is needed. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
JohnHalleyGotway
added a commit
that referenced
this pull request
Feb 15, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * Fix typo, missing one * to make SciPy bold in appendixF.rst * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Bugfix #3054 main_v12.0 parusr (#3068) * Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that I checked all other calls to c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, revert emplace_back() to its original push_back() to make the bugfix diffs as limited as possible. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Per #3070, updates for the 12.0.1 bugfix release. (#3071) * Updating file for 12.0.1 installation for NCO * Updating to 12.0.1 for NCO * Update and rename 12.0.0_acorn to 12.0.1_acorn for NCO * Rename 12.0.0.lua_wcoss2 to 12.0.1.lua_wcoss2 for NCO * Update 12.0.0_hercules * Update install_met_env.hercules * Update compiler and MET version in install_met_env.orion * Update compiler and MET version in 12.0.0_orion * Bugfix #3075 main_v12.0 optimization (#3076) * Per #3075, update get_att_value_chars() utility function to store the attribute value in a string rather than a fixed-length character array for which overflow may occur. * Per #3075, switch from compiling MET in Docker using the -g debug flag to using -O2 optimization since that's what how we configure installations on supported platforms. This makes the testing environment more simliar to the deployed versions. And we've found some bugs due to unexpected behavior when compiled with -O2 optimization. * Per #3075, remove accidentally committed log file * Per #3075, update TrackInfo::diag_name() to return a string rather than a pointer to temporary memory to solve the problem with diagnostic names in unit_tc_pairs.xml when compiled with optimization enabled. * Per #3075, update read_netcdf_logic() to store pointers to class members rather than local variables which go out of scope. * Per #3075, don't need to use local variables at all. * Per #3075, switch to using STL vectors for memory management * Per #3075, reimplement month_name_to_m() with stl strings to avoid variable length arrays. * Per #3075, update MetNcFile::readFile() to use stl vectors instead of variable length arrays * Per #3075, update NcCfFile member functions to use stl vectors instead of variable length arrays * Per #3075, update is_netcdf_file() to use stl vectors instead of variable length arrays * Per #3075, update 3d_conv.cc to use stl vectors instead of variable length arrays * Per #3075, update the vx_util library to use stl vectors instead of variable length arrays * Per #3075, update ensemble_stat to use stl vectors instead of variable length arrays * Per #3075, update decode_lat_lon() to use stl vectors instead of variable length arrays * Per #3075, update grid_diag to use stl vectors instead of variable length arrays * Per #3075, update ioda2nc to use stl vectors instead of variable length arrays * Per #3075, update madis2nc to use stl vectors instead of variable length arrays * Per #3075, update mode_graphics to use stl vectors instead of variable length arrays * Per #3075, update the vx_nc_obs library to use stl vectors instead of variable length arrays * Per #3075, update plot_point_obs to use stl vectors instead of variable length arrays * Per #3075, update point_stat to use stl vectors instead of variable length arrays * Per #3075, update wavelet_stat to use stl vectors instead of variable length arrays * Per #3075, no real code change, just whitespace * Per #3075, removing commented out code * Per #3075, need to add 2 to account for time_count being initialized to -1. An array of length 0 is different from a vector of length 0. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, update looping logic * Per #3075, eliminate all instances of vector<vector<type>> since it's not stored in contiguous memory and therefore not useful for reading data from the NetCDF files. * Per #303075, bit more madis2nc changes. * Per #3075, fix Nx typo * Per #3075, fix chaNetCDF attribute character type * Per #3075, minor changes to satisfy SonarQube findings. * Per #3075, when sizing vectors of type <char> add one for the trailing null. * Per #3075, remove debugging code. * Per #3075, unit_ioda2nc.xml fails when compiled with Intel since there are issues parsing NC_STRING attribute types. Reverting back to the previous logic from main_v12.0 since that works. * Per #3750, back out the change to using -O2 in development.docker. With it, differences on flagged by GHA. I'd like to make sure all the changes on this branch cause NO differences before switching to using -O2... most likely in the develop branch rather than main_v12.0. * Per #3075, getting segfault from point2grid. Null terminating character vectors after reading NetCDF attributes just to be safe. * Unrelated to #3075, only whitespace changes. * Per #3075, fix logic of the write_nc(...) function so that all variable attributes and added and defined prior to writing the data for that variable. Writing attributes AFTER the data, as we had been doing, causes unexpected failures, as found when compiled with Intel. * Per #3075, update args to write_nc(...) to minimize regression test diffs. * Per #3075, fix madis2nc i_buf definition problem. * Per #3075, more refinement of i_buf definition in madis2nc for acars and raob inputs. * Per #3075, remove FFLAGS from development.docker becuase there's no good reason to add it. * replace raw array with std array * replace raw array with std array in 2 other places * Per #3075, fix clear bug in vx_bool_calc/tokenizer.cc where the = assignment operator is used when the == comparison operator is needed. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #3075, adding a hotfix to main_v12.0 that should have been included in the #3076 pull request. The logic for converting to camel case was in the wrong order and the details of this are described in this comment: #3078 (comment) * Bugfix #3077 main_v12.0 yyerror (#3083) * Per #3077, back out the yystate patching logic added to Makefile.am for MET #2408 to allow for the parsing of empty configuration file. This has caused a 'shift/reduce conflict' warning message from the Intel compiler, is flagged as problem by '-fsanitize=address', and very well may be causing the sporadic yyerror failures described in #3077. Recommend testing with this change to see if the yyerror's go away, but also re-testing MET #2408 to assess the handling of empty configuration files. * Per #3075, update config.tab.yy/.cc to allow for empty inputs but allowing for an empty input in the statement_list. * Feature #3081 v12.0.2 (#3085) * Per #3081, roll the MET version number from 12.0.1 to 12.0.2 and add release notes. * Per #3081, improve the title for issue #3075 --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
bikegeek
added a commit
that referenced
this pull request
May 14, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * initial YAML config file used to perform benchmarking on METplus use cases * initial version of code used to run use case on MET that has CTRACK macro enables * Fix typo, missing one * to make SciPy bold in appendixF.rst * Add support to run the use case multiple times, based on the num_runs setting in the yaml config file * Add support for determining the path where the YAML file is located * Allow a list of wrapper config files, create subdirectories named after use case config files, add more assertion checks * Add list to wrapper config file * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Removed incorrectly placed asserts and fixed incorrect comment * Fixed assertion that checks output file size * Remove assert in check, checking for a single config file when a list is expected. * clean up comments, mv summary_output.txt and detail_output.txt files after finishing, format the information file * Removed option to save output as csv, tabular or both. Both formats are saved. * Fixed name of config file * Fix the command to get the directory where the script has been invoked * Incorrectly passing in the use case path instead of name into generate_info() * copy rather than move the summary and detail files generated by CTRACK * Modified to write summary and detail information to files * using CTRACK macro * using CTRACK macro to benchmark code * Use CTRACK macro to perform benchmarking * Updated to address seg fault when running * Fixes to address seg fault when running the tool * Create README.md first version * Added call to ctrack to write out results in the "main" function met_main() * ctrack updated to write summary and details to files * Update README.md added reminder to add ctrack::result_print(); to ensure the summary and detail files are written * CTRACK include file * Removed hard-coded paths and replaced output_path with benchmark_output_path to separate output from METplus Use case output * Clean up parsing yaml config, read the benchmark_output_path in YAML instead of output_path * Write out ctrack information to files * Write out ctrack results to files * Update README.md Finished description of settings in YAML config * Update README.md update to instructions- yaml config file doesn't need to be specified at the command line * Adding support to read in updates to YAML file to allow running MET command * Replace latest_benchmark.yaml with benchmark.yaml * Added config support for specifying running MET directly * Removed remaining reference to latest_benchmark.yaml * Added setting for specifying a subdirectory to save output when running via MET * Refactoring the code for running usecases and adding support for running MET directly * fixed indentation error * Remove CTRACK macro and include * Removing ctrack header to move to vx_util * Remove CTRACK macro and include * Removed and placed under the vx_util directory * Added ctrack include, use macro for pre- and post-processing functions * Add the CTRACK benchmarking tool * Update checking and saving information to be generic and support running via METplus wrappers or MET directly * Modify check_settings to check settings for running use cases or running MET directly * Instrument with CTRACK macro for benchmarking * Support setting a description (optional) which will appear in the information file * Include instructions to save the benchmarking summary and detail information to files * Fix setting name from met_cmd to met_command to match what is in the YAML file * Pass MET command as detail into the info.txt file * Convert full_subdir_path to string * Pass in the full subdir path to extract_summary_info and extract_detail_info to get correct behavior * Make the MET final output subdirectory (where consolidated files go) if it doesn't exist * Remove call to printing the ctrack report to file * Fix input to generate_info: replace "met_command" with met_command * Add the METPLUS_ environment information to the information text file when running the MET command directly * set additional text to None if there is no additional information (i.e. use case run) * Fixed order for generate_info args inside run_met_cli * Add the values of the METplus ENV vars used in run_met_cli * Removed conflict * Updates added for running MET commands * Adding constants for time units. This will be used when calculating the mean metric when multiple runs are performed for stress-testing. * Latest updates to first iteration of the benchmarking tool * Clean up of format and instructions * Clean up some grammar * Fix some formatting of code snippets * Code snippets cannot have any italics or bold text * Still have issues with formatting code snippets * New lines not being kept in code * more experimentation with formatting code snippet * Remove special characters around code snippet * more experimenting with additional newlines to format code * Removed all code block syntax and use only newlines and indentation to format code blocks * Re-arrange instructions for printing reports * Update ensemble_stat.cc Add the code to met_main() to save the CTRACK metrics * Added ifdef WITH_PROFILER for turning on CTRACK at compile time * Instrument code for benchmarking with CTRACK * Adding documentation for running the code profiler used in benchmarking * updated for use in the Contributor's Guide * Moved content to the Contributor's Guide * Replaced formatting and cleaned up content * Clean up some formatting used by the README.md but incompatible with RST * Added missing information for environments for running MET commands, cleaned up formatting * Rearrange some text for clarity * Fixed the introduced conflicts from earlier pull * Reorganized to reduce clutter and redundant content. * Added clarification to Compile MET code section * Replace the 'r' in the regex compile that was somehow removed * Open the summary_out.txt and detail_out.txt files with append to prevent clobbering metrics when switching from profiling functions from different files. * Fixed merging of summary and detail data to ensure the matching file/function/lines are merged. * Renamed new index to profiling_index for clarity * Updates with information about the num_runs setting and where the summary and detail files are moved. * Removing, the correct version is located in the MET/internal/scripts/benchmark directory * Inadvertently added but not relevant information for this commit * No longer necessary, this information is in the Contributor's Guide and developers automatically refer to the ReadTheDocs documentation. * Delete summary_output.txt should not be in source code * Delete detail_output.txt output should not be included with the source code * Delete internal/scripts/benchmark/summary_output.txt output should not be included with source code * Delete internal/scripts/benchmark/detail_output.txt output should not be included in the source code * Update system.conf remove hard-coded values * Delete internal/scripts/benchmark/constants.py not used for this code * Update benchmark.yaml num_of_runs should match instructions in the Contributor's Guide * Delete internal/scripts/benchmark/ctrack.hpp Correct location is in the src/basic/vx_util directory * Update example_MET.yaml remove hard-coded paths and set the num_of_runs to 1 to match instructions in documentation * Update example_usecase.yaml remove hard-coded values and set the num_of_runs to 1 to be consistent with instructions * Output files should not be included with source code * update to version in the develop branch * Revert to version in develop, undoing whitespace changes * Revert to version in develop branch, eliminating whitespace changes * Reduced the number of profiling points to just the two major top level methods * Per #3065, fix scripts/Makefile to ignore ENABLE_PROFILER when determining which tests to run. * Update code_profiling.rst No more error message during make test. Update comiler section accordingly * Update docs/Contributors_Guide/code_profiling.rst Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Update docs/Contributors_Guide/code_profiling.rst Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Jul 21, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * Fix typo, missing one * to make SciPy bold in appendixF.rst * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Bugfix #3054 main_v12.0 parusr (#3068) * Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that I checked all other calls to c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, revert emplace_back() to its original push_back() to make the bugfix diffs as limited as possible. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Per #3070, updates for the 12.0.1 bugfix release. (#3071) * Updating file for 12.0.1 installation for NCO * Updating to 12.0.1 for NCO * Update and rename 12.0.0_acorn to 12.0.1_acorn for NCO * Rename 12.0.0.lua_wcoss2 to 12.0.1.lua_wcoss2 for NCO * Update 12.0.0_hercules * Update install_met_env.hercules * Update compiler and MET version in install_met_env.orion * Update compiler and MET version in 12.0.0_orion * Bugfix #3075 main_v12.0 optimization (#3076) * Per #3075, update get_att_value_chars() utility function to store the attribute value in a string rather than a fixed-length character array for which overflow may occur. * Per #3075, switch from compiling MET in Docker using the -g debug flag to using -O2 optimization since that's what how we configure installations on supported platforms. This makes the testing environment more simliar to the deployed versions. And we've found some bugs due to unexpected behavior when compiled with -O2 optimization. * Per #3075, remove accidentally committed log file * Per #3075, update TrackInfo::diag_name() to return a string rather than a pointer to temporary memory to solve the problem with diagnostic names in unit_tc_pairs.xml when compiled with optimization enabled. * Per #3075, update read_netcdf_logic() to store pointers to class members rather than local variables which go out of scope. * Per #3075, don't need to use local variables at all. * Per #3075, switch to using STL vectors for memory management * Per #3075, reimplement month_name_to_m() with stl strings to avoid variable length arrays. * Per #3075, update MetNcFile::readFile() to use stl vectors instead of variable length arrays * Per #3075, update NcCfFile member functions to use stl vectors instead of variable length arrays * Per #3075, update is_netcdf_file() to use stl vectors instead of variable length arrays * Per #3075, update 3d_conv.cc to use stl vectors instead of variable length arrays * Per #3075, update the vx_util library to use stl vectors instead of variable length arrays * Per #3075, update ensemble_stat to use stl vectors instead of variable length arrays * Per #3075, update decode_lat_lon() to use stl vectors instead of variable length arrays * Per #3075, update grid_diag to use stl vectors instead of variable length arrays * Per #3075, update ioda2nc to use stl vectors instead of variable length arrays * Per #3075, update madis2nc to use stl vectors instead of variable length arrays * Per #3075, update mode_graphics to use stl vectors instead of variable length arrays * Per #3075, update the vx_nc_obs library to use stl vectors instead of variable length arrays * Per #3075, update plot_point_obs to use stl vectors instead of variable length arrays * Per #3075, update point_stat to use stl vectors instead of variable length arrays * Per #3075, update wavelet_stat to use stl vectors instead of variable length arrays * Per #3075, no real code change, just whitespace * Per #3075, removing commented out code * Per #3075, need to add 2 to account for time_count being initialized to -1. An array of length 0 is different from a vector of length 0. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, update looping logic * Per #3075, eliminate all instances of vector<vector<type>> since it's not stored in contiguous memory and therefore not useful for reading data from the NetCDF files. * Per #303075, bit more madis2nc changes. * Per #3075, fix Nx typo * Per #3075, fix chaNetCDF attribute character type * Per #3075, minor changes to satisfy SonarQube findings. * Per #3075, when sizing vectors of type <char> add one for the trailing null. * Per #3075, remove debugging code. * Per #3075, unit_ioda2nc.xml fails when compiled with Intel since there are issues parsing NC_STRING attribute types. Reverting back to the previous logic from main_v12.0 since that works. * Per #3750, back out the change to using -O2 in development.docker. With it, differences on flagged by GHA. I'd like to make sure all the changes on this branch cause NO differences before switching to using -O2... most likely in the develop branch rather than main_v12.0. * Per #3075, getting segfault from point2grid. Null terminating character vectors after reading NetCDF attributes just to be safe. * Unrelated to #3075, only whitespace changes. * Per #3075, fix logic of the write_nc(...) function so that all variable attributes and added and defined prior to writing the data for that variable. Writing attributes AFTER the data, as we had been doing, causes unexpected failures, as found when compiled with Intel. * Per #3075, update args to write_nc(...) to minimize regression test diffs. * Per #3075, fix madis2nc i_buf definition problem. * Per #3075, more refinement of i_buf definition in madis2nc for acars and raob inputs. * Per #3075, remove FFLAGS from development.docker becuase there's no good reason to add it. * replace raw array with std array * replace raw array with std array in 2 other places * Per #3075, fix clear bug in vx_bool_calc/tokenizer.cc where the = assignment operator is used when the == comparison operator is needed. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #3075, adding a hotfix to main_v12.0 that should have been included in the #3076 pull request. The logic for converting to camel case was in the wrong order and the details of this are described in this comment: #3078 (comment) * Bugfix #3077 main_v12.0 yyerror (#3083) * Per #3077, back out the yystate patching logic added to Makefile.am for MET #2408 to allow for the parsing of empty configuration file. This has caused a 'shift/reduce conflict' warning message from the Intel compiler, is flagged as problem by '-fsanitize=address', and very well may be causing the sporadic yyerror failures described in #3077. Recommend testing with this change to see if the yyerror's go away, but also re-testing MET #2408 to assess the handling of empty configuration files. * Per #3075, update config.tab.yy/.cc to allow for empty inputs but allowing for an empty input in the statement_list. * Feature #3081 v12.0.2 (#3085) * Per #3081, roll the MET version number from 12.0.1 to 12.0.2 and add release notes. * Per #3081, improve the title for issue #3075 * Update install_met_env.gaea * Update 12.0.0_gaea * Feature 3092 main v12.0 comp script (#3093) * Per #3092, updating compilation script and removing old wcoss file * Per #3092, updated config and modulefiles as necessary * Per #2891, updating reference to 12.0.0 to 12.0.2 * Bugfix #3095 main_v12.0 ascii2nc_tab_delimited (#3103) * Per #3095, when checking the IABP file type, just use the existing DataLine object rather than re-parsing based on whitespace. Also call is_eq() to check for equality of doubles rather than precise equality with ==. * Per dtcenter/METplus#2920 discussion (incl with issue #3095), update the is_eq(a,b) logic to return true if both values are NaN. * Per #3095, reorder for efficiency. * Feature METplus2771 main_v12.0 subprojects (#3116) * Adding ToC for subprojects * Removing references to feature branch * Per #3100, tweaks to issue and pull request templates to reflect project board updates. * Fixing zenodo link * Update grid-stat.rst for Zenodo link (#3124) Zenodo link was not working properly. Modified to use correct link. * Updating to reflect the setting of MET_PYTHON_EXE which is actually occurring on Casper * Minor update to the internal nightly build script (test_nightly.sh) to revise the email list and update the 'latest' link after each successful run. * #3107 Added GribEntryMatch and APIs (is_eq, match, is_new_entry) (#3135) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #3112 netcdf dimension main v12.0 (#3139) * #3112 Removed the number of dimensions. Exit if not found a data variable * #3112 Changed Error log to Warning * #3112 Fixed a bug with a index. Changed back error and stop --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Hotfix to develop remove write permission for groups others for nightly build runs and also log updates to the latest link. * Hotfix to develop branch to get NB latest link to update correctly. ci-skip-all * Per dtcenter/METbaseimage#32, hotfix to the MET main_v12.0 branch to make the docker build enviornment more portable. * Per dtcenter/METbaseimage#32, add workflow to build Docker images for MET releases on a schedule. * Per dtcenter/METbaseimage#32, update the release_docker_images.yml workflow to build releases using Dockerfile instead of Dockerfile.copy. * Per dtcenter/METbaseimage#32, omit the leading v for released versions in the Docker Hub tag. * Per METbaseimage#32, store logs for docker builds. * Update the Docker image logic to add X.Y-latest * Update release-docker-images workflow to do a shallow checkout of the Dockerfile for the version to be built, but run scripts from the triggered branch. * Avoid overwritting checkout * Fix Dockerfile checkout #ci-skip-unit * Update schedule to rebuild supported releases weekly instead of daily. * Feature #3198 main_v12.0 scan_for_CVEs (#3200) * Per #3198, port over CVE scanning logic from METbaseimage to MET. * Per #3198, use SOURCE_BRANCH instead of GITHUB_NAME for the scan_docker_image.sh script. * Fix typo * Update CVE log message * Per #3198, update the versions listed in the release-docker-images.yml workflow for the develop branch. * Per #3198, update the versions listed in the release-docker-images.yml workflow for the main_v6.0 branch. * Feature #3204 v12.0.3 (#3205) * Per #3204, update to version number 12.0.3. * Per #3204, update from METbaseimage version 3.3 to 3.3.1. * Per #3204, add v12.0.3 release notes. * Per #3204, update GHA automation to use the secret. * Per #3204, update release-docker-images.yml workflow to build the new 12.0.3 images instead of the previous 12.0.2. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Pratiman <31694629+pratiman-91@users.noreply.github.com> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
JohnHalleyGotway
added a commit
that referenced
this pull request
Jul 24, 2025
* use custom GitHub Action to trigger METplus use cases * Updating values * Bugfix #3020 main_v12.0 grid_stat_seeps (#3022) * Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file. * Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide. * Per #3020, no real changes. Just whitespace * Update to reflect usage of oneAPI compilers * Updating file to reflect usage of oneAPI compilers * Hotfix to the main_v12.0 branch after PR #3022 fixed a SEEPS bug. The GridStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests. * Adding In Memoriam * Feature #3032 main_v12.0 docs data type (#3040) * Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> * Making a superficial change in the main_v12.0 branch to trigger GHA to create and push an updated test output image. * Feature #3033 v12.0.0 (#3042) * Per #3033, update version info, consolidate release notes, and add upgrade instructions. * Per #3033, remove all instances of 'Bugfix: ' from the release notes since it's redundant with the dropdown name * Per #3030, based on request from Randy Pierce, also add MTD header columns to met_header_columns_v12.0.txt to make it easier to parse the output from MET. * Per #3033, fix typo and correct alignment in table * Update install_met_env.acorn Removing reference to beta version * Update install_met_env.cactus Remove references to beta version * Update install_met_env.cactus Update paths for eckit and atlas * Update install_met_env.wcoss2 Remove beta references * Fix typo, missing one * to make SciPy bold in appendixF.rst * Per #3051, update unit tests so that installed files are found relative to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3052) * Bugfix #3054 main_v12.0 parusr (#3068) * Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that I checked all other calls to c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, revert emplace_back() to its original push_back() to make the bugfix diffs as limited as possible. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Per #3070, updates for the 12.0.1 bugfix release. (#3071) * Updating file for 12.0.1 installation for NCO * Updating to 12.0.1 for NCO * Update and rename 12.0.0_acorn to 12.0.1_acorn for NCO * Rename 12.0.0.lua_wcoss2 to 12.0.1.lua_wcoss2 for NCO * Update 12.0.0_hercules * Update install_met_env.hercules * Update compiler and MET version in install_met_env.orion * Update compiler and MET version in 12.0.0_orion * Bugfix #3075 main_v12.0 optimization (#3076) * Per #3075, update get_att_value_chars() utility function to store the attribute value in a string rather than a fixed-length character array for which overflow may occur. * Per #3075, switch from compiling MET in Docker using the -g debug flag to using -O2 optimization since that's what how we configure installations on supported platforms. This makes the testing environment more simliar to the deployed versions. And we've found some bugs due to unexpected behavior when compiled with -O2 optimization. * Per #3075, remove accidentally committed log file * Per #3075, update TrackInfo::diag_name() to return a string rather than a pointer to temporary memory to solve the problem with diagnostic names in unit_tc_pairs.xml when compiled with optimization enabled. * Per #3075, update read_netcdf_logic() to store pointers to class members rather than local variables which go out of scope. * Per #3075, don't need to use local variables at all. * Per #3075, switch to using STL vectors for memory management * Per #3075, reimplement month_name_to_m() with stl strings to avoid variable length arrays. * Per #3075, update MetNcFile::readFile() to use stl vectors instead of variable length arrays * Per #3075, update NcCfFile member functions to use stl vectors instead of variable length arrays * Per #3075, update is_netcdf_file() to use stl vectors instead of variable length arrays * Per #3075, update 3d_conv.cc to use stl vectors instead of variable length arrays * Per #3075, update the vx_util library to use stl vectors instead of variable length arrays * Per #3075, update ensemble_stat to use stl vectors instead of variable length arrays * Per #3075, update decode_lat_lon() to use stl vectors instead of variable length arrays * Per #3075, update grid_diag to use stl vectors instead of variable length arrays * Per #3075, update ioda2nc to use stl vectors instead of variable length arrays * Per #3075, update madis2nc to use stl vectors instead of variable length arrays * Per #3075, update mode_graphics to use stl vectors instead of variable length arrays * Per #3075, update the vx_nc_obs library to use stl vectors instead of variable length arrays * Per #3075, update plot_point_obs to use stl vectors instead of variable length arrays * Per #3075, update point_stat to use stl vectors instead of variable length arrays * Per #3075, update wavelet_stat to use stl vectors instead of variable length arrays * Per #3075, no real code change, just whitespace * Per #3075, removing commented out code * Per #3075, need to add 2 to account for time_count being initialized to -1. An array of length 0 is different from a vector of length 0. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, can't use 2D vectors to read data from NetCDF files into a contiguous block of memory. * Per #3075, update looping logic * Per #3075, eliminate all instances of vector<vector<type>> since it's not stored in contiguous memory and therefore not useful for reading data from the NetCDF files. * Per #303075, bit more madis2nc changes. * Per #3075, fix Nx typo * Per #3075, fix chaNetCDF attribute character type * Per #3075, minor changes to satisfy SonarQube findings. * Per #3075, when sizing vectors of type <char> add one for the trailing null. * Per #3075, remove debugging code. * Per #3075, unit_ioda2nc.xml fails when compiled with Intel since there are issues parsing NC_STRING attribute types. Reverting back to the previous logic from main_v12.0 since that works. * Per #3750, back out the change to using -O2 in development.docker. With it, differences on flagged by GHA. I'd like to make sure all the changes on this branch cause NO differences before switching to using -O2... most likely in the develop branch rather than main_v12.0. * Per #3075, getting segfault from point2grid. Null terminating character vectors after reading NetCDF attributes just to be safe. * Unrelated to #3075, only whitespace changes. * Per #3075, fix logic of the write_nc(...) function so that all variable attributes and added and defined prior to writing the data for that variable. Writing attributes AFTER the data, as we had been doing, causes unexpected failures, as found when compiled with Intel. * Per #3075, update args to write_nc(...) to minimize regression test diffs. * Per #3075, fix madis2nc i_buf definition problem. * Per #3075, more refinement of i_buf definition in madis2nc for acars and raob inputs. * Per #3075, remove FFLAGS from development.docker becuase there's no good reason to add it. * replace raw array with std array * replace raw array with std array in 2 other places * Per #3075, fix clear bug in vx_bool_calc/tokenizer.cc where the = assignment operator is used when the == comparison operator is needed. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #3075, adding a hotfix to main_v12.0 that should have been included in the #3076 pull request. The logic for converting to camel case was in the wrong order and the details of this are described in this comment: #3078 (comment) * Bugfix #3077 main_v12.0 yyerror (#3083) * Per #3077, back out the yystate patching logic added to Makefile.am for MET #2408 to allow for the parsing of empty configuration file. This has caused a 'shift/reduce conflict' warning message from the Intel compiler, is flagged as problem by '-fsanitize=address', and very well may be causing the sporadic yyerror failures described in #3077. Recommend testing with this change to see if the yyerror's go away, but also re-testing MET #2408 to assess the handling of empty configuration files. * Per #3075, update config.tab.yy/.cc to allow for empty inputs but allowing for an empty input in the statement_list. * Feature #3081 v12.0.2 (#3085) * Per #3081, roll the MET version number from 12.0.1 to 12.0.2 and add release notes. * Per #3081, improve the title for issue #3075 * Update install_met_env.gaea * Update 12.0.0_gaea * Feature 3092 main v12.0 comp script (#3093) * Per #3092, updating compilation script and removing old wcoss file * Per #3092, updated config and modulefiles as necessary * Per #2891, updating reference to 12.0.0 to 12.0.2 * Bugfix #3095 main_v12.0 ascii2nc_tab_delimited (#3103) * Per #3095, when checking the IABP file type, just use the existing DataLine object rather than re-parsing based on whitespace. Also call is_eq() to check for equality of doubles rather than precise equality with ==. * Per dtcenter/METplus#2920 discussion (incl with issue #3095), update the is_eq(a,b) logic to return true if both values are NaN. * Per #3095, reorder for efficiency. * Feature METplus2771 main_v12.0 subprojects (#3116) * Adding ToC for subprojects * Removing references to feature branch * Per #3100, tweaks to issue and pull request templates to reflect project board updates. * Fixing zenodo link * Update grid-stat.rst for Zenodo link (#3124) Zenodo link was not working properly. Modified to use correct link. * Updating to reflect the setting of MET_PYTHON_EXE which is actually occurring on Casper * Minor update to the internal nightly build script (test_nightly.sh) to revise the email list and update the 'latest' link after each successful run. * #3107 Added GribEntryMatch and APIs (is_eq, match, is_new_entry) (#3135) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #3112 netcdf dimension main v12.0 (#3139) * #3112 Removed the number of dimensions. Exit if not found a data variable * #3112 Changed Error log to Warning * #3112 Fixed a bug with a index. Changed back error and stop --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Hotfix to develop remove write permission for groups others for nightly build runs and also log updates to the latest link. * Hotfix to develop branch to get NB latest link to update correctly. ci-skip-all * Per dtcenter/METbaseimage#32, hotfix to the MET main_v12.0 branch to make the docker build enviornment more portable. * Per dtcenter/METbaseimage#32, add workflow to build Docker images for MET releases on a schedule. * Per dtcenter/METbaseimage#32, update the release_docker_images.yml workflow to build releases using Dockerfile instead of Dockerfile.copy. * Per dtcenter/METbaseimage#32, omit the leading v for released versions in the Docker Hub tag. * Per METbaseimage#32, store logs for docker builds. * Update the Docker image logic to add X.Y-latest * Update release-docker-images workflow to do a shallow checkout of the Dockerfile for the version to be built, but run scripts from the triggered branch. * Avoid overwritting checkout * Fix Dockerfile checkout #ci-skip-unit * Update schedule to rebuild supported releases weekly instead of daily. * Feature #3198 main_v12.0 scan_for_CVEs (#3200) * Per #3198, port over CVE scanning logic from METbaseimage to MET. * Per #3198, use SOURCE_BRANCH instead of GITHUB_NAME for the scan_docker_image.sh script. * Fix typo * Update CVE log message * Per #3198, update the versions listed in the release-docker-images.yml workflow for the develop branch. * Per #3198, update the versions listed in the release-docker-images.yml workflow for the main_v6.0 branch. * Feature #3204 v12.0.3 (#3205) * Per #3204, update to version number 12.0.3. * Per #3204, update from METbaseimage version 3.3 to 3.3.1. * Per #3204, add v12.0.3 release notes. * Per #3204, update GHA automation to use the secret. * Per #3204, update release-docker-images.yml workflow to build the new 12.0.3 images instead of the previous 12.0.2. * Bugfix #3210 main_v12.0 WDIR (#3213) * Fix jobstring to write -out_wind_logic correctly instead of reporting the value for cnt_logic instead. * Restore logic in aggr_mpr_wind_lines() removed for #2395 when STAT-Analysis aggregates MPR lines to compute WDIR output. In main_v11.0, that job automatically removes 0 vectors when computing ROW_MEAN_WDIR output. * Per #3210, update the revision history * Per #3210, update the WDIR documentation * Per #3210, add one more WDIR job to STAT-Analysis using all inputs, not just the non-zero MPR vectors. * Unrelated to #3210, but noted by @CPKalb, note that MET can read raw WRF output. * Per #3210, solve one additional bug in the MPR -> VCNT aggregation. Since the local v_info object was declared outside the loop, when d_diff is bad data, the values from the previous iteration were being using. This cause the TOTAL_DIR value to be reported as the TOTAL value and not properly skip over the zero length vectors as it should... and as Point-Stat does. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Pratiman <31694629+pratiman-91@users.noreply.github.com> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as PR #3021 but for main_v12.0 instead of develop. Please review this one at the same time as #3021.