Update develop-ref after #3187 #3197
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Per #2395, add new columns to VL1L2, VAL1L2, and VCNT line types for wind direction statistics. Work still in progress. * Per #2395, write the new VCNT columns to the output and document the additions to the VL1L2, VAL1L2, and VCNT columns. * Per #2395, add the definition of new statistics to Appendix G. * Per #2395, update file version history. * Per #2395, tweak warning message about zero wind vectors and update grid-stat and point-stat to log calls to the do_vl1l2() function. * Per #2395, refine the weights for wind direction stats, ignoring the undefined directions. * Update src/tools/core/stat_analysis/aggr_stat_line.cc * Update src/tools/core/stat_analysis/parse_stat_line.cc * Update src/tools/core/stat_analysis/aggr_stat_line.cc
… broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge develop into develop-ref directly, use an intermediate update_truth_for_develop branch.
* Per #2280, update to support probability threshold strings like ==8, where 8 is the number of ensemble members, to create probability bins centered on the n/8 for n = 0 ... 8. * Per #2280, update docs about probability threshold settings. * Per #2280, use a loose tolerance when checking for consistent bin widths. * Per #2280, add a new unit test for grid_stat to demonstrate processing the output from gen_ens_prod. * Per #2280, when verifying NMEP probability forecasts, smooth the obs data first. * Per #2280, only request STAT output for the PCT line type to match unit_grid_stat.xml and minimize the new output files. * Per #2280, update config option docs. * Per #2280, update config option docs.
…qube_beta3_namespace
…ullptr Feature 2673 sonarqube beta4 nullptr
…eturn Feature 2673 sonarqube beta4 return
…qube_beta3_namespace
* Per #3096, update string_array.h/.cc to reduce Intel OneAPI compiler warnings and SonarQube code smells. * Per #3096, add support for specifying range/azimuth grid strings and document the details in Appendix B. * Per #3096, update the vx_data2d_nc_met library code the handle both the tc_rmw and tc_diag conventions for writing Range/Azimuth time and location information. * Per #3096, consistent spacing and remove some unused variables * Per #3059, update vx_data2d_nc_met library to read TrackLat_mean and TrackLon_mean from the NetCDF output of RMW-Analysis. * Per #3059, refine spacing and log messages written from RMW-Analysis. Also, update it to write the TrackLat_mean and TrackLon_mean location information. * Per #3096, update MET library code to read and write range/azimuth grids to/from NetCDF output files. * Per #3096, update unit_tc_rmw.xml to demonstrate that plot_data_plane can read the range/azimuth NetCDF output generated by grid_stat. * Per #3096, add plot title. * Per #3096, replace snprintf with fixed lenght character arrays with calls to ConcatString::format(...) to reduce SonarQube code smells. * Per #3096, replace snprintf with fixed length character arrays with calls to ConcatString::format(...) in the vx_grid library to reduce SonarQube code smells. * Per #3096, the ENUM_TO_STRING functionality in Makefile.am was not working as expected because of extra leading spaces. This corrects that issue so that all of the *_to_string.h/.cc files are removed by make clean when configured with true set. * Per #3096, update wwmca_tool/Makefile.am to use the ConcatString output option for ENUM_TO_STRING for gridhemisphere_to_string.h/.cc and modify the application code to handle the change. * Per #3096, may changes throughout the codebase to switch from formatting strings using snprintf() with fixed length character arrays to using the ConcatString::format(...) option to reduce the number of SonarQube code smells. * Per #3096, one more removal of fixed width character array. * Per #3096, eliminate more fixed length character arrays. * Per #3096, more elmination of fixed-length character arrays. * Per #3096, reduce more SonarQube code smells and also add a draft TC-RMW call to Series-Analysis that isn't quite working yet. * Per #3096, unset MET_DEVELOPMENT so that enum_to_string output won't be removed. * Per #3096, fix 2 parsing bugs. * Per #3096, add back in time_string(...) function but writing to a ConcatString to avoid diffs. * Per #3096, update tc_rmw and rmw_analysis to write common global attributes about the command used to generate the file. * Per #3096, update Series-Analysis to handle range/azimuth regridding logic, enable a call to Series-Analysis in unit_tc_rmw.xml, and tweak its configuration. * Per #3096, print warning message about unsupported @value notation in the vx_data2d_nc_met library. * Per #3096, fix typo in unit_tc_rmw.xml and drive down SonarQube code smells. * Per #3096, add support for and documentation of passing range/azimuth grids via python embedding. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* #3106 Saves the exception to log file * #3106 Saves the exception to log file * #3106 Changed Py_SetProgramName to PyConfig_SetString * #3106 SonarQube: moved include directives to top * #3106 SonarQube: moved include directive to top * #3106 SonarQube: resolved code smaells * #3106 Replaced deprecated PySys_SetArgv to PyConfig_SetArgv * #3106 Changed for loop for SonarQube findings * #3106 SonarQube fix * #3106 SonarQube fix (added const) --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
* Per #2940, to support python embedding, only retrieve the Grid from the input file AFTER the data has been read. For python embedding, the grid is defined at the same time the data is read. * Per #2940, add a new unit test to demonstrate python embedding for multivariate MODE. * Per #2940, no real code changes. Just whitespace. * Per #2940, only compute/print debugging information about split mode objects for -v 4 or higher. * Per #2940, increase the verbosity level for superobject point information from -v 1 to -v 4. * Per #2940, merging does not apply when processing superobjects with no intensity information. Switch hard-coded setting from using threshold merging to doing no merging. * Per #2940, refine settings for new unit test. ci-run-unit * Per #2940, put back in the setting of the merge_thresh value, even if we're not actually doing threshold merging, to avoid the threshold from the default config file being used. * Per #2940, consistent formatting and make the grid process more concise by avoiding an uneeded local copy. * Per #2940, some small differences were flagged by GHA in the UGRD_Z10_ENS_NMEP_ltOCDP25_NBRHD25_GAUSSIAN1 variable of gen_ens_prod_WITH_CTRL_20120410_120000V.nc. Moving call to interp_gaussian_dp() OUTSIDE of the parallel section, where it should be. Perhaps that'll help. * Per #2940, fix compilation error. * Per #2940, back out changes to mode_exec.cc since they are unrelated to Python-embedding. I do however recommend that we test multi-var MODE to confirm that double-threshold merging is working as expected. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* Unrelated to #3120 but fix config_file.h warning message about dump() overriding the base class. * Per #3120, fix OpenMP Intel compilation errors about reduction variables not being specified as shared and parallelize one more loop. * Per #3120, silence namespace std intel warning. * Per #3120, mark UscrnHandler class final to silence intel warning. * Per #3120, remove Debug(9) log message causing an intel compiler warning. * Per #3120, move namespace declaration after includes to get rid of intel compiler warning. * Per #3120, get rid of concat_string.cc compiler warning. * Per #3120, fix diag_file.cc intel compilation warning. * Per #3120, move namespace declaration after includes to get rid of intel compiler warning. * Per #3120, fix compilation error about std::string in iadp_handler.h * Per #3120, the gaussian smoothing does not belong in the parallel section. * Unrelated to #3120, correct config file link in the RMW-Analysis chapter. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* Updates to documentation * Added the start and end time in the detail table to be consistent with the summary table * Used for converting time units * Refactored to support multiple runs of MET command * Updated yaml config instructions for num_runs * Remove extraneous dataframe used in debugging * Removed another file that was used in debugging * Instrumented two functions * Per #3109, no real code changes here, just applying consistent whitespace formatting and removing commented out code. * Included grid_diag code as list of instrumented code and removed note regarding running only once. * Add a check (assertion) in the check_settings function to verify that the script is invoked in the correct directory * Committing a generic version of the config file * Added a note about the AssertionError that is raised when benchmark.py is run outside the MET/internal/script/benchmark directory --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
* Per #3165, correct two poorly formed OpenMP pragma statements. * Per #3165, add missing array include to ensemble_stat, point_stat, and plot_point_obs that's appearently needed when compiling with the Intel OneAPI compilers and with the --disable-opemp configuration option. * Per #3165, move 'using namespace std' after the includes in the all the test code and enum_parser.yy to avoid Intel OneAPI compilation warnings about it being implicitly defined. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, only compile the kdtree test programs when ENABLE_UGRID is true. * Per #3165, move 'using namespace std' after the includes in the all the test code. * Per #3165, add DISABLE_OPENMP option to compilation script * Per #3165, updating beta2 reference to rc1 * Per #3165, fix typo in internal/test_util/libcode/vx_grid/Makefile.am * Per #3165, removing typo in acorn config file * Per #3165, Test whether disabling custom reductions from the vx_series_data library enables MET to compile on WCOSS2 with Intel classic compilers. --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
…gex which works with other compilers (#3176)
* Per #3179, update the logic to not search the parent when parsing file_type from climatology dictionaries. (#3180) * Per #3179, modify the climo_name file_type logic to parse it from the top-level climo dictionary rather than parsing it separately for each field array entry. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* #3164 Removed duplicated code * #3164 Added more API to handle the slice by value * Merged fixes for the SonarQube * #3164 Fix the SonarQUbe findings * #3164 Corrected const declarations * #3164 Corrected const declarations * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings: const --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #3164 Suuport @lower-@upper notation * #3164 Renamed error_code_bad_offset to error_code_out_of_index * Bugfix 3164 slice range sonarqube (#3173) * #3164 Fix the SonarQUbe findings * #3164 Corrected const declarations * #3164 Corrected const declarations * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings * #3164 Fixes for SonarQube findings: const * Renamed error_code_bad_offset to error_code_out_of_index --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #3164 Initialize cur_time_index and cur_z_index * #3164 Changed offset 0 to time string. Changed two @ notation level * #3164 Corrected typo * #3164 SonarQube code smell * #3164 SonarQube code smell * #3164 CHnaged warning to error for bad offset for Z --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Fix typo in code comment. No real code changes.
Make the setting of SQLITE_INCLUDE_DIR and SQLITE_LIB_DIR more portable.
Revert the development.docker environment back to setting MAKE_ARGS="-j 5" that was used for version 12.0.0. Use unlimited "-j" causes an error when building ecKit, likely consuming too many resources in the Docker image.
* Updated note in UGRID documentation explaining Point-Stat does not need regridding. * utilized secondary opinion on language update * Update language to remove ambiguity Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
* updates to test scripts to replace comp_dir.R with comp_dir.py * fixed saving of diff files (sort of) * fixed how diff files are saved to diff directory; general tidying * updated .gitignore to only ignore top level bin dir, not other files/dirs containig bin * updated how path for METplus module is supplied * changed test_regression.sh to call comp_dir.py directly instead of wrapper script and removed comp_dir.sh--wrapper not necessary * updating run_diff_tests.sh in GH jobs to use metplus diff * updating run_diff_tests.sh in GH jobs to use metplus diff * updated github diff scripts to run metplus-envs:diff container * minor fix in docker command * more fixes to run diff tests in docker * more fixes to run diff tests in docker * more fixes to run diff tests in docker * updated copying diff files * remove ref to copy_diff_files.log * added log statements for copying diff files * added print statements * added time tracking to comp_dir, changed metplus branch for testing * trivial change to logging output * updated to use develop branch of metplus for tests * tidying & attempt at programmatic way to get diff docker version * tidying * tidying, continued * updated METPLUS_DIR env var for nightly build tests --------- Co-authored-by: Natalie Babij <nbabij@seneca.rap.ucar.edu>
…ree rather than /home/met_test.
…he PATH so that the comp_dir.py script uses the intended Python version.
* Per dtcenter/METbaseimage#32, port over changes from main_v12.0 to develop, but update the schedule to rebuild images weekly rather than daily. * Better comment
* Per #3148, add new is_directory() utility function. * Per #3148, add new is_ascii_file_list() utility function. * Per #3148, add new is_ascii_file_list() utility function. * Per #3148, add new -inputrx command line option and add logic to parse input files on the command, from a file list, or by searching a directory. * Per #3148, update the get_filenames() functions to check input regular expressions directly rather than a filename prefix and suffix that it converts to regular expressions. * Per #3148, update all calls to get_filenames() to pass regular expressions as arguments rather then file prefix and suffix strings. * Per #3148, use more consistent logging levels. * Per #3148, update ascii2nc unit tests to demonstrate providing inputs with file lists or searching input directories. * Per #3148, update the ascii2nc documentaiton. * Per #3148, refine the ascii2nc usage statement, refine log messages, and make sure python inputs work. * Per #3148, update ascii2nc to support multiple python inputs all specified as quoted strings on the command line, rather than only one. * Per #3148, correct the usage statement in the User's Guide. * Per #3148, fix typo in ascii2nc unit test. * Per #3148, define atcf_suffix and atcf_reg_exp in one spot in the vx_tc_util library instead of redefining it in each TC application. * Per #3148, address easy SonarQube code smells.g * Per #3148, correct logic in get_input_files()... had an if that should be else if. * Per #3148, whitespace diffs. * Per #3148, update the ascii2nc error-handling logic. Rather than a bad or missing filename causing ascii2nc to error out completely, process as many input files as it can and print warning messages about problematic inputs. * Not released to #3148, but coordinated with Howard Soh to update the formatting of the method_name strings that appear in log messages for consistency. * Per #3148, remove unneeded exclamation points * Per #3148, for file handlers, generally, change Error log messages to Warning since the top level file handler no longer exits with bad status when failing to read an input file. Instead, a warning is printed since processing will continue to the next file. * Per #3148, add DEBUG(2) log message to list the number of successful input file reads. * Per #3148, update ascii2nc to print the -format option being applied when the user has not provided one on the command line. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds new ascii2nc output file
ascii2nc/USCRN_Boulder_file_listto demonstrate ascii2nc's new support for file lists.Created by @JohnHalleyGotway