Update develop-ref after #3114 #3117
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Per #2395, add new columns to VL1L2, VAL1L2, and VCNT line types for wind direction statistics. Work still in progress. * Per #2395, write the new VCNT columns to the output and document the additions to the VL1L2, VAL1L2, and VCNT columns. * Per #2395, add the definition of new statistics to Appendix G. * Per #2395, update file version history. * Per #2395, tweak warning message about zero wind vectors and update grid-stat and point-stat to log calls to the do_vl1l2() function. * Per #2395, refine the weights for wind direction stats, ignoring the undefined directions. * Update src/tools/core/stat_analysis/aggr_stat_line.cc * Update src/tools/core/stat_analysis/parse_stat_line.cc * Update src/tools/core/stat_analysis/aggr_stat_line.cc
… broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge develop into develop-ref directly, use an intermediate update_truth_for_develop branch.
* Per #2280, update to support probability threshold strings like ==8, where 8 is the number of ensemble members, to create probability bins centered on the n/8 for n = 0 ... 8. * Per #2280, update docs about probability threshold settings. * Per #2280, use a loose tolerance when checking for consistent bin widths. * Per #2280, add a new unit test for grid_stat to demonstrate processing the output from gen_ens_prod. * Per #2280, when verifying NMEP probability forecasts, smooth the obs data first. * Per #2280, only request STAT output for the PCT line type to match unit_grid_stat.xml and minimize the new output files. * Per #2280, update config option docs. * Per #2280, update config option docs.
…qube_beta3_namespace
…ullptr Feature 2673 sonarqube beta4 nullptr
…eturn Feature 2673 sonarqube beta4 return
…qube_beta3_namespace
…ve to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3053)
* Per #1019, initial setup for supporting -format uscrn. It compiles but I still need to make it work for the variety of USCRN inputs. * Per #1019, add NumArray constructor using a vector of doubles. * Per #1019, saving progress after handling monthly, daily, and hourly input files. Need to complete support for other format types and handle the unit strings * Per #1019, consistent spacing. * Per #1019, tweak log messages so that the file being read is logged before it's actually read so that an error in parsing the data will indicate which file caused it. * Per #1019, update DataLine and LineDataFile classes to support parsing .csv files. Get rid of the unneeded Offsets vector. Add AllowEmptyColumns option to the DataLine class so that multiple delimiters in a row will be treated as separate columns. Since the default delim is whitespace, it makes sense that you'd want to parse multiple delims in a group. But for .csv files, each comma indicates a new column. * Per #1019, update USCRN handler code to support all 7 input variants, including .csv files. This required updates to the DataLine and LineDataFile classes to parse the .csv data properly. Still need to enhance ascii2nc to write units * Per #1019, add units to the Observation class. * Per #1019, add units string to the SummaryObs and SummaryKey classes. * Per #1019, update USCRN format to write units. Consider defining units for all the other ascii file types as well. * Per #1019, add StringArray::all_empty() member function to check for a list of all empty strings. This is used in ascii2nc to determine if observation units and descriptions should be written. * Per #1019, update library code to handle the independent writing of point observation descriptions. Previously, if units were present then descriptions (usually empty ones) were added. Now, units and descriptions and handled independently. * Per #1019, add descriptions for all USCRN observations, pulled from the USCRN website. * Per #1019, need to subtract 1900 from the year and 1 from the month to make this work. Seems like we should ADD these numbers where needed rather than subtracting them everywhere else! * Per #1019, get rid of USCRNHandler::_readHeaderInfo() since it compilicated the logic for ignoring the first line from csv files. * Per #1019, rather than always skipping the first line of USCRN csv files, just skip any lines where the station ID begins with 'WBAN'. That'll handle files being concatenated together and is simpler logic. * Per #1019, add an ascii2nc unit test for USCRN point observations. * Per #1019, doc-only change to indicate the prefix/suffix of the input USCRN files are used to the determine the specific format. * Per #1019, update USCRN code to no longer specify a default _qcOffset value since it conflicts with the initialization. While the GHA compiler is fine with it, the SonarQube one is not. These changes should enble to SonarQube build to complete. * Per #1019, for SonarQube replace ALL instances of push_back() with emplace_back() which SonarQube prefers for efficiency. * Per #1019, more changes to address SonarQube code smells and reduce the overall number of them lower than what's in the develop branch. * Per #1019, second pass through to further reduce SonarQube findings. * Per #1019, revert back to protected members in file_handler.h * Per #1019, one last round of minor SonarQube code smell remediation.
* Per #3047, modify the intel oneapi settings for compiler variable * Per #3047, modifications for ATLAS and ECKIT * Per 3047, creating a new development environment file for MET for seneca using the necessary values for the Intel oneAPI compilers on seneca * Per #3047, uploading config file used to install MET on seneca with Intel oneAPI * Per #3047, updated development file to use appropriate compiler settings * Per #3047, removing paths to compilers * Per #3047, removing paths to compilers * Per #3047, added BUFRLIB_NAME and GRIB2CLIB_NAME * Per #3047, adding paths back in for now * Per #3047, removing paths to compilers * Per #3047, commenting out source of setvars.sh and adding paths to older Intel oneAPI compilers * Per #2047, updates for compilation however a pb2nc unit test still fails * Per #3047, updating file for the 2024.2.1.100 compilers * Per #3047, updating some ADDTL_DIRs --------- Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
* Per #3006, add new pair_stat tool as a full copy of the point_stat tool with all instances of point_stat renamed as pair_stat. * Per #3006, add pair_stat to the list of things for which no 'make test' command is run. * Per #3006, saving work in progress prior to seneca reboot * Per #3006, revert back to using FileType instead of GrdFileType. That change was not meaningful or warranted. * Per #3006, revert back to using FileType instead of GrdFileType. That change was not meaningful or warranted. * Per #3006, committing changes since the code is compiling. Added IODADataConfig_default file to store default settings for reading IODA data. * Per #3006, starting to tweak config options. Saving progress while it's successfully compiling * Per #3006, add fcst.pairs and obs.pairs config entries. * #3007 Added vx_ioda * #3007 Added vx_ioda * #3007 Added vx_ioda * #3007 Derived from IODADataConfInfo * #3007 Reduced the code smells (SonarQube findings) * #3007 Added station_value_base_t and point_pair_t * Initial release * #3007 Changed ack the location of nc_point_obs.set_nc_out_data * Changed station_value_base_t::clear() to station_value_base_t::clear_base() * Changed bAPI names * #3007 Reduced code smells * #3007 CLeanup * #3007 Cahnged API for IODADataConfInfo * #3007 Renamed ioda_file to ioda_reader * #3007 Corrected comment * #3007 Added -lvx_statistics again * #3007 Added get_nc_data(NcVar *, unixtime) * #3007 Cleanup * #3007 Added add_to_unixtime((unixtime) * #3007 Reduced the complexiity of read_time. Added read_time_as_number * #3007 Added read_time_as_number * #3007 Added add_to_unixtime(unixtime) * #3007 Cleanup * #3007 Set bad_data_int to qc_buf * #3007 Cleanup * Per #3006, define new GrdFileType::FileType_Pairs enumerated value to be used in the pair_stat tool. * Per #3006, update pair_stat to use the newly added GrdFileType::FileType_Pairs enumerated value. * #3007 Temporarily removed pair_stat * Per #3006, rerun bootstrap on seneca to incorporate the compilation of the vx_ioda library. * Per #3006, make docs build without warning * Per #3006, saving compiling state * Per #3006, use ConcatString instead of std::string for consistency. * Per #3006, work in progress * Unrelated to #3006, but fix typo in log message. * Per #3006, default_column_union was defined in 2 spots. Renaming one of them to avoid compilation conflict. * Per #3006, move StatHdrInfo out of aggr_stat_line.h/.cc and into vx_stat_out/stat_hdr_info.h/.cc. This make it available to both Stat-Analysis and the Pair-Stat tool to track the unique STAT headers elements read. * Per #3006, remove the unused land/topo/msg_type type config options from the pair_stat tool's configuration file and code that parses it. If needed, we can add it back in the future. * Per #3006, update VarInfoPairs::set_dict() to also call VarInfo::set_magic(). * Per #3006, since python_line.h lives in src/basic/vx_util, the vx_util library now also depends on the * Per #3006, saving off version that compiles before trying changes that may not. * #3007 Deleted commented out cpde * Changed data typo (float to double) * #3007 Resio;lved SonarQube finding * Per #3006, added logic to track ck unique header input columns like Stat-Analysis does. Committing the current state of this branch prior to merging in changes from the #3007 feature branch and completing development for the beta1 cycle. * Per #3006, fix indexing for vx_opt * Per #3006, error out for -format ioda and -format python * Per #3006, make -format ioda or -format python error out, but add unit tests demonstrating those errors. Also note this in the user's guide. * Per #3006, replace -outdir with -out and remove output_prefix config option. * Per #3006, remove output_prefix from Pair-Stat config files. * Per #3006, update unit_pair_stat.xml to use the -out option. * Per #3006, working version. However, the filtering by grid is not working all that well right now because it's based on G004. * Per #3006, expand the Pair-Stat example. * Per #3006, switch from using global 0.5 degree reference grid to 0.1 degree. This still isn't good enough though. Instead, we need to get rid of the reference grid altogether and keep track of the grid information separately for each mask. * Per #3006, fix for loop typo in 3 spots * Per #3006, remove one line from bad merge * Per #3006, SonarQube updates. * Per #3006, more SonarQube fixes * Per #3006, remove desctrutor as recommended by SonarQube --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* #3056 Initial release * #3056 Added UGridConfig_lfric2 * #3056 Moved some APIs (for init_time) to nc_utils.cc from nc_cf_file.cc * #3056 Get ther InitTime from the forecast_reference_time variable or filename * #3056 Added an unittest for LFric dataset, point_stat_ugrid_lfric2_config * #3056 Moved back getting init_time and valid_time from ghe filename * #3056 Corrected version * #3056 Cleanup * #3056 Resolved SonarQUbe findings * #3056 Cleanup * Per #3056, rename 'ugrid_lfric2' as 'ugrid_lfric_pressure_levels' throughout to more clearly indicate that this configuration is for lfric data that's been interpolated in the vertical to pressure levels. * Per #3056, tweak wording of UGrid config file comments * Per #3056, reconfigure the UGrid LFRic pressure levels test to make more realistic comparisons at 2 pressure levels and don't write .stat and _mpr.txt because the data is duplicated. --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* Unrelated to #3037, update the Python embedding Appendix to list gen_ens_prod. * Per #3037, add 12.1.0-beta1 release notes. * Per #3037, roll the copyright date from 2024 to 2025 * Per #3037, proposing this updated structure and content for the upgrade instructions * Per #3037, add upgrade instruction details. * Per #3037, tweak wording * Per #3037, updating format of intended bulleted list --------- Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* Per #3066, add Data-IO section about file lists and reference it in all tools that use them. * Per #3060, fix typo * Update docs/Users_Guide/data_io.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Update docs/Users_Guide/data_io.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Per #3066, update usage statements in the docs and application code based on PR feedback. * Per #3066, missed a handful of additional updates. --------- Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
* Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that checked all other usage of c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
…t_hira.xml from which they were missing and that was causing a runtime warning message from python/unit.py.'
* Per #3077, update yacc files to address the Intel warning: 'POSIX yacc reserves %type to nonterminals [-Wyacc]' * Per #3077, update the color parser to allow for empty inputs in the same way we allow for them in config.tab.yy. Also use the opportunity to refine the code based on SonarQube feedback - removing commented out code and declaring variables on dedicated lines. * Per #3077, update vx_config library to back out the insufficient fix for MET #2408 with a yystate patch in Makefile.am that was causing other intermittent failures. Update config.tab.yy to allow for empty inputs in the statement_list rule. * Per #3077, changes to config.tab.yy to address some SonarQube code smells of having each variable declared on its own line. * Per #3077, consistent formatting of error messages * Per #3077, consistent formatting of error messages * Per #3077, no real change, just whitespace. * Per #3077, fix a few more SonarQube code smells that are easily fixed and are low risk.
* Per #3087, update logic in VarInfoNcMet::set_magic(...) to actuall store the requested level string to allow for discriminating between multiple U/V vertical level matches. * Unrelated to #3087, delete unneeded 'int errno;' local variable from temp_file.cc that caused an unexpected copmilation error with GCC 9.4.0 on Ubuntu as described in the dtcenter/METplus#2897 discussion. * Per #3087, tweak logic to handle '*' and fix resolve regression test differences. * Per #3087, add regrid_data_plane and grid_stat unit test to demostrate creating vector pairs at multiple levels from NetCDF input files. * Per #3087, forgot to add the Grid-Stat config file needed for the new unit test. * Per #3075, refine name and logic the new tests. * Per #3087, update ConcatString class to simplify from a pointer to a string to just a string itself. This is based on SonarQube code smells, but the implementation is much simpler and easier to maintain. * Per #3087, drive down a few more SonarQube code smells. * Per #3087, back out the ConcatString changes to switch from enum to enum class and the use of explicit since those had huge and wide-ranging impacts. Touching that many files is not worth it to reduce SonarQube code smells. * Per #3087, modify the existing point2grid_pb2nc_big_input test in unit_point2grid.xml by switching from requesting the 'Z2' level to using '*', like all the other simliar point2grid tests. Note that I DID actually test to confirm that 'Z2' and '*' produce the same result. So specifying Z2 does NOT actually filter the obs data as you'd expect it would. With this change, the diff of the output from the test should go away for PR #3088. * Unrelated to #3087, but pointed out by @j-opatz, removing an outdated line from the Ensemble-Stat chapter of the MET User's Guide referencing the 'ens' dictionary which was removed at the same time Gen-Ens-Prod was created.
… which now depends on it.
* Per #3095, when checking the IABP file type, just use the existing DataLine object rather than re-parsing based on whitespace. Also call is_eq() to check for equality of doubles rather than precise equality with ==. * Per #3095, add ascii2nc unit test for tab-delimited inputs. * Per dtcenter/METplus#2920 discussion (incl with issue #3095), update the is_eq(a,b) logic to return true if both values are NaN. * Per #3095, reorder for efficiency.
* per dtcenter/METbaseimage#30, update variables to reflect change from python 3.10 to 3.12.0 * set met base tag to test * remove warning that $ is invalid * print exception error so it is easier to debug failures in script * change np.NaN to np.nan because the former was deprecated in numpy 2.0 * convert python embedding attributes to a string before converting to json to prevent error handling float32 values * define json encoder to convert numpy types to python -- see gist https://gist.github.com/jonathanlurie/1b8d12f938b400e54c1ed8de21269b65 * move numpy json encoder to common area where it can be referenced by 2 calls to json.dump * add encoder to python embedding script (to test, should move later) * update /nrit/ral python to use 3.12 * convert numpy types to python in the attributes to fix issue introduced in fire weather use case moving to numpy 2.X * revert back to python 3.10 to test that changes to python embedding logic are backwards compatible * revert back to python 3.10 to test * change python settings back to 3.10 for testing on seneca * per PR feedback from @hsoh-u, removing duplicate class and instead import it from the other location * Revert "per PR feedback from @hsoh-u, removing duplicate class and instead import it from the other location" This reverts commit b489e79. * fix SonarQube complaints
* per dtcenter/METbaseimage#30, update variables to reflect change from python 3.10 to 3.12.0 * set met base tag to test * remove warning that $ is invalid * print exception error so it is easier to debug failures in script * change np.NaN to np.nan because the former was deprecated in numpy 2.0 * convert python embedding attributes to a string before converting to json to prevent error handling float32 values * define json encoder to convert numpy types to python -- see gist https://gist.github.com/jonathanlurie/1b8d12f938b400e54c1ed8de21269b65 * move numpy json encoder to common area where it can be referenced by 2 calls to json.dump * add encoder to python embedding script (to test, should move later) * update /nrit/ral python to use 3.12 * convert numpy types to python in the attributes to fix issue introduced in fire weather use case moving to numpy 2.X * use v3.4 METbaseimage that uses Python 3.12.0 * per #3105 PR review from @JohnHalleyGotway, update versions to MET base image 3.4 and Python 3.12
* Per #3064, add framework for supporting RA datasets. Still need to implement the library code though. * Per #3064, add is_ncmetra_file() logic. * Per #3064, adding vx_data2d_nc_met_ra as just a copy of vx_data2d_nc_met for now. It compiles but I obviously need to change the functionality of the library. Also recommend migrating I/O logic from the tc_rmw and tc_diag application code into this common library. * Per #3064, minor change to declaration of local variables to satisfy SonarQube. * Per #3064, reverting the addition of the vx_data2d_nc_met_ra library in an attempt to support these files via the existing vx_data2d_nc_met library instead. * Per #3064, reverting the addition of the vx_data2d_nc_met_ra library in an attempt to support these files via the existing vx_data2d_nc_met library instead. * Per #3064, reverting the addition of the vx_data2d_nc_met_ra library in an attempt to support these files via the existing vx_data2d_nc_met library instead. * Unrelated to #3064, add met_version to the standard Start and Finish DEBUG(1) log messages. * Unrelated to #3064, add met_version to the standard Start and Finish DEBUG(1) log messages. * Per #3064, still a work in progress, but updates to vx_data2d_factory, vx_data2d_nc_met, and vx_grid to more fully and directly support the range/azimuth grid. * Per #3064, add TcrmwGrid::dump() * Per #3064, do not plot map data for range/azimuth grids. * Per #3064, fix typo in log message * Per #3064, update vx_data2d_nc_met to update the Range/Azimuth grid information and timing information after actually reading data from Range/Azimuth files. * Per #3064, for SonarQube, remove unused variables and switch to const NcFile * where possible. * Per #3064, more SonarQube findings. * Per #3064, address some SonarQube code smells in vx_grid. * Per #3064, rename TcrmwGrid and TcrmwData as RngAziGrid and RngAziData, repsectively. * Per #3064, remove RngAziData pointer from 'tc' to 'ra' after renaming from TcrmwData to RngAziData. * Per #3064, consistent variable names in tc_rmw. * Per #3064, update Grid-Stat to actually read Range/Azimuth data in order rder to fully define the input grid with it's lat/lon centerpoint. * Per #3064, more consistent naming conventions of local variables * Per #3064, reimplement RngAziGrid::xy_to_latlon() and latlon_to_xy() functions to support regridding. * Per #3064, update RngAziGrid:::wrap_lon() to return true since the 'longitudes' here are really the azimuths which by definition include a full 360 degrees and should be wrapped. * Per #3064, adding full support for writing NetCDF Range/Azimuth files for all MET tools is too big of a task and beyond the scope of this issue. Enhance vx_nc_util to error out with a useful error message about that fact and write up a new issue to describe the desired enhancement. * Per #3064, remove trailing separator from the end of RngAziGrid::serialize(). * Per #3064, revert change in data_plane_plot.cc to include the map data by default since it actually does provide useful information after fixing the projection issues. * Per #3064, update plot_point_obs to redefine the grid after reading gridding data since the grid definition is only finalized for range/azimuth grids after data is read. * Per #3064, add tests to run plot_data_plane, plot_point_obs, grid_stat, and point_stat using tc_rmw output as input. * Per #3064, drive down SonarQube code smells. * Per #3064, fix unit_tc_rmw.xml to use point observations from the correct date * Per #3064, after updating tc_rmw and tc_diag to use the RngAziGrid, no longer need to reverse the range values when writing to NetCDF output files, which is much simpler. * Per #3064, switch from locally managed memory to using STL::vectors for tc_rmw.
* Trying out subproject ToC functionality * Per #2771, add ToC scroll bar and sticky logo * Adjust values for ToC and sticky logo * Reduce space between ToC and sticky logo * Reduce space between ToC and sticky logo * Reduce space between ToC and sticky logo * Slightly increase space between ToC and sticky logo * Menu updates * Hide the sticky logo so that it behaves like the original (disappearing into the hamburger menu)
* Per #3059, add vx_ioda.h header file to the vx_ioda library. * Per #3059, rename IODA-related classes from ioadReader/iodaMetadata/iodaHeaders to IODAReader/IODAMetadata/IODAHeaders. * Per #3059, add vector<string> constructor for StringArray. * Per #3059, migrate logic from ioda2nc.cc check_core_data() into IODAReader::validate_metadata() to make it accessible from pair_stat. * Per #3059, update IODAReader to parse the requested variable/group names rather than handling it in the application code. * Per #3059, work in progress adding IODA support to pair_stat. * Per #3059, work in progress. Writing STAT output for IODA inputs but still need to populate the header columns in a sensical manner. * Per #3059, update vx_ioda library to parse a global attribute reference time from input files and interpret it the same way as a model initialization time. * Per #3059, update StatHdrInfo to track header entries using pair info instead of just STATLine inputs. Do still need some more work in the vx_stat_out library for Pair-Stat to avoid false alarm warning messages. * Per #3059, update the point_pair_t struct to store the forecast lead time for each pair. * Per #3059, pair_stat work in progress. Next, I need to keep track of the forecast lead time for each pair which will require changes to the PairBase class. * Per #3059, update IODAReader::get_point_pairs() to handle empty strings. * Per #3059, update PairDataPoint to store the lead time for each matched pair. Note that this feature is currently only used by Pair-Stat when processing IODA inputs where the forecast lead times can vary. * Per #3059, update the default IODADataConfig metadata_map to read the station ID name from an input stationIdentifier variable name. * Per #3059, update Pair-Stat to store header column values for IODA inputs and update the StatHdrInfo::get_shc() logic to not print a warning message if those header column values are manually defined. * Per #3059, update expected pair_stat output file names. * Per #3059, remove unused obs_window config option and add many time filtering options over from Stat-Analysis to Pair-Stat. * Per #3059, add definitions for missing fcst/obs_init/valid_inc/exc configuration options. * Per #3059, add write_css_hhmmss(const IntArray &) utility function. * Per #3059, add, parse, and log time filtering options. Still need to actually apply those filters to the data though. * Per #3059, filter input data by fcst/lead time, init time, and valid time. * Per #3059, updating the read_ascii_mpr.py script to read the now 39 MPR input columns rather than the previous 37 number of them. * Per #3059, update pair_stat to support -format python and add a unit test to demonstrate that functionality. * Per #3059, fix typo in log message * Per #3059, add climo_mean.regrid to the pair_stat config file and update unit_pair_stat.xml to test with IODA inputs with climo input. * Per #3059, update read_climo.h/.cc to SET tTORE the grid read from the climo data. This is needed for pair_stat but the verification grid is already defined for all the other application code. * Per #3059, update PairBase to allow the climo_pnt_info to be defined separately, which is needed for pair_stat." * Per #3059, update ioda.cc to parse the timestamps from sample_hofx_output_sondes.nc4 correctly. * Per #3059, update pair_stat application code to incorporate climo data. * Per #3059, changes to address SonarQube bugs and (easily fixed) code smells flagged for PR #3114. * Per #3059, complete documentation for the Pair-Stat tool. * Per #3059, try to eliminate one more SonarQube code smell. * Per #3059, squash a few more SonarQube code smells. * Fix typo in appendixF.rst * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Per #3059, tweak Pair-Stat documentation based on PR feedback. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
* Per #3059, add vx_ioda.h header file to the vx_ioda library. * Per #3059, rename IODA-related classes from ioadReader/iodaMetadata/iodaHeaders to IODAReader/IODAMetadata/IODAHeaders. * Per #3059, add vector<string> constructor for StringArray. * Per #3059, migrate logic from ioda2nc.cc check_core_data() into IODAReader::validate_metadata() to make it accessible from pair_stat. * Per #3059, update IODAReader to parse the requested variable/group names rather than handling it in the application code. * Per #3059, work in progress adding IODA support to pair_stat. * Per #3059, work in progress. Writing STAT output for IODA inputs but still need to populate the header columns in a sensical manner. * Per #3059, update vx_ioda library to parse a global attribute reference time from input files and interpret it the same way as a model initialization time. * Per #3059, update StatHdrInfo to track header entries using pair info instead of just STATLine inputs. Do still need some more work in the vx_stat_out library for Pair-Stat to avoid false alarm warning messages. * Per #3059, update the point_pair_t struct to store the forecast lead time for each pair. * Per #3059, pair_stat work in progress. Next, I need to keep track of the forecast lead time for each pair which will require changes to the PairBase class. * Per #3059, update IODAReader::get_point_pairs() to handle empty strings. * Per #3059, update PairDataPoint to store the lead time for each matched pair. Note that this feature is currently only used by Pair-Stat when processing IODA inputs where the forecast lead times can vary. * Per #3059, update the default IODADataConfig metadata_map to read the station ID name from an input stationIdentifier variable name. * Per #3059, update Pair-Stat to store header column values for IODA inputs and update the StatHdrInfo::get_shc() logic to not print a warning message if those header column values are manually defined. * Per #3059, update expected pair_stat output file names. * Per #3059, remove unused obs_window config option and add many time filtering options over from Stat-Analysis to Pair-Stat. * Per #3059, add definitions for missing fcst/obs_init/valid_inc/exc configuration options. * Per #3059, add write_css_hhmmss(const IntArray &) utility function. * Per #3059, add, parse, and log time filtering options. Still need to actually apply those filters to the data though. * Per #3059, filter input data by fcst/lead time, init time, and valid time. * Per #3059, updating the read_ascii_mpr.py script to read the now 39 MPR input columns rather than the previous 37 number of them. * Per #3059, update pair_stat to support -format python and add a unit test to demonstrate that functionality. * Per #3059, fix typo in log message * Per #3059, add climo_mean.regrid to the pair_stat config file and update unit_pair_stat.xml to test with IODA inputs with climo input. * Per #3059, update read_climo.h/.cc to SET tTORE the grid read from the climo data. This is needed for pair_stat but the verification grid is already defined for all the other application code. * Per #3059, update PairBase to allow the climo_pnt_info to be defined separately, which is needed for pair_stat." * Per #3059, update ioda.cc to parse the timestamps from sample_hofx_output_sondes.nc4 correctly. * Per #3059, update pair_stat application code to incorporate climo data. * Per #3059, changes to address SonarQube bugs and (easily fixed) code smells flagged for PR #3114. * Per #3059, complete documentation for the Pair-Stat tool. * Per #3059, try to eliminate one more SonarQube code smell. * Per #3059, squash a few more SonarQube code smells. * Fix typo in appendixF.rst * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Update docs/Users_Guide/pair-stat.rst Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Per #3059, tweak Pair-Stat documentation based on PR feedback. * Per #3059, fix the MPR Python embedding logic to read 37 columns prior to version 12 and 39 columns for version 12 and above. * Per #3059, tweak processing logic for header columns. model and desc are set to an empty string in the config file by default, and those values should only be written to the output when they are set to a non-empty string. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New and modified output from unit_pair_stat.xml
These differences flagged by this GHA Testing Workflow run are all expected:
Created by @JohnHalleyGotway