Skip to content

Conversation

github-actions[bot]
Copy link
Contributor

Adds new ascii2nc/USCRN_Boulder_20240801.nc output file and modifies existing airnow output files by adding the obs_desc variable.
Created by @JohnHalleyGotway

Howard Soh and others added 30 commits February 2, 2024 16:58
* Per #2395, add new columns to VL1L2, VAL1L2, and VCNT line types for wind direction statistics. Work still in progress.

* Per #2395, write the new VCNT columns to the output and document the additions to the VL1L2, VAL1L2, and VCNT columns.

* Per #2395, add the definition of new statistics to Appendix G.

* Per #2395, update file version history.

* Per #2395, tweak warning message about zero wind vectors and update grid-stat and point-stat to log calls to the do_vl1l2() function.

* Per #2395, refine the weights for wind direction stats, ignoring the undefined directions.

* Update src/tools/core/stat_analysis/aggr_stat_line.cc

* Update src/tools/core/stat_analysis/parse_stat_line.cc

* Update src/tools/core/stat_analysis/aggr_stat_line.cc
… broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge develop into develop-ref directly, use an intermediate update_truth_for_develop branch.
* Per #2280, update to support probability threshold strings like ==8, where 8 is the number of ensemble members, to create probability bins centered on the n/8 for n = 0 ... 8.

* Per #2280, update docs about probability threshold settings.

* Per #2280, use a loose tolerance when checking for consistent bin widths.

* Per #2280, add a new unit test for grid_stat to demonstrate processing the output from gen_ens_prod.

* Per #2280, when verifying NMEP probability forecasts, smooth the obs data first.

* Per #2280, only request STAT output for the PCT line type to match unit_grid_stat.xml and minimize the new output files.

* Per #2280, update config option docs.

* Per #2280, update config option docs.
…ullptr

Feature 2673 sonarqube beta4 nullptr
…eturn

Feature 2673 sonarqube beta4 return
JohnHalleyGotway and others added 29 commits November 12, 2024 16:07
* Per #2729, get rid of extra blank line in log message about manually deleting temp files.

* Per #2729, fix log message describing the range/azimuth grid.

* Per #2729, Update TC-Diag Python driver code to convert rotation angles from degree clockwise from due East to radians counter-clockwise from due East.

* Per #2729, complete the TC-Diag documentation.

* Per #2729, more tweaks

* Per #2729, clarify that some diagnostics are directly extracted from the ATCF track file rather than being computed from the cylindrical coordinates grid.

* Update docs/Users_Guide/tc-diag.rst

Co-authored-by: KathrynNewman <knewman@ucar.edu>

* Update docs/Users_Guide/tc-diag.rst

Co-authored-by: KathrynNewman <knewman@ucar.edu>

* Update docs/Users_Guide/tc-diag.rst

Co-authored-by: KathrynNewman <knewman@ucar.edu>

* Update docs/Users_Guide/tc-diag.rst

Co-authored-by: KathrynNewman <knewman@ucar.edu>

* Update docs/Users_Guide/tc-diag.rst

Co-authored-by: KathrynNewman <knewman@ucar.edu>

---------

Co-authored-by: KathrynNewman <knewman@ucar.edu>
* Per #2750, user-provided should be hyphenated.

* Per #2750, add units to MTD output tables and specify order of forecast minus observation differences, where applicable. Also update some wording for consistency.

* Unrelated to #2750, but prior to the v12.0.0 release, create a 12.0.0 flowchart, save as PNG, and use it to update the overview figure.

* Per #2750, put X_DOT and Y_DOT back on separate lines since RTD interprets X_,Y_DOT as being a poorly defined link.
* Update data_io.rst

Updated netCDF reading documentation

* Update data_io.rst

* Update data_io.rst

* Update data_io.rst

updated link

* Update data_io.rst

Fixed typo
* Per #3015, update the release version number and add release notes.

* Per #3015, add one upgrade instruction
* Per #3020, add missing GridStatNcOutInfo::do_seeps flag and use it to determine if SEEPS information should be written to the Grid-Stat NetCDF matched pairs output file.

* Unrelated to #3020, fix broken NetCDF cf-conventions links in the User's Guide.

* Per #3020, no real changes. Just whitespace
…idStatConfig_SEEPS config file needs to be updated with nc_pairs_flag.seeps = TRUE in order for the same output to be produced by the unit tests.
* Per #2966, add new solar_time() function to the vx_solar library.

* Per #2966, add support for new solar_time masking type. Also make log messages for consistent and eliminate the warning about -thresh not being specified becuase its fine to not specify a threshold.

* Per #2966, add a units attribute to the output NetCDF mask variable.

* Per #2966, modify solar azimuth and altitude strings to make the log messages align well.

* Per #2966, add gen_vx_mask unit test to demonstrate the solar_time masking type.

* Per #2966, add documentation about the -solar_time option

* Per #2966, reduce SonarQube code smells in gen_vx_mask

* Per #2966, reduce SonarQube findings

* Per #2966, support multiple mask types with the same mask field being supported in a single run. Still need to update the user's guide.

* Per #2966, add UTC

* Per #2966, update gen_vx_mask docs about supporting multiple -type options in a single run

* Per #2966, update logic to fix using data masking twice, add a unit test to demonstrate, and update the mask_type attribute to include the magic string for the gridded data used for data masking.

* Per #2966, adjust the logic slightly to revert to existing behavior where we only write the timing information of the input data to the gen_vx_mask output when no threshold was applied. This should reduce the number of diffs flagged by PR #3008

* Per #2966, update details about the -union, -intersection, and -symdiff options in the usage statement and documentation as recommended by @CPKalb.

---------

Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* #3012 Initial release

* #3012 Added search_3d_kdtree.cc

* #3012 Initial release

* #3012 Added check_llh_to_ecef and test_llh_to_ecef

* #3012Added search_3d_kdtree_api

* #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km

* #3012 Compare points_XYZ or points_latlon

* #3012 Commented out latlonalt_to_xy & xy_to_latlonalt

* #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km. Added llh_to_ecef APIs

* Per #3012, update the .gitignore settings to ignore the newly created test executables in the test_util/libcode/vx_grid directory.

* Per #3012, changes to the vx_grid library to address SonarQube code smells in New Code and hopefully reduce the Overall number of code smells.

* Per #3012, couple more minor SonarQube tweaks

---------

Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* Per #3038, add debug and warning log messages to tell the user which NetCDF dimension indices are being used.

* Per #3038, fix a few SonarQube code smells to reduce the overall number of them for the #3039 PR.
* Source code changes for multiple radii/thresholds in multivariate mode

* Slight cleanup

* added multiple thresh/radii functionality for the mvmode superobjects only situation

* mvmode log message improvements

* Mods to the mode users guide for multiple convolution thresholds and radii in multivariate mode

* Added v12.1

* Minor bugfix and tweak

* Per #2709, updates to fix SonarQube code smells.

* Per #2709, fix compilation problem I introduced with SonarQube fixes.

* Per #2709, eliminate SonarQube bug that was flagged

* Per #2709, fix typo in docs

---------

Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
* Per #3032, add data type column to all of the output tables

* Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names.

* Update docs/Users_Guide/gsi-tools.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

---------

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
…' based on 1/9/2025 decision at the METplus-Analysis Tools project meeting. Also update MET to link to the METplus User's Guide.
* Per #3024, roll the version number from 12.0 to 12.1 prior to adding new columns to the existing GRAD line type.

* Per #3024, return bad data for the square root of a negative number

* Per #3024, update Grid-Stat to write 4 new columns to the GRAD line type.

* Per #3024, tweak laplace MSE computation

* Per #3024, add appendixC.rst equations for the new statistics

* Per #3024, escape embedded underscores in equations only for latex formatting, not html

* Per #3024, add reference to paper and summary of the new gradient stats

* Per #3030, update docs by moving the gradient config entry from Grid-Stat to the common area and then referencing it in both Grid-Stat and Series-Analysis.

* Per #3030, update Series-Analysis config file by adding the gradient dictionary and an entry for output_stats.gradient. Update the conf_info source code to parse them. Still need to update OTHER Series-Analysis config files and also update the logic in series_analysis.cc to compute GRAD statistics.

* Per #3030, add descriptions of GRAD statistics for inclusion in the long_name attribute of the Series-Analysis output files.

* Per #3030, add the GRADInfo::get_stat() member function to be called by Series-Analysis

* Per #3030, fix parsing logic for the gradient dictionary.

* Per #3030, add gradient entries to existing Series-Analysis config files

* Per #3030, add GRADInfo::set_stat() member function

* Per #3030, enhance Series-Analysis to aggregate the GRAD line type incrementally across multiple runs. However, this can only be done when requesting that 'ALL' GRAD columns be written.

* Per #3030, reduce the volume of SonarQube code smells.

* Updated syntax

* Updated syntax

* Updated syntax

---------

Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
…ve to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3053)
* Per #1019, initial setup for supporting -format uscrn. It compiles but I still need to make it work for the variety of USCRN inputs.

* Per #1019, add NumArray constructor using a vector of doubles.

* Per #1019, saving progress after handling monthly, daily, and hourly input files. Need to complete support for other format types and handle the unit strings

* Per #1019, consistent spacing.

* Per #1019, tweak log messages so that the file being read is logged before it's actually read so that an error in parsing the data will indicate which file caused it.

* Per #1019, update DataLine and LineDataFile classes to support parsing .csv files. Get rid of the unneeded Offsets vector. Add AllowEmptyColumns option to the DataLine class so that multiple delimiters in a row will be treated as separate columns. Since the default delim is whitespace, it makes sense that you'd want to parse multiple delims in a group. But for .csv files, each comma indicates a new column.

* Per #1019, update USCRN handler code to support all 7 input variants, including .csv files. This required updates to the DataLine and LineDataFile classes to parse the .csv data properly. Still need to enhance ascii2nc to write units

* Per #1019, add units to the Observation class.

* Per #1019, add units string to the SummaryObs and SummaryKey classes.

* Per #1019, update USCRN format to write units. Consider defining units for all the other ascii file types as well.

* Per #1019, add StringArray::all_empty() member function to check for a list of all empty strings. This is used in ascii2nc to determine if observation units and descriptions should be written.

* Per #1019, update library code to handle the independent writing of point observation descriptions. Previously, if units were present then descriptions (usually empty ones) were added. Now, units and descriptions and handled independently.

* Per #1019, add descriptions for all USCRN observations, pulled from the USCRN website.

* Per #1019, need to subtract 1900 from the year and 1 from the month to make this work. Seems like we should ADD these numbers where needed rather than subtracting them everywhere else!

* Per #1019, get rid of USCRNHandler::_readHeaderInfo() since it compilicated the logic for ignoring the first line from csv files.

* Per #1019, rather than always skipping the first line of USCRN csv files, just skip any lines where the station ID begins with 'WBAN'. That'll handle files being concatenated together and is simpler logic.

* Per #1019, add an ascii2nc unit test for USCRN point observations.

* Per #1019, doc-only change to indicate the prefix/suffix of the input USCRN files are used to the determine the specific format.

* Per #1019, update USCRN code to no longer specify a default _qcOffset value since it conflicts with the initialization. While the GHA compiler is fine with it, the SonarQube one is not. These changes should enble to  SonarQube build to complete.

* Per #1019, for SonarQube replace ALL instances of push_back() with emplace_back() which SonarQube prefers for efficiency.

* Per #1019, more changes to address SonarQube code smells and reduce the overall number of them lower than what's in the develop branch.

* Per #1019, second pass through to further reduce SonarQube findings.

* Per #1019, revert back to protected members in file_handler.h

* Per #1019, one last round of minor SonarQube code smell remediation.
@JohnHalleyGotway JohnHalleyGotway merged commit 47065a2 into develop-ref Jan 22, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants