Skip to content

Bugfix 2387 pb2nc obs count #2400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jan 13, 2023
Merged

Bugfix 2387 pb2nc obs count #2400

merged 4 commits into from
Jan 13, 2023

Conversation

hsoh-u
Copy link
Collaborator

@hsoh-u hsoh-u commented Jan 12, 2023 •
edited
Loading

Expected Differences

The total obs count at the log message should match with the obs count at the pb2nc output. The counts are for each PREPBUFT . The total observation count is added to the log message

Minor change at nc_obs_util.cc because ConcatString does not accept the integer to add() API.

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

Before:

export MASK_GRID=""
export MASK_POLY=""
export QUALITY_MARK_THRESH="2"
export STATION_ID=""

/d1/projects/MET/MET_regression/develop/NB20230112/MET-develop/bin/pb2nc /d1/projects/METplus/METplus_Data.v5.0/met_test/data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr obs_count.nc /d1/personal/hsoh/git/bugfixes/bugfix_2387_pb2nc_obs_count/MET/internal/test_unit/config//internal/test_unit/config/PB2NCConfig -v 2

DEBUG 2: Total Messages processed               = 69833
...
DEBUG 2: Rejected based on zero observations    = 13435
DEBUG 2: Total Messages retained                = 13614
DEBUG 2: Total observations retained or derived = 186394

ncdump -h obs_count.nc | grep "nobs ="
        nobs = 137075 ;

After:

export MASK_GRID=""
export MASK_POLY=""
export QUALITY_MARK_THRESH="2"
export STATION_ID=""

/d1/personal/hsoh/git/bugfixes/bugfix_2387_pb2nc_obs_count/MET/bin/pb2nc /d1/projects/METplus/METplus_Data.v5.0/met_test/data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr obs_count.nc /d1/personal/hsoh/git/bugfixes/bugfix_2387_pb2nc_obs_count/MET/internal/test_unit/config/PB2NCConfig -v 2

DEBUG 2: Messages processed                     = 69833
...
DEBUG 2: Messages retained                      = 13614
DEBUG 2: Observations retained or derived       = 137075
DEBUG 2:
DEBUG 2: Total Observations retained or derived = 137075

ncdump -h obs_count.nc | grep "nobs ="
   nobs = 137075 ;

Total observation count with two PREPBUFR inputs:

/d1/personal/hsoh/git/bugfixes/bugfix_2387_pb2nc_obs_count/MET/bin/pb2nc /d1/projects/METplus/METplus_Data.v5.0/met_test/data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr obs_count.nc /d1/personal/hsoh/git/bugfixes/bugfix_2387_pb2nc_obs_count/MET/internal/test_unit/config/PB2NCConfig -pbfile /d1/projects/METplus/METplus_Data.v5.0/met_test/data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr -v 2

DEBUG 2: Observations retained or derived       = 137075
...
DEBUG 2: Observations retained or derived       = 137075
...
DEBUG 2: Total Observations retained or derived = 274150


ncdump -h obs_count.nc | grep "nobs ="
   nobs = 274150 ;

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No]

  • Do these changes include sufficient testing updates? [No]

  • Will this PR result in changes to the test suite? [No]

    If yes, describe the new output and/or changes to the existing output:

  • Please complete this pull request review by [Fill in date].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s)
    Select: Organization level software support Project or Repository level development cycle Project
    Select: Milestone as the version that will include these changes
  • After submitting the PR, select Development issue with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@hsoh-u hsoh-u requested a review from georgemccabe January 12, 2023 20:02
@hsoh-u hsoh-u linked an issue Jan 12, 2023 that may be closed by this pull request
Bugfix: Fix PB2NC to report accurate total observation counts in log messages #2387
Closed
22 tasks
georgemccabe
georgemccabe approved these changes Jan 12, 2023
View reviewed changes
Copy link
Collaborator

@georgemccabe georgemccabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes look good and the new output looks as expected.

@hsoh-u hsoh-u merged commit 0163b9b into develop Jan 13, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.1.0 milestone Jan 25, 2023
@JohnHalleyGotway JohnHalleyGotway deleted the bugfix_2387_pb2nc_obs_count branch February 28, 2023 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@georgemccabe georgemccabe georgemccabe approved these changes

Assignees
No one assigned
Labels
None yet
Projects
Coordinated METplus-5.0 Support
Status: ✅ Done
MET-11.1.0 Development
Status: ✅ Done
Milestone
MET-11.1.0
Development

Successfully merging this pull request may close these issues.

Bugfix: Fix PB2NC to report accurate total observation counts in log messages
3 participants
@hsoh-u @georgemccabe @JohnHalleyGotway