Skip to content

Bugfix: Fix inconsistent ASCII2NC AIRNOW location lookup logic #2452

@JohnHalleyGotway

Description

@JohnHalleyGotway

Describe the Problem

Development for issue #2402 flagged unexpected differences in the ascii2nc output for AIRNOW observations:

file1: /data/output/met_test_truth/ascii2nc/airnow/HourlyData_20220312.nc
ERROR: NetCDF headers differ:

Investigating this difference further, I was able to narrow it down and locate a bug in the AirnowLocations::lookupLatLonElev() member function. Running the same command multiple times yielded inconsistent results.

Expected Behavior

All AIRNOW data for which the station location is defined should appear in the output. Running the same command multiple times should produce consistent results.

Environment

Difference flagged via GHA. Reproduced on my Mac Laptop.

To Reproduce

Describe the steps to reproduce the behavior. Replicate the behavior by running a very stripped down run of ascii2nc.

  1. Use data for a single station:
cat data_for_124000050314.dat
03/12/22|00:00|124000050314|Québec - Charlesbour|-5|OZONE|PPB|39|Canada-Quebec1
03/12/22|00:00|124000050314|Québec - Charlesbour|-5|PM2.5|UG/M3|5.2|Canada-Quebec1
  1. Define the airnow station file for only that station:
cat airnow_monitoring_site_locations_v2_124000050314.txt
StationID|AQSID|FullAQSID|Parameter|MonitorType|SiteCode|SiteName|Status|AgencyID|AgencyName|EPARegion|Latitude|Longitude|Elevation|GMTOffset|CountryFIPS|CBSA_ID|CBSA_Name|StateAQSCode|StateAbbreviation|CountyAQSCode|CountyName
124000050314|000050314|124000050314|O3|Permanent|0314|Québec - Charlesbourg|Active|QC1|Canada-Quebec1|CA|46.861500|-71.257200||-5.00|CA||||||
124000050314|000050314|124000050314|PM2.5|Permanent|0314|Québec - Charlesbourg|Active|QC1|Canada-Quebec1|CA|46.861500|-71.257200||-5.00|CA||||||
  1. Set $MET_AIRNOW_STATIONS env var:
MET_AIRNOW_STATIONS=./airnow_monitoring_site_locations_v2_124000050314.txt
  1. Run ascii2nc for MET version 11.0.0:
./ascii2nc data_for_124000050314.dat data_for_124000050314.nc
  1. Observe inconsistent results when run several times:
DEBUG 2: Finished processing 2 observations for 1 headers.

versus

DEBUG 2: Finished processing 0 observations for 0 headers.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issue
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.

Metadata

Metadata

Type

No type

Projects

Status

✅ Done

Relationships

None yet

Development

No branches or pull requests

Issue actions