Describe the New Feature

Per dtcenter/METplus#1515 and @PerryShafran-NOAA, the EPA will switch from providing BUFR to providing ASCII data to NOAA. The new feature is to add support for this new dataset to ASCII2NC.

Acceptance Testing

Sample files exist on seneca here: /home/dadriaan/projects/airnow/shafran_data/

From dtcenter/METplus#1515:

There are several files here to look at, but they both provide the same data. The file with all the data is the HourlyAQObs data. The HourlyData is a neater looking file with the same data, but the lat/lon of each station is not provided there, so you'll have to use the HourlyAQObs file, which is a CSV file.

It would be great to be able to also read in the daily file, which I think we can use to validate daily model items. That is the daily_data.dat file, and there are two of them. Unfortunately, they are in the HourlyData format, but you should be able to use the lat/lon of the HourlyAQObs file to find these stations. Here's the fact sheet on the daily file:

As you can see, they don't make this easy. However, we could just stick with hourly data and just sum/average them the way we are doing now in PB2NC.

There are four file types:

1. "HourlyAQObs" (Docs: HourlyAQObsFactSheet.pdf)
2. "HourlyData" (Docs: HourlyDataFactSheet.pdf)
3. "daily_data" (Docs: unknown)
4. "daily_data_v2" (Docs: DailyDataFactSheet.pdf)

The documentation for file type 4) only describes "daily_data_v2", and no information was provided about the "daily_data" file from the user. @JohnHalleyGotway notes:

...looking at a diff of the two, I recommend using daily_data_v2.dat. The former does NOT contain lat/lon info for each site and would require a lookup table for that. The v2 version DOES contain the lat/lon info (it would seems) and would therefore be simpler to process.

Time Estimate

1 day of work

Sub-Issues

• Document different file type metadata
• Add support for reading these files in ASCII2NC
• Test support

Relevant Deadlines

NONE.

Funding Source

2792541

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus: need to update ASCII2NC wrapper

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.