Describe the New Feature

tc_gen_probabilistic_algorithm_v2.pdf

Note: During development, an issue with the handling of the (lat, lon) shapefile points was discovered. When applying them to a grid, after converting from (lat, lon) to grid (x, y), the (x, y) points were rounded to the nearest grid point. There is no specific need to do this rounding and it has been removed in this feature branch. This results in changes to the output of shapefile masking in gen_vx_mask. Skipping the rounding step produces a more accurate result.

Please see the attached slides to illustrate 2 main changes that are required for TC-Genesis verification. This issue describes the second of those 2 enhancements. Enhance MET to verify the NHC tropical weather outlook files.

The logic for this verification is described in the attached PDF. The task is to read a custom ASCII file which summarizes probability forecasts through time associated with each disturbance. If the disturbance did eventually develop into a storm, then the corresponding BEST track id is listed for that storm. If not, a sequence of 9's replaces that BEST track id.

For each line, the columns indicate the probability of storm development within 48, 120, and 168 hours, although those timesteps are hard-coded and NOT actually included in the metadata anywhere. Note that earlier versions of these files only had columns for 48 and 120 hours. So the tools should support 1, 2, or 3 numeric columns of probabilities, followed by a column for forecaster initials. The result should be probabilistic contingency table counts and statistics.

@halperin-erau has provided some sample data containing these tropical weather outlook summary files, which are only used internally within NHC. There are 2 reasonable implementation options... enhance tc_gen to process these files via a new command line option, or create an entirely new tool for this novel data format.

The advantage to the former is reusing many config options that would be needed. The advantage to the latter is avoiding confusion by users outside of NHC would won't have access to data in this format anyway.

However, as of May 2021, their format is still under development. In the existing and historical versions of these files, both the lat,lon location and valid timestamps are absent. If any of those columns are missing from the input data, tc_gen should print a warning message and ignore that input.

Be sure to subset output by basin, time window, and perhaps forecaster initials.

Acceptance Testing

List input data types and sources.
Describe tests required for new functionality.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

The project for 7790901 technically ends August, 2021. However, @halperin-erau plans to request a 6-month no-cost extension.

Funding Source

7790901

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
  Recommend adding a METplus use-case to demonstrate this new functionality, assuming enough data is available to do so.

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.