Describe the New Feature

The MET Office will need the computation of the HiRA method applied to ensembles.

This task is to enhance Ensemble-Stat to incorporate HiRA methodology into Ensemble-Stat.

In Point-Stat, the HiRA logic is controlled by the "hira" dictionary in the config file. It defines a whole separate set of processing logic and write ensemble output statistics and line types.

The incorporation in Ensemble-Stat should be much simpler. Ensemble-Stat already writes ensemble output statistics and line types. The only wrinkle added by HiRA is providing more flexibility as to which ensemble member values are used. That logic is currently controlled by the "interp" dictionary. Output statistics are generated for each interpolation type defined in the config file. Right now, all of these methods compute a SINGLE forecast value for each point observation value (e.g. the NEAREST, MIN, MAX, UW_MEAN, or DW_MEAN). Instead of computing this summary over nearby forecast grid points, we just want to use ALL of the nearby grid point values. That increases the ensemble size from N ensemble members to N members * M points in the HiRA neighborhood.

Rather than adding a "hira" dictionary to the Ensemble-Stat config file, recommend supporting a new "interp.type.method" option. See the existing supported methods listed here. Plan to name the new option "HIRA" to be specific about its intended use. But that name could be something else, like "ALL" for "all grid points in the neighborhood".

Most of the code changes to support this should lie in the PairDataEnsemble class, although there will likely be changes to the Ensemble-Stat application code. Also, add checks to make sure that "method = HIRA" is ONLY supported by Ensemble-Stat. All other tools should error out if the method is requested.

Note that Ensemble-Stat can be run with both point and gridded observations. If interp.type.method = HIRA, compute output when verifying against POINT observations, but do not compute output when verifying against GRIDDED analyses. Just log a message stating the HIRA only applies to points and the gridded vx step is being skipped for that interpolation method.

The config entry would look something like this:

interp = {
   field = OBS;
   vld_thresh = 1.0;

   type = [
      { method = HIRA;
        width  = 3;
        shape = SQUARE; }
   ];
}

@mpm-meto please review this logic and advise.

Acceptance Testing

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

Jun 2021

Funding Source

2799991 Met Office

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
• Loading HiRA output with many columns into METdatadb is problematic and will require code changes. The current upper limit for METdatadb is 150 columns.
• METdatadb issue for handling very long lines: Handle variable length lines with more than 125 columns after the header, especially for HiRA METdataio#78

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s), Project(s), Milestone, and Linked issues
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.