Describe the Enhancement

As discussed on Nov 18, 2024 by @DanielAdriaansen and @JohnHalleyGotway (see meeting notes), consider adding statistics to quantify "sharpness", especially in AI/ML output, using the methods defined in this paper:
An Investigation of Metrics to Evaluate the Sharpness in AI-Generated Meteorological Imagery

Grid-Stat already writes a gradient (GRAD) output line type that contains the S1 score and it's components. The existing statistics are computed by summing the X- and Y-gradients. The methods described in this paper use the X- and Y-gradients to form a gradient vector, and then compute the magnitude of those vectors in both the forecast and observation fields.

Consider adding 4 new GRAD columns for the following methods:

1. Mean Gradient Magnitude (grad-mag) of the forecast and observation fields is the average of norm of the gradient vectors in each field.
2. Gradient RMSE (grad-rmse) is computed by comparing the forecast and observation gradient vector magnitudes.
3. Laplace RMSE (laplace-rmse) is the RMSE of the divergence of each gradient vector rather than the magnitude.

Also enhance Grid-Stat to include the gradient vector magnitude and divergence fields, if nc_pairs_flag.gradient = TRUE in the Grid-Stat config file.

Also enhance Stat-Analysis to aggregate these new statistics properly.

Prior to adding new columns, update the version number of MET from 12.0 to 12.1.

Recommend NOT adding the following:

1. Image Intensity reports the min/max/mean of the values within each image. The CNT Line Type already includes the mean forecast FBAR and observation OBAR values. While the min/max data values are not actually written, they don't really belong in the gradient line type since they are NOT based on gradients.
2. Root Mean Squared Error is already included in the CNT line type.
3. Total Variation is summed across an image and not normalized by its size. Recommend not including it because its dependence on the number of grid points makes it much less useful.
4. Gradient Total Variation is also summed and dependent on the number of grid points.

Requires further investigation:

1. Structural Similarity Index Measure (ssim) is a similarity measure between two images based on a weighted combination of three simpler comparisons: luminance, contrast, and structure. If this were well-defined, we could consider adding it to the CNT line type.

• See reference on Wikipedia

I'd also recommend that we review the existing implementation of the S1 score to confirm that the Appendix C Equations (and also the WGNE website) actually match the computation of sums starting on this line of code.

      // Gradient sums
      fgbar += wgt * (fabs(fgx_na[i]) + fabs(fgy_na[i]));
      ogbar += wgt * (fabs(ogx_na[i]) + fabs(ogy_na[i]));
      mgbar += wgt * (max(fabs(fgx_na[i]), fabs(ogx_na[i])) +
                      max(fabs(fgy_na[i]), fabs(ogy_na[i])));
      egbar += wgt * (fabs(fgx_na[i] - ogx_na[i]) +
                      fabs(fgy_na[i] - ogy_na[i]));
      total++;

Where the forecast gradient is given by (fgx_na[i], fgy_na[i]) and observed gradient is given by (ogx_na[i], ogy_na[i]).

Time Estimate

2 days?

Sub-Issues

Consider breaking the enhancement down into sub-issues.
None needed.

Relevant Deadlines

NRL charging must be completed on 12/30/24

Funding Source

FY25 Q1 NRL METplus 7730022

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as a MET-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
• For a MET-X.Y.Z version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
• Need METdataio issue to update database schema.
• Need METviewer issue to plot these new statistics.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: MET-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.