Describe the Enhancement

On 2/25/21, the METplus team met with NOAA/EMC staff to discuss the use of climatology data in MET. Here are the notes:
https://docs.google.com/document/d/1kp12JXuCk94UQCX2So3HdqcpossHetGNlwpGewdHBrE/edit

During this meeting, we identified a difference in how the climatological Brier Score is computed by MET and VSDB. This issue is to modify MET's method to match what's used by NOAA/EMC.

Existing logic: Derive the climatological probability by calling derive_climo_prob() to

// Derive climatological probabilities
if(cmn_flag) climo_prob = derive_climo_prob(pd.cmn_na, pd.csd_na, fthresh[i]);

New logic: Modify derive_climo_prob() or add a new function.
Instead of:

prob = normal_cdf(othresh.get_value(), mn_na[i], sd_na[i]);

Sample equal-area spaced values from the climo distribution and compute the climo probability as the event frequency among the climo values.

John Opatz and John HG had a long discussion about this. It does seem a bit odd to not just use the climo mean/stdev directly to compute the frequency of the climo event. By sampling from the climo distribution, we're really just coarsening the probability resolution down to 1/n, where n is the number of climo bins. But the point here is to make the climo probability logic commensurate with the forecast ensemble probability logic. And the forecast ensemble probabilities are limited by the number of ensemble members.

The number of climo bins is configurable in Ensemble-Stat (climo_cdf.n_bins). Perhaps it would be good practice for users to select the number of bins equal to the size of the forecast ensemble? That would make the forecast and climo probabilities as equitable as possible.

Time Estimate

1 day

Sub-Issues

Consider breaking the enhancement down into sub-issues.
No sub-issues required.

Relevant Deadlines

Must be in met-10.0.0.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: John HG
• Select scientist(s) or no scientist required: John O

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s), Project(s), Milestone, and Linked issues
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.