Skip to content

Enhance Ensemble-Stat to derive CRPSCL_EMP and CRPSS_EMP from a deterministic reference model. #1685

@JohnHalleyGotway

Description

@JohnHalleyGotway

Describe the Enhancement

On 2/25/21, the METplus team met with NOAA/EMC staff to discuss the use of climatology data in MET. Here are the notes:
https://docs.google.com/document/d/1kp12JXuCk94UQCX2So3HdqcpossHetGNlwpGewdHBrE/edit

During this meeting, Binbin pointed out that when doing regional ensemble verification, the CRPSS skill score is computed relative to a single deterministic forecast and NOT using a climatology mean/standard deviation. This task is to also support that logic in MET.

To avoid confusion, recommend triggering some logic based on the climo_cdf.n_bin setting.

  • If climo_mean and climo_stdev are both provided, let climo_cdf.n_bin determine the number of climo values used to compute CRPSCL_EMP and CPRSS_EMP.
  • If climo_mean is provided, climo_stdev is NOT and climo_cdf.n_bin > 1, print a warning message that the climo_stdev is required when n_bin > 1 and compute CRPSCL_EMP = CRPSS_EMP = NA.
  • If climo_mean is provided, climo_stdev is NOT and climo_cdf.n_bin = 1, use the EMC VSDB method to compute CRPSCL_EMP and CRPSS_EMP.

Binbin is sending sample code to demonstrate the logic.

Time Estimate

1 day

Sub-Issues

Consider breaking the enhancement down into sub-issues.
No sub-issues required.

Relevant Deadlines

Must be in met-10.0.0.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required: John HG
  • Select scientist(s) or no scientist required: John O

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
  • Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project(s), Milestone, and Linked issues
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions