Skip to content

Add LEPS and SEEPS to MET probabilistic verification. #563

@dwfncar

Description

@dwfncar

Replace italics below with details for this issue.

Describe the New Feature

On 10/29/2015, Geoff Dimego at NCEP inquired as to whether MET can compute SEEPS. Tara responded that no, it can't, but it could be added in the next release if we can find funding. Here's some more details on it...

According to Beth Ebert's website (http://www.cawcr.gov.au/projects/verification/) both LEPS and SEEPS are computed relative to climatology. And they require knowledge of the climatological distribution. In met-5.1, we read the climatological mean but not the spread. Adding LEPS and SEEPS would require us to read the spread. If I understand correctly, we convert the actual observed event to the likelihood that value would occur based on the climatological PDF (likely assuming a normal distribution). And we use that observed likelihood to compute these scores.

I suppose we would add LEPS and SEEPS to the PSTD line type... although they can't be derived from the Nx2 probabilistic contingency table.

Also, if these require the climatological spread, we'd need to add that to the MPR line type so that SEEPS and LEPS could be computed from the MPR line type. And if we add the climo spread to MPR, we should add it to ORANK to keep things consistent.

Marion Mittermaier says Rachael North can help with development and testing. She also pointed out that Rachael now has a method for SEEPS using TRMM so we will need to implement in Grid Stat as well. Finally, Marion mentioned that LEPS is not really related to SEEPS. It's a continuous statistics.

Stat-Analysis will also need to be modified to add this into the WMO/CBS format

Acceptance Testing

List input data types and sources.
Describe tests required for new functionality.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

MET-10.0

Funding Source

2799991 - Met Office

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required RACHAEL NORTH

Labels

  • Select component(s)
  • Select priority
  • [ X Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
  • Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

New Feature Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project(s), Milestone, and Linked issues
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.

On 10/29/2015, Geoff Dimego at NCEP inquired as to whether MET can compute SEEPS. Tara responded that no, it can't, but it could be added in the next release if we can find funding. Here's some more details on it...


According to Beth Ebert's website (http://www.cawcr.gov.au/projects/verification/) both LEPS and SEEPS are computed relative to climatology. And they require knowledge of the climatological distribution. In met-5.1, we read the climatological mean but not the spread. Adding LEPS and SEEPS would require us to read the spread. If I understand correctly, we convert the actual observed event to the likelihood that value would occur based on the climatological PDF (likely assuming a normal distribution). And we use that observed likelihood to compute these scores.

I suppose we would add LEPS and SEEPS to the PSTD line type... although they can't be derived from the Nx2 probabilistic contingency table.

Also, if these require the climatological spread, we'd need to add that to the MPR line type so that SEEPS and LEPS could be computed from the MPR line type. And if we add the climo spread to MPR, we should add it to ORANK to keep things consistent. [MET-563] created by johnhg

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions