Describe the Problem

This issue arose via dtcenter/METplus#2298 and is described in this comment. Recommend enhancing the logic for the percentile() and percentile_f() functions in ptile.cc to check bounds and prevent reading beyond the end of the array.

Expected Behavior

Requesting the 100-th percentile from these functions should return the maximum value and not read past the end of the array.

Environment

Describe your runtime environment:
Reported by @malloryprow when testing on the WCOSS2 dogwood machine. However, this behavior is only reproducible when running multiple Point-Stat jobs via cron. It does not appear when running on the command line. This transient behavior is consistent with a memory problem.

I was never able to directly reproduce this problem myself. However, adding one line to the percentile() function did trigger the exact same error message that @malloryprow reported:

double percentile(const double *ordered_array, const int n, const double t)

{

int index;
double delta;
double p = bad_data_double;

// JHG, add this line for testing
if(is_eq(t,1.0)) return ( p );

Produces these warning/error messages:

DEBUG 3: The observation threshold value ">20.0" represents the 100-th percentile.
DEBUG 3: For forecast threshold "==FBIAS1" with type ">" update the requested percentile from 100 to 100.
WARNING: 
WARNING: Simple_Node::set_perc() -> the requested percentile (100) for threshold "==FBIAS1" differs from the actual percentile (0) by 100.
WARNING: This is common for small samples or data that contains ties.
WARNING: 
DEBUG 3: The requested percentile (95) for threshold ">=SOP95(2.2)" includes 95.6522% of the data.
DEBUG 2: Computing Categorical Statistics.
ERROR  : 
ERROR  : Simple_Node::check(double, double, double) const -> percentile threshold "==FBIAS1" used before it was set.
ERROR  : 

This implies that @malloryprow's call to the percentile function is returning bad data or nan. And its most likely the case that nan is being computed by reading past the end of the array.

Relevant Deadlines

Fix this in main_v11.0 prior to cutting an 11.0.3 bugfix release.
Also fix this in main_v11.1 and develop.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Organization level Project for support of the current coordinated release
• Select Repository level Project for development toward the next official release or add alert: NEED CYCLE ASSIGNMENT label
• Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Organization level software support Project for the current coordinated release
  Select: Milestone as the next bugfix version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Development issue
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Close this issue.