Describe the Problem

This issue arose via METplus Discussions dtcenter/METplus#1076. @lindsayrblank reported the problem and provided data and Stat-Analysis commands to demonstrate the problem.

The void ClimoCDFInfo::set_cdf_ta(int n_bin, bool &center) function fails to initialize the cdf_ta array. This is called once for each "case" (i.e. OBS_SID + FCST_VAR) encountered by Stat-Analysis. And each time, that array grows by 21 elements. That's what's hogging all the memory.

Recommend fixing this in the main_v10.0 branch with a one-line fix.
Recommend fixing this in the develop branch with that same one-line fix but also add a DEBUG(3) log message for each new "case" this is created, like this:

DEBUG 3: [4756] Adding new case for key = CWBZ:TMP

Also consider re-implementing the NumArray class as an STL vector since it's default memory allocation is 1000 elements. Lindsay's jobs use less than 50 of those 1000 array elements.

Expected Behavior

The job listed below should complete in a reasonable amount of time and use MUCH less than 20GB of memory.

Environment

Describe your runtime environment:
1. Machine: Mac Laptop
2. OS: MacOS
3. Software version number(s): MET version 10.0

To Reproduce

Describe the steps to reproduce the behavior:
Use this input file to run the job below:
TMP_72hr.txt

stat_analysis -lookin TMP_72hr.txt -job aggregate_stat -line_type MPR -out_line_type CNT -out_stat TMP_72hr_output.stat -by OBS_SID,FCST_VAR -set_hdr VX_MASK OBS_SID -set_hdr DESC CASE -v 3 

That job consumes well over 20GB of memory on my Mac laptop!

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: John HG
• Select scientist(s) or no scientist required: no scientist needed

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Organization level Project for support of the current coordinated release
• Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Organization level software support Project for the current coordinated release
  Select: Milestone as the next bugfix version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Close this issue.