Describe the Enhancement

The -set_hdr job command option can be specified to manually override the columns of data written by the Stat-Analysis tool. By default, when aggregating STAT lines, Stat-Analysis writes a comma-separated list of the unique input values. For example, when aggregating data for the northern and southern hemispheres (denoted by VX_MASK set to NH or SH in the input), it will write the output VX_MASK column as NH,SH. In this case, the -set_hdr VX_MASK GLOBAL job command option can be used to override that comma-separated string.

Currently, the -set_hdr option only affects the output of jobs that perform aggregation. They do not currently apply to the data written to the -dump_row output file, which is where the filtered input data used to perform the job is written.

This task is to update Stat-Analysis to actually apply the -set_hdr option to the -dump_row output but ONLY when running a filter type job (-job filter). A filter job reads input STAT data, applies the user-specified filtering criteria, and writes the filtered data to the -dump_row output file. Update the logic to use -set_hdr to manually override the columns when writing the -dump_row output file.

This issue arose from the DTC Space Weather project. @jvigh would like to use Stat-Analysis to replace the default NA string in the DESC column with a date-specific index category computed elsewhere. Enhancing Stat-Analysis filter jobs in this way will may this easier to do.

Time Estimate

4 hours.

Sub-Issues

Consider breaking the enhancement down into sub-issues.
None needed.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as the next official version or Backlog of Development Ideas
• For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
  No impacts to other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: MET-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.