Describe the Enhancement

There is room for improvement in the logic used to compare numeric values in metplus/util/diff_util.py, in particular around precision limits for comparing values.

There is currently a DEFAULT_ROUNDING_PRECISION = 6 global variable to limit the number of decimal places to compare, which can be overridden by a keyword search with the ROUNDING_OVERRIDES list.

#3019 added global variable SIG_FIG = 7 to limit the number of significant figures to compare. This was added so that this differencing logic could be applied to MET tests, which previously used logic that limited to 6 significant figures.

On some MET tests run though the GHA Testing workflow, an error is raised due to the difference between values 103.88155 and 103.88154, which differ by 1 in their 7th digit. The question of how best to handle this raised larger questions about what kind of precision limits should be applied and how.

Some things to consider:

• Do we want to keep both a rounding precision and a significant figure limit?
• Do we want to compare the same number of sig figs on all test output? Or should we implement a varying sig fig limit, depending on ... [keywords, file type, value type, ?] ?
• If we have a single sig fig limit, what should it be? Is 7 good? Would 6 suffice?
• Should we allow the least significant figure that we are including to differ by 1?

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as a METplus-Wrappers-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
• For a METplus-Wrappers-X.Y.Z version, select the METplus-Wrappers-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Add any new Python packages to the METplus Components Python Requirements table.
• For any new datasets, an entry to the METplus Verification Datasets Guide.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: METplus-Wrappers-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.