Skip to content

Bugfix: Fix the parsing logic when setting the file_type config option for the forecast or observation but not the climatology #3179

@JohnHalleyGotway

Description

@JohnHalleyGotway

Describe the Problem

This issue was raised by @CPKalb and @j-opatz while working with the model_applications/s2s_soil_moisture/GridStat_fcstSFSGSL_obsERA5Land_SoilMoisture METplus Use Case. This use case configures and runs Grid-Stat and Series-Analysis in almost identical ways but those two tools behave differently when parsing climatology data. The following inputs are used:

  • Forecast data is supplied through Python embedding (e.g. file_type = PYTHON_NUMPY).
  • Observation data is CF-Compliant NetCDF (e.g. file_type = NETCDF_NCCF).
  • Climatology data is MET NetCDF (e.g. file_type = NETCDF_MET).

Grid-Stat runs and parses the climatology data without problem. However, Series-Analysis requires SERIES_ANALYSIS_FCST_CLIMO_MEAN_VAR1_OPTIONS = file_type = NETCDF_MET; to be set in order for it to be run. Otherwise, it attempts to read the NetCDF files using the Python embedding library code.

Expected Behavior

The behavior of Grid-Stat and Series-Analysis reading climo data should be as consistent as possible.

Here is an explanation for this behavior:

  • MET#2924 enhanced MET version 12.0.0 to support reading separate forecast and observation climatology inputs.
  • For this, the parsing of the climo_mean and climo_stdev dictionaries was moved from the top-level of config file context, to being parsed separately from the fcst and obs dictionaries.
  • Python embedding works slightly differently in Grid-Stat and Series-Analysis.
  • For Series-Analysis, fcst.file_type = PYTHON_NUMPY; is set to enable Python embedding.
  • For Grid-Stat, fcst.file_type is NOT set.
  • When parsing MET config options, the settings of the parent (e.g. fcst or obs) are inherited by its children (e.g. climo_mean and climo_stdev).
  • So fcst.file_type = PYTHON_NUMPY is parsed and applied to the fcst.climo_mean and fcst.climo_stdev dictionaries.

The fix is to explicitly disable searching the "parent" when parsing the climatology file_type config setting.

To Reproduce

Describe the steps to reproduce the behavior:
On seneca, run /d1/projects/METplus/discussions/kalb_20250612/METplus/run_sa.sh.
The run_sa_v12.0.2.log file illustrates the runtime errors.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as the next bugfix version
  • Select METplus-X.Y Support project for support of the current coordinated release
  • Select MET-X.Y Development project for development toward the next coordinated release

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next bugfix version
    Select: METplus-X.Y Support project for support of the current coordinated release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y Development project for development toward the next coordinated release
  • Close this issue.

Metadata

Metadata

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions