Skip to content

Slow runtime for regrid_data_plane MAX interpolation in MET-10.0.0_beta5 #1778

@LoganDawson-NOAA

Description

@LoganDawson-NOAA

Replace italics below with details for this issue.

Describe the Problem

When testing the beta5 release for METplus-4.0.0 and MET-10.0.0, I discovered a regrid_data_plane MAX interpolation step that is taking ~8x longer (~12 minutes vs. 90 seconds) to run with MET-10.0.0_beta5 than with MET-9.1. When I initially ran this step in a batch job and from the command line, I thought MET was hanging and unable to complete the interpolation. The input file for this step is a file output by regrid_data_plane at a prior step, so I knew MET shouldn't have a problem reading the file. I ran the command again and realized the interpolation command is simply taking longer to run.

Expected Behavior

With MET-9.1, this regrid_data_plane MAX interpolation step took on the order of 90 seconds to complete. With MET-10.0.0_beta5, the same step took upwards of 12 minutes to complete.

Environment

Describe your runtime environment:
1. Machine: WCOSS Dell
2. OS: Linux
3. MET-10.0.0_beta5

To Reproduce

Describe the steps to reproduce the behavior:
1. Interpolate the GRIB2 data to G227 using the following command:
met/10.0.0-beta5/bin/regrid_data_plane -v 5 -method BUDGET -width 2 -field 'name="MergedReflectivityQCComposite"; level="Z500";' -name MergedReflectivityQCComposite MergedReflectivityQCComposite_00.50_20210504-190000.grib2 G227.nc MergedReflectivityQCComposite_00.50_20210504-190000_g227.nc

2. Run MAX interpolate on the prior step's output using the following command:
met/10.0.0-beta5/bin/regrid_data_plane -v 5 -method MAX -width 17 -field 'name="MergedReflectivityQCComposite"; level="Z500";' -name MergedReflectivityQCComposite MergedReflectivityQCComposite_00.50_20210504-190000_g227.nc G227.nc MergedReflectivityQCComposite_MAX40_20210504-190000_g227.nc

3. See the significantly slower runtime for the second command. Running the command(s) within a script that can provide timing info would be useful.

4. Running the same two commands with a MET-9.1 executable will show that the runtime is approximately the same for the first interpolation step, but the second step is considerably slower.

Both the input file (for step 1 and step 2) and the "to grid" file have been posted to the ftp.rap.ucar.edu site.

These commands were generated by METplus-4.0.0_beta5 and METplus-3.1 in my test environment. I can provide the METplus conf files and commands if those would be helpful, but the issue appears isolated to MET itself. I have however attached METplus log files, which provide the output I've seen and the timing information as well.

master_metplus.METplus-3.1.log.20210505164635.txt
master_metplus.METplus-4.0.0-beta5.log.20210504230632.txt

Relevant Deadlines

Investigating before MET-10.0.0 official release would be helpful.

Funding Source

NONE

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
  • Select milestone to relevant bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project(s), Milestone, and Linked issues
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
  • Close this issue.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions