Skip to content

Conversation

jprestop
Copy link
Collaborator

@jprestop jprestop commented May 8, 2025

This pull request adds OpenMP parallelization to the MET application code for Pcp-Combine (7 parallelizations added) and Gen-Vx-Mask (12 parallelizations added). The feature_3140_openmp_apps_pcp_genvxmask branch was created from the feature_3132_openmp_vx_util branch (PR #3136).

The changes for this PR include OpenMP parallelization and SonarQube code smell remediation. In particular, the following code smells were addressed:

  • Add the 'static' storage specifier to this declaration
  • Replace the redundant type with "auto"
  • Convert this string literal to a raw string literal
  • Function 'usage' could be declared with attribute 'noreturn'

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

    Validation Testing:

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    Please review code changes for accuracy and consistency.
    Review the validation testing (links above) to see the speedup.
    This branch is built on seneca at /d1/projects/MET/MET_pull_requests/met-12.1.0/rc1/MET-feature_3140_openmp_apps_pcp_genvxmask in case you want to do any testing.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

  • Do these changes include sufficient testing updates? [Yes]

  • Will this PR result in changes to the MET test suite? [No]

    If yes, describe the new output and/or changes to the existing output:

  • Will this PR result in changes to existing METplus Use Cases? [No]

    If yes, create a new Update Truth METplus issue to describe them.

  • Do these changes introduce new SonarQube findings? [No]

    If yes, please describe:
    While some differences are flagged in "new code" the overall number of SonarQube code smells are reduced by 44 (from 17,114 for develop to 17,070 for this PR).

  • Please complete this pull request review by [20250515].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the version that will include these changes
    Select: METplus-X.Y Support project for bugfix releases or MET-X.Y Development project for the next coordinated release
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

JohnHalleyGotway and others added 30 commits April 17, 2025 21:43
… in the vx_regrid library to avoid duplication.
… using stl::vector's instead of manual memory management. This required introducing a DataPlaneArray::at(int) accessor function. Modify some library and application code to get things compiling.
…ike apply_mask(). Doing so produced a Grid-Stat runtime failure.
… parallelized regridding actually runs well and produces the expected output.
…runtime segfault in the unit_regrid.xml unit test.
… the maximum number of available threads. ci-run-unit
…nce that triggers a failure when ncdiff is used to compute differences: libgomp: Invalid value for environment variable OMP_NUM_THREADS ... ci-run-unit
…te unit_met_test_scripts.xml to add missing expected output file names. ci-run-unit
…uction(+:n) to avoid race conditions updating the counter. ci-run-unit
@jprestop jprestop added this to the MET-12.1.0 milestone May 8, 2025
@github-project-automation github-project-automation bot moved this to 🩺 Needs Triage in METplus-6.1 Development May 8, 2025
@jprestop jprestop moved this from 🩺 Needs Triage to 🔎 In review in METplus-6.1 Development May 8, 2025
@jprestop jprestop requested a review from JohnHalleyGotway May 8, 2025 20:31
@jprestop jprestop linked an issue May 8, 2025 that may be closed by this pull request
8 tasks
@jprestop jprestop marked this pull request as ready for review May 8, 2025 20:32
@JohnHalleyGotway JohnHalleyGotway changed the title Feature 3140 openmp apps pcp genvxmask Feature #3140 openmp apps pcp genvxmask May 8, 2025
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jprestop I approve of these changes. They look great!

Thanks for linking to your tests in runtime improvement. I confirmed that no diffs are flagged in the by the testing workflow and that the overall number of SonarQube code smells are reduced. I reviewed the code changes and they match what I was expecting.

} // end for y
} // end for x
} // end of parallel

if(!put_nc_data_with_dims(&mask_var, mask_data.data(), grid.ny(), grid.nx())) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jprestop I'll note that I'm planning to replace this section with a call to a new put_nc_data_plane_float() utility function that I'm adding for #3145.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@jprestop jprestop merged commit 15b5afb into develop May 9, 2025
39 of 42 checks passed
@github-project-automation github-project-automation bot moved this from 🔎 In review to 🏁 Done in METplus-6.1 Development May 9, 2025
@jprestop jprestop deleted the feature_3140_openmp_apps_pcp_genvxmask branch May 9, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: 🏁 Done
Development

Successfully merging this pull request may close these issues.

Apply OpenMP to parallelize loops over grid dimensions in MET Applications
2 participants