Skip to content

Conversation

pp-mo
Copy link
Member

@pp-mo pp-mo commented Feb 24, 2025

Rework replaces #6331, addresses #6203

Copy link

codecov bot commented Feb 24, 2025

Codecov Report

Attention: Patch coverage is 85.26316% with 14 lines in your changes missing coverage. Please review.

Project coverage is 89.78%. Comparing base (8745465) to head (44bb2c6).
Report is 81 commits behind head on main.

Files with missing lines Patch % Lines
lib/iris/util.py 59.09% 5 Missing and 4 partials ⚠️
lib/iris/_combine.py 90.56% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6334      +/-   ##
==========================================
- Coverage   89.80%   89.78%   -0.02%     
==========================================
  Files          90       90              
  Lines       23589    23632      +43     
  Branches     4402     4411       +9     
==========================================
+ Hits        21183    21217      +34     
- Misses       1662     1667       +5     
- Partials      744      748       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pp-mo pp-mo linked an issue Feb 25, 2025 that may be closed by this pull request
@pp-mo
Copy link
Member Author

pp-mo commented Feb 25, 2025

Status update 2025-02-25

  • functional code and APIs are basically complete and correct
  • basic API documentation is now complete + correct
  • doctests and examples need review
  • all existing tests are passing
  • most additional tests are now present
  • some extra testing may be needed

The currently-failing doctests in the docsring of the combine_cubes() utility routine are just a placeholder.
(We do need a working example there...)

My current details to-do list

(maybe not exhaustive) ...

  • add unit tests for
    • LoadPolicy.context()
    • equalise_cubes_kwargscontrol
      • what happens if this contains invalid settings -- does that need further handling ?
        (update: no I don't think this needs handling)
    • merge_unique control
    • CubeList combine methods
  • review options + settings explanations for
    • `CombineOptions
      • especially "settings" are more directly appropriate for loading -- needs saying, at least
    • LoadPolicy
      • needs (at least) strong hint to refer to settings -- documented under CombineOptions. ?use example?
  • add a proper example/doctests for
    • CombineOptions
    • LOAD_POLICY
    • combine_cubes
  • consider using enums for switchable API options (at least the merge_concat_sequence values)
  • cross-reference user documentation between (?all-of ?) ...
    • load functions
    • iris main module
    • LoadPolicy
    • user-guide loading chapter

@pp-mo pp-mo force-pushed the combine_cubes_2 branch 3 times, most recently from 557076f to 7ca4d8e Compare February 25, 2025 23:34
@pp-mo
Copy link
Member Author

pp-mo commented Feb 26, 2025

I'm going to mark this "ready for review", since it is (I think) functionally complete.
But I must admit the list of to-dos is a bit daunting.

For a functional query, I'm still a bit unsure of the relationship complexity between CombineOptions and LoadPolicy,
not least because the default for the options of a combine_cubes call is defined by LOAD_POLICY.
And you can see that in the subsidiary _combine_options_asdict routine which I needed to pull out for sharing,which 'ought' to live in iris._combine, but actually relies on iris.loading .
Hmmm
(update : resolved -- see below}

Hopefully I can get a big chunk of the TODOs today.

@pp-mo pp-mo marked this pull request as ready for review February 26, 2025 09:53
@pp-mo
Copy link
Member Author

pp-mo commented Feb 26, 2025

Note:

Following considerable offline discussion with @trexfeathers @stephenworsley ,
We decided it was better to re-merge the definitions of CombineOptions and LoadPolicy
So I've now done that : f7ecf84

In short, LOAD_POLICY and LoadPolicy are now just aliases for COMBINE_POLICY (object) and CombineOptions (class).
The notable reasons were :

  • since all APIs (internal included) took settings dictionaries and not settings objects, there was no need ever to create a CombineOptions object
    • ... and clearly it was over-complex to make a separate global control object for combine_cubes, so I was going to be using LOAD_POLICY to control that too
  • in order to make either or both of CombineOptions and LoadPolicydocs legible, I was finding I had to replicate documentation writeup between the two
  • the "settings" classes of a CombineObject wanted to be the same as those of a LoadPolicy, but these were determined by the needs of the loading modes -- and notably tje distinction between "default" and "legacy" which is meaningless for combine_cubes)

So in the end ... it makes more sense to have a single definition, parts of which are irrelevant to one use, than to separate the two for the sake of a small difference

@pp-mo pp-mo moved this from 🚀 In Progress to 👀 In Review in 🦔 v3.12.0 Feb 27, 2025
@pp-mo pp-mo moved this from 👀 In Review to 🚀 In Progress in 🦔 v3.12.0 Feb 27, 2025
@pp-mo
Copy link
Member Author

pp-mo commented Feb 27, 2025

Update 2025-02-26

I finally completed my checklist for adding tests 🍾

I've also added a whatsnew, and some really minimal redirection links from the iris main module API and the userguide loading chapter.
HOWEVER I have also realised that we are missing (probably need) a mention of combine in the UG chapter on merge+concatenate. I don't know quite how to contemplate that at this point.

For now, I'm just awaiting comments

@pp-mo pp-mo moved this from 🚀 In Progress to 👀 In Review in 🦔 v3.12.0 Feb 27, 2025
Copy link
Contributor

@stephenworsley stephenworsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. I'm pretty happy with the decision to simplify the API by making LOAD_POLICY a copy of COMBINE_POLICY, though I think it's worth adding another test to ensure that modifying one does indeed modify the other (and vice versa). I just have a couple suggestions for the documentation for the moment, I plan to go look through the testing next. So far, the code itself looks like a definite improvement.

@pp-mo
Copy link
Member Author

pp-mo commented Mar 3, 2025

making LOAD_POLICY a copy of COMBINE_POLICY, though I think it's worth adding another test to ensure that modifying one does indeed modify the other

I don't think there's a real point to answer for this one, as they aren't copies but different names for the actual same object.
Both references are set to the same thing here

@pp-mo
Copy link
Member Author

pp-mo commented Mar 3, 2025

Update : thanks @stephenworsley
I think I addressed all comments to-date
(which is not to say there is not more to come)

please re-review !

@stephenworsley
Copy link
Contributor

I don't think there's a real point to answer for this one, as they aren't copies but different names for the actual same object.
Both references are set to the same thing here

ah yes, that's what I meant, I was just imprecise (or incorrect) in my language. I ask because I have seen some confusion in the past over how to expect configuration objects to behave when being modified or set (e.g. code of the form iris.COMBINE_POLICY = ... rather than iris.COMBINE_POLICY[key] = ..., which I imagine is not what we would expect to support). I think I recall that for some non-standard use cases, it mattered how these objects were imported so I wondered if there was any call for tests for what happens to the same object imported with different names. If there was any non-trivial interaction we wanted to ensure was supported, that might be worth considering a test for, but it may be the case as you say that everything is either trivial or ought to be unsupported.

Copy link
Contributor

@stephenworsley stephenworsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, Just a couple small notes from the tests.

@pp-mo
Copy link
Member Author

pp-mo commented Mar 7, 2025

I don't think there's a real point to answer for this one, as they aren't copies but different names for the actual same object.
Both references are set to the same thing here

ah yes, that's what I meant, I was just imprecise (or incorrect) in my language. I ask because I have seen some confusion in the past over how to expect configuration objects to behave when being modified or set (e.g. code of the form iris.COMBINE_POLICY = ... rather than iris.COMBINE_POLICY[key] = ..., which I imagine is not what we would expect to support). I think I recall that for some non-standard use cases, it mattered how these objects were imported so I wondered if there was any call for tests for what happens to the same object imported with different names. If there was any non-trivial interaction we wanted to ensure was supported, that might be worth considering a test for, but it may be the case as you say that everything is either trivial or ought to be unsupported.

OK I think I do see what you mean now.
I will add a warning about this to the docs where the control usage is explained.
See 857a4f5

@pp-mo pp-mo force-pushed the combine_cubes_2 branch 2 times, most recently from 6bfe028 to 0c11048 Compare March 7, 2025 01:39
@pp-mo
Copy link
Member Author

pp-mo commented Mar 7, 2025

Update 2024-03-6

Thanks @stephenworsley responded to comments so-far + up to date, I hope.
Please re-check !

Note that while other PRs are merging at present, unfortunately every merge tends to create conflict in the whastnew/latest.rst.

Copy link
Contributor

@stephenworsley stephenworsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple TODOs to remove.

@pp-mo
Copy link
Member Author

pp-mo commented Mar 7, 2025

OK I had to rebase yet again, but hopefully that addresses the stale TODOs you pointed out.

@pp-mo pp-mo force-pushed the combine_cubes_2 branch from f82fb0c to 44bb2c6 Compare March 7, 2025 16:54
Copy link
Contributor

@stephenworsley stephenworsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I think this is mergable now.

@stephenworsley stephenworsley merged commit cf85915 into SciTools:main Mar 7, 2025
21 checks passed
@github-project-automation github-project-automation bot moved this from 👀 In Review to 🏁 Done in 🦔 v3.12.0 Mar 7, 2025
@pp-mo
Copy link
Member Author

pp-mo commented Mar 7, 2025

Many thanks @stephenworsley !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏁 Done
Development

Successfully merging this pull request may close these issues.

Make _combine_cubes public, expand API
2 participants