-
Notifications
You must be signed in to change notification settings - Fork 321
Add better support for dependency management (aka BOM) #3097
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add better support for dependency management (aka BOM) #3097
Conversation
64fffe6
to
a1f37b3
Compare
6cdf73c
to
4d52c37
Compare
cc22c69
to
a8173b4
Compare
This comment was marked as resolved.
This comment was marked as resolved.
5eec3c5
to
945798d
Compare
3bfd3d5
to
c89f5b1
Compare
This comment was marked as outdated.
This comment was marked as outdated.
b2aefad
to
027a3e6
Compare
cc @eed3si9n who dived a quite a bit in dependency resolver intricacies and @lefou who manifested interest for BOM support in #1390. Also cc-ing @lihaoyi and @adpi2 given this should impact both Mill and sbt, and @jin for rules_jvm_external |
I haven't reviewed the code, but the PR description looks like exactly what we'd want, since it makes Coursier resolutions behave the same as Maven or Gradle or other tools |
I'm probably going to have a look at this next: adding an equivalent to |
I think that matches my perception. Local dependency declarations override management overrides parent poms. There are a lot of precedence rules in Maven with regard to dependency management. I think the following page list some or all (?) of them: It also mentions a new packaging type In case Coursier will follow the Maven semantics, we should name the new task in Mill rather |
cc @shs96c for rules_jvm_external's BOM support. We currently ask folks who need BOMs to switch to a different Aether based resolver (https://github.com/bazel-contrib/rules_jvm_external?tab=readme-ov-file#experimental-support-for-maven-bom-files), so it would definitely be nice to have this. We will be happy to try out a build of coursier with our tests (https://github.com/bazel-contrib/rules_jvm_external/blob/3ea96498725cb21e3ac18490cc3f3b75db2259ca/MODULE.bazel#L618, https://github.com/bazel-contrib/rules_jvm_external/blob/master/MODULE.bazel#L432) |
027a3e6
to
6dcf08a
Compare
Like Maven is supposed to do
6dcf08a
to
45e49fa
Compare
@lefou Thanks for all these details. Let's discuss these Mill-specific points in an upcoming Mill issue or early PR that I'll open about this |
@eed3si9n Thanks for mentioning this. I'm well aware of this point, coursier does follow this rule when using dependency management. I didn't mention it to keep my explanations simple, but this will have to be taken into account when generating POM files: in sbt, if |
Going to merge this, with the feature enabled by default. I should mention it in the next release notes, but those changes can be easily disabled, and you might want to allow sbt / Mill / rules_jvm_external users to do so. From the API, it can be disabled via the Resolve()
.mapResolutionParams(_.withEnableDependencyOverrides(Some(false)))
// or
Fetch()
.mapResolutionParams(_.withEnableDependencyOverrides(Some(false))) On the CLI, |
For |
cc @Gedochao, as Scala CLI resolves dependencies and publishes them too. IIRC, Scala CLI uses coursier via coursier-interface. Options to disable the feature here should be added to it too, to allow Scala CLI to disable it too. Maybe some BOM support could be added to Scala CLI too (see discussion above) |
What these changes are about
This PR changes things related to the so called Maven "BOMs" or "dependency management" support. It changes the way versions are picked for many dependencies of various JVM ecosystems (Android, Spark, Spring, …), hence the "sensitiveness" of these changes.
What BOMs are
BOMs are basically POM files listing versions to be used for dependencies. They look like this: this POM contains a
dependencyManagement
section, listing many dependencies. These dependencies' version and sometimes exclusions matter to us here.A standard Maven dependency can pull a BOM either by declaring it as a "parent", like at the top of this POM, or by declaring an "import" dependency in its
dependencies
ordependencyManagement
sections, like in this POM. Note that parents POMs can themselves have parents and "import" dependencies, whose values are taken into account too. Note also that POMs can also have a<dependencyManagement>
section, acting as a BOM too, like this POM - BOM values don't always come from other POM files.When added to a Maven project, Maven BOMs are basically expected to be dependency override lists. If project A has B:1.2 in its BOM, it is expected either: not to pull B (if neither A or its transitive dependencies depend on B), or to pull B version
1.2
(and no other version). No matter what version of B a transitive dependency of A depends on, if pulled via A, B should have version1.2
, because of A's BOM.Also, it's common practice in Maven projects having a BOM not to put versions in dependencies in their
dependencies
section. These versions are supposed to be filled in with values from the BOM. We can see that clearly in this POM.BOM support in coursier
Up to now, coursier used BOM values for the latter case (filling up empty versions of direct dependencies of the dependency that pulls the BOM), not the former (dependency overrides, applying to transitive dependencies too). Early checks of mine (around… 2015 😅) didn't display the former behavior, so I assumed all those BOM values where only meant for the latter case, when one pulls Maven dependencies.
This PR changes that: it uses the BOM values as "dependency overrides" too, applying to transitive dependencies.
Impact of those changes
One can see in the diff of this PR how many resolutions are impacted (or not impacted…). Many resolutions' results are hard-coded under
modules/tests/shared/src/test/resources/resolutions
. We can see in the diff here that Spark or Quarkus dependencies are impacted, among others. Overall, these changes are consistent with the new BOM values handling (all individual values I looked at in detail were fine at least).Should these changes be enabled by default right now?
The new way of computing versions gives more consistent dependencies to me, more in line with what Maven users use when building their projects. These changes ought to be welcomed, yet they might surprise some users.
I'm thinking these could be enabled by default, but advertized loudly in release notes, and users need to be able to disable them if they want to.
Dependency overrides handling in build tools
BOMs give a "first class citizen" status to "dependency overrides". BOMs are dependency overrides that can be published, and pulled by downstream users.
In sbt for example, the values of
dependencyOverrides
are equivalent to BOM values. As a consequence, these can be published. These can go in a<dependencyManagement>
section of a module POM file, like in this POM. Then, when users pull that sbt-published module, coursier applies the dependency overrides from the<dependencyManagement>
section to transitive dependencies pulled by the module.AFAIK, Mill doesn't have an equivalent target.
A simple setting like
dependencyOverrides
in sbt or an equivalent target in Mill only correspond todependencyManagement
sections in library POMs. They don't allow to publish shared dependency overrides via "parent POMs" and "dependency imports", nor to pull such overrides from Maven Central. Gradle apparently allows such uses.