#113225 uncovered a coming snarl of dependencies between cloud.google.com/go and google.golang.org/genproto

Because several generic libraries use the cloud.google.com/go/compute/metadata package, the following chain makes it almost impossible to avoid adding all cloud.google.com/go submodules into the dependency graph for all downstream consumers:

• {golang.org/x/oauth2, google.golang.org/api, google.golang.org/grpc} --> cloud.google.com/go/compute --> cloud.google.com/go --> google.golang.org/genproto --> cloud.google.com/go/*

It's possible we can avoid the updates that expanded these cross-references in the short-term with judicious upstream changes back to older genproto versions, but we should get ahead of this if we want to avoid being forced to pull in these dependency graph increases in the future for a critical bugfix.

Recommendations

1. Isolate cloud.google.com/go/compute/metadata

The cloud.google.com/go/compute/metadata package (https://github.com/googleapis/google-cloud-go/tree/main/compute/metadata) only uses stdlib dependencies, but lives inside cloud.google.com/go/compute, which has many other references (https://github.com/googleapis/google-cloud-go/blob/main/compute/go.mod)

1. Isolate cloud.google.com/go/compute/metadata to its own zero-dependency module

   • compute/metadata: make its own module googleapis/google-cloud-go#6311 / https://pkg.go.dev/cloud.google.com/go/compute/metadata
   • drop backreference to cloud.google.com/go/compute from https://github.com/googleapis/google-cloud-go/blob/main/compute/metadata/go.mod#L5 to reach zero-dependencies - chore: re-drop weak refs to parent modules and tag googleapis/google-cloud-go#9545
   • tag a release of cloud.google.com/go/compute/metadata - compute/metadata/v0.3.0

2. Sweep dependencies using cloud.google.com/go/compute/metadata and update them to use the zero-dependency alternative

   • golang.org/x/oauth2 - v0.20.0+
   • google.golang.org/api - v0.174.0+
   • google.golang.org/grpc - v1.65.0+
   • github.com/google/cadvisor (transitive via golang.org/x/oauth2, google.golang.org/api, google.golang.org/grpc)
     • bump containerd related deps: api v1.8.0; errdefs v1.0.0; ttrpc v1.2.6 google/cadvisor#3622
     • tagged
   • go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc (transitive via golang.org/x/oauth2, google.golang.org/grpc)
     • committed - [chore] updating google.golang.org/genproto open-telemetry/opentelemetry-go-contrib#3968
     • tagged

2. Prune use of cloud genproto packages in non-cloud.google.com modules

1. Update github.com/googleapis/gax-go/v2 test to not require the google.golang.org/genproto/googleapis/cloud/location package
   • committed - chore(v2): switch ProtoJSONStream test to use non-cloud API googleapis/gax-go#237
   • tagged - https://github.com/googleapis/gax-go/releases/tag/v2.7.0
   • Use a non-cloud type for nested unit test googleapis/gax-go#270
   • tag release (>=v2.9.1)

3. Isolate generic genproto packages

• make the types in the following generic packages available without dragging in all of cloud.google.com/go

  • google.golang.org/genproto/googleapis/api/...
  • google.golang.org/genproto/googleapis/bytestream
  • google.golang.org/genproto/googleapis/rpc/...
  • Submodules created in Isolate submodules, take 2 googleapis/go-genproto#1012
  • Backreferences dropped in chore(all): Drop submodule backreferences googleapis/go-genproto#1087, chore(all): Update to google.golang.org/genproto/googleapis/rpc without backreference googleapis/go-genproto#1093

4. Drop use of google.golang.org/genproto/protobuf/field_mask

Update to use canonical google.golang.org/protobuf/types/known/fieldmaskpb package instead

• google.golang.org/genproto
  • Eliminate use of alias field_mask package googleapis/go-genproto#1008
• cloud.google.com/go
  • chore(all): Use fieldmask types directly instead of field_mask genproto alias googleapis/google-cloud-go#7962
• github.com/grpc-ecosystem/grpc-gateway/v2
  • Switch from aliased field_mask package to canonical package grpc-ecosystem/grpc-gateway#3317
• github.com/grpc-ecosystem/grpc-gateway
  • Mark grpc-gateway v1 as an unwanted dependency #118119
  • etcd depends on two different versions of the same package - grpc-gateway etcd-io/etcd#14499 (used by go.etcd.io/etcd/api/v3 / go.etcd.io/etcd/server/v3)
    • Upgrade grpc-gateway from v1 to v2 etcd-io/etcd#16595
    • etcd client release (3.6)
    • update kubernetes to use etcd client 3.6+ - etcd 3.6 client update #128419

/priority important-longterm
/area code-organization
/sig architecture
/cc @dims
fyi @codyoss @bertinatto