[Core][Autoscaler] Update the Autoscaler Resource Requests Data Model for Scheduling for Label Selector #51771

MengjinYan · 2025-03-27T21:36:29Z

Why are these changes needed?

This PR updated the resource request & gang resource request data model for communication between GCS and Autoscaler.

This is to enable GCS to send the associated node label requirements to Autoscaler to support the labeled scheduling feature.

The PR added the label selectors to the resource requests and Bundle selectors for the gang resource requests.

Reference: Public label based scheduling API

Related issue number

#51564

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

brucez-anyscale · 2025-03-28T21:11:29Z

src/ray/protobuf/autoscaler.proto

+  LABEL_OPERATOR_IN = 1;
+  // This is to support not equal or not in semantics.
+  LABEL_OPERATOR_NOT_IN = 2;


so exist or not exist are not supported at the first place.

Yes from my understanding here: https://docs.google.com/document/d/1DKhPuZERlLbsU4TIQZZ6POCsm1pVMBgN_yn5r0OmaDI/edit?tab=t.0#heading=h.428qclpgsw2d

cc: @edoakes to confirm

sg. The pr lg. Will let Janet take a look as well

correct, we can add in the future

MengjinYan · 2025-03-31T17:35:03Z

@edoakes Wondering if you can help to take a look as well? Thanks!

src/ray/protobuf/autoscaler.proto

edoakes · 2025-03-31T18:37:50Z

src/ray/protobuf/autoscaler.proto

+  // The node label requirements for the request. Multiple label selectors are for
+  // fallback mechanism. When trying to find a node that satisfies the label
+  // requirements, the first label selector should be tried first, if not found,
+  // the second label selector should be tried, and so on.
+  repeated LabelSelector label_selectors = 3;


how would this work with coordinated fallback between resources and labels? That is, overriding resource requirements depending on the label selector that's selected.

we don't necessarily need to make the change to support this immediately if there is a path to support it.

My idea is that if we want to do coordinated fallback between resources and labels, we can do similar things as the what we are doing with the placement group bundles:

deprecate the current resources_bundle field

add a new proto message ResourceSelector with 1 map<string, double> field

add a new field with type repeated ResourceSelector in the current ResourceRequest proto message

In this way, the newly added resource_selectors and the existing label_selectors can work together to support the coordinated fallback mechanism. What do you think?

This makes sense to me -- we should align the interfaces for PG and regular tasks/actors as much as possible

Do you think we should do it now, or can be added when we take on the work to do resource fallback?

I think we can do that when we do the resource fallback work.

edoakes · 2025-03-31T18:39:08Z

src/ray/protobuf/autoscaler.proto

+message BundleSelector {
+  // The list of resource requests that should be allocated together.
+  repeated ResourceRequest resource_requests = 1;
+}


Can you provide an example of how this would be used for fallback?

Actually the note about the fallback mechanism for GangResourceRequest is added inside the comment in GangResourceRequest: https://github.com/ray-project/ray/pull/51771/files#diff-79033515bd7d0b4d966220d9c45290447e0aac8afd71b048a817b0d9bdfaa052R114-R117

// The bundle requests. Multiple bundle selectors are for fallback mechanism. // When trying to find nodes that satisfies the bundle selector, the first bundle // selector should be tried first, if not found, the second bundle selector should be // tried, and so on.

I also just added the more notes in ResourceRequest indicating that in bundle selector case, the fallback will be done on the bundle selector level and in resource request, there should only be 1 label selector.

Wondering if this is something you were looking for?

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

MengjinYan · 2025-04-14T21:48:03Z

@edoakes All tests are passed. Wondering if you can help merge the PR? Thanks!

update the autoscaler schema for scheduling with label selector

c7e646e

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

MengjinYan mentioned this pull request Mar 27, 2025

[Core] Ray Label Selector API Implementation Tracker #51564

Open

23 tasks

MengjinYan requested review from edoakes and brucez-anyscale March 27, 2025 21:39

brucez-anyscale reviewed Mar 28, 2025

View reviewed changes

janetli19 approved these changes Mar 28, 2025

View reviewed changes

brucez-anyscale approved these changes Mar 28, 2025

View reviewed changes

Merge branch 'master' into issue-51564

8966a65

MengjinYan added the go add ONLY when ready to merge, run all tests label Mar 31, 2025

edoakes reviewed Mar 31, 2025

View reviewed changes

MengjinYan and others added 2 commits March 31, 2025 11:54

Update sequence of fields in label selector

d67f91c

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

Update comment on label selector

44c5218

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

cszhu added the core Issues that should be addressed in Ray Core label Apr 3, 2025

MengjinYan added 2 commits April 11, 2025 09:02

add more note on fallback

afb7a8c

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>

Merge branch 'master' into issue-51564

2f8892a

edoakes approved these changes Apr 11, 2025

View reviewed changes

Merge branch 'master' into issue-51564

64a4abd

edoakes enabled auto-merge (squash) April 11, 2025 13:18

Merge branch 'master' into issue-51564

2605e72

github-actions bot disabled auto-merge April 11, 2025 21:42

edoakes merged commit b7b7645 into ray-project:master Apr 15, 2025
5 checks passed

hainesmichaelc added the community-backlog label May 22, 2025

[Core][Autoscaler] Update the Autoscaler Resource Requests Data Model for Scheduling for Label Selector #51771

[Core][Autoscaler] Update the Autoscaler Resource Requests Data Model for Scheduling for Label Selector #51771

Uh oh!

Conversation

MengjinYan commented Mar 27, 2025

Why are these changes needed?

Related issue number

Checks

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MengjinYan commented Mar 31, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MengjinYan commented Apr 14, 2025

Uh oh!

Uh oh!

Uh oh!