Skip to content

Extending the Azure DevOps Agent scaling - possibility to "ignore" demands #5579

@jan-mrm

Description

@jan-mrm

Proposal

I would like the possibility within the azure-pipelines trigger of a ScaledObject and ScaledJob to differentiate between demands which are required for scaling up and demands which are ignore/accepted but not important for actually triggering the scaling.

Idea:

demands: 'cap-8gb'
requireAllDemands: true
demandsToIgnore: 'npm' # new configuration

This is because we are not fully in control of the demands which are requested by an Azure DevOps pipeline. Demands can be defined in the pipeline definition. But demands are also just added automatically when using certain tasks.

Use-Case

We currently use Keda to scale our Azure DevOps agents using the azure-pipelines trigger, with the following setup:

  • We have two kinds of agents which differ in the resources (more kinds or other differentiation could come). Lets say the two kinds are 4GB and 8GB. So as demand something like cap-4gb and cap-8gb.
  • We set requireAllDemands: false for the 4GB agent, so that it would be the default, since a pipeline job with no demands at all will trigger this agent (and it is also triggered when a pipeline has the demand cap-4gb).
demands: 'cap-4gb'
requireAllDemands: false
  • We set requireAllDemands: true for the 8GB agent, because we want to avoid scaling both agents when a pipeline job with no demands is issued
demands: 'cap-8gb'
requireAllDemands: true

This worked out for us until we investigated a scaling issue where a job with demand cap-8gb would not trigger Keda to scale. It turned out the pipeline job uses a Npm@1 task which as stated at the bottom under Requirements will add the demand npm to the pipeline just through using that task.
This does conflict with requireAllDemands: true since it checks for the exact match (in this case the job will have demands cap-8gb and npm).

Is this a feature you are interested in implementing yourself?

Yes

Anything else?

Quick fixes are

  • adding a third agent with demands cap-8gb,npm and requireAllDemands: true
  • adding another trigger for both kinds of agents which account for npm

Both solutions work for now, but I'm not sure if there are more pipeline task that add demands because the possibilities that would be needed to be covered by trigger combinations would grow fast.

Would love some feedback about how you feel about adding something like demandsToIgnore.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Ready To Ship

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions