Skip to content

Make Ci::Runner#token routable

What does this MR do and why?

This builds upon !168833 (closed) and allow Runner tokens to be routable (i.e. the Cells HTTP Router will be able to decode it and use the information to route a request to a particular cell), for Group runners and Project runners only (i.e. not Instance runners).

The change is behind the routable_runner_token feature flag, which can be enabled for specific users.

See https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/cells/routable_tokens/ for more information on Routable Tokens.

MR acceptance checklist

The change being behind a feature flag, it should be safe to deploy it, and I will perform various checks with my user before enabling it further.

Steps to reproduce locally

  1. Checkout the branch
  2. Open a Rails console:

With the feature flag disabled

# Ensure the feature flag is disabled
> Feature.disable(:routable_runner_token)

# Create a runner
> runner = Ci::Runner.create!(runner_type: :instance_type)

# Check the token shape
> runner.token
=> "t1_ckEgrixABm-RwHj8ToeZ"

# Base-64 decode the payload, it doesn't contain routing information
> Base64.urlsafe_decode64(runner.token.delete_prefix('t1_'))
=> "rA \xAE,@\x06o\x91\xC0x\xFCN\x87\x99"
# Create a runner for Group.first
> runner = Ci::Runner.create!(runner_type: :group_type, groups: [Group.first], sharding_key_id: Group.first.id)

# Check the token shape
> runner.token
=> "t2_QzrmsQpwaooknRSy-AUC"

# Base-64 decode the payload, it doesn't contain routing information
> Base64.urlsafe_decode64(runner.token.delete_prefix('t2_'))
=> "C:\xE6\xB1\npj\x8A$\x9D\x14\xB2\xF8\x05\x02"
# Create a runner for Project.first
> runner = Ci::Runner.create!(runner_type: :project_type, projects: [Project.first], sharding_key_id: Project.first.id)

# Check the token shape
> runner.token
=> "t3_Qn8_ceAytDD4rNFAm9fn"

# Base-64 decode the payload, it doesn't contain routing information
> Base64.urlsafe_decode64(runner.token.delete_prefix('t3_'))
=> "B\x7F?q\xE02\xB40\xF8\xAC\xD1@\x9B\xD7\xE7"

With the feature flag enabled

# Enable the feature flag for the first user
> Feature.enable(:routable_runner_token, User.first)

# Create a runner created by User.first
> runner = Ci::Runner.create!(runner_type: :instance_type, creator: User.first)

# Check the token shape
> runner.token
=> "t1_cjoHKk3N3UMeC8Kt8Fk0jlCVCnU6MQ"

# Base-64 decode the payload, it doesn't contain routing information
> Base64.urlsafe_decode64(runner.token.delete_prefix('t1_'))
=> "\x0Ex\xF1V\xF4!\x99p\f\xC7Q\xC0\xCA\x14\xF1"
# Enable the feature flag for the first group
> Feature.enable(:routable_runner_token, Group.first)

# Create a runner created by User.first
> runner = Ci::Runner.create!(runner_type: :group_type, groups: [Group.first], sharding_key_id: Group.first.id)

# Check the token shape
> runner.token
=> "t2_YzoxCm86McBZvbkDPhEYLTOsgNjxLaQQ.0w0rpsm6v"

# Base-64 decode the payload, it contains routing information
> decoded_payload = Base64.urlsafe_decode64(runner.token.delete_prefix('t2_')[...-10])
=> "c:1\no:1\xC0Y\xBD\xB9\x03>\x11\x18-3\xAC\x80\xD8\xF1-\xA4\x10"

> routing_payload_length = decoded_payload.size - decoded_payload[-1].unpack1('C') - 1
=> 7

> decoded_payload[0, routing_payload_length].lines(chomp: true).map { |l| l.split(':') }.to_h
=> {"c"=>"1", "o"=>"1"}
# Enable the feature flag for the first project
> Feature.enable(:routable_runner_token, Project.first)

# Create a runner created by User.first
> runner = Ci::Runner.create!(runner_type: :project_type, groups: [Project.first], sharding_key_id: Project.first.id)

# Check the token shape
> runner.token
=> "t3_YzoxCm86MfSW16832Qmg3aCkgSuq6SMQ.0w0ez9zrg"

# Base-64 decode the payload, it contains routing information
> decoded_payload = Base64.urlsafe_decode64(runner.token.delete_prefix('t3_')[...-10])
=> "c:1\no:1\xF4\x96\xD7\xAF7\xD9\t\xA0\xDD\xA0\xA4\x81+\xAA\xE9#\x10"

> routing_payload_length = decoded_payload.size - decoded_payload[-1].unpack1('C') - 1
=> 7

> decoded_payload[0, routing_payload_length].lines(chomp: true).map { |l| l.split(':') }.to_h
=> {"c"=>"1", "o"=>"1"}
# Enable the feature flag globally
> Feature.enable(:routable_runner_token)

# Create a runner for Project.first
> runner = Ci::Runner.create!(runner_type: :project_type, creator: User.first, projects: [Project.first], sharding_key_id: Project.first.id)

# Check the token shape
> runner.token
=> "t3_YzoxCm86Ma3mwr3ZipLEPxz9gckewFgQ.0w1qgubo8"

# Base-64 decode the payload, it contains routing information
> decoded_payload = Base64.urlsafe_decode64(runner.token.delete_prefix('t3_')[...-10])
=> "c:1\no:1\xAD\xE6\xC2\xBD\xD9\x8A\x92\xC4?\x1C\xFD\x81\xC9\x1E\xC0X\x10"

> routing_payload_length = decoded_payload.size - decoded_payload[-1].unpack1('C') - 1
=> 7

> decoded_payload[0, routing_payload_length].lines(chomp: true).map { |l| l.split(':') }.to_h
=> {"c"=>"1", "o"=>"1"}

Related to #489847 (closed).

Edited by Rémy Coutable

Merge request reports

Loading