Skip to content

mimirtool: Failed to pick the correct backend when running convert #1622

@slim-bean

Description

@slim-bean

Describe the bug

I ran the convert tool on my existing app and it picked an incorrect backend value post conversions

To Reproduce

My config:

auth_enabled: false

server:
  http_listen_port: 9009

  # Configure the server to allow messages up to 100MB.
  grpc_server_max_recv_msg_size: 104857600
  grpc_server_max_send_msg_size: 104857600
  grpc_server_max_concurrent_streams: 1000

distributor:
  shard_by_all_labels: true
  pool:
    health_check_ingesters: true

ingester_client:
  grpc_client_config:
    # Configure the client to allow messages up to 100MB.
    max_recv_msg_size: 104857600
    max_send_msg_size: 104857600
    grpc_compression: gzip

ingester:
  lifecycler:
    # The address to advertise for this ingester.  Will be autodiscovered by
    # looking up address on eth0 or en0; can be specified if this fails.
    # address: 127.0.0.1

    # We want to start immediately and flush on shutdown.
    join_after: 0
    min_ready_duration: 0s
    final_sleep: 0s
    num_tokens: 512

    # Use an in memory ring store, so we don't need to launch a Consul.
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1

storage:
  engine: blocks

blocks_storage:
  tsdb:
    dir: /tmp/cortex/tsdb

  bucket_store:
    sync_dir: /tmp/cortex/tsdb-sync

  # You can choose between local storage and Amazon S3, Google GCS and Azure storage. Each option requires additional configuration
  # as shown below. All options can be configured via flags as well which might be handy for secret inputs.
  backend: s3 # s3, gcs, azure or filesystem are valid options
  s3:
    bucket_name: bucket_name
    endpoint: s3.us-east-1.wasabisys.com
    # Configure your S3 credentials below.
    secret_access_key: secretkey
    access_key_id:     accessKeyId
#  gcs:
#    bucket_name: cortex
#    service_account: # if empty or omitted Cortex will use your default service account as per Google's fallback logic
#  azure:
#    account_name:
#    account_key:
#    container_name:
#    endpoint_suffix:
#    max_retries: # Number of retries for recoverable errors (defaults to 20)
# filesystem:
#   dir: ./data/tsdb

compactor:
  data_dir: /tmp/cortex/compactor
  sharding_ring:
    kvstore:
      store: inmemory

frontend_worker:
  match_max_concurrent: true

ruler:
  enable_api: true
  enable_sharding: false

ruler_storage:
  backend: local
  local:
    directory: /tmp/cortex/rules

Running this command ./mimirtool config convert --yaml-file=current.yaml --update-defaults --yaml-out=mimir.yaml

Produced

❯ cat mimir.yaml
blocks_storage:
    backend: filesystem
    bucket_store:
        sync_dir: /tmp/cortex/tsdb-sync
    s3:
        access_key_id: accessKeyId
        bucket_name: bucket_name
        endpoint: s3.us-east-1.wasabisys.com
        secret_access_key: secretkey
    tsdb:
        dir: /tmp/cortex/tsdb
compactor:
    data_dir: /tmp/cortex/compactor
    sharding_ring:
        kvstore:
            store: inmemory
distributor:
    pool:
        health_check_ingesters: true
ingester:
    ring:
        final_sleep: 0s
        join_after: 0s
        kvstore:
            store: inmemory
        min_ready_duration: 0s
        num_tokens: 512
        replication_factor: 1
ingester_client:
    grpc_client_config:
        grpc_compression: gzip
        max_recv_msg_size: 104857600
        max_send_msg_size: 104857600
multitenancy_enabled: false
ruler:
    enable_api: true
ruler_storage:
    backend: local
    local:
        directory: /tmp/cortex/rules
server:
    grpc_server_max_concurrent_streams: 1000
    grpc_server_max_recv_msg_size: 104857600
    grpc_server_max_send_msg_size: 104857600
    http_listen_port: 9009

Which shows the blocks_storage: backend: filesystem which should be s3

Environment

❯ ./mimirtool version
Mimirtool, version 2.0.0 (branch: release-2.0, revision: 9fd2da5d3)
  go version:       go1.17.8
  platform:         linux/amd64
checking latest version... A newer version of mimirtool is available, please update to mimir-2.0.0

Additional Context

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions