Skip to content

New ceph implementation required permissions unclear without admin user #1711

@ibot3

Description

@ibot3

With the 6.10 update, the ceph integration in our cluster broke (with the fix from today already applied):

When starting a VM, this error occurs:
Failed setting up disk device "root": Failed to query ceph config for auth_service_required: Failed to run: ceph --cluster ceph config get client.incus auth_service_required: exit status 1 (2025-03-01T13:53:07.250+0100 7efc748886c0 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2] [errno 1] RADOS permission error (error connecting to the cluster))

This is a result of this call:

// Check that cephx is enabled.
authType, err := callCeph(
"--cluster", cluster,
"config", "get", client, "auth_service_required",
)
if err != nil {
return "", fmt.Errorf(
"Failed to query ceph config for auth_service_required: %w",
err,
)
}

If I understand it correctly, incus tries to directly obtain the keyring from ceph instead of using the provided keyring in /etc/ceph.
As the call is made without --id or --name, the call is tried with the ceph admin user by default.

However, this does not work, as /etc/ceph only contains the incus ceph user in our deployment.
We don't want incus to act as the admin user, as the ceph cluster is also used for other applications.

While we theoretically could set the user with CEPH_ARGS='-n client.incus' env var, this still would not work, as I am not really aware which permissions the ceph user needs in order to run the required commands.
So this still results in Error EACCES: access denied (User has not the permissions to access the config)

Incus ignoring the given username for the ceph pool seems to be problematic.
And I could not find documentation that states which permissions the ceph user needs.

With 6.9 it worked fine with

  • mon profile rbd
  • osd profile rbd pool=incus

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions