-
-
Notifications
You must be signed in to change notification settings - Fork 343
Description
With the 6.10 update, the ceph integration in our cluster broke (with the fix from today already applied):
When starting a VM, this error occurs:
Failed setting up disk device "root": Failed to query ceph config for auth_service_required: Failed to run: ceph --cluster ceph config get client.incus auth_service_required: exit status 1 (2025-03-01T13:53:07.250+0100 7efc748886c0 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2] [errno 1] RADOS permission error (error connecting to the cluster))
This is a result of this call:
incus/internal/server/storage/drivers/utils_ceph.go
Lines 190 to 200 in 4905664
// Check that cephx is enabled. | |
authType, err := callCeph( | |
"--cluster", cluster, | |
"config", "get", client, "auth_service_required", | |
) | |
if err != nil { | |
return "", fmt.Errorf( | |
"Failed to query ceph config for auth_service_required: %w", | |
err, | |
) | |
} |
If I understand it correctly, incus tries to directly obtain the keyring from ceph instead of using the provided keyring in /etc/ceph
.
As the call is made without --id
or --name
, the call is tried with the ceph admin user by default.
However, this does not work, as /etc/ceph
only contains the incus ceph user in our deployment.
We don't want incus to act as the admin user, as the ceph cluster is also used for other applications.
While we theoretically could set the user with CEPH_ARGS='-n client.incus'
env var, this still would not work, as I am not really aware which permissions the ceph user needs in order to run the required commands.
So this still results in Error EACCES: access denied
(User has not the permissions to access the config)
Incus ignoring the given username for the ceph pool seems to be problematic.
And I could not find documentation that states which permissions the ceph user needs.
With 6.9 it worked fine with
mon profile rbd
osd profile rbd pool=incus