SIGN IN SIGN UP

migrations: add a guided cephx service key rotation helper

Ceph Squid 19.2.6 and Ceph Tentacle 20.2.4 add the aes256k cephx
cipher and report every key still using the old aes one as insecure.
Two of those health checks are errors, so an upgraded cluster sits in
HEALTH_ERR until its service keys are migrated. Nothing is broken
meanwhile, but the state is noisy and it hides real errors.

Rotating those keys by hand is a long sequence per daemon, and getting
it wrong strands one: an OSD reads its key from the bluestore label
rather than from the keyring file, so writing only the file works
until the next reboot rebuilds the data directory from that label.

The helper walks the cluster from one node and drives the rest over
SSH. It prints a plan and changes nothing unless '--apply' is given,
and '--only' narrows a run to one daemon type or a single daemon.

The key of a manager, metadata server or OSD is swapped while the
daemon keeps running: a pending key is staged, handed to the daemon,
and then promoted, so nothing is stopped and clients stay connected.
One that does not take it that way, and one an earlier run left
stopped, is stopped, rotated and started again instead, with Ceph
asked before every stop. '--restart-daemons' takes that path
throughout.

Running it again resumes. The key every daemon used is recorded on the
cluster file system before anything is rotated, together with the
progress of the run, so an interrupted run can be continued, a daemon
it left stopped is picked up even after Ceph stopped listing it, and a
'noout' flag it left behind is cleared.

Client keys are opt-in, because whether one can move depends on what
reads it: an in-kernel RBD or CephFS client needs kernel 7.0 or newer.
'--rotate-client-keys' covers the bootstrap keys and client.crash, '--
rotate-admin-key' covers client.admin and every copy Proxmox VE keeps
of it, and '--rotate-storage-key' covers one storage with its own
user. '--rotate-mon-key' additionally rotates the shared 'mon.' key,
which restarts the monitors one at a time.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
T
Thomas Lamprecht committed
9cec18a7f7047cdc49393212c8dbb667097b3419
Parent: aaf02d0