migrations: add a guided cephx service key rotation helper
Ceph Squid 19.2.6 and Ceph Tentacle 20.2.4 add the aes256k cephx cipher and report every key still using the old aes one as insecure. Two of those health checks are errors, so an upgraded cluster sits in HEALTH_ERR until its service keys are migrated. Nothing is broken meanwhile, but the state is noisy and it hides real errors. Rotating those keys by hand is a long sequence per daemon, and getting it wrong strands one: an OSD reads its key from the bluestore label rather than from the keyring file, so writing only the file works until the next reboot rebuilds the data directory from that label. The helper walks the cluster from one node and drives the rest over SSH. It prints a plan and changes nothing unless '--apply' is given, and '--only' narrows a run to one daemon type or a single daemon. The key of a manager, metadata server or OSD is swapped while the daemon keeps running: a pending key is staged, handed to the daemon, and then promoted, so nothing is stopped and clients stay connected. One that does not take it that way, and one an earlier run left stopped, is stopped, rotated and started again instead, with Ceph asked before every stop. '--restart-daemons' takes that path throughout. Running it again resumes. The key every daemon used is recorded on the cluster file system before anything is rotated, together with the progress of the run, so an interrupted run can be continued, a daemon it left stopped is picked up even after Ceph stopped listing it, and a 'noout' flag it left behind is cleared. Client keys are opt-in, because whether one can move depends on what reads it: an in-kernel RBD or CephFS client needs kernel 7.0 or newer. '--rotate-client-keys' covers the bootstrap keys and client.crash, '-- rotate-admin-key' covers client.admin and every copy Proxmox VE keeps of it, and '--rotate-storage-key' covers one storage with its own user. '--rotate-mon-key' additionally rotates the shared 'mon.' key, which restarts the monitors one at a time. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
T
Thomas Lamprecht committed
9cec18a7f7047cdc49393212c8dbb667097b3419
Parent: aaf02d0