1. Use Ubuntu 20.04.1 (Raspbian is only available as armhf images ; we need arm64).
  2. In /etc/modules, add libceph and rbd
  3. Install K3s
  4. Clear the disks you want to use as Ceph storages
  5. As root:
sudo apt install ceph-common
git clone https://github.com/rook/rook.git
cd rook

Switch to the raspbernetes images instead of the default ones. Currently the default ones are not all built as multi-arch yet and therefore don't all work on arm64.

diff --git a/cluster/examples/kubernetes/ceph/operator.yaml b/cluster/examples/kubernetes/ceph/operator.yaml
index 7935b6bf..14f07ec8 100644
--- a/cluster/examples/kubernetes/ceph/operator.yaml
+++ b/cluster/examples/kubernetes/ceph/operator.yaml
@@ -45,12 +45,12 @@ data:
   # The default version of CSI supported by Rook will be started. To change the version
   # of the CSI driver to something other than what is officially supported, change
   # these images to the desired release of the CSI driver.
-  # ROOK_CSI_CEPH_IMAGE: "quay.io/cephcsi/cephcsi:v3.1.0"
-  # ROOK_CSI_REGISTRAR_IMAGE: "quay.io/k8scsi/csi-node-driver-registrar:v1.2.0"
-  # ROOK_CSI_RESIZER_IMAGE: "quay.io/k8scsi/csi-resizer:v0.4.0"
-  # ROOK_CSI_PROVISIONER_IMAGE: "quay.io/k8scsi/csi-provisioner:v1.6.0"
-  # ROOK_CSI_SNAPSHOTTER_IMAGE: "quay.io/k8scsi/csi-snapshotter:v2.1.1"
-  # ROOK_CSI_ATTACHER_IMAGE: "quay.io/k8scsi/csi-attacher:v2.1.0"
+  ROOK_CSI_CEPH_IMAGE: "raspbernetes/ceph-csi:v3.1.0"
+  ROOK_CSI_REGISTRAR_IMAGE: "raspbernetes/csi-node-driver-registrar:1.3.0"
+  ROOK_CSI_RESIZER_IMAGE: "raspbernetes/csi-external-resizer:0.5.0"
+  ROOK_CSI_PROVISIONER_IMAGE: "raspbernetes/csi-external-provisioner:1.6.0"
+  ROOK_CSI_SNAPSHOTTER_IMAGE: "raspbernetes/csi-external-snapshotter:2.1.1"
+  ROOK_CSI_ATTACHER_IMAGE: "raspbernetes/csi-external-attacher:2.2.0"
 
   # (Optional) set user created priorityclassName for csi plugin pods.
   # CSI_PLUGIN_PRIORITY_CLASSNAME: "system-node-critical"
cd cluster/examples/kubernetes/ceph/
kubectl create -f ./common.yml
kubectl create -f ./operator.yml
kubectl create -f ./cluster.yml
kubectl create -f ./toolbox.yml
kubectl -n rook-ceph get pod  # check the result