blob: 9871daa04e7895b3b413edfd605fe92d5fcf0445 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
|
# USB Key Mounting for ZFS Encryption
The f-hosts keep raw ZFS encryption keys on per-host UFS USB sticks mounted at
`/keys`. All four sticks are labeled `F3S_KEYS` and hold all 8 key files as
cross-host backups.
Do **not** mount `/keys` from `/etc/fstab`. A missing or corrupt key stick must
not block the FreeBSD base OS from booting.
## Managed Files
Source files live in the conf repo:
```text
f3s/freebsd-hosts/keys/
f3s-mount-keys
f3s-load-zfs-keys
f3skeys.rc
```
Installed paths on each f-host:
```text
/usr/local/sbin/f3s-mount-keys
/usr/local/sbin/f3s-load-zfs-keys
/etc/rc.d/f3skeys
```
`f3skeys` runs before FreeBSD's built-in `zfskeys` service. If the USB stick is
missing or `fsck_ufs -p` fails, the helper logs the problem and exits
successfully so boot continues. Encrypted datasets stay locked until the stick
is repaired and `/usr/local/sbin/f3s-load-zfs-keys` is run manually.
## Setup
Format a new key stick:
```sh
doas newfs -L F3S_KEYS /dev/da0
doas mkdir -p /keys
doas mount -t ufs -o ro /dev/ufs/F3S_KEYS /keys
```
Label an existing stick without rebuilding it:
```sh
doas umount /keys
doas tunefs -L F3S_KEYS /dev/da0
```
Keep the old `/etc/fstab` line commented on all f-hosts:
```fstab
# /dev/da0 /keys ufs rw 0 2
```
Enable boot loading:
```sh
doas sysrc f3skeys_enable=YES
doas sysrc zfskeys_enable=YES
```
Current `zfskeys_datasets` values:
```sh
# f0
doas sysrc zfskeys_datasets="zdata/enc zdata/enc/nfsdata zroot/bhyve zroot/garage"
# f1
doas sysrc zfskeys_datasets="zdata/enc zroot/bhyve zroot/garage zdata/sink/f0/zdata/enc/nfsdata"
# f2
doas sysrc zfskeys_datasets="zdata/enc zroot/bhyve zroot/garage zroot/sink/f3/zroot/bhyve/freebsd"
# f3
doas sysrc zfskeys_datasets="zroot/bhyve"
```
Replicated sinks with raw encryption need explicit file keylocations:
```sh
# f1
doas zfs set keylocation=file:///keys/f0.lan.buetow.org:zdata.key \
zdata/sink/f0/zdata/enc/nfsdata
# f2
doas zfs set keylocation=file:///keys/f3.lan.buetow.org:bhyve.key \
zroot/sink/f3/zroot/bhyve/freebsd
```
Manual recovery after boot:
```sh
doas /usr/local/sbin/f3s-mount-keys --strict
doas /usr/local/sbin/f3s-load-zfs-keys
```
## Verification
```sh
mount | grep ' /keys '
sysrc -n f3skeys_enable
sysrc -n zfskeys_enable
sysrc -n zfskeys_datasets
doas /usr/local/sbin/f3s-load-zfs-keys
zfs list -H -o name,encryption,keylocation,keystatus,mounted |
awk '$2 != "off" { print }'
```
Full reboot validation was run on f0, f1, f2, and f3 on 2026-05-30 after this
change.
Note: `zroot/sink/f3/zroot/bhyve/freebsd` on f2 has `mountpoint=none`; the
reboot check expects its key to be `available`, but it is not mounted because it
has no filesystem mountpoint.
|
