diff options
Diffstat (limited to 'gemfeed')
18 files changed, 1261 insertions, 557 deletions
diff --git a/gemfeed/.gitignore b/gemfeed/.gitignore deleted file mode 100644 index 1e107f52..00000000 --- a/gemfeed/.gitignore +++ /dev/null @@ -1 +0,0 @@ -examples diff --git a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi index 0d67f9ba..41e5feaa 100644 --- a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi +++ b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi @@ -397,6 +397,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-) Other *BSD related posts are: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi index 1efa22e1..88481536 100644 --- a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi +++ b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi @@ -676,6 +676,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-) Other *BSD related posts are: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi index 7b3759b7..b1057a1a 100644 --- a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi +++ b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi @@ -53,6 +53,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-) Other *BSD related posts are: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi index c79cb573..7eb69d32 100644 --- a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi +++ b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi @@ -300,6 +300,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-) Other *BSD and KISS related posts are: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi index 0d01ac9d..9b9a0ebc 100644 --- a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi +++ b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi @@ -14,6 +14,7 @@ These are all the posts so far: => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -162,6 +163,7 @@ Read the next post of this series: Other *BSD-related posts: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi index 70036e8a..827d4ff8 100644 --- a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi +++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi @@ -1,4 +1,4 @@ - f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation +# f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation > Published at 2024-12-02T23:48:21+02:00 @@ -14,6 +14,7 @@ These are all the posts so far: => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -23,6 +24,7 @@ Let's continue... ## Table of Contents +* ⇢ f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation * ⇢ Deciding on the hardware * ⇢ ⇢ Not ARM but Intel N100 * ⇢ ⇢ Beelink unboxing @@ -302,6 +304,7 @@ Read the next post of this series: Other *BSD-related posts: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl index 03dff1d0..cc9b2903 100644 --- a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl +++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl @@ -1,4 +1,4 @@ - f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation +# f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation > Published at 2024-12-02T23:48:21+02:00 diff --git a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi index 65c1637b..14b9b35e 100644 --- a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi +++ b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi @@ -10,6 +10,7 @@ This is the third blog post about my f3s series for my self-hosting demands in m => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -364,6 +365,7 @@ Read the next post of this series: Other BSD related posts are: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi index e5c3a5ae..9f532d6a 100644 --- a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi +++ b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi @@ -10,6 +10,7 @@ This is the fourth blog post about the f3s series for self-hosting demands in a => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs (You are currently reading this) => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -510,6 +511,7 @@ Read the next post of this series: Other *BSD-related posts: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs (You are currently reading this) diff --git a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi index f22ddc15..ae88f848 100644 --- a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi +++ b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi @@ -14,6 +14,7 @@ These are all the posts so far: => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network (You are currently reading this) => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -931,6 +932,7 @@ Read the next post of this series: Other *BSD-related posts: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network (You are currently reading this) => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi index ba315359..77117a8d 100644 --- a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi +++ b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi @@ -10,6 +10,7 @@ This is the sixth blog post about the f3s series for self-hosting demands in a h => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage (You are currently reading this) +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -719,7 +720,7 @@ Whereas: Next, update `/etc/hosts` on all nodes (`f0`, `f1`, `f2`, `r0`, `r1`, `r2`) to resolve the VIP hostname: ``` -192.168.1.138 f3s-storage-ha f3s-storage-ha.lan f3s-storage-ha.lan.buetow.org +192.168.2.138 f3s-storage-ha f3s-storage-ha.wg0 f3s-storage-ha.wg0.wan.buetow.org ``` This allows clients to connect to `f3s-storage-ha` regardless of which physical server is currently the MASTER. @@ -1400,7 +1401,7 @@ To mount NFS through the stunnel encrypted tunnel, we run: clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1) # For persistent mount, add to /etc/fstab: -127.0.0.1:/data/nfs/k3svolumes /data/nfs/k3svolumes nfs4 port=2323,_netdev 0 0 +127.0.0.1:/k3svolumes /data/nfs/k3svolumes nfs4 port=2323,_netdev,soft,timeo=10,retrans=2,intr 0 0 ``` Note: The mount uses localhost (`127.0.0.1`) because stunnel is listening locally and forwarding the encrypted traffic to the remote server. @@ -1650,10 +1651,13 @@ MooseFS is a fault-tolerant, distributed file system that could provide proper h Both technologies could run on top of our encrypted ZFS volumes, combining ZFS's data integrity and encryption features with distributed storage capabilities. This would be particularly interesting for workloads that need either S3-compatible APIs (MinIO) or transparent distributed POSIX storage (MooseFS). What about Ceph and GlusterFS? Unfortunately, there doesn't seem to be great native FreeBSD support for them. However, other alternatives also appear suitable for my use case. -I'm looking forward to the next post in this series, where we will set up k3s (Kubernetes) on the Linux VMs. +Read the next post of this series: + +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments Other *BSD-related posts: +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage (You are currently reading this) => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs diff --git a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi.tpl b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi.tpl index d62100cd..d0843866 100644 --- a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi.tpl +++ b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi.tpl @@ -1602,7 +1602,9 @@ MooseFS is a fault-tolerant, distributed file system that could provide proper h Both technologies could run on top of our encrypted ZFS volumes, combining ZFS's data integrity and encryption features with distributed storage capabilities. This would be particularly interesting for workloads that need either S3-compatible APIs (MinIO) or transparent distributed POSIX storage (MooseFS). What about Ceph and GlusterFS? Unfortunately, there doesn't seem to be great native FreeBSD support for them. However, other alternatives also appear suitable for my use case. -I'm looking forward to the next post in this series, where we will set up k3s (Kubernetes) on the Linux VMs. +Read the next post of this series: + +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments Other *BSD-related posts: diff --git a/gemfeed/DRAFT-kubernetes-with-freebsd-part-7.gmi b/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi index a9414bcc..c9f8c2b5 100644 --- a/gemfeed/DRAFT-kubernetes-with-freebsd-part-7.gmi +++ b/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi @@ -1,19 +1,22 @@ -# f3s: Kubernetes with FreeBSD - Part 7: First pod deployments +# f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments -This is the seventh blog post about the f3s series for self-hosting demands in a home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution used on FreeBSD-based physical machines. +> Published at 2025-10-02T11:27:19+03:00 + +This is the seventh blog post about the f3s series for my self-hosting demands in a home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution I use on FreeBSD-based physical machines. => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage -=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 Deciding on the hardware +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network => ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage +=> ./2025-10-02-f3s-kubernetes-with-freebsd-part-7.gmi 2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments (You are currently reading this) => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo ## Table of Contents -* ⇢ f3s: Kubernetes with FreeBSD - Part 7: First pod deployments +* ⇢ f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments * ⇢ ⇢ Introduction * ⇢ ⇢ Updating * ⇢ ⇢ Installing k3s @@ -29,26 +32,30 @@ This is the seventh blog post about the f3s series for self-hosting demands in a * ⇢ ⇢ ⇢ Prepare the NFS-backed storage * ⇢ ⇢ ⇢ Install (or upgrade) the chart * ⇢ ⇢ ⇢ Allow nodes and workstations to trust the registry -* ⇢ ⇢ ⇢ Push and pull images +* ⇢ ⇢ ⇢ Pushing and pulling images * ⇢ ⇢ Example: Anki Sync Server from the private registry * ⇢ ⇢ ⇢ Build and push the image -* ⇢ ⇢ ⇢ Create the secret and storage on the cluster +* ⇢ ⇢ ⇢ Create the Anki secret and storage on the cluster * ⇢ ⇢ ⇢ Deploy the chart * ⇢ ⇢ NFSv4 UID mapping for Postgres-backed (and other) apps * ⇢ ⇢ ⇢ Helm charts currently in service ## Introduction +In this blog post, I am finally going to install k3s (the Kubernetes distribution I use) to the whole setup and deploy the first workloads (helm charts, and a private registry) to it. + +=> https://k3s.io + ## Updating -On all three Rocky Linux 9 boxes `r0`, `r1`, and `r2`: +Before proceeding, I bring all systems involved up-to-date. On all three Rocky Linux 9 boxes `r0`, `r1`, and `r2`: ```sh dnf update -y reboot ``` -On the FreeBSD hosts, upgrading from FreeBSD 14.2 to 14.3-RELEASE, running this on all three hosts `f0`, `f1` and `f2`: +On the FreeBSD hosts, I upgraded from FreeBSD 14.2 to 14.3-RELEASE, running this on all three hosts `f0`, `f1` and `f2`: ```sh paul@f0:~ % doas freebsd-update fetch @@ -80,7 +87,7 @@ FreeBSD f0.lan.buetow.org 14.3-RELEASE FreeBSD 14.3-RELEASE ### Generating `K3S_TOKEN` and starting the first k3s node -I generated the k3s token on my Fedora laptop with `pwgen -n 32` and selected one of the results. Then, on all three `r` hosts (replace SECRET_TOKEN with the actual secret before running the following command) run: +I generated the k3s token on my Fedora laptop with `pwgen -n 32` and selected one of the results. Then, on all three `r` hosts, I ran the following (replace SECRET_TOKEN with the actual secret): ```sh [root@r0 ~]# echo -n SECRET_TOKEN > ~/.k3s_token @@ -90,7 +97,7 @@ The following steps are also documented on the k3s website: => https://docs.k3s.io/datastore/ha-embedded -We run this on `r0`: +To bootstrap k3s on the first node, I ran this on `r0`: ```sh [root@r0 ~]# curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s_token) \ @@ -105,7 +112,7 @@ We run this on `r0`: ### Adding the remaining nodes to the cluster -Then we run on the other two nodes `r1` and `r2`: +Then I ran on the other two nodes `r1` and `r2`: ```sh [root@r1 ~]# curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s_token) \ @@ -121,7 +128,7 @@ Then we run on the other two nodes `r1` and `r2`: ``` -Once done, we've got a three-node Kubernetes cluster control plane: +Once done, I had a three-node Kubernetes cluster control plane: ```sh [root@r0 ~]# kubectl get nodes @@ -143,7 +150,7 @@ kube-system svclb-traefik-411cec5b-twrd7 2/2 Running 0 kube-system traefik-c98fdf6fb-lt6fx 1/1 Running 0 4m58s ``` -In order to connect with `kubectl` from my Fedora laptop, I had to copy `/etc/rancher/k3s/k3s.yaml` from `r0` to `~/.kube/config` and then replace the value of the server field with `r0.lan.buetow.org`. kubectl can now manage the cluster. Note that this step has to be repeated when we want to connect to another node of the cluster (e.g. when `r0` is down). +In order to connect with `kubectl` from my Fedora laptop, I had to copy `/etc/rancher/k3s/k3s.yaml` from `r0` to `~/.kube/config` and then replace the value of the server field with `r0.lan.buetow.org`. kubectl can now manage the cluster. Note that this step has to be repeated when I want to connect to another node of the cluster (e.g. when `r0` is down). ## Test deployments @@ -240,7 +247,7 @@ apache-service ClusterIP 10.43.249.165 <none> 80/TCP 4s Now let's create an ingress: -> Note: I've modified the hosts listed in this example after I published this blog post. This is to ensure that there aren't any bots scraping it. +> Note: I've modified the hosts listed in this example after I published this blog post to ensure that there aren't any bots scraping it. ```sh > ~ cat <<END > apache-ingress.yaml @@ -313,10 +320,9 @@ Events: <none> Notes: -* I've modified the ingress hosts after I'd published this blog post. This is to ensure that there aren't any bots scraping it. -* In the ingress we use plain HTTP (web) for the Traefik rule, as all the "production" traffic will be routed through a WireGuard tunnel anyway, as we will see later. +* In the ingress, I use plain HTTP (web) for the Traefik rule, as all the "production" traffic will be routed through a WireGuard tunnel anyway, as I will show later. -So let's test the Apache web server through the ingress rule: +So I tested the Apache web server through the ingress rule: ```sh > ~ curl -H "Host: www.f3s.foo.zone" http://r0.lan.buetow.org:80 @@ -325,7 +331,7 @@ So let's test the Apache web server through the ingress rule: ### Test deployment with persistent volume claim -So let's modify the Apache example to serve the `htdocs` directory from the NFS share we created in the previous blog post. We use the following manifests. Most of them are the same as before, except for the persistent volume claim and the volume mount in the Apache deployment. +Next, I modified the Apache example to serve the `htdocs` directory from the NFS share I created in the previous blog post. I used the following manifests. Most of them are the same as before, except for the persistent volume claim and the volume mount in the Apache deployment. ```sh > ~ cat <<END > apache-deployment.yaml @@ -383,7 +389,7 @@ metadata: traefik.ingress.kubernetes.io/router.entrypoints: web spec: rules: - - host: f3s.buetow.org + - host: f3s.foo.zone http: paths: - path: / @@ -393,7 +399,7 @@ spec: name: apache-service port: number: 80 - - host: standby.f3s.buetow.org + - host: standby.f3s.foo.zone http: paths: - path: / @@ -403,7 +409,7 @@ spec: name: apache-service port: number: 80 - - host: www.f3s.buetow.org + - host: www.f3s.foo.zone http: paths: - path: / @@ -466,7 +472,7 @@ spec: END ``` -Let's apply the manifests: +I applied the manifests: ```sh > ~ kubectl apply -f apache-persistent-volume.yaml @@ -475,7 +481,7 @@ Let's apply the manifests: > ~ kubectl apply -f apache-ingress.yaml ``` -Looking at the deployment, we can see it failed because the directory doesn't exist yet on the NFS share (note that we also increased the replica count to 2 so if one node goes down there's already a replica running on another node for faster failover): +Looking at the deployment, I could see it failed because the directory didn't exist yet on the NFS share (note that I also increased the replica count to 2 so if one node goes down there's already a replica running on another node for faster failover): ```sh > ~ kubectl get pods @@ -494,7 +500,7 @@ Events: /data/nfs/k3svolumes/example-apache is not a directory ``` -That's intentional—we need to create the directory on the NFS share first, so let's do that (e.g. on `r0`): +That's intentional—I needed to create the directory on the NFS share first, so I did that (e.g. on `r0`): ```sh [root@r0 ~]# mkdir /data/nfs/k3svolumes/example-apache-volume-claim/ @@ -518,7 +524,7 @@ The `index.html` file gives us some actual content to serve. After deleting the ```sh > ~ kubectl delete pod apache-deployment-5b96bd6b6b-fv2jx -> ~ curl -H "Host: www.f3s.buetow.org" http://r0.lan.buetow.org:80 +> ~ curl -H "Host: www.f3s.foo.zone" http://r0.lan.buetow.org:80 <!DOCTYPE html> <html> <head> @@ -533,7 +539,7 @@ The `index.html` file gives us some actual content to serve. After deleting the ### Scaling Traefik for faster failover -Traefik ships with a single replica by default, but for faster failover I bumped it to two replicas so each worker node runs one pod. That way, if a node disappears, the service stays up while Kubernetes schedules a replacement. Here's the command I used: +Traefik (used for ingress on k3s) ships with a single replica by default, but for faster failover I bumped it to two replicas so each worker node runs one pod. That way, if a node disappears, the service stays up while Kubernetes schedules a replacement. Here's the command I used: ```sh > ~ kubectl -n kube-system scale deployment traefik --replicas=2 @@ -549,7 +555,7 @@ kube-system traefik-c98fdf6fb-9npg2 1/1 Running 11 (53d ago) 61d ## Make it accessible from the public internet -Next, we should make this accessible through the public internet via the `www.f3s.foo.zone` hosts. As a reminder, refer back to part 1 of this series and review the section titled "OpenBSD/relayd to the rescue for external connectivity": +Next, I made this accessible through the public internet via the `www.f3s.foo.zone` hosts. As a reminder from part 1 of this series, I reviewed the section titled "OpenBSD/relayd to the rescue for external connectivity": => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi f3s: Kubernetes with FreeBSD - Part 1: Setting the stage @@ -570,11 +576,11 @@ Next, we should make this accessible through the public internet via the `www.f3 <html><body><h1>It works!</h1></body></html> ``` -How does that work in `relayd.conf` on OpenBSD? Read on... +This is how it works in `relayd.conf` on OpenBSD: ### OpenBSD relayd configuration -The OpenBSD edge relays keep the Kubernetes-facing addresses for the f3s ingress endpoints in a shared backend table so TLS traffic for every `f3s` hostname lands on the same pool of k3s nodes: +The OpenBSD edge relays keep the Kubernetes-facing addresses for the f3s ingress endpoints in a shared backend table so TLS traffic for every `f3s` hostname lands on the same pool of k3s nodes (pointing to the WireGuard IP addresses of those nodes - remember, they are running locally in my LAN, wheras the OpenBSD edge relays operate in the public internet): ``` table <f3s> { @@ -584,7 +590,7 @@ table <f3s> { } ``` -Inside the `http protocol "https"` block each public hostname gets its Let's Encrypt certificate and is matched to that backend table. Besides the primary trio, every service-specific hostname (`anki`, `bag`, `flux`, `audiobookshelf`, `gpodder`, `radicale`, `vault`, `syncthing`, `uprecords`) and their `www` / `standby` aliases reuse the same pool so new apps can go live just by publishing an ingress rule: +Inside the `http protocol "https"` block each public hostname gets its Let's Encrypt certificate and is matched to that backend table. Besides the primary trio, every service-specific hostname (`anki`, `bag`, `flux`, `audiobookshelf`, `gpodder`, `radicale`, `vault`, `syncthing`, `uprecords`) and their `www` / `standby` aliases reuse the same pool so new apps can go live just by publishing an ingress rule, whereas they will all map to a service running in k3s: ``` http protocol "https" { @@ -676,9 +682,9 @@ As not all Docker images I want to deploy are available on public Docker registr All manifests for the f3s stack live in my configuration repository: -=> https://codeberg.org/snonux/conf/f3s snonux/conf/f3s +=> https://codeberg.org/snonux/conf/src/branch/master/f3s codeberg.org/snonux/conf/f3s -Within that repo, the `examples/conf/f3s/registry/` directory contains the Helm chart, a `Justfile`, and a detailed README. Here's the condensed walkthrough I used to roll out the registry with Helm. +Within that repo, the `examples/conf/f3s/registry/` directory contains the Helm chart, a `Justfile`, and a detailed `README`. Here's the condensed walkthrough I used to roll out the registry with Helm. ### Prepare the NFS-backed storage @@ -698,7 +704,7 @@ $ cd conf/f3s/examples/conf/f3s/registry $ helm upgrade --install registry ./helm-chart --namespace infra --create-namespace ``` -Helm creates the `infra` namespace if it does not exist, provisions a `PersistentVolume`/`PersistentVolumeClaim` pair that points at `/data/nfs/k3svolumes/registry`, and spins up a single `registry:2` pod exposed via the `docker-registry-service` NodePort (`30001`). Verify everything is up before continuing: +Helm creates the `infra` namespace if it does not exist, provisions a `PersistentVolume`/`PersistentVolumeClaim` pair that points at `/data/nfs/k3svolumes/registry`, and spins up a single registry pod exposed via the `docker-registry-service` NodePort (`30001`). Verify everything is up before continuing: ```sh $ kubectl get pods --namespace infra @@ -716,6 +722,7 @@ The registry listens on plain HTTP, so both Docker daemons on workstations and t * I don't store any secrets in the images * I access the registry this way only via my LAN +* I may will change it later on... On my Fedora workstation where I build images: @@ -750,9 +757,9 @@ systemctl restart k3s" > done ``` -Thanks to the relayd configuration earlier in the post, the external hostnames (`f3s.foo.zone`, etc.) can already reach NodePort `30001`, so publishing the registry later to the outside world is just a matter of wiring the DNS the same way as the ingress hosts. But by default, that's not enabled for now. +Thanks to the relayd configuration earlier in the post, the external hostnames (`f3s.foo.zone`, etc.) can already reach NodePort `30001`, so publishing the registry later to the outside world is just a matter of wiring the DNS the same way as the ingress hosts. But by default, that's not enabled for now due to security reasons. -### Push and pull images +### Pushing and pulling images Tag any locally built image with one of the node IPs on port `30001`, then push it. I usually target whichever node is closest to me, but any of the three will do: @@ -775,7 +782,7 @@ $ kubectl run registry-test \ > --restart=Never -n test --command -- sleep 300 ``` -If the pod pulls successfully, the private registry is ready for use by the rest of the workloads. +If the pod pulls successfully, the private registry is ready for use by the rest of the workloads. Note, that the commands above actually don't work, they are only for illustration purpose mentioned here. ## Example: Anki Sync Server from the private registry @@ -795,7 +802,7 @@ $ docker push r0.lan.buetow.org:30001/anki-sync-server:25.07.5b Because every k3s node treats `registry.lan.buetow.org:30001` as an insecure mirror (see above), the push succeeds regardless of which node answers. If you prefer the shortcut, `just f3s` in that directory performs the same build/tag/push sequence. -### Create the secret and storage on the cluster +### Create the Anki secret and storage on the cluster The Helm chart expects the `services` namespace, a pre-created NFS directory, and a Kubernetes secret that holds the credentials the upstream container understands: @@ -807,8 +814,6 @@ $ kubectl create secret generic anki-sync-server-secret \ -n services ``` -You may reuse the same credentials you had on the old VM—`SYNC_USER1` follows the `username:password` format, and additional user pairs can be added later via `kubectl edit`. - If the `services` namespace already exists, you can skip that line or let Kubernetes tell you the namespace is unchanged. ### Deploy the chart @@ -830,12 +835,12 @@ containers: mountPath: /anki_data ``` -Once the release comes up, verify that the pod pulled the freshly pushed image and that the ingress we configured earlier resolves through relayd just like the Apache example. The default chart routes `anki.f3s.buetow.org`—adjust the hostnames if you prefer the `foo.zone` variants we used earlier: +Once the release comes up, verify that the pod pulled the freshly pushed image and that the ingress we configured earlier resolves through relayd just like the Apache example. ```sh $ kubectl get pods -n services $ kubectl get ingress anki-sync-server-ingress -n services -$ curl https://anki.f3s.buetow.org/health +$ curl https://anki.f3s.foo.zone/health ``` All of this runs solely on first-party images that now live in the private registry, proving the full flow from local bild to WireGuard-exposed service. @@ -874,12 +879,13 @@ paul@f0:~ % doas pw useradd postgres -u 999 -g postgres \ |