diff options
| author | Paul Buetow <paul@buetow.org> | 2024-12-02 23:46:50 +0200 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2024-12-02 23:46:50 +0200 |
| commit | 3bd48af1e730cca5f8e2e941b93a40f46c6af082 (patch) | |
| tree | 19e6195ff11a71f01136030c7f90c094dd26da40 /gemfeed | |
| parent | b2681a2763405d140dcff3ba62b8c8dbf6a1a6cc (diff) | |
Update content for gemtext
Diffstat (limited to 'gemfeed')
16 files changed, 932 insertions, 341 deletions
diff --git a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi index 78dc2553..13ce5c0e 100644 --- a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi +++ b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi @@ -403,5 +403,6 @@ Other *BSD related posts are: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ../ Back to the main site diff --git a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi index ac003c81..c1598066 100644 --- a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi +++ b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi @@ -682,5 +682,6 @@ Other *BSD related posts are: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ../ Back to the main site diff --git a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi index abb05ae7..3125f3d3 100644 --- a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi +++ b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi @@ -57,5 +57,6 @@ Other *BSD related posts are: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD (You are currently reading this) => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ../ Back to the main site diff --git a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi index 60b18948..5f2fdb7b 100644 --- a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi +++ b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi @@ -308,5 +308,6 @@ Other *BSD and KISS related posts are: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD (You are currently reading this) => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ../ Back to the main site diff --git a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi index a0723df0..d727985b 100644 --- a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi +++ b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi @@ -6,7 +6,10 @@ This is the first blog post about my f3s series for my self-hosting demands in m I will post a new entry every month or so (there are too many other side projects for more frequent updates—I bet you can understand). +These are all the posts so far: + => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage (You are currently reading this) +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -149,6 +152,10 @@ For now, though, I'm focused on completing the migration from AWS ECS and gettin What's your take on self-hosting? Are you planning to move away from managed cloud services? Stay tuned for the second part of this series, where I will likely write about the hardware and the OS setups. +Read the next post of this series: + +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation + Other *BSD-related posts: => ./2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi 2016-04-09 Jails and ZFS with Puppet on FreeBSD @@ -157,6 +164,7 @@ Other *BSD-related posts: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage (You are currently reading this) +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation E-Mail your comments to `paul@nospam.buetow.org` :-) diff --git a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi.tpl b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi.tpl index cd8e05b1..31459951 100644 --- a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi.tpl +++ b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi.tpl @@ -6,7 +6,9 @@ This is the first blog post about my f3s series for my self-hosting demands in m I will post a new entry every month or so (there are too many other side projects for more frequent updates—I bet you can understand). -<< template::inline::index f3s-kubernetes-with-freebsd-part +These are all the posts so far: + +<< template::inline::index f3s => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -134,6 +136,10 @@ For now, though, I'm focused on completing the migration from AWS ECS and gettin What's your take on self-hosting? Are you planning to move away from managed cloud services? Stay tuned for the second part of this series, where I will likely write about the hardware and the OS setups. +Read the next post of this series: + +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation + Other *BSD-related posts: << template::inline::index bsd diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi new file mode 100644 index 00000000..dcce9b0a --- /dev/null +++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi @@ -0,0 +1,289 @@ +# f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation + +> Published at 2024-12-02T23:46:16+02:00 + +This is the second blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution I will use on FreeBSD-based physical machines. + +We set the stage last time; this time, we will set up the hardware for this project. + +These are all the posts so far: + +=> ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation (You are currently reading this) + +=> ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo + +> Logo was generated by ChatGPT. + +Let's continue... + +## Table of Contents + +* ⇢ f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation +* ⇢ ⇢ Deciding on the hardware +* ⇢ ⇢ ⇢ Not ARM but Intel N100 +* ⇢ ⇢ ⇢ Beelink unboxing +* ⇢ ⇢ ⇢ Network switch +* ⇢ ⇢ Installing FreeBSD +* ⇢ ⇢ ⇢ Base install +* ⇢ ⇢ ⇢ Latest patch level and customizing `/etc/hosts` +* ⇢ ⇢ ⇢ Additional packages after install +* ⇢ ⇢ Hardware check +* ⇢ ⇢ ⇢ Ethernet +* ⇢ ⇢ ⇢ RAM +* ⇢ ⇢ ⇢ CPUs +* ⇢ ⇢ ⇢ CPU throttling +* ⇢ ⇢ Conclusion + +## Deciding on the hardware + +Note that the OpenBSD VMs included in the F3S setup (which will be used later in this blog series for internet ingress) are already there. These are virtual machines that I rent at OpenBSD Amsterdam and Hetzner. + +=> https://openbsd.amsterdam +=> https://hetzner.cloud + +This means that the FreeBSD boxes need to be covered, which will later be running k3s in Linux VMs via bhyve hypervisor. + +I've been considering whether to use Raspberry Pis or look for alternatives. It turns out that complete N100-based mini-computers aren't much more expensive than Raspberry Pi 5s, and they don't require assembly. Furthermore, I like that they are AMD64 and not ARM-based, which increases compatibility with some applications (e.g., I might want to virtualize Windows (via bhyve) on one of those, though that's out of scope for this blog series). + +### Not ARM but Intel N100 + +I needed something compact, efficient, and capable enough to handle the demands of a small-scale Kubernetes cluster and preferably something I don't have to assemble a lot. After researching, I decided on the Beelink S12 Pro with Intel N100 CPUs. + +=> https://www.bee-link.com/products/beelink-mini-s12-pro-n100 Beelink Min S12 Pro N100 official page + +The Intel N100 CPUs are built on the "Alder Lake-N" architecture. These chips are designed to balance performance and energy efficiency well. With four cores, they're more than capable of running multiple containers, even with moderate workloads. Plus, they consume only 6W of power, keeping the electricity bill low and the setup quiet - perfect for 24/7 operation. + +=> ./f3s-kubernetes-with-freebsd-part-2/f3s-collage1.jpg Beelink preparation + +The Beelink comes with the following specs: + +* 12th Gen Intel N100 processor, with four cores and four threads, and a maximum frequency of up to 3.4 GHz. +* 16 GB of DDR4 RAM, with a maximum (official) size of 16 GB (but people could install 32 GB on it). +* 500 GB M.2 SSD, with the option to install a 2nd 2.5 SSD drive (which I want to use later in this blog series). +* GBit ethernet +* Four USB 3.2 Gen2 ports (maybe I want to mount something externally at some point) +* Dimensions and weight: 115*102*39mm, 280g +* Silent cooling system. +* HDMI output (needed only for the initial installation) +* Auto power on via WoL (may make use of it) +* Wi-Fi (not going to use it) + +I bought three (3) of them for the cluster I intend to build. + +### Beelink unboxing + +Unboxing was uneventful. Every Beelink PC came with: + +* An AC power adapter +* An HDMI cable +* A VESA mount with screws (not using it as of now) +* Some manuals +* The pre-assembled Beelink PC itself. +* A "Hello" post card (??) + +Overall, I love the small form factor. + +### Network switch + +I went with the TP-Link mini 5-port switch, as I had a spare one available. That switch will be plugged into my wall Ethernet port, which connects directly to my fiber internet router with 100 Mbit/s down and 50 Mbit/s upload speed. + +=> ./f3s-kubernetes-with-freebsd-part-2/switch.jpg Switch + +## Installing FreeBSD + +### Base install + +First, I downloaded the boot-only ISO of the latest FreeBSD release and dumped it on a USB stick on my Fedora laptop: + +```sh +[paul@earth]~/Downloads% sudo dd \ + if=FreeBSD-14.1-RELEASE-amd64-bootonly.iso \ + of=/dev/sda conv=sync +``` + +Next, I plugged the Beelinks (one after another) into my monitor via HDMI (the resolution of the FreeBSD text console seems strangely stretched, as I am using the LG Dual Up monitor), connected Ethernet, an external USB keyboard, and the FreeBSD USB stick, and booted the devices up. With F7, I entered the boot menu and selected the USB stick for the FreeBSD installation. + +The installation was uneventful. I selected: + +* Guided ZFS on root (pool `zroot`) +* Unencrypted ZFS (I will encrypt separate datasets later; I want it to be able to boot without human interaction) +* Static IP configuration (to ensure that the boxes always have the same IPs, even after switching the router/DHCP server) +* I decided to enable the SSH daemon, NTP server, and NTP time synchronization at boot, and I also enabled powerd for automatic CPU frequency scaling. +* In addition to `root,` I added a personal user, `paul,` whom I placed in the `wheel` group. + +After doing all that three times (once for each Beelink PC), I had three ready-to-use FreeBSD boxes! Their hostnames are `f0`, `f1` and `f2`! + +=> ./f3s-kubernetes-with-freebsd-part-2/f3s-collage2.jpg Beelink installation + +### Latest patch level and customizing `/etc/hosts` + +After the first boot, I upgraded to the latest FreeBSD patch level as follows: + +```sh +root@f0:~ # freebsd-update fetch +root@f0:~ # freebsd-update install +root@f0:~ # freebsd-update reboot +``` + +I also added the following entries for the three FreeBSD boxes to the `/etc/hosts` file: +```sh +root@f0:~ # cat <<END >>/etc/hosts +192.168.1.130 f0 f0.lan f0.lan.buetow.org +192.168.1.131 f1 f1.lan f1.lan.buetow.org +192.168.1.132 f2 f2.lan f2.lan.buetow.org +END +``` + +### Additional packages after install + +After that, I installed the following additional packages: + +```sh +root@f0:~ # pkg install helix doas zfs-periodic uptimed +``` + +Helix? It's my favourite text editor. I have nothing against `vi` but like `hx` (Helix) more! + +`doas`? It's a pretty neat (and KISS) replacement for `sudo`. It has far fewer features than `sudo`, which is supposed to make it more secure. Its origin is the OpenBSD project. For `doas`, I accepted the default configuration (where users in the `wheel` group are allowed to run commands as `root`): + +```sh +root@f0:~ # cp /usr/local/etc/doas.conf.sample /usr/local/etc/doas.conf +``` + +`zfs-periodic` is a nifty tool for automatically creating ZFS snapshots. I decided to go with the following configuration here: + +```sh +root@f0:~ # sysrc daily_zfs_snapshot_enable=YES +daily_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc daily_zfs_snapshot_pools=zroot +daily_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc daily_zfs_snapshot_keep=7 +daily_zfs_snapshot_keep: -> 7 +root@f0:~ # sysrc weekly_zfs_snapshot_enable=YES +weekly_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc weekly_zfs_snapshot_pools=zroot +weekly_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc weekly_zfs_snapshot_keep=5 +weekly_zfs_snapshot_keep: -> 5 +root@f0:~ # sysrc monthly_zfs_snapshot_enable=YES +monthly_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc monthly_zfs_snapshot_pools=zroot +monthly_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc weekly_zfs_snapshot_keep=2 +weekly_zfs_snapshot_keep: 5 -> 2 +``` + +`uptimed`? I like to track my uptimes. This is how I configured the daemon: + +```sh +root@f0:~ # cp /usr/local/mimecast/etc/uptimed.conf-dist \ + /usr/local/mimecast/etc/uptimed.conf +root@f0:~ # hx /usr/local/mimecast/etc/uptimed.conf +``` + +In the Helix editor session, I changed `LOG_MAXIMUM_ENTRIES` to `0` to keep all uptime entries forever and not cut off at 50 (the default config). After that, I enabled and started `uptimed`: + +```sh +root@f0:~ # service uptimed enable +root@f0:~ # service uptimed start +``` + +To check the current uptime stats, I can now run `uprecords`: + +```sh + root@f0:~ # uprecords + # Uptime | System Boot up +----------------------------+--------------------------------------------------- +-> 1 0 days, 00:07:34 | FreeBSD 14.1-RELEASE Mon Dec 2 12:21:44 2024 +----------------------------+--------------------------------------------------- +NewRec 0 days, 00:07:33 | since Mon Dec 2 12:21:44 2024 + up 0 days, 00:07:34 | since Mon Dec 2 12:21:44 2024 + down 0 days, 00:00:00 | since Mon Dec 2 12:21:44 2024 + %up 100.000 | since Mon Dec 2 12:21:44 2024 +``` + +## Hardware check + +### Ethernet + +Works. Nothing eventful, really. It's a cheap Realtek chip, but it will do what it is supposed to do. + +```sh +paul@f0:~ % ifconfig re0 +re0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 + options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> + ether e8:ff:1e:d7:1c:ac + inet 192.168.1.130 netmask 0xffffff00 broadcast 192.168.1.255 + inet6 fe80::eaff:1eff:fed7:1cac%re0 prefixlen 64 scopeid 0x1 + inet6 fd22:c702:acb7:0:eaff:1eff:fed7:1cac prefixlen 64 detached autoconf + inet6 2a01:5a8:304:1d5c:eaff:1eff:fed7:1cac prefixlen 64 autoconf pltime 10800 vltime 14400 + media: Ethernet autoselect (1000baseT <full-duplex>) + status: active + nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> +``` + +### RAM + +All there: + +```sh +paul@f1:~ % sysctl hw.physmem +hw.physmem: 16902905856 + +``` + +### CPUs + +Work: + +```sh +paul@f0:~ % sysctl dev.cpu | grep freq: +dev.cpu.3.freq: 705 +dev.cpu.2.freq: 705 +dev.cpu.1.freq: 604 +dev.cpu.0.freq: 604 +``` + +### CPU throttling + +With `powerd` running, CPU freq is dowthrottled when the box isn't jam-packed. To stress it a bit, I run `ubench` to see the frequencies being unthrottled again: + +```sh +paul@f0:~ % doas pkg install ubench +paul@f0:~ % rehash # For tcsh to find the newly installed command +paul@f0:~ % ubench & +paul@f0:~ % sysctl dev.cpu | grep freq: +dev.cpu.3.freq: 2922 +dev.cpu.2.freq: 2922 +dev.cpu.1.freq: 2923 +dev.cpu.0.freq: 2922 +``` + +Idle, all three Beelinks plus the switch consumed 26.2W. But with `ubench` stressing all the CPUs, it went up to 38.8W. + +=> ./f3s-kubernetes-with-freebsd-part-2/watt.jpg Idle consumption. + +## Conclusion + +The Beelink S12 Pro with Intel N100 CPUs checks all the boxes for a k3s project: compact, efficient, expandable, and affordable. Its compatibility with both Linux and FreeBSD makes it versatile for other use cases, whether as part of your cluster or as a standalone system. If you’re looking for hardware that punches above its weight for Kubernetes, this little device deserves a spot on your shortlist. + +=> ./f3s-kubernetes-with-freebsd-part-2/3beelinks.jpg Beelinks stacked + +To ease cable management, I need to get shorter Ethernet cables. I will place the tower on my shelf, where most of the cables will be hidden (together with a UPS, which will also be added to the setup). + +What will be covered in the next post of this series? The bhyve/Rocky Linux and WireGuard setup as described in part 1 of this series. + +Other *BSD-related posts: + +=> ./2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi 2016-04-09 Jails and ZFS with Puppet on FreeBSD +=> ./2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi 2022-07-30 Let's Encrypt with OpenBSD and Rex +=> ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD +=> ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD +=> ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD +=> ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation (You are currently reading this) + +E-Mail your comments to `paul@nospam.buetow.org` :-) + +=> ../ Back to the main site diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl new file mode 100644 index 00000000..5ce123a5 --- /dev/null +++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi.tpl @@ -0,0 +1,264 @@ +# f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation + +This is the second blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution I will use on FreeBSD-based physical machines. + +We set the stage last time; this time, we will set up the hardware for this project. + +These are all the posts so far: + +<< template::inline::index f3s + +=> ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo + +> Logo was generated by ChatGPT. + +Let's continue... + +<< template::inline::toc + +## Deciding on the hardware + +Note that the OpenBSD VMs included in the F3S setup (which will be used later in this blog series for internet ingress) are already there. These are virtual machines that I rent at OpenBSD Amsterdam and Hetzner. + +=> https://openbsd.amsterdam +=> https://hetzner.cloud + +This means that the FreeBSD boxes need to be covered, which will later be running k3s in Linux VMs via bhyve hypervisor. + +I've been considering whether to use Raspberry Pis or look for alternatives. It turns out that complete N100-based mini-computers aren't much more expensive than Raspberry Pi 5s, and they don't require assembly. Furthermore, I like that they are AMD64 and not ARM-based, which increases compatibility with some applications (e.g., I might want to virtualize Windows (via bhyve) on one of those, though that's out of scope for this blog series). + +### Not ARM but Intel N100 + +I needed something compact, efficient, and capable enough to handle the demands of a small-scale Kubernetes cluster and preferably something I don't have to assemble a lot. After researching, I decided on the Beelink S12 Pro with Intel N100 CPUs. + +=> https://www.bee-link.com/products/beelink-mini-s12-pro-n100 Beelink Min S12 Pro N100 official page + +The Intel N100 CPUs are built on the "Alder Lake-N" architecture. These chips are designed to balance performance and energy efficiency well. With four cores, they're more than capable of running multiple containers, even with moderate workloads. Plus, they consume only 6W of power, keeping the electricity bill low and the setup quiet - perfect for 24/7 operation. + +=> ./f3s-kubernetes-with-freebsd-part-2/f3s-collage1.jpg Beelink preparation + +The Beelink comes with the following specs: + +* 12th Gen Intel N100 processor, with four cores and four threads, and a maximum frequency of up to 3.4 GHz. +* 16 GB of DDR4 RAM, with a maximum (official) size of 16 GB (but people could install 32 GB on it). +* 500 GB M.2 SSD, with the option to install a 2nd 2.5 SSD drive (which I want to use later in this blog series). +* GBit ethernet +* Four USB 3.2 Gen2 ports (maybe I want to mount something externally at some point) +* Dimensions and weight: 115*102*39mm, 280g +* Silent cooling system. +* HDMI output (needed only for the initial installation) +* Auto power on via WoL (may make use of it) +* Wi-Fi (not going to use it) + +I bought three (3) of them for the cluster I intend to build. + +### Beelink unboxing + +Unboxing was uneventful. Every Beelink PC came with: + +* An AC power adapter +* An HDMI cable +* A VESA mount with screws (not using it as of now) +* Some manuals +* The pre-assembled Beelink PC itself. +* A "Hello" post card (??) + +Overall, I love the small form factor. + +### Network switch + +I went with the TP-Link mini 5-port switch, as I had a spare one available. That switch will be plugged into my wall Ethernet port, which connects directly to my fiber internet router with 100 Mbit/s down and 50 Mbit/s upload speed. + +=> ./f3s-kubernetes-with-freebsd-part-2/switch.jpg Switch + +## Installing FreeBSD + +### Base install + +First, I downloaded the boot-only ISO of the latest FreeBSD release and dumped it on a USB stick on my Fedora laptop: + +```sh +[paul@earth]~/Downloads% sudo dd \ + if=FreeBSD-14.1-RELEASE-amd64-bootonly.iso \ + of=/dev/sda conv=sync +``` + +Next, I plugged the Beelinks (one after another) into my monitor via HDMI (the resolution of the FreeBSD text console seems strangely stretched, as I am using the LG Dual Up monitor), connected Ethernet, an external USB keyboard, and the FreeBSD USB stick, and booted the devices up. With F7, I entered the boot menu and selected the USB stick for the FreeBSD installation. + +The installation was uneventful. I selected: + +* Guided ZFS on root (pool `zroot`) +* Unencrypted ZFS (I will encrypt separate datasets later; I want it to be able to boot without human interaction) +* Static IP configuration (to ensure that the boxes always have the same IPs, even after switching the router/DHCP server) +* I decided to enable the SSH daemon, NTP server, and NTP time synchronization at boot, and I also enabled powerd for automatic CPU frequency scaling. +* In addition to `root,` I added a personal user, `paul,` whom I placed in the `wheel` group. + +After doing all that three times (once for each Beelink PC), I had three ready-to-use FreeBSD boxes! Their hostnames are `f0`, `f1` and `f2`! + +=> ./f3s-kubernetes-with-freebsd-part-2/f3s-collage2.jpg Beelink installation + +### Latest patch level and customizing `/etc/hosts` + +After the first boot, I upgraded to the latest FreeBSD patch level as follows: + +```sh +root@f0:~ # freebsd-update fetch +root@f0:~ # freebsd-update install +root@f0:~ # freebsd-update reboot +``` + +I also added the following entries for the three FreeBSD boxes to the `/etc/hosts` file: +```sh +root@f0:~ # cat <<END >>/etc/hosts +192.168.1.130 f0 f0.lan f0.lan.buetow.org +192.168.1.131 f1 f1.lan f1.lan.buetow.org +192.168.1.132 f2 f2.lan f2.lan.buetow.org +END +``` + +### Additional packages after install + +After that, I installed the following additional packages: + +```sh +root@f0:~ # pkg install helix doas zfs-periodic uptimed +``` + +Helix? It's my favourite text editor. I have nothing against `vi` but like `hx` (Helix) more! + +`doas`? It's a pretty neat (and KISS) replacement for `sudo`. It has far fewer features than `sudo`, which is supposed to make it more secure. Its origin is the OpenBSD project. For `doas`, I accepted the default configuration (where users in the `wheel` group are allowed to run commands as `root`): + +```sh +root@f0:~ # cp /usr/local/etc/doas.conf.sample /usr/local/etc/doas.conf +``` + +`zfs-periodic` is a nifty tool for automatically creating ZFS snapshots. I decided to go with the following configuration here: + +```sh +root@f0:~ # sysrc daily_zfs_snapshot_enable=YES +daily_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc daily_zfs_snapshot_pools=zroot +daily_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc daily_zfs_snapshot_keep=7 +daily_zfs_snapshot_keep: -> 7 +root@f0:~ # sysrc weekly_zfs_snapshot_enable=YES +weekly_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc weekly_zfs_snapshot_pools=zroot +weekly_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc weekly_zfs_snapshot_keep=5 +weekly_zfs_snapshot_keep: -> 5 +root@f0:~ # sysrc monthly_zfs_snapshot_enable=YES +monthly_zfs_snapshot_enable: -> YES +root@f0:~ # sysrc monthly_zfs_snapshot_pools=zroot +monthly_zfs_snapshot_pools: -> zroot +root@f0:~ # sysrc weekly_zfs_snapshot_keep=2 +weekly_zfs_snapshot_keep: 5 -> 2 +``` + +`uptimed`? I like to track my uptimes. This is how I configured the daemon: + +```sh +root@f0:~ # cp /usr/local/mimecast/etc/uptimed.conf-dist \ + /usr/local/mimecast/etc/uptimed.conf +root@f0:~ # hx /usr/local/mimecast/etc/uptimed.conf +``` + +In the Helix editor session, I changed `LOG_MAXIMUM_ENTRIES` to `0` to keep all uptime entries forever and not cut off at 50 (the default config). After that, I enabled and started `uptimed`: + +```sh +root@f0:~ # service uptimed enable +root@f0:~ # service uptimed start +``` + +To check the current uptime stats, I can now run `uprecords`: + +```sh + root@f0:~ # uprecords + # Uptime | System Boot up +----------------------------+--------------------------------------------------- +-> 1 0 days, 00:07:34 | FreeBSD 14.1-RELEASE Mon Dec 2 12:21:44 2024 +----------------------------+--------------------------------------------------- +NewRec 0 days, 00:07:33 | since Mon Dec 2 12:21:44 2024 + up 0 days, 00:07:34 | since Mon Dec 2 12:21:44 2024 + down 0 days, 00:00:00 | since Mon Dec 2 12:21:44 2024 + %up 100.000 | since Mon Dec 2 12:21:44 2024 +``` + +## Hardware check + +### Ethernet + +Works. Nothing eventful, really. It's a cheap Realtek chip, but it will do what it is supposed to do. + +```sh +paul@f0:~ % ifconfig re0 +re0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 + options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> + ether e8:ff:1e:d7:1c:ac + inet 192.168.1.130 netmask 0xffffff00 broadcast 192.168.1.255 + inet6 fe80::eaff:1eff:fed7:1cac%re0 prefixlen 64 scopeid 0x1 + inet6 fd22:c702:acb7:0:eaff:1eff:fed7:1cac prefixlen 64 detached autoconf + inet6 2a01:5a8:304:1d5c:eaff:1eff:fed7:1cac prefixlen 64 autoconf pltime 10800 vltime 14400 + media: Ethernet autoselect (1000baseT <full-duplex>) + status: active + nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> +``` + +### RAM + +All there: + +```sh +paul@f1:~ % sysctl hw.physmem +hw.physmem: 16902905856 + +``` + +### CPUs + +Work: + +```sh +paul@f0:~ % sysctl dev.cpu | grep freq: +dev.cpu.3.freq: 705 +dev.cpu.2.freq: 705 +dev.cpu.1.freq: 604 +dev.cpu.0.freq: 604 +``` + +### CPU throttling + +With `powerd` running, CPU freq is dowthrottled when the box isn't jam-packed. To stress it a bit, I run `ubench` to see the frequencies being unthrottled again: + +```sh +paul@f0:~ % doas pkg install ubench +paul@f0:~ % rehash # For tcsh to find the newly installed command +paul@f0:~ % ubench & +paul@f0:~ % sysctl dev.cpu | grep freq: +dev.cpu.3.freq: 2922 +dev.cpu.2.freq: 2922 +dev.cpu.1.freq: 2923 +dev.cpu.0.freq: 2922 +``` + +Idle, all three Beelinks plus the switch consumed 26.2W. But with `ubench` stressing all the CPUs, it went up to 38.8W. + +=> ./f3s-kubernetes-with-freebsd-part-2/watt.jpg Idle consumption. + +## Conclusion + +The Beelink S12 Pro with Intel N100 CPUs checks all the boxes for a k3s project: compact, efficient, expandable, and affordable. Its compatibility with both Linux and FreeBSD makes it versatile for other use cases, whether as part of your cluster or as a standalone system. If you’re looking for hardware that punches above its weight for Kubernetes, this little device deserves a spot on your shortlist. + +=> ./f3s-kubernetes-with-freebsd-part-2/3beelinks.jpg Beelinks stacked + +To ease cable management, I need to get shorter Ethernet cables. I will place the tower on my shelf, where most of the cables will be hidden (together with a UPS, which will also be added to the setup). + +What will be covered in the next post of this series? The bhyve/Rocky Linux and WireGuard setup as described in part 1 of this series. + +Other *BSD-related posts: + +<< template::inline::index bsd + +E-Mail your comments to `paul@nospam.buetow.org` :-) + +=> ../ Back to the main site diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-hardware-setup.gmi b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-hardware-setup.gmi index a161a375..f96cd65f 100644 --- a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-hardware-setup.gmi +++ b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-hardware-setup.gmi @@ -7,6 +7,7 @@ This is the second blog post about my f3s series for my self-hosting demands in I will post a new entry every month or so (there are too many other side projects for more frequent updates—I bet you can understand). => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo @@ -33,6 +34,7 @@ Other *BSD-related posts: => ./2024-01-13-one-reason-why-i-love-openbsd.gmi 2024-01-13 One reason why I love OpenBSD => ./2024-04-01-KISS-high-availability-with-OpenBSD.gmi 2024-04-01 KISS high-availability with OpenBSD => ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage +=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation E-Mail your comments to `paul@nospam.buetow.org` :-) diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index 4860a696..88372c64 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,12 +1,357 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2024-12-01T12:52:29+02:00</updated> + <updated>2024-12-02T23:46:16+02:00</updated> <title>foo.zone feed</title> <subtitle>To be in the .zone!</subtitle> <link href="gemini://foo.zone/gemfeed/atom.xml" rel="self" /> <link href="gemini://foo.zone/" /> <id>gemini://foo.zone/</id> <entry> + <title>f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation</title> + <link href="gemini://foo.zone/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi" /> + <id>gemini://foo.zone/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi</id> + <updated>2024-12-02T23:46:16+02:00</updated> + <author> + <name>Paul Buetow aka snonux</name> + <email>paul@dev.buetow.org</email> + </author> + <summary>This is the second blog post about my f3s series for my self-hosting demands in my home lab. f3s? The 'f' stands for FreeBSD, and the '3s' stands for k3s, the Kubernetes distribution I will use on FreeBSD-based physical machines.</summary> + <content type="xhtml"> |
