From 50ba608e5ce00b828d5e8be6f2fa89e4d5ed55e8 Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Thu, 14 Aug 2025 23:31:38 +0300 Subject: Update content for md --- ...5-03-projects-i-currently-dont-have-time-for.md | 219 -------- ...024-12-03-f3s-kubernetes-with-freebsd-part-2.md | 226 --------- ...25-01-01-posts-from-october-to-december-2024.md | 60 --- ...025-02-01-f3s-kubernetes-with-freebsd-part-3.md | 376 -------------- ...025-05-11-f3s-kubernetes-with-freebsd-part-5.md | 91 ---- .../2025-07-01-posts-from-january-to-june-2025.md | 52 -- ...025-07-14-f3s-kubernetes-with-freebsd-part-6.md | 52 -- gemfeed/2025-08-05-local-coding-llm-with-ollama.md | 157 ------ gemfeed/2025-08-15-random-weird-things-iii.md | 2 +- gemfeed/DRAFT-kubernetes-with-freebsd-part-7.md | 563 --------------------- gemfeed/DRAFT-totalrecall.md | 271 ---------- 11 files changed, 1 insertion(+), 2068 deletions(-) (limited to 'gemfeed') diff --git a/gemfeed/2024-05-03-projects-i-currently-dont-have-time-for.md b/gemfeed/2024-05-03-projects-i-currently-dont-have-time-for.md index e71adedf..8badb762 100644 --- a/gemfeed/2024-05-03-projects-i-currently-dont-have-time-for.md +++ b/gemfeed/2024-05-03-projects-i-currently-dont-have-time-for.md @@ -217,225 +217,6 @@ I am abandoning this project because my workplace has stopped the annual pet pro I value privacy. It would be great to run my own Matrix server for communication within my family. I have yet to have time to look into this more closely. -# Projects I currently don't have time for - -> Published at 2024-05-03T16:23:03+03:00 - -Over the years, I have collected many ideas for my personal projects and noted them down. I am currently in the process of cleaning up all my notes and reviewing those ideas. I don’t have time for the ones listed here and won’t have any soon due to other commitments and personal projects. So, in order to "get rid of them" from my notes folder, I decided to simply put them in this blog post so that those ideas don't get lost. Maybe I will pick up one or another idea someday in the future, but for now, they are all put on ice in favor of other personal projects or family time. - -``` -Art by Laura Brown - -.'`~~~~~~~~~~~`'. -( .'11 12 1'. ) -| :10 \ 2: | -| :9 @-> 3: | -| :8 4; | -'. '..7 6 5..' .' - ~-------------~ ldb - -``` - -## Table of Contents - -* [⇢ Projects I currently don't have time for](#projects-i-currently-don-t-have-time-for) -* [⇢ ⇢ Hardware projects I don't have time for](#hardware-projects-i-don-t-have-time-for) -* [⇢ ⇢ ⇢ I use Arch, btw!](#i-use-arch-btw) -* [⇢ ⇢ ⇢ OpenBSD home router](#openbsd-home-router) -* [⇢ ⇢ ⇢ Pi-Hole server](#pi-hole-server) -* [⇢ ⇢ ⇢ Infodash](#infodash) -* [⇢ ⇢ ⇢ Reading station](#reading-station) -* [⇢ ⇢ ⇢ Retro station](#retro-station) -* [⇢ ⇢ ⇢ Sound server](#sound-server) -* [⇢ ⇢ ⇢ Project Freekat](#project-freekat) -* [⇢ ⇢ Programming projects I don't have time for](#programming-projects-i-don-t-have-time-for) -* [⇢ ⇢ ⇢ CLI-HIVE](#cli-hive) -* [⇢ ⇢ ⇢ Enhanced KISS home photo albums](#enhanced-kiss-home-photo-albums) -* [⇢ ⇢ ⇢ KISS file sync server with end-to-end encryption](#kiss-file-sync-server-with-end-to-end-encryption) -* [⇢ ⇢ ⇢ A language that compiles to `bash`](#a-language-that-compiles-to-bash) -* [⇢ ⇢ ⇢ A language that compiles to `sed`](#a-language-that-compiles-to-sed) -* [⇢ ⇢ ⇢ Renovate VS-Sim](#renovate-vs-sim) -* [⇢ ⇢ ⇢ KISS ticketing system](#kiss-ticketing-system) -* [⇢ ⇢ ⇢ A domain-specific language (DSL) for work](#a-domain-specific-language-dsl-for-work) -* [⇢ ⇢ Self-hosting projects I don't have time for](#self-hosting-projects-i-don-t-have-time-for) -* [⇢ ⇢ ⇢ My own Matrix server](#my-own-matrix-server) -* [⇢ ⇢ ⇢ Ampache music server](#ampache-music-server) -* [⇢ ⇢ ⇢ Librum eBook reader](#librum-ebook-reader) -* [⇢ ⇢ ⇢ Memos - Note-taking service](#memos---note-taking-service) -* [⇢ ⇢ ⇢ Bepasty server](#bepasty-server) -* [⇢ ⇢ Books I don't have time to read](#books-i-don-t-have-time-to-read) -* [⇢ ⇢ ⇢ Fluent Python](#fluent-python) -* [⇢ ⇢ ⇢ Programming Ruby](#programming-ruby) -* [⇢ ⇢ ⇢ Peter F. Hamilton science fiction books](#peter-f-hamilton-science-fiction-books) -* [⇢ ⇢ New websites I don't have time for](#new-websites-i-don-t-have-time-for) -* [⇢ ⇢ ⇢ Create a "Why Raku Rox" site](#create-a-why-raku-rox-site) -* [⇢ ⇢ Research projects I don't have time for](#research-projects-i-don-t-have-time-for) -* [⇢ ⇢ ⇢ Project secure](#project-secure) -* [⇢ ⇢ ⇢ CPU utilisation is all wrong](#cpu-utilisation-is-all-wrong) - -## Hardware projects I don't have time for - -### I use Arch, btw! - -The idea was to build the ultimate Arch Linux setup on an old ThinkPad X200 booting with the open-source LibreBoot firmware, complete with a tiling window manager, dmenu, and all the elite tools. This is mainly for fun, as I am pretty happy (and productive) with my Fedora Linux setup. I ran EndeavourOS (close enough to Arch) on an old ThinkPad for a while, but then I switched back to Fedora because the rolling releases were annoying (there were too many updates). - -### OpenBSD home router - -In my student days, I operated a 486DX PC with OpenBSD as my home DSL internet router. I bought the setup from my brother back then. The router's hostname was `fishbone`, and it performed very well until it became too slow for larger broadband bandwidth after a few years of use. - -I had the idea to revive this concept, implement `fishbone2`, and place it in front of my proprietary ISP router to add an extra layer of security and control in my home LAN. It would serve as the default gateway for all of my devices, including a Wi-Fi access point, would run a DNS server, Pi-hole proxy, VPN client, and DynDNS client. I would also implement high availability using OpenBSD's CARP protocol. - -[https://openbsdrouterguide.net](https://openbsdrouterguide.net) -[https://pi-hole.net/](https://pi-hole.net/) -[https://www.OpenBSD.org](https://www.OpenBSD.org) -[https://www.OpenBSD.org/faq/pf/carp.html](https://www.OpenBSD.org/faq/pf/carp.html) - -However, I am putting this on hold as I have opted for an OpenWRT-based solution, which was much quicker to set up and runs well enough. - -[https://OpenWRT.org/](https://OpenWRT.org/) - -### Pi-Hole server - -Install Pi-hole on one of my Pis or run it in a container on Freekat. For now, I am putting this on hold as the primary use for this would be ad-blocking, and I am avoiding surfing ad-heavy sites anyway. So there's no significant use for me personally at the moment. - -[https://pi-hole.net/](https://pi-hole.net/) - -### Infodash - -The idea was to implement my smart info screen using purely open-source software. It would display information such as the health status of my personal infrastructure, my current work tracker balance (I track how much I work to prevent overworking), and my sports balance (I track my workouts to stay within my quotas for general health). The information would be displayed on a small screen in my home office, on my Pine watch, or remotely from any terminal window. - -I don't have this, and I haven't missed having it, so I guess it would have been nice to have it but not provide any value other than the "fun of tinkering." - -### Reading station - -I wanted to create the most comfortable setup possible for reading digital notes, articles, and books. This would include a comfy armchair, a silent barebone PC or Raspberry Pi computer running either Linux or *BSD, and an e-Ink display mounted on a flexible arm/stand. There would also be a small table for my paper journal for occasional note-taking. There are a bunch of open-source software available for PDF and ePub reading. It would have been neat, but I am currently using the most straightforward solution: a Kobo Elipsa 2E, which I can use on my sofa. - -### Retro station - -I had an idea to build a computer infused with retro elements. It wouldn't use actual retro hardware but would look and feel like a retro machine. I would call this machine HAL or Retron. - -I would use an old ThinkPad laptop placed on a horizontal stand, running NetBSD, and attaching a keyboard from ModelFkeyboards. I use WindowMaker as a window manager and run terminal applications through Retro Term. For the monitor, I would use an older (black) EIZO model with large bezels. - -[https://www.NetBSD.org](https://www.NetBSD.org) -[https://www.modelfkeyboards.com](https://www.modelfkeyboards.com) -[https://github.com/Swordfish90/cool-retro-term)](https://github.com/Swordfish90/cool-retro-term)) - -The computer would occasionally be used to surf the Gemini space, take notes, blog, or do light coding. However, I have abandoned the project for now because there isn't enough space in my apartment, as my daughter will have a room for herself. - -### Sound server - -My idea involved using a barebone mini PC running FreeBSD with the Navidrome sound server software. I could remotely connect to it from my phone, workstation/laptop to listen to my music collection. The storage would be based on ZFS with at least two drives for redundancy. The app would run in a Linux Docker container under FreeBSD via Bhyve. - -[https://github.com/navidrome/navidrome](https://github.com/navidrome/navidrome) -[https://wiki.freebsd.org/bhyve](https://wiki.freebsd.org/bhyve) - -### Project Freekat - -My idea involved purchasing the Meerkat mini PC from System76 and installing FreeBSD. Like the sound-server idea (see previous idea), it would run Linux Docker through Bhyve. I would self-host a bunch of applications on it: - -* Wallabag -* Ankidroid -* Miniflux & Postgres -* Audiobookshelf -* ... - -All of this would be within my LAN, but the services would also be accessible from the internet through either Wireguard or SSH reverse tunnels to one of my OpenBSD VMs, for example: - -* `wallabag.awesome.buetow.org` -* `ankidroid.awesome.buetow.org` -* `miniflux.awesome.buetow.org` -* `audiobookshelf.awesome.buetow.org` -* ... - -I am abandoning this project for now, as I am currently hosting my apps on AWS ECS Fargate under `*.cool.buetow.org`, which is "good enough" for the time being and also offers the benefit of learning to use AWS and Terraform, knowledge that can be applied at work. - -[My personal AWS setup](./2024-02-04-from-babylon5.buetow.org-to-.cloud.md) - -## Programming projects I don't have time for - -### CLI-HIVE - -This was a pet project idea that my brother and I had. The concept was to collect all shell history of all servers at work in a central place, apply ML/AI, and return suggestions for commands to type or allow a fuzzy search on all the commands in the history. The recommendations for the commands on a server could be context-based (e.g., past occurrences on the same server type). - -You could decide whether to share your command history with others so they would receive better suggestions depending on which server they are on, or you could keep all the history private and secure. The plan was to add hooks into zsh and bash shells so that all commands typed would be pushed to the central location for data mining. - -### Enhanced KISS home photo albums - -I don't use third-party cloud providers such as Google Photos to store/archive my photos. Instead, they are all on a ZFS volume on my home NAS, with regular offsite backups taken. Thus, my project would involve implementing the features I miss most or finding a solution simple enough to host on my LAN: - -* A feature I miss presents me with a random day from the past and some photos from that day. This project would randomly select a day and generate a photo album for me to view and reminisce about memories. -* Another feature I miss is the ability to automatically deduplicate all the photos, as I am sure there are tons of duplicates on my NAS. -* Auto-enhancing the photos (perhaps using ImageMagick?) -* I already have a simple `photoalbum.sh` script that generates an album based on an input directory. However, it would be great also to have a timeline feature to enable browsing through different dates. - -[KISS static web photo albums with `photoalbum.sh`](./2023-10-29-kiss-static-web-photo-albums-with-photoalbum.sh.md) - -### KISS file sync server with end-to-end encryption - -I aimed to have a simple server to which I could sync notes and other documents, ensuring that the data is fully end-to-end encrypted. This way, only the clients could decrypt the data, while an encrypted copy of all the data would be stored on the server side. There are a few solutions (e.g., NextCloud), but they are bloated or complex to set up. - -I currently use Syncthing for encrypted file sync across all my devices; however, the data is not end-to-end encrypted. It's a good-enough setup, though, as my Syncthing server is in my home LAN on an encrypted file system. - -[https://syncthing.net](https://syncthing.net) - -I also had the idea of using this as a pet project for work and naming it `Cryptolake`, utilizing post-quantum-safe encryption algorithms and a distributed data store. - -### A language that compiles to `bash` - -I had an idea to implement a higher-level language with strong typing that could be compiled into native Bash code. This would make all resulting Bash scripts more robust and secure by default. The project would involve developing a parser, lexer, and a Bash code generator. I planned to implement this in Go. - -I had previously implemented a tiny scripting language called Fype (For Your Program Execution), which could have served as inspiration. - -[The Fype Programming Language](./2010-05-09-the-fype-programming-language.md) - -### A language that compiles to `sed` - -This is similar to the previous idea, but the difference is that the language would compile into a sed script. Sed has many features, but the brief syntax makes scripts challenging to read. The higher-level language would mimic sed but in a form that is easier for humans to read. - -### Renovate VS-Sim - -VS-Sim is an open-source simulator programmed in Java for distributed systems. VS-Sim stands for "Verteilte Systeme Simulator," the German translation for "Distributed Systems Simulator." The VS-Sim project was my diploma thesis at Aachen University of Applied Sciences. - -[https://codeberg.org/snonux/vs-sim](https://codeberg.org/snonux/vs-sim) - -The ideas I had was: - -* Translate the project into English. -* Modernise the Java codebase to be compatible with the latest JDK. -* Make it compile to native binaries using GraalVM. -* Distribute the project using AppImages. - -I have put this project on hold for now, as I want to do more things in Go and fewer in Java in my personal time. - -### KISS ticketing system - -My idea was to program a KISS (Keep It Simple, Stupid) ticketing system for my personal use. However, I am abandoning this project because I now use the excellent Taskwarrior software. You can learn more about it at: - -[https://taskwarrior.org/](https://taskwarrior.org/) - -### A domain-specific language (DSL) for work - -At work, an internal service allocates storage space for our customers on our storage clusters. It automates many tasks, but many tweaks are accessible through APIs. I had the idea to implement a Ruby-based DSL that would make using all those APIs for ad-hoc changes effortless, e.g.: - -```ruby -Cluster :UK, :uk01 do - Customer.C1A1.segments.volumes.each do |volume| - puts volume.usage_stats - volume.move_off! if volume.over_subscribed? - end -end -``` - -I am abandoning this project because my workplace has stopped the annual pet project competition, and I have other more important projects to work on at the moment. - -[Creative universe (Work pet project contests)](./2022-04-10-creative-universe.md) - -## Self-hosting projects I don't have time for - -### My own Matrix server - -I value privacy. It would be great to run my own Matrix server for communication within my family. I have yet to have time to look into this more closely. - [https://matrix.org](https://matrix.org) ### Ampache music server diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.md b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.md index afd68911..1cc803ed 100644 --- a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.md +++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.md @@ -223,232 +223,6 @@ NewRec 0 days, 00:07:33 | since Mon Dec 2 12:21:44 2024 %up 100.000 | since Mon Dec 2 12:21:44 2024 ``` -This is how I track the uptimes for all of my host: -# f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation - -> Published at 2024-12-02T23:48:21+02:00 - -This is the second blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution I will use on FreeBSD-based physical machines. - -We set the stage last time; this time, we will set up the hardware for this project. - -These are all the posts so far: - -[2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage](./2024-11-17-f3s-kubernetes-with-freebsd-part-1.md) -[2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation (You are currently reading this)](./2024-12-03-f3s-kubernetes-with-freebsd-part-2.md) -[2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts](./2025-02-01-f3s-kubernetes-with-freebsd-part-3.md) -[2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) -[2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network](./2025-05-11-f3s-kubernetes-with-freebsd-part-5.md) -[2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage](./2025-07-14-f3s-kubernetes-with-freebsd-part-6.md) - -[![f3s logo](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png "f3s logo")](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png) - -> ChatGPT generated logo.. - -Let's continue... - -## Table of Contents - -* [⇢ f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation](#f3s-kubernetes-with-freebsd---part-2-hardware-and-base-installation) -* [⇢ Deciding on the hardware](#deciding-on-the-hardware) -* [⇢ ⇢ Not ARM but Intel N100 ](#not-arm-but-intel-n100-) -* [⇢ ⇢ Beelink unboxing](#beelink-unboxing) -* [⇢ ⇢ Network switch](#network-switch) -* [⇢ Installing FreeBSD](#installing-freebsd) -* [⇢ ⇢ Base install](#base-install) -* [⇢ ⇢ Latest patch level and customizing `/etc/hosts`](#latest-patch-level-and-customizing-etchosts) -* [⇢ ⇢ After install](#after-install) -* [⇢ ⇢ ⇢ Helix editor](#helix-editor) -* [⇢ ⇢ ⇢ `doas`](#doas) -* [⇢ ⇢ ⇢ Periodic ZFS snapshotting](#periodic-zfs-snapshotting) -* [⇢ ⇢ ⇢ Uptime tracking](#uptime-tracking) -* [⇢ Hardware check](#hardware-check) -* [⇢ ⇢ Ethernet](#ethernet) -* [⇢ ⇢ RAM](#ram) -* [⇢ ⇢ CPUs](#cpus) -* [⇢ ⇢ CPU throttling](#cpu-throttling) -* [⇢ Conclusion](#conclusion) - -# Deciding on the hardware - -Note that the OpenBSD VMs included in the f3s setup (which will be used later in this blog series for internet ingress - as you know from the first part of this blog series) are already there. These are virtual machines that I rent at OpenBSD Amsterdam and Hetzner. - -[https://openbsd.amsterdam](https://openbsd.amsterdam) -[https://hetzner.cloud](https://hetzner.cloud) - -This means that the FreeBSD boxes need to be covered, which will later be running k3s in Linux VMs via bhyve hypervisor. - -I've been considering whether to use Raspberry Pis or look for alternatives. It turns out that complete N100-based mini-computers aren't much more expensive than Raspberry Pi 5s, and they don't require assembly. Furthermore, I like that they are AMD64 and not ARM-based, which increases compatibility with some applications (e.g., I might want to virtualize Windows (via bhyve) on one of those, though that's out of scope for this blog series). - -## Not ARM but Intel N100 - -I needed something compact, efficient, and capable enough to handle the demands of a small-scale Kubernetes cluster and preferably something I don't have to assemble a lot. After researching, I decided on the Beelink S12 Pro with Intel N100 CPUs. - -[Beelink Mini S12 Pro N100 official page](https://www.bee-link.com/products/beelink-mini-s12-pro-n100) - -The Intel N100 CPUs are built on the "Alder Lake-N" architecture. These chips are designed to balance performance and energy efficiency well. With four cores, they're more than capable of running multiple containers, even with moderate workloads. Plus, they consume only around 8W of power (ok, that's more than the Pis...), keeping the electricity bill low enough and the setup quiet - perfect for 24/7 operation. - -[![Beelink preparation](./f3s-kubernetes-with-freebsd-part-2/f3s-collage1.jpg "Beelink preparation")](./f3s-kubernetes-with-freebsd-part-2/f3s-collage1.jpg) - -The Beelink comes with the following specs: - -* 12th Gen Intel N100 processor, with four cores and four threads, and a maximum frequency of up to 3.4 GHz. -* 16 GB of DDR4 RAM, with a maximum (official) size of 16 GB (but people could install 32 GB on it). -* 500 GB M.2 SSD, with the option to install a 2nd 2.5 SSD drive (which I want to make use of later in this blog series). -* GBit ethernet -* Four USB 3.2 Gen2 ports (maybe I want to mount something externally at some point) -* Dimensions and weight: 115*102*39mm, 280g -* Silent cooling system. -* HDMI output (needed only for the initial installation and maybe for troubleshooting later) -* Auto power on via WoL (may make use of it) -* Wi-Fi (not going to use it) - -I bought three (3) of them for the cluster I intend to build. - -## Beelink unboxing - -Unboxing was uneventful. Every Beelink PC came with: - -* An AC power adapter -* An HDMI cable -* A VESA mount with screws (not using it as of now) -* Some manuals -* The pre-assembled Beelink PC itself. -* A "Hello" post card (??) - -Overall, I love the small form factor. - -## Network switch - -I went with the tp-link mini 5-port switch, as I had a spare one available. That switch will be plugged into my wall ethernet port, which connects directly to my fiber internet router with 100 Mbit/s down and 50 Mbit/s upload speed. - -[![Switch](./f3s-kubernetes-with-freebsd-part-2/switch.jpg "Switch")](./f3s-kubernetes-with-freebsd-part-2/switch.jpg) - -# Installing FreeBSD - -## Base install - -First, I downloaded the boot-only ISO of the latest FreeBSD release and dumped it on a USB stick via my Fedora laptop: - -```sh -[paul@earth]~/Downloads% sudo dd \ - if=FreeBSD-14.1-RELEASE-amd64-bootonly.iso \ - of=/dev/sda conv=sync -``` - -Next, I plugged the Beelinks (one after another) into my monitor via HDMI (the resolution of the FreeBSD text console seems strangely stretched, as I am using the LG Dual Up monitor), connected Ethernet, an external USB keyboard, and the FreeBSD USB stick, and booted the devices up. With F7, I entered the boot menu and selected the USB stick for the FreeBSD installation. - -The installation was uneventful. I selected: - -* Guided ZFS on root (pool `zroot`) -* Unencrypted ZFS (I will encrypt separate datasets later; I want it to be able to boot without manual interaction) -* Static IP configuration (to ensure that the boxes always have the same IPs, even after switching the router/DHCP server) -* I decided to enable the SSH daemon, NTP server, and NTP time synchronization at boot, and I also enabled `powerd` for automatic CPU frequency scaling. -* In addition to `root,` I added a personal user, `paul,` whom I placed in the `wheel` group. - -After doing all that three times (once for each Beelink PC), I had three ready-to-use FreeBSD boxes! Their hostnames are `f0`, `f1` and `f2`! - -[![Beelink installation](./f3s-kubernetes-with-freebsd-part-2/f3s-collage2.jpg "Beelink installation")](./f3s-kubernetes-with-freebsd-part-2/f3s-collage2.jpg) - -## Latest patch level and customizing `/etc/hosts` - -After the first boot, I upgraded to the latest FreeBSD patch level as follows: - -```sh -root@f0:~ # freebsd-update fetch -root@f0:~ # freebsd-update install -root@f0:~ # freebsd-update reboot -``` - -I also added the following entries for the three FreeBSD boxes to the `/etc/hosts` file: - -```sh -root@f0:~ # cat <>/etc/hosts -192.168.1.130 f0 f0.lan f0.lan.buetow.org -192.168.1.131 f1 f1.lan f1.lan.buetow.org -192.168.1.132 f2 f2.lan f2.lan.buetow.org -END -``` - -You might wonder why bother using the hosts file? Why not use DNS properly? The reason is simplicity. I don't manage 100 hosts, only a few here and there. Having an OpenWRT router in my home, I could also configure everything there, but maybe I'll do that later. For now, keep it simple and straightforward. - -## After install - -After that, I installed the following additional packages: - -```sh -root@f0:~ # pkg install helix doas zfs-periodic uptimed -``` - -### Helix editor - -Helix? It's my favourite text editor. I have nothing against `vi` but like `hx` (Helix) more! - -[https://helix-editor.com/](https://helix-editor.com/) - -### `doas` - -`doas`? It's a pretty neat (and KISS) replacement for `sudo`. It has far fewer features than `sudo`, which is supposed to make it more secure. Its origin is the OpenBSD project. For `doas`, I accepted the default configuration (where users in the `wheel` group are allowed to run commands as `root`): - -```sh -root@f0:~ # cp /usr/local/etc/doas.conf.sample /usr/local/etc/doas.conf -``` - -[https://man.openbsd.org/doas](https://man.openbsd.org/doas) - -### Periodic ZFS snapshotting - -`zfs-periodic` is a nifty tool for automatically creating ZFS snapshots. I decided to go with the following configuration here: - -```sh -root@f0:~ # cat <>/etc/periodic.conf -daily_zfs_snapshot_enable="YES" -daily_zfs_snapshot_pools="zroot,zdata" -daily_zfs_snapshot_keep="7" -weekly_zfs_snapshot_enable="YES" -weekly_zfs_snapshot_pools="zroot,zdata" -weekly_zfs_snapshot_keep="5" -monthly_zfs_snapshot_enable="YES" -monthly_zfs_snapshot_pools="zroot,zdata" -monthly_zfs_snapshot_keep="6" -END -``` - -[https://github.com/ross/zfs-periodic](https://github.com/ross/zfs-periodic) - -Note: We have also added `zdata` to the list of snapshot pools. Currently, this pool does not exist yet, but it will be created later in this blog series. - -### Uptime tracking - -`uptimed`? I like to track my uptimes. This is how I configured the daemon: - -```sh -root@f0:~ # cp /usr/local/mimecast/etc/uptimed.conf-dist \ - /usr/local/mimecast/etc/uptimed.conf -root@f0:~ # hx /usr/local/mimecast/etc/uptimed.conf -``` - -In the Helix editor session, I changed `LOG_MAXIMUM_ENTRIES` to `0` to keep all uptime entries forever and not cut off at 50 (the default config). After that, I enabled and started `uptimed`: - -```sh -root@f0:~ # service uptimed enable -root@f0:~ # service uptimed start -``` - -To check the current uptime stats, I can now run `uprecords`: - -```sh - root@f0:~ # uprecords - # Uptime | System Boot up -----------------------------+--------------------------------------------------- --> 1 0 days, 00:07:34 | FreeBSD 14.1-RELEASE Mon Dec 2 12:21:44 2024 -----------------------------+--------------------------------------------------- -NewRec 0 days, 00:07:33 | since Mon Dec 2 12:21:44 2024 - up 0 days, 00:07:34 | since Mon Dec 2 12:21:44 2024 - down 0 days, 00:00:00 | since Mon Dec 2 12:21:44 2024 - %up 100.000 | since Mon Dec 2 12:21:44 2024 -``` - This is how I track the uptimes for all of my host: [Unveiling `guprecords.raku`: Global Uptime Records with Raku-](./2023-05-01-unveiling-guprecords:-uptime-records-with-raku.md) diff --git a/gemfeed/2025-01-01-posts-from-october-to-december-2024.md b/gemfeed/2025-01-01-posts-from-october-to-december-2024.md index f9ea7bd7..a44041ee 100644 --- a/gemfeed/2025-01-01-posts-from-october-to-december-2024.md +++ b/gemfeed/2025-01-01-posts-from-october-to-december-2024.md @@ -13,66 +13,6 @@ These are from Mastodon and LinkedIn. Have a look at my about page for my social ## Table of Contents -* [⇢ Posts from October to December 2024](#posts-from-october-to-december-2024) -* [⇢ ⇢ October 2024](#october-2024) -* [⇢ ⇢ ⇢ First on-call experience in a startup. Doesn't ...](#first-on-call-experience-in-a-startup-doesn-t-) -* [⇢ ⇢ ⇢ Reviewing your own PR or MR before asking ...](#reviewing-your-own-pr-or-mr-before-asking-) -* [⇢ ⇢ ⇢ Fun with defer in `#golang`, I did't know, that ...](#fun-with-defer-in-golang-i-did-t-know-that-) -* [⇢ ⇢ ⇢ I have been in incidents. Understandably, ...](#i-have-been-in-incidents-understandably-) -* [⇢ ⇢ ⇢ Little tips using strings in `#golang` and I ...](#little-tips-using-strings-in-golang-and-i-) -* [⇢ ⇢ ⇢ Reading this post about `#rust` (especially the ...](#reading-this-post-about-rust-especially-the-) -* [⇢ ⇢ ⇢ The opposite of `#ChaosMonkey` ... ...](#the-opposite-of-chaosmonkey--) -* [⇢ ⇢ November 2024](#november-2024) -* [⇢ ⇢ ⇢ I just became a Silver Patreon for OSnews. What ...](#i-just-became-a-silver-patreon-for-osnews-what-) -* [⇢ ⇢ ⇢ Until now, I wasn't aware, that Go is under a ...](#until-now-i-wasn-t-aware-that-go-is-under-a-) -* [⇢ ⇢ ⇢ These are some book notes from "Staff Engineer" ...](#these-are-some-book-notes-from-staff-engineer-) -* [⇢ ⇢ ⇢ Looking at `#Kubernetes`, it's pretty much ...](#looking-at-kubernetes-it-s-pretty-much-) -* [⇢ ⇢ ⇢ There has been an outage at the upstream ...](#there-has-been-an-outage-at-the-upstream-) -* [⇢ ⇢ ⇢ One of the more confusing parts in Go, nil ...](#one-of-the-more-confusing-parts-in-go-nil-) -* [⇢ ⇢ ⇢ Agreeably, writing down with Diagrams helps you ...](#agreeably-writing-down-with-diagrams-helps-you-) -* [⇢ ⇢ ⇢ I like the idea of types in Ruby. Raku is ...](#i-like-the-idea-of-types-in-ruby-raku-is-) -* [⇢ ⇢ ⇢ So, `#Haskell` is better suited for general ...](#so-haskell-is-better-suited-for-general-) -* [⇢ ⇢ ⇢ At first, functional options add a bit of ...](#at-first-functional-options-add-a-bit-of-) -* [⇢ ⇢ ⇢ Revamping my home lab a little bit. `#freebsd` ...](#revamping-my-home-lab-a-little-bit-freebsd-) -* [⇢ ⇢ ⇢ Wondering to which `#web` `#browser` I should ...](#wondering-to-which-web-browser-i-should-) -* [⇢ ⇢ ⇢ eks-node-viewer is a nifty tool, showing the ...](#eks-node-viewer-is-a-nifty-tool-showing-the-) -* [⇢ ⇢ ⇢ Have put more Photos on - On my static photo ...](#have-put-more-photos-on---on-my-static-photo-) -* [⇢ ⇢ ⇢ In Go, passing pointers are not automatically ...](#in-go-passing-pointers-are-not-automatically-) -* [⇢ ⇢ ⇢ Myself being part of an on-call rotations over ...](#myself-being-part-of-an-on-call-rotations-over-) -* [⇢ ⇢ ⇢ Feels good to code in my old love `#Perl` again ...](#feels-good-to-code-in-my-old-love-perl-again-) -* [⇢ ⇢ ⇢ This is an interactive summary of the Go ...](#this-is-an-interactive-summary-of-the-go-) -* [⇢ ⇢ December 2024](#december-2024) -* [⇢ ⇢ ⇢ Thats unexpected, you cant remove a NaN key ...](#thats-unexpected-you-cant-remove-a-nan-key-) -* [⇢ ⇢ ⇢ My second blog post about revamping my home lab ...](#my-second-blog-post-about-revamping-my-home-lab-) -* [⇢ ⇢ ⇢ Very insightful article about tech hiring in ...](#very-insightful-article-about-tech-hiring-in-) -* [⇢ ⇢ ⇢ for `#bpf` `#ebpf` performance debugging, have ...](#for-bpf-ebpf-performance-debugging-have-) -* [⇢ ⇢ ⇢ 89 things he/she knows about Git commits is a ...](#89-things-heshe-knows-about-git-commits-is-a-) -* [⇢ ⇢ ⇢ I found that working on multiple side projects ...](#i-found-that-working-on-multiple-side-projects-) -* [⇢ ⇢ ⇢ Agreed? Agreed. Besides `#Ruby`, I would also ...](#agreed-agreed-besides-ruby-i-would-also-) -* [⇢ ⇢ ⇢ Plan9 assembly format in Go, but wait, it's not ...](#plan9-assembly-format-in-go-but-wait-it-s-not-) -* [⇢ ⇢ ⇢ This is a neat blog post about the Helix text ...](#this-is-a-neat-blog-post-about-the-helix-text-) -* [⇢ ⇢ ⇢ This blog post is basically a rant against ...](#this-blog-post-is-basically-a-rant-against-) -* [⇢ ⇢ ⇢ Quick trick to get Helix themes selected ...](#quick-trick-to-get-helix-themes-selected-) -* [⇢ ⇢ ⇢ Example where complexity attacks you from ...](#example-where-complexity-attacks-you-from-) -* [⇢ ⇢ ⇢ LLMs for Ops? Summaries of logs, probabilities ...](#llms-for-ops-summaries-of-logs-probabilities-) -* [⇢ ⇢ ⇢ Excellent article about your dream Product ...](#excellent-article-about-your-dream-product-) -* [⇢ ⇢ ⇢ I just finished reading all chapters of CPU ...](#i-just-finished-reading-all-chapters-of-cpu-) -* [⇢ ⇢ ⇢ Indeed, useful to know this stuff! `#sre` ...](#indeed-useful-to-know-this-stuff-sre-) -* # Posts from October to December 2024 - -> Published at 2024-12-31T18:09:58+02:00 - -Happy new year! - -These are my social media posts from the last three months. I keep them here to reflect on them and also to not lose them. Social media networks come and go and are not under my control, but my domain is here to stay. - -These are from Mastodon and LinkedIn. Have a look at my about page for my social media profiles. This list is generated with Gos, my social media platform sharing tool. - -[My about page](../about/index.md) -[https://codeberg.org/snonux/gos](https://codeberg.org/snonux/gos) - -## Table of Contents - * [⇢ Posts from October to December 2024](#posts-from-october-to-december-2024) * [⇢ ⇢ October 2024](#october-2024) * [⇢ ⇢ ⇢ First on-call experience in a startup. Doesn't ...](#first-on-call-experience-in-a-startup-doesn-t-) diff --git a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.md b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.md index 9942e524..f11a637f 100644 --- a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.md +++ b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.md @@ -364,382 +364,6 @@ Read the next post of this series: Other BSD related posts are: -[2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage](./2025-07-14-f3s-kubernetes-with-freebsd-part-6.md) -[2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network](./2025-05-11-f3s-kubernetes-with-freebsd-part-5.md) -[2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) -[2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts (You are currently reading this)](./2025-02-01-f3s-kubernetes-with-freebsd-part-3.md) -[2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation](./2024-12-03-f3s-kubernetes-with-freebsd-part-2.md) -[2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage](./2024-11-17-f3s-kubernetes-with-freebsd-part-1.md) -[2024-04-01 KISS high-availability with OpenBSD](./2024-04-01-KISS-high-availability-with-OpenBSD.md) -[2024-01-13 One reason why I love OpenBSD](./2024-01-13-one-reason-why-i-love-openbsd.md) -[2022-10-30 Installing DTail on OpenBSD](./2022-10-30-installing-dtail-on-openbsd.md) -[2022-07-30 Let's Encrypt with OpenBSD and Rex](./2022-07-30-lets-encrypt-with-openbsd-and-rex.md) -# f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts - -> Published at 2025-01-30T09:22:06+02:00 - -This is the third blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution we will use on FreeBSD-based physical machines. - -[2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage](./2024-11-17-f3s-kubernetes-with-freebsd-part-1.md) -[2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation](./2024-12-03-f3s-kubernetes-with-freebsd-part-2.md) -[2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts (You are currently reading this)](./2025-02-01-f3s-kubernetes-with-freebsd-part-3.md) -[2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) -[2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network](./2025-05-11-f3s-kubernetes-with-freebsd-part-5.md) -[2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage](./2025-07-14-f3s-kubernetes-with-freebsd-part-6.md) - -[![f3s logo](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png "f3s logo")](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png) - -## Table of Contents - -* [⇢ f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts](#f3s-kubernetes-with-freebsd---part-3-protecting-from-power-cuts) -* [⇢ ⇢ Introduction](#introduction) -* [⇢ ⇢ Changes since last time](#changes-since-last-time) -* [⇢ ⇢ ⇢ FreeBSD upgrade from 14.1 to 14.2](#freebsd-upgrade-from-141-to-142) -* [⇢ ⇢ ⇢ A new home (behind the TV)](#a-new-home-behind-the-tv) -* [⇢ ⇢ The UPS hardware](#the-ups-hardware) -* [⇢ ⇢ Configuring FreeBSD to Work with the UPS](#configuring-freebsd-to-work-with-the-ups) -* [⇢ ⇢ ⇢ USB Device Detection](#usb-device-detection) -* [⇢ ⇢ ⇢ `apcupsd` Installation](#apcupsd-installation) -* [⇢ ⇢ ⇢ UPS Connectivity Test](#ups-connectivity-test) -* [⇢ ⇢ APC Info on Partner Nodes:](#apc-info-on-partner-nodes) -* [⇢ ⇢ ⇢ Installation on partners](#installation-on-partners) -* [⇢ ⇢ Power outage simulation](#power-outage-simulation) -* [⇢ ⇢ ⇢ Pulling the plug](#pulling-the-plug) -* [⇢ ⇢ ⇢ Restoring power](#restoring-power) -* [⇢ ⇢ Conclusion](#conclusion) - -## Introduction - -In this blog post, we are setting up the UPS for the cluster. A UPS, or Uninterruptible Power Supply, safeguards my cluster from unexpected power outages and surges. It acts as a backup battery that kicks in when the electricity cuts out—especially useful in my area, where power cuts are frequent—allowing for a graceful system shutdown and preventing data loss and corruption. This is especially important since I will also store some of my data on the f3s nodes. - -## Changes since last time - -### FreeBSD upgrade from 14.1 to 14.2 - -There has been a new release since the last blog post in this series. The upgrade from 14.1 was as easy as: - -```sh -paul@f0: ~ % doas freebsd-update fetch -paul@f0: ~ % doas freebsd-update install -paul@f0: ~ % doas freebsd-update -r 14.2-RELEASE upgrade -paul@f0: ~ % doas freebsd-update install -paul@f0: ~ % doas shutdown -r now -``` - -And after rebooting, I ran: - -```sh -paul@f0: ~ % doas freebsd-update install -paul@f0: ~ % doas pkg update -paul@f0: ~ % doas pkg upgrade -paul@f0: ~ % doas shutdown -r now -``` - -And after another reboot, I was on 14.2: - -```sh -paul@f0:~ % uname -a -FreeBSD f0.lan.buetow.org 14.2-RELEASE FreeBSD 14.2-RELEASE - releng/14.2-n269506-c8918d6c7412 GENERIC amd64 -``` - -And, of course, I ran this on all 3 nodes! - -### A new home (behind the TV) - -I've put all the infrastructure behind my TV, as plenty of space is available. The TV hides most of the setup, which drastically improved the SAF (spouse acceptance factor). - -[![New hardware placement arrangement](./f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg "New hardware placement arrangement")](./f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg) - -I got rid of the mini-switch I mentioned in the previous blog post. I have the TP-Link EAP615-Wall mounted on the wall nearby, which is my OpenWrt-powered Wi-Fi hotspot. It also has 3 Ethernet ports, to which I connected the Beelink nodes. That's the device you see at the very top. - -The Ethernet cables go downward through the cable boxes to the Beelink nodes. In addition to the Beelink f3s nodes, I connected the TP-Link to the UPS as well (not discussed further in this blog post, but the positive side effect is that my Wi-Fi will still work during a power loss for some time—and during a power cut, the Beelink nodes will still be able to communicate with each other). - -On the very left (the black box) is the UPS, with four power outlets. Three go to the Beelink nodes, and one goes to the TP-Link. A USB output is also connected to the first Beelink node, `f0`. - -On the very right (halfway hidden behind the TV) are the 3 Beelink nodes stacked on top of each other. The only downside (or upside?) is that my 14-month-old daughter is now chaos-testing the Beelink nodes, as the red power buttons (now reachable for her) are very attractive for her to press when passing by randomly. :-) Luckily, that will only cause graceful system shutdowns! - -## The UPS hardware - -I wanted a UPS that I could connect to via FreeBSD, and that would provide enough backup power to operate the cluster for a couple of minutes (it turned out to be around an hour, but this time will likely be shortened after future hardware upgrades, like additional drives and a backup enclosure) and to automatically initiate the shutdown of all the f3s nodes. - -I decided on the APC Back-UPS BX750MI model because: - -* Zero noise level when there is no power cut (some light noise when the battery is in operation during a power cut). -* Cost: It is relatively affordable (not costing thousands). -* USB connectivity: Can be connected via USB to one of the FreeBSD hosts to read the UPS status. -* A power output of 750VA (or 410 watts), suitable for an hour of runtime for my f3s nodes (plus the Wi-Fi router). -* Multiple power outlets: Can connect all 3 f3s nodes directly. -* User-replaceable batteries: I can replace the batteries myself after two years or more (depending on usage). -* Its compact design. Overall, I like how it looks. - -[![The APC Back-UPS BX750MI in operation.](./f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg "The APC Back-UPS BX750MI in operation.")](./f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg) - -## Configuring FreeBSD to Work with the UPS - -### USB Device Detection - -Once plugged in via USB on FreeBSD, I could see the following in the kernel messages: - -```sh -paul@f0: ~ % doas dmesg | grep UPS -ugen0.2: at usbus0 -``` - -### `apcupsd` Installation - -To make use of the USB connection, the `apcupsd` package had to be installed: - -```sh -paul@f0: ~ % doas install apcupsd -``` - -I have made the following modifications to the configuration file so that the UPS can be used via the USB interface: - -```sh -paul@f0:/usr/local/etc/apcupsd % diff -u apcupsd.conf.sample apcupsd.conf ---- apcupsd.conf.sample 2024-11-01 16:40:42.000000000 +0200 -+++ apcupsd.conf 2024-12-03 10:58:24.009501000 +0200 -@@ -31,7 +31,7 @@ - # 940-1524C, 940-0024G, 940-0095A, 940-0095B, - # 940-0095C, 940-0625A, M-04-02-2000 - # --UPSCABLE smart -+UPSCABLE usb - - # To get apcupsd to work, in addition to defining the cable - # above, you must also define a UPSTYPE, which corresponds to -@@ -88,8 +88,10 @@ - # that apcupsd binds to that particular unit - # (helpful if you have more than one USB UPS). - # --UPSTYPE apcsmart --DEVICE /dev/usv -+UPSTYPE usb -+DEVICE - - # POLLTIME - # Interval (in seconds) at which apcupsd polls the UPS for status. This -``` - -I left the remaining settings as the default ones; for example, the following are of main interest: - -``` -# If during a power failure, the remaining battery percentage -# (as reported by the UPS) is below or equal to BATTERYLEVEL, -# apcupsd will initiate a system shutdown. -BATTERYLEVEL 5 - -# If during a power failure, the remaining runtime in minutes -# (as calculated internally by the UPS) is below or equal to MINUTES, -# apcupsd, will initiate a system shutdown. -MINUTES 3 -``` - -I then enabled and started the daemon: - -```sh -paul@f0:/usr/local/etc/apcupsd % doas sysrc apcupsd_enable=YES -apcupsd_enable: -> YES -paul@f0:/usr/local/etc/apcupsd % doas service apcupsd start -Starting apcupsd. -``` - -### UPS Connectivity Test - -And voila, I could now access the UPS information via the `apcaccess` command; how convenient :-) (I also read through the manual page, which provides a good understanding of what else can be done with it!). - -```sh -paul@f0:~ % apcaccess -APC : 001,035,0857 -DATE : 2025-01-26 14:43:27 +0200 -HOSTNAME : f0.lan.buetow.org -VERSION : 3.14.14 (31 May 2016) freebsd -UPSNAME : f0.lan.buetow.org -CABLE : USB Cable -DRIVER : USB UPS Driver -UPSMODE : Stand Alone -STARTTIME: 2025-01-26 14:43:25 +0200 -MODEL : Back-UPS BX750MI -STATUS : ONLINE -LINEV : 230.0 Volts -LOADPCT : 4.0 Percent -BCHARGE : 100.0 Percent -TIMELEFT : 65.3 Minutes -MBATTCHG : 5 Percent -MINTIMEL : 3 Minutes -MAXTIME : 0 Seconds -SENSE : Medium -LOTRANS : 145.0 Volts -HITRANS : 295.0 Volts -ALARMDEL : No alarm -BATTV : 13.6 Volts -LASTXFER : Automatic or explicit self test -NUMXFERS : 0 -TONBATT : 0 Seconds -CUMONBATT: 0 Seconds -XOFFBATT : N/A -SELFTEST : NG -STATFLAG : 0x05000008 -SERIALNO : 9B2414A03599 -BATTDATE : 2001-01-01 -NOMINV : 230 Volts -NOMBATTV : 12.0 Volts -NOMPOWER : 410 Watts -END APC : 2025-01-26 14:44:06 +0200 -``` - -## APC Info on Partner Nodes: - -So far, so good. Host `f0` would shut down itself when short on power. But what about the `f1` and `f2` nodes? They aren't connected directly to the UPS and, therefore, wouldn't know that their power is about to be cut off. For this, `apcupsd` running on the `f1` and `f2` nodes can be configured to retrieve UPS information via the network from the `apcupsd` server running on the `f0` node, which is connected directly to the APC via USB. - -Of course, this won't work when `f0` is down. In this case, no operational node would be connected to the UPS via USB; therefore, the current power status would not be known. However, I consider this a rare circumstance. Furthermore, in case of an `f0` system crash, sudden power outages on the two other nodes would occur at different times making real data loss (the main concern here) less likely. - -And if `f0` is down and `f1` and `f2` receive new data and crash midway, it's likely that a client (e.g., an Android app or another laptop) still has the data stored on it, making data recoverable and data loss overall nearly impossible. I'd receive an alert if any of the nodes go down (more on monitoring later in this blog series). - -### Installation on partners - -To do this, I installed `apcupsd` via `doas pkg install apcupsd` on `f1` and `f2`, and then I could connect to it this way: - -```sh -paul@f1:~ % apcaccess -h f0.lan.buetow.org | grep Percent -LOADPCT : 12.0 Percent -BCHARGE : 94.0 Percent -MBATTCHG : 5 Percent -``` - -But I want the daemon to be configured and enabled in such a way that it connects to the master UPS node (the one with the UPS connected via USB) so that it can also initiate a system shutdown when the UPS battery reaches low levels. For that, `apcupsd` itself needs to be aware of the UPS status. - -On `f1` and `f2`, I changed the configuration to use `f0` (where `apcupsd` is listening) as a remote device. I also changed the `MINUTES` setting from 3 to 6 and the `BATTERYLEVEL` setting from 5 to 10 to ensure that the `f1` and `f2` nodes could still connect to the `f0` node for UPS information before `f0` decides to shut down itself. So `f1` and `f2` must shut down earlier than `f0`: - -```sh -paul@f2:/usr/local/etc/apcupsd % diff -u apcupsd.conf.sample apcupsd.conf ---- apcupsd.conf.sample 2024-11-01 16:40:42.000000000 +0200 -+++ apcupsd.conf 2025-01-26 15:52:45.108469000 +0200 -@@ -31,7 +31,7 @@ - # 940-1524C, 940-0024G, 940-0095A, 940-0095B, - # 940-0095C, 940-0625A, M-04-02-2000 - # --UPSCABLE smart -+UPSCABLE ether - - # To get apcupsd to work, in addition to defining the cable - # above, you must also define a UPSTYPE, which corresponds to -@@ -52,7 +52,6 @@ - # Network Information Server. This is used if the - # UPS powering your computer is connected to a - # different computer for monitoring. --# - # snmp hostname:port:vendor:community - # SNMP network link to an SNMP-enabled UPS device. - # Hostname is the ip address or hostname of the UPS -@@ -88,8 +87,8 @@ - # that apcupsd binds to that particular unit - # (helpful if you have more than one USB UPS). - # --UPSTYPE apcsmart --DEVICE /dev/usv -+UPSTYPE net -+DEVICE f0.lan.buetow.org:3551 - - # POLLTIME - # Interval (in seconds) at which apcupsd polls the UPS for status. This -@@ -147,12 +146,12 @@ - # If during a power failure, the remaining battery percentage - # (as reported by the UPS) is below or equal to BATTERYLEVEL, - # apcupsd will initiate a system shutdown. --BATTERYLEVEL 5 -+BATTERYLEVEL 10 - - # If during a power failure, the remaining runtime in minutes - # (as calculated internally by the UPS) is below or equal to MINUTES, - # apcupsd, will initiate a system shutdown. --MINUTES 3 -+MINUTES 6 - - # If during a power failure, the UPS has run on batteries for TIMEOUT - # many seconds or longer, apcupsd will initiate a system shutdown. - -``` -So I also ran the following commands on `f1` and `f2`: - -```sh -paul@f1:/usr/local/etc/apcupsd % doas sysrc apcupsd_enable=YES -apcupsd_enable: -> YES -paul@f1:/usr/local/etc/apcupsd % doas service apcupsd start -Starting apcupsd. -``` - -And then I was able to connect to localhost via the `apcaccess` command: - -```sh -paul@f1:~ % doas apcaccess | grep Percent -LOADPCT : 5.0 Percent -BCHARGE : 95.0 Percent -MBATTCHG : 5 Percent -``` - -## Power outage simulation - -### Pulling the plug - -I simulated a power outage by removing the power input from the APC. Immediately, the following message appeared on all the nodes: - -```` -Broadcast Message from root@f0.lan.buetow.org - (no tty) at 15:03 EET... - -Power failure. Running on UPS batteries. -``` - -I ran the following command to confirm the available battery time: - -```sh -paul@f0:/usr/local/etc/apcupsd % apcaccess -p TIMELEFT -63.9 Minutes -``` - -And after around one hour (`f1` and `f2` a bit earlier, `f0` a bit later due to the different `BATTERYLEVEL` and `MINUTES` settings outlined earlier), the following broadcast was sent out: - -``` -Broadcast Message from root@f0.lan.buetow.org - (no tty) at 15:08 EET... - - *** FINAL System shutdown message from root@f0.lan.buetow.org *** - -System going down IMMEDIATELY - -apcupsd initiated shutdown -``` - -And all the nodes shut down safely before the UPS ran out of battery! - -### Restoring power - -After restoring power, I checked the logs in `/var/log/daemon.log` and found the following on all 3 nodes: - -``` -Jan 26 17:36:24 f2 apcupsd[2159]: Power failure. -Jan 26 17:36:30 f2 apcupsd[2159]: Running on UPS batteries. -Jan 26 17:36:30 f2 apcupsd[2159]: Battery charge below low limit. -Jan 26 17:36:30 f2 apcupsd[2159]: Initiating system shutdown! -Jan 26 17:36:30 f2 apcupsd[2159]: User logins prohibited -Jan 26 17:36:32 f2 apcupsd[2159]: apcupsd exiting, signal 15 -Jan 26 17:36:32 f2 apcupsd[2159]: apcupsd shutdown succeeded -``` - -All good :-) - -## Conclusion - -I have the same UPS (but with a bit more capacity) for my main work setup, which powers my 28" screen, music equipment, etc. It has already been helpful a couple of times during power outages here, so I am sure that the smaller UPS for the F3s setup will be of great use. - -Read the next post of this series: - -[f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) - -Other BSD related posts are: - [2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage](./2025-07-14-f3s-kubernetes-with-freebsd-part-6.md) [2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network](./2025-05-11-f3s-kubernetes-with-freebsd-part-5.md) [2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) diff --git a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.md b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.md index 67bc6c50..1dca3454 100644 --- a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.md +++ b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.md @@ -89,97 +89,6 @@ We could have used Tailscale for an easy to set up and manage the WireGuard netw [https://www.wireguard.com/](https://www.wireguard.com/) [https://tailscale.com/](https://tailscale.com/) -# f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network - -> Published at 2025-05-11T11:35:57+03:00 - -This is the fifth blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution I will use on FreeBSD-based physical machines. - -I will post a new entry every month or so (there are too many other side projects for more frequent updates — I bet you can understand). - -These are all the posts so far: - -[2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage](./2024-11-17-f3s-kubernetes-with-freebsd-part-1.md) -[2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation](./2024-12-03-f3s-kubernetes-with-freebsd-part-2.md) -[2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts](./2025-02-01-f3s-kubernetes-with-freebsd-part-3.md) -[2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs](./2025-04-05-f3s-kubernetes-with-freebsd-part-4.md) -[2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network (You are currently reading this)](./2025-05-11-f3s-kubernetes-with-freebsd-part-5.md) -[2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage](./2025-07-14-f3s-kubernetes-with-freebsd-part-6.md) - -[![f3s logo](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png "f3s logo")](./f3s-kubernetes-with-freebsd-part-1/f3slogo.png) - -> ChatGPT generated logo. - -Let's begin... - -## Table of Contents - -* [⇢ f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network](#f3s-kubernetes-with-freebsd---part-5-wireguard-mesh-network) -* [⇢ ⇢ Introduction](#introduction) -* [⇢ ⇢ ⇢ Expected traffic flow](#expected-traffic-flow) -* [⇢ ⇢ Deciding on WireGuard](#deciding-on-wireguard) -* [⇢ ⇢ Base configuration](#base-configuration) -* [⇢ ⇢ ⇢ FreeBSD](#freebsd) -* [⇢ ⇢ ⇢ Rocky Linux](#rocky-linux) -* [⇢ ⇢ ⇢ OpenBSD](#openbsd) -* [⇢ ⇢ WireGuard configuration](#wireguard-configuration) -* [⇢ ⇢ ⇢ Example `wg0.conf`](#example-wg0conf) -* [⇢ ⇢ ⇢ NAT traversal and keepalive](#nat-traversal-and-keepalive) -* [⇢ ⇢ ⇢ Preshared key](#preshared-key) -* [⇢ ⇢ Mesh network generator](#mesh-network-generator) -* [⇢ ⇢ ⇢ `wireguardmeshgenerator.yaml`](#wireguardmeshgeneratoryaml) -* [⇢ ⇢ ⇢ `wireguardmeshgenerator.rb` overview](#wireguardmeshgeneratorrb-overview) -* [⇢ ⇢ Invoking the mesh network generator](#invoking-the-mesh-network-generator) -* [⇢ ⇢ ⇢ Generating the `wg0.conf` files and keys](#generating-the-wg0conf-files-and-keys) -* [⇢ ⇢ ⇢ Installing the `wg0.conf` files](#installing-the-wg0conf-files) -* [⇢ ⇢ ⇢ Re-generating mesh and installing the `wg0.conf` files again](#re-generating-mesh-and-installing-the-wg0conf-files-again) -* [⇢ ⇢ Happy WireGuard-ing](#happy-wireguard-ing) -* [⇢ ⇢ Conclusion](#conclusion) - -## Introduction - -By default, traffic within my home LAN, including traffic inside a k3s cluster, is not encrypted. While it resides in the "secure" home LAN, adopting a zero-trust policy means encryption is still preferable to ensure confidentiality and security. So we decide to secure all the traffic of all f3s participating hosts by building a mesh network of all participating hosts: - -[![Full mesh network](./f3s-kubernetes-with-freebsd-part-5/wireguard-full-mesh.svg "Full mesh network")](./f3s-kubernetes-with-freebsd-part-5/wireguard-full-mesh.svg) - -Whereas `f0`, `f1`, and `f2` are the FreeBSD base hosts, `r0`, `r1`, and `r2` are the Rocky Linux Bhyve VMs, and `blowfish` and `fishfinger` are two OpenBSD systems running on the internet (as mentioned in the first blog of this series—these systems are already built; in fact, this very blog is served by those OpenBSD systems). - -As we can see from the graph, it is a true full-mesh network, where every host has a VPN tunnel to every other host. The benefit is that we do not need to route traffic through intermediate hosts (significantly simplifying the routing configuration). However, the downside is that there is some overhead in configuring and managing all the tunnels. - -For simplicity, we also establish VPN tunnels between `f0 <-> r0`, `f1 <-> r1`, and `f2 <-> r2`. Technically, this wouldn't be strictly required since the VMs `rN` are running on the hosts `fN`, and no network traffic is leaving the box. However, it simplifies the configuration as we don't have to account for exceptions, and we are going to automate the mesh network configuration anyway (read on). - -### Expected traffic flow - -The traffic is expected to flow between the host groups through the mesh network as follows: - -* `fN <-> rN`: The traffic between the FreeBSD hosts and the Rocky Linux VMs will be routed through the VPN tunnels for persistent storage. In a later post in this series, we will set up an NFS server on the `fN` hosts. -* `fN <-> blowfish,fishfinger`: The traffic between the FreeBSD hosts and the OpenBSD host `blowfish,fishfinger` will be routed through the VPN tunnels for management. We may want to log in via the internet to set it up remotely. The VPN tunnel will also be used for monitoring purposes. -* `rN <-> blowfish,fishfinger`: The traffic between the Rocky Linux VMs and the OpenBSD host `blowfish,fishfinger` will be routed through the VPN tunnels for usage traffic. Since k3s will be running on the `rN` hosts, the OpenBSD servers will route the traffic through `relayd` to the services running in Kubernetes. -* `fN <-> fM`: The traffic between the FreeBSD hosts may be later used for data replication for the NFS storage. -* `rN <-> rM`: The traffic between the Rocky Linux VMs will later be used by the k3s cluster itself, as every `rN` will be a Kubernetes worker node. -* `blowfish <-> fishfinger`: The traffic between the OpenBSD hosts isn't strictly required for this setup, but I set it up anyway for future use cases. - -We won't cover all the details in this blog post, as we only focus on setting up the Mesh network in this blog post. Subsequent posts in this series will cover the other details. - -## Deciding on WireGuard - -I have decided to use WireGuard as the VPN technology for this purpose. - -WireGuard is a lightweight, modern, and secure VPN protocol designed for simplicity, speed, and strong cryptography. It is an excellent choice due to its minimal codebase, ease of configuration, high performance, and robust security, utilizing state-of-the-art encryption standards. WireGuard is supported on various operating systems, and its implementations are compatible with each other. Therefore, establishing WireGuard VPN tunnels between FreeBSD, Linux, and OpenBSD is seamless. This cross-platform availability makes it suitable for setups like the one described in this blog series. - -We could have used Tailscale for an easy to set up and manage the WireGuard network, but the benefits of creating our own mesh network are: - -* Learning about WireGuard configuration details -* Have full control over the setup -* Don't rely on an external provider like Tailscale (even if some of the components are open-source) -* Have even more fun along the way -* WireGuard is easy to configure on my target operating systems and, therefore, easier to maintain in the long run. -* There are no official Tailscale packages available for OpenBSD and FreeBSD. However, getting Tailscale running on these systems is still possible, though some tinkering would be required. Instead, we use that tinkering time to set up WireGuard tunnels ourselves. - -[https://en.wikipedia.org/wiki/WireGuard](https://en.wikipedia.org/wiki/WireGuard) -[https://www.wireguard.com/](https://www.wireguard.com/) -[https://tailscale.com/](https://tailscale.com/) - [![WireGuard Logo](./f3s-kubernetes-with-freebsd-part-5/wireguard.svg "WireGuard Logo")](./f3s-kubernetes-with-freebsd-part-5/wireguard.svg) ## Base configuration diff --git a/gemfeed/2025-07-01-posts-from-january-to-june-2025.md b/gemfeed/2025-07-01-posts-from-january-to-june-2025.md index ca5b8816..0620a68b 100644 --- a/gemfeed/2025-07-01-posts-from-january-to-june-2025.md +++ b/gemfeed/2025-07-01-posts-from-january-to-june-2025.md @@ -11,58 +11,6 @@ These are from Mastodon and LinkedIn. Have a look at my about page for my social ## Table of Contents -* [⇢ Posts from January to June 2025](#posts-from-january-to-june-2025) -* [⇢ ⇢ January 2025](#january-2025) -* [⇢ ⇢ ⇢ I am currently binge-listening to the Google ...](#i-am-currently-binge-listening-to-the-google-) -* [⇢ ⇢ ⇢ Recently, there was a >5000 LOC `#bash` ...](#recently-there-was-a-5000-loc-bash-) -* [⇢ ⇢ ⇢ Ghostty is a terminal emulator that was ...](#ghostty-is-a-terminal-emulator-that-was-) -* [⇢ ⇢ ⇢ Go is not an easy programming language. Don't ...](#go-is-not-an-easy-programming-language-don-t-) -* [⇢ ⇢ ⇢ How will AI change software engineering (or has ...](#how-will-ai-change-software-engineering-or-has-) -* [⇢ ⇢ ⇢ Eliminating toil - Toil is not always a bad ...](#eliminating-toil---toil-is-not-always-a-bad-) -* [⇢ ⇢ ⇢ Fun read. How about using the character ...](#fun-read-how-about-using-the-character-) -* [⇢ ⇢ ⇢ Thats unexpected, you cant remove a NaN key ...](#thats-unexpected-you-cant-remove-a-nan-key-) -* [⇢ ⇢ ⇢ Nice refresher for `#shell` `#bash` `#zsh` ...](#nice-refresher-for-shell-bash-zsh-) -* [⇢ ⇢ ⇢ I think discussing action items in incident ...](#i-think-discussing-action-items-in-incident-) -* [⇢ ⇢ ⇢ At first, functional options add a bit of ...](#at-first-functional-options-add-a-bit-of-) -* [⇢ ⇢ ⇢ In the "Working with an SRE Interview" I have ...](#in-the-working-with-an-sre-interview-i-have-) -* [⇢ ⇢ ⇢ Small introduction to the `#Android` ...](#small-introduction-to-the-android-) -* [⇢ ⇢ ⇢ Helix 2025.01 has been released. The completion ...](#helix-202501-has-been-released-the-completion-) -* [⇢ ⇢ ⇢ I found these are excellent examples of how ...](#i-found-these-are-excellent-examples-of-how-) -* [⇢ ⇢ ⇢ LLMs for Ops? Summaries of logs, probabilities ...](#llms-for-ops-summaries-of-logs-probabilities-) -* [⇢ ⇢ ⇢ Enjoying an APC Power-UPS BX750MI in my ...](#enjoying-an-apc-power-ups-bx750mi-in-my-) -* [⇢ ⇢ ⇢ "Even in the projects where I'm the only ...](#even-in-the-projects-where-i-m-the-only-) -* [⇢ ⇢ ⇢ Connecting an `#UPS` to my `#FreeBSD` cluster ...](#connecting-an-ups-to-my-freebsd-cluster-) -* [⇢ ⇢ ⇢ So, the Co-founder and CTO of honeycomb.io and ...](#so-the-co-founder-and-cto-of-honeycombio-and-) -* [⇢ ⇢ February 2025](#february-2025) -* [⇢ ⇢ ⇢ I don't know about you, but at work, I usually ...](#i-don-t-know-about-you-but-at-work-i-usually-) -* [⇢ ⇢ ⇢ Great proposal (got accepted by the Goteam) for ...](#great-proposal-got-accepted-by-the-goteam-for-) -* [⇢ ⇢ ⇢ My Gemtexter has only 1320 LOC.... The Biggest ...](#my-gemtexter-has-only-1320-loc-the-biggest-) -* [⇢ ⇢ ⇢ Against /tmp - He is making a point `#unix` ...](#against-tmp---he-is-making-a-point-unix-) -* [⇢ ⇢ ⇢ Random Weird Things Part 2: `#blog` ...](#random-weird-things-part-2-blog-) -* [⇢ ⇢ ⇢ As a former `#Pebble` user and fan, thats ...](#as-a-former-pebble-user-and-fan-thats-) -* [⇢ ⇢ ⇢ I think I am slowly getting the point of Cue. ...](#i-think-i-am-slowly-getting-the-point-of-cue-) -* [⇢ ⇢ ⇢ Jonathan's reflection of 10 years of ...](#jonathan-s-reflection-of-10-years-of-) -* [⇢ ⇢ ⇢ Really enjoyed reading this. Easily digestible ...](#really-enjoyed-reading-this-easily-digestible-) -* [⇢ ⇢ ⇢ Some great advice from 40 years of experience ...](#some-great-advice-from-40-years-of-experience-) -* [⇢ ⇢ ⇢ I enjoyed t