From 638aa9523d57c7071d311909ef805bdda80098b3 Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Tue, 1 Jul 2025 22:48:14 +0300 Subject: new post --- about/resources.gmi | 192 ++--- ...5-01-01-posts-from-october-to-december-2024.gmi | 1 + .../DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi | 182 ++++ ...RAFT-f3s-kubernetes-with-freebsd-part-6.gmi.tpl | 230 +++++ gemfeed/atom.xml | 941 +++++++++++++++++---- gemfeed/index.gmi | 1 + index.gmi | 3 +- uptime-stats.gmi | 54 +- 8 files changed, 1294 insertions(+), 310 deletions(-) diff --git a/about/resources.gmi b/about/resources.gmi index 2334d0d8..f2a10981 100644 --- a/about/resources.gmi +++ b/about/resources.gmi @@ -35,105 +35,105 @@ You won't find any links on this site because, over time, the links will break. In random order: -* The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional -* Systemprogrammierung in Go; Frank Müller; dpunkt -* 97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly -* The Kubernetes Book; Nigel Poulton; Unabridged Audiobook -* Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly -* The Pragmatic Programmer; David Thomas; Addison-Wesley -* Pro Puppet; James Turnbull, Jeffrey McCune; Apress -* Modern Perl; Chromatic ; Onyx Neon Press -* Hands-on Infrastructure Monitoring with Prometheus; Joel Bastos, Pedro Araujo; Packt -* Higher Order Perl; Mark Dominus; Morgan Kaufmann +* Polished Ruby Programming; Jeremy Evans; Packt Publishing +* Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press +* Leanring eBPF; Liz Rice; O'Reilly +* Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers * Perl New Features; Joshua McAdams, brian d foy; Perl School -* Site Reliability Engineering; How Google runs production systems; O'Reilly -* Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press * Object-Oriented Programming with ANSI-C; Axel-Tobias Schreiner -* Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf +* The Kubernetes Book; Nigel Poulton; Unabridged Audiobook +* Java ist auch eine Insel; Christian Ullenboom; +* DNS and BIND; Cricket Liu; O'Reilly +* Effective awk programming; Arnold Robbins; O'Reilly +* Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press +* Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers +* Ultimate Go Notebook; Bill Kennedy * The Docker Book; James Turnbull; Kindle -* Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly -* Systems Performance Tuning; Gian-Paolo D. Musumeci and others...; O'Reilly -* Clusterbau mit Linux-HA; Michael Schwartzkopff; O'Reilly -* 100 Go Mistakes and How to Avoid Them; Teiva Harsanyi; Manning Publications +* The Pragmatic Programmer; David Thomas; Addison-Wesley +* The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional +* Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson +* Concurrency in Go; Katherine Cox-Buday; O'Reilly * 21st Century C: C Tips from the New School; Ben Klemens; O'Reilly +* The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis; Audible +* Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly +* 97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly * Terraform Cookbook; Mikael Krief; Packt Publishing -* Raku Recipes; J.J. Merelo; Apress -* Funktionale Programmierung; Peter Pepper; Springer * Raku Fundamentals; Moritz Lenz; Apress -* Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press -* Ultimate Go Notebook; Bill Kennedy -* Data Science at the Command Line; Jeroen Janssens; O'Reilly -* Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly -* Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers -* Effective awk programming; Arnold Robbins; O'Reilly -* The Practise of System and Network Administration; Thomas A. Limoncelli, Christina J. Hogan, Strata R. Chalup; Addison-Wesley Professional Pro Git; Scott Chacon, Ben Straub; Apress -* The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton -* DNS and BIND; Cricket Liu; O'Reilly -* Java ist auch eine Insel; Christian Ullenboom; -* Polished Ruby Programming; Jeremy Evans; Packt Publishing * Amazon Web Services in Action; Michael Wittig and Andreas Wittig; Manning Publications -* The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis; Audible -* DevOps And Site Reliability Engineering Handbook; Stephen Fleming; Audible +* Pro Puppet; James Turnbull, Jeffrey McCune; Apress +* Funktionale Programmierung; Peter Pepper; Springer +* Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly +* Modern Perl; Chromatic ; Onyx Neon Press * Developing Games in Java; David Brackeen and others...; New Riders +* Systemprogrammierung in Go; Frank Müller; dpunkt +* 100 Go Mistakes and How to Avoid Them; Teiva Harsanyi; Manning Publications +* Data Science at the Command Line; Jeroen Janssens; O'Reilly +* Raku Recipes; J.J. Merelo; Apress +* Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly * C++ Programming Language; Bjarne Stroustrup; +* Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf +* Higher Order Perl; Mark Dominus; Morgan Kaufmann * Effective Java; Joshua Bloch; Addison-Wesley Professional -* Concurrency in Go; Katherine Cox-Buday; O'Reilly -* Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson -* Leanring eBPF; Liz Rice; O'Reilly -* Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers +* Hands-on Infrastructure Monitoring with Prometheus; Joel Bastos, Pedro Araujo; Packt +* Systems Performance Tuning; Gian-Paolo D. Musumeci and others...; O'Reilly +* Site Reliability Engineering; How Google runs production systems; O'Reilly +* The Practise of System and Network Administration; Thomas A. Limoncelli, Christina J. Hogan, Strata R. Chalup; Addison-Wesley Professional Pro Git; Scott Chacon, Ben Straub; Apress +* DevOps And Site Reliability Engineering Handbook; Stephen Fleming; Audible +* The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton +* Clusterbau mit Linux-HA; Michael Schwartzkopff; O'Reilly ## Technical references I didn't read them from the beginning to the end, but I am using them to look up things. The books are in random order: -* Relayd and Httpd Mastery; Michael W Lucas -* BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley +* Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley * Implementing Service Level Objectives; Alex Hidalgo; O'Reilly -* Groovy Kurz & Gut; Joerg Staudemeier; O'Reilly * Go: Design Patterns for Real-World Projects; Mat Ryer; Packt * Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly +* Groovy Kurz & Gut; Joerg Staudemeier; O'Reilly * The Linux Programming Interface; Michael Kerrisk; No Starch Press -* Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley +* Relayd and Httpd Mastery; Michael W Lucas +* BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley ## Self-development and soft-skills books In random order: -* Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne -* Psycho-Cybernetics; Maxwell Maltz; Perigee Books -* 101 Essays that change the way you think; Brianna Wiest; Audiobook -* Solve for Happy; Mo Gawdat (RE-READ 1ST TIME) -* Eat That Frog!; Brian Tracy; Hodder Paperbacks -* Influence without Authority; A. Cohen, D. Bradford; Wiley -* The 7 Habits Of Highly Effective People; Stephen R. Covey; Simon & Schuster UK -* Eat That Frog; Brian Tracy -* So Good They Can't Ignore You; Cal Newport; Business Plus +* Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion +* Deep Work; Cal Newport; Piatkus * Coders at Work - Reflections on the craft of programming, Peter Seibel and Mitchell Dorian et al., Audiobook -* Atomic Habits; James Clear; Random House Business -* The Joy of Missing Out; Christina Crook; New Society Publishers -* The Good Enough Job; Simone Stolzoff; Ebury Edge +* Getting Things Done; David Allen +* Slow Productivity; Cal Newport; Penguin Random House +* Meditation for Mortals, Oliver Burkeman, Audiobook +* Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook +* The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd +* Solve for Happy; Mo Gawdat (RE-READ 1ST TIME) +* Never Split the Difference; Chris Voss, Tahl Raz; Random House Business * Stop starting, start finishing; Arne Roock; Lean-Kanban University -* Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press +* Soft Skills; John Sommez; Manning Publications +* Ultralearning; Anna Laurent; Self-published via Amazon +* The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books +* The Joy of Missing Out; Christina Crook; New Society Publishers * The Complete Software Developer's Career Guide; John Sonmez; Unabridged Audiobook -* Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook -* Meditation for Mortals, Oliver Burkeman, Audiobook -* Ultralearning; Scott Young; Thorsons * Digital Minimalism; Cal Newport; Portofolio Penguin +* The Bullet Journal Method; Ryder Carroll; Fourth Estate +* So Good They Can't Ignore You; Cal Newport; Business Plus +* Ultralearning; Scott Young; Thorsons +* Influence without Authority; A. Cohen, D. Bradford; Wiley +* Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne * The Off Switch; Mark Cropley; Virgin Books (RE-READ 1ST TIME) -* Never Split the Difference; Chris Voss, Tahl Raz; Random House Business -* Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion +* Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press +* The Good Enough Job; Simone Stolzoff; Ebury Edge +* Psycho-Cybernetics; Maxwell Maltz; Perigee Books +* The 7 Habits Of Highly Effective People; Stephen R. Covey; Simon & Schuster UK +* The Power of Now; Eckhard Tolle; Yellow Kite +* Eat That Frog; Brian Tracy +* Buddah and Einstein walk into a Bar; Guy Joseph Ale, Claire Bloom; Blackstone Publishing +* 101 Essays that change the way you think; Brianna Wiest; Audiobook +* Eat That Frog!; Brian Tracy; Hodder Paperbacks * Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly * The Phoenix Project - A Novel About IT, DevOps, and Helping your Business Win; Gene Kim and Kevin Behr; Trade Select -* The Bullet Journal Method; Ryder Carroll; Fourth Estate -* The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd -* Ultralearning; Anna Laurent; Self-published via Amazon -* Soft Skills; John Sommez; Manning Publications -* Buddah and Einstein walk into a Bar; Guy Joseph Ale, Claire Bloom; Blackstone Publishing -* Getting Things Done; David Allen -* Slow Productivity; Cal Newport; Penguin Random House -* The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books -* Deep Work; Cal Newport; Piatkus -* The Power of Now; Eckhard Tolle; Yellow Kite +* Atomic Habits; James Clear; Random House Business => ../notes/index.gmi Here are notes of mine for some of the books @@ -141,30 +141,30 @@ In random order: Some of these were in-person with exams; others were online learning lectures only. In random order: +* AWS Immersion Day; Amazon; 1-day interactive online training +* Ultimate Go Programming; Bill Kennedy; O'Reilly Online +* The Well-Grounded Rubyist Video Edition; David. A. Black; O'Reilly Online * The Ultimate Kubernetes Bootcamp; School of Devops; O'Reilly Online -* Functional programming lecture; Remote University of Hagen -* Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online +* Cloud Operations on AWS - Learn how to configure, deploy, maintain, and troubleshoot your AWS environments; 3-day online live training with labs; Amazon * Apache Tomcat Best Practises; 3-day on-site training -* Ultimate Go Programming; Bill Kennedy; O'Reilly Online -* Protocol buffers; O'Reilly Online * Structure and Interpretation of Computer Programs; Harold Abelson and more...; -* F5 Loadbalancers Training; 2-day on-site training; F5, Inc. * Scripting Vim; Damian Conway; O'Reilly Online +* Functional programming lecture; Remote University of Hagen +* F5 Loadbalancers Training; 2-day on-site training; F5, Inc. +* Protocol buffers; O'Reilly Online +* Developing IaC with Terraform (with Live Lessons); O'Reilly Online +* Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online * Linux Security and Isolation APIs Training; Michael Kerrisk; 3-day on-site training -* Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need) * MySQL Deep Dive Workshop; 2-day on-site training -* Developing IaC with Terraform (with Live Lessons); O'Reilly Online -* Cloud Operations on AWS - Learn how to configure, deploy, maintain, and troubleshoot your AWS environments; 3-day online live training with labs; Amazon -* The Well-Grounded Rubyist Video Edition; David. A. Black; O'Reilly Online -* AWS Immersion Day; Amazon; 1-day interactive online training +* Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need) ## Technical guides These are not whole books, but guides (smaller or larger) which I found very useful. in random order: * How CPUs work at https://cpu.land -* Raku Guide at https://raku.guide * Advanced Bash-Scripting Guide +* Raku Guide at https://raku.guide ## Podcasts @@ -172,57 +172,57 @@ These are not whole books, but guides (smaller or larger) which I found very use In random order: -* Dev Interrupted * Backend Banter -* The Changelog Podcast(s) -* Deep Questions with Cal Newport * Hidden Brain -* Fork Around And Find Out -* Cup o' Go [Golang] +* Fallthrough [Golang] * The Pragmatic Engineer Podcast +* Fork Around And Find Out * BSD Now [BSD] -* The ProdCast (Google SRE Podcast) +* Deep Questions with Cal Newport * Maintainable -* Fallthrough [Golang] -* Modern Mentor * Pratical AI +* The ProdCast (Google SRE Podcast) +* Cup o' Go [Golang] +* Dev Interrupted +* The Changelog Podcast(s) +* Modern Mentor ### Podcasts I liked I liked them but am not listening to them anymore. The podcasts have either "finished" (no more episodes) or I stopped listening to them due to time constraints or a shift in my interests. * Go Time (predecessor of fallthrough) +* Java Pub House * Ship It (predecessor of Fork Around And Find Out) * CRE: Chaosradio Express [german] * FLOSS weekly -* Java Pub House * Modern Mentor ## Newsletters I like This is a mix of tech and non-tech newsletters I am subscribed to. In random order: -* Changelog News -* VK Newsletter -* Applied Go Weekly Newsletter -* Andreas Brandhorst Newsletter (Sci-Fi author) -* Register Spill -* Monospace Mentor * Ruby Weekly * The Valuable Dev -* The Pragmatic Engineer +* VK Newsletter +* Applied Go Weekly Newsletter * The Imperfectionist * byteSizeGo +* The Pragmatic Engineer +* Changelog News +* Register Spill +* Andreas Brandhorst Newsletter (Sci-Fi author) +* Monospace Mentor * Golang Weekly ## Magazines I like(d) This is a mix of tech I like(d). I may not be a current subscriber, but now and then, I buy an issue. In random order: -* LWN (online only) * Linux User * freeX (not published anymore) * Linux Magazine +* LWN (online only) # Formal education diff --git a/gemfeed/2025-01-01-posts-from-october-to-december-2024.gmi b/gemfeed/2025-01-01-posts-from-october-to-december-2024.gmi index 2183a7bf..d59f5981 100644 --- a/gemfeed/2025-01-01-posts-from-october-to-december-2024.gmi +++ b/gemfeed/2025-01-01-posts-from-october-to-december-2024.gmi @@ -327,6 +327,7 @@ My New Year's resolution is not to start any new non-fiction books (or only very Other related posts: +=> ./2025-07-01-posts-from-january-to-june-2025.gmi 2025-07-01 Posts from January to June 2025 => ./2025-01-01-posts-from-october-to-december-2024.gmi 2025-01-01 Posts from October to December 2024 (You are currently reading this) E-Mail your comments to `paul@nospam.buetow.org` :-) diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi index 039ba6b3..6e3f9395 100644 --- a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi +++ b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi @@ -22,6 +22,15 @@ This is the sixth blog post about the f3s series for self-hosting demands in a h * ⇢ ⇢ ⇢ Configuring `zdata` ZFS pool and encryption * ⇢ ⇢ ⇢ Migrating Bhyve VMs to encrypted `bhyve` ZFS volume * ⇢ ⇢ CARP +* ⇢ ⇢ ZFS Replication with zrepl +* ⇢ ⇢ ⇢ Installing zrepl +* ⇢ ⇢ ⇢ Checking ZFS pools +* ⇢ ⇢ ⇢ Configuring zrepl with WireGuard tunnel +* ⇢ ⇢ ⇢ Configuring zrepl on f0 (source) +* ⇢ ⇢ ⇢ Configuring zrepl on f1 (sink) +* ⇢ ⇢ ⇢ Enabling and starting zrepl services +* ⇢ ⇢ ⇢ Verifying replication +* ⇢ ⇢ ⇢ Monitoring replication ## Introduction @@ -189,6 +198,179 @@ next, copied that script /usr/local/bin/carpcontrol.sh and adjusted the disk to reboot or run doas kldload carp0 +## ZFS Replication with zrepl + +In this section, we'll set up automatic ZFS replication from f0 to f1 using zrepl. This ensures our data is replicated across nodes for redundancy. + +### Installing zrepl + +First, install zrepl on both hosts: + +```sh +# On f0 +paul@f0:~ % doas pkg install -y zrepl + +# On f1 +paul@f1:~ % doas pkg install -y zrepl +``` + +### Checking ZFS pools + +Verify the pools and datasets on both hosts: + +```sh +# On f0 +paul@f0:~ % doas zpool list +NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +zdata 928G 1.03M 928G - - 0% 0% 1.00x ONLINE - +zroot 472G 26.7G 445G - - 0% 5% 1.00x ONLINE - + +paul@f0:~ % doas zfs list -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc 200K 899G 200K /data/enc + +# On f1 +paul@f1:~ % doas zpool list +NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +zdata 928G 956K 928G - - 0% 0% 1.00x ONLINE - +zroot 472G 11.7G 460G - - 0% 2% 1.00x ONLINE - + +paul@f1:~ % doas zfs list -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc 200K 899G 200K /data/enc +``` + +### Configuring zrepl with WireGuard tunnel + +Since we have a WireGuard tunnel between f0 and f1, we'll use TCP transport over the secure tunnel instead of SSH. First, check the WireGuard IP addresses: + +```sh +# Check WireGuard interface IPs +paul@f0:~ % ifconfig wg0 | grep inet + inet 192.168.2.130 netmask 0xffffff00 + +paul@f1:~ % ifconfig wg0 | grep inet + inet 192.168.2.131 netmask 0xffffff00 +``` + +### Configuring zrepl on f0 (source) + +Create the zrepl configuration on f0: + +```sh +paul@f0:~ % doas tee /usr/local/etc/zrepl/zrepl.yml <<'EOF' +global: + logging: + - type: stdout + level: info + format: human + +jobs: + - name: "f0_to_f1" + type: push + connect: + type: tcp + address: "192.168.2.131:8888" + filesystems: { + "zdata/enc": true + } + send: + encrypted: true + snapshotting: + type: periodic + prefix: zrepl_ + interval: 10m + pruning: + keep_sender: + - type: last_n + count: 10 + - type: grid + grid: 1x1h(keep=all) | 24x1h | 7x1d | 4x7d | 6x30d + regex: "^zrepl_.*" + keep_receiver: + - type: grid + grid: 1x1h(keep=all) | 24x1h | 7x1d | 4x7d | 6x30d + regex: "^zrepl_.*" +EOF +``` + +### Configuring zrepl on f1 (sink) + +Create the zrepl configuration on f1: + +```sh +paul@f1:~ % doas tee /usr/local/etc/zrepl/zrepl.yml <<'EOF' +global: + logging: + - type: stdout + level: info + format: human + +jobs: + - name: "sink" + type: sink + serve: + type: tcp + listen: "192.168.2.131:8888" + clients: + "192.168.2.130": "f0" + recv: + placeholder: + encryption: inherit + root_fs: "zdata/enc" +EOF +``` + +### Enabling and starting zrepl services + +Enable and start zrepl on both hosts: + +```sh +# On f0 +paul@f0:~ % doas sysrc zrepl_enable=YES +zrepl_enable: -> YES +paul@f0:~ % doas service zrepl start +Starting zrepl. + +# On f1 +paul@f1:~ % doas sysrc zrepl_enable=YES +zrepl_enable: -> YES +paul@f1:~ % doas service zrepl start +Starting zrepl. +``` + +### Verifying replication + +Check the replication status: + +```sh +# On f0, check zrepl status +paul@f0:~ % doas zrepl status + +# Check for zrepl snapshots +paul@f0:~ % doas zfs list -t snapshot -r zdata/enc | grep zrepl + +# On f1, verify the replicated datasets +paul@f1:~ % doas zfs list -r zdata/enc + +# Check zrepl logs for any errors +paul@f0:~ % doas tail -f /var/log/zrepl.log +``` + +### Monitoring replication + +You can monitor the replication progress with: + +```sh +# Real-time status +paul@f0:~ % doas zrepl status --mode interactive + +# Check specific job details +paul@f0:~ % doas zrepl status --job f0_to_f1 +``` + +With this setup, zdata/enc on f0 will be automatically replicated to f1 every 10 minutes, with encrypted snapshots preserved on both sides. The pruning policy ensures that we keep recent snapshots while managing disk space efficiently. + ZFS auto scrubbing....~? Backup of the keys on the key locations (all keys on all 3 USB keys) diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi.tpl b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi.tpl index a2b7ea5f..64fa1cf1 100644 --- a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi.tpl +++ b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-6.gmi.tpl @@ -176,6 +176,236 @@ next, copied that script /usr/local/bin/carpcontrol.sh and adjusted the disk to reboot or run doas kldload carp0 +## ZFS Replication with zrepl + +In this section, we'll set up automatic ZFS replication from f0 to f1 using zrepl. This ensures our data is replicated across nodes for redundancy. + +### Installing zrepl + +First, install zrepl on both hosts: + +```sh +# On f0 +paul@f0:~ % doas pkg install -y zrepl + +# On f1 +paul@f1:~ % doas pkg install -y zrepl +``` + +### Checking ZFS pools + +Verify the pools and datasets on both hosts: + +```sh +# On f0 +paul@f0:~ % doas zpool list +NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +zdata 928G 1.03M 928G - - 0% 0% 1.00x ONLINE - +zroot 472G 26.7G 445G - - 0% 5% 1.00x ONLINE - + +paul@f0:~ % doas zfs list -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc 200K 899G 200K /data/enc + +# On f1 +paul@f1:~ % doas zpool list +NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +zdata 928G 956K 928G - - 0% 0% 1.00x ONLINE - +zroot 472G 11.7G 460G - - 0% 2% 1.00x ONLINE - + +paul@f1:~ % doas zfs list -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc 200K 899G 200K /data/enc +``` + +### Configuring zrepl with WireGuard tunnel + +Since we have a WireGuard tunnel between f0 and f1, we'll use TCP transport over the secure tunnel instead of SSH. First, check the WireGuard IP addresses: + +```sh +# Check WireGuard interface IPs +paul@f0:~ % ifconfig wg0 | grep inet + inet 192.168.2.130 netmask 0xffffff00 + +paul@f1:~ % ifconfig wg0 | grep inet + inet 192.168.2.131 netmask 0xffffff00 +``` + +### Configuring zrepl on f0 (source) + +Create the zrepl configuration on f0: + +```sh +paul@f0:~ % doas tee /usr/local/etc/zrepl/zrepl.yml <<'EOF' +global: + logging: + - type: stdout + level: info + format: human + +jobs: + - name: f0_to_f1 + type: push + connect: + type: tcp + address: "192.168.2.131:8888" + filesystems: + "zdata/enc": true + send: + encrypted: true + snapshotting: + type: periodic + prefix: zrepl_ + interval: 10m + pruning: + keep_sender: + - type: last_n + count: 10 + keep_receiver: + - type: last_n + count: 10 +EOF +``` + +### Configuring zrepl on f1 (sink) + +Create the zrepl configuration on f1: + +```sh +paul@f1:~ % doas tee /usr/local/etc/zrepl/zrepl.yml <<'EOF' +global: + logging: + - type: stdout + level: info + format: human + +jobs: + - name: "sink" + type: sink + serve: + type: tcp + listen: "192.168.2.131:8888" + clients: + "192.168.2.130": "f0" + recv: + placeholder: + encryption: inherit + root_fs: "zdata/enc" +EOF +``` + +### Enabling and starting zrepl services + +Enable and start zrepl on both hosts: + +```sh +# On f0 +paul@f0:~ % doas sysrc zrepl_enable=YES +zrepl_enable: -> YES +paul@f0:~ % doas service zrepl start +Starting zrepl. + +# On f1 +paul@f1:~ % doas sysrc zrepl_enable=YES +zrepl_enable: -> YES +paul@f1:~ % doas service zrepl start +Starting zrepl. +``` + +### Verifying replication + +Check the replication status: + +```sh +# On f0, check zrepl status (use raw mode for non-tty) +paul@f0:~ % doas zrepl status --mode raw | grep -A2 "Replication" +"Replication":{"StartAt":"2025-07-01T22:31:48.712143123+03:00"... + +# Check if services are running +paul@f0:~ % doas service zrepl status +zrepl is running as pid 2649. + +paul@f1:~ % doas service zrepl status +zrepl is running as pid 2574. + +# Check for zrepl snapshots on source +paul@f0:~ % doas zfs list -t snapshot -r zdata/enc | grep zrepl +zdata/enc@zrepl_20250701_193148_000 0B - 176K - + +# On f1, verify the replicated datasets +paul@f1:~ % doas zfs list -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc 776K 899G 200K /data/enc +zdata/enc/f0 576K 899G 200K none +zdata/enc/f0/zdata 376K 899G 200K none +zdata/enc/f0/zdata/enc 176K 899G 176K none + +# Check replicated snapshots on f1 +paul@f1:~ % doas zfs list -t snapshot -r zdata/enc +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc/f0/zdata/enc@zrepl_20250701_193148_000 0B - 176K - +``` + +### Monitoring replication + +You can monitor the replication progress with: + +```sh +# Real-time status +paul@f0:~ % doas zrepl status --mode interactive + +# Check specific job details +paul@f0:~ % doas zrepl status --job f0_to_f1 +``` + +With this setup, zdata/enc on f0 will be automatically replicated to f1 every 10 minutes, with encrypted snapshots preserved on both sides. The pruning policy ensures that we keep the last 10 snapshots while managing disk space efficiently. + +The replicated data appears on f1 under `zdata/enc/f0/zdata/enc` to maintain the source dataset hierarchy. The replication uses the WireGuard tunnel for secure, encrypted transport between nodes. + +### Quick status check commands + +Here are the essential commands to monitor replication status: + +```sh +# On the source node (f0) - check if replication is active +paul@f0:~ % doas zrepl status --job f0_to_f1 | grep -E '(State|Last)' +State: done +LastError: + +# List all zrepl snapshots on source +paul@f0:~ % doas zfs list -t snapshot -r zdata/enc | grep zrepl +zdata/enc@zrepl_20250701_193148_000 0B - 176K - +zdata/enc@zrepl_20250701_194148_000 0B - 176K - + +# On the sink node (f1) - verify received datasets +paul@f1:~ % doas zfs list -r zdata/enc/f0 +NAME USED AVAIL REFER MOUNTPOINT +zdata/enc/f0 576K 899G 200K none +zdata/enc/f0/zdata 376K 899G 200K none +zdata/enc/f0/zdata/enc 176K 899G 176K none + +# Check received snapshots on sink +paul@f1:~ % doas zfs list -t snapshot -r zdata/enc | wc -l + 2 + +# Monitor replication progress in real-time (on source) +paul@f0:~ % doas zrepl status --mode interactive + +# Check last replication time (on source) +paul@f0:~ % doas zrepl status --job f0_to_f1 | grep -A1 "Replication" +Replication: + Status: Idle (last run: 2025-07-01T22:41:48) + +# View zrepl logs for troubleshooting +paul@f0:~ % doas tail -20 /var/log/zrepl.log | grep -E '(error|warn|replication)' +``` + +These commands provide a quick way to verify that: +- Replication jobs are running without errors +- Snapshots are being created on the source +- Data is being received on the sink +- The replication schedule is being followed + ZFS auto scrubbing....~? Backup of the keys on the key locations (all keys on all 3 USB keys) diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index 7d3fd79e..55a54ef4 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,11 +1,764 @@ - 2025-06-29T13:29:59+03:00 + 2025-07-01T22:39:30+03:00 foo.zone feed To be in the .zone! gemini://foo.zone/ + + Posts from January to June 2025 + + gemini://foo.zone/gemfeed/2025-07-01-posts-from-january-to-june-2025.gmi + 2025-07-01T22:39:29+03:00 + + Paul Buetow aka snonux + paul@dev.buetow.org + + These are my social media posts from the last six months. I keep them here to reflect on them and also to not lose them. Social media networks come and go and are not under my control, but my domain is here to stay. + +
+

Posts from January to June 2025


+
+These are my social media posts from the last six months. I keep them here to reflect on them and also to not lose them. Social media networks come and go and are not under my control, but my domain is here to stay.
+
+These are from Mastodon and LinkedIn. Have a look at my about page for my social media profiles. This list is generated with Gos, my social media platform sharing tool.
+
+My about page
+https://codeberg.org/snonux/gos
+
+

Table of Contents


+
+
+

January 2025


+
+

I am currently binge-listening to the Google ...


+
+I am currently binge-listening to the Google #SRE ProdCast. It's really great to learn about the stories of individual SREs and their journeys. It is not just about SREs at Google; there are also external guests.
+
+sre.google/prodcast/
+
+

Recently, there was a >5000 LOC #bash ...


+
+Recently, there was a >5000 LOC #bash codebase at work that reported the progress of a migration, nobody understood it and it was wonky (sometimes it would not return the desired results). On top of that, the coding style was very bad as well (I could rant forever here). The engineer who wrote it left the company. I rewrote it in #Perl in about 300 LOC. Colleagues asked why not Python. Perl is the perfect choice here—it's even in its name: Practical Extraction and Report Language!
+
+

Ghostty is a terminal emulator that was ...


+
+Ghostty is a terminal emulator that was recently released publicly as open-source. I love that it works natively on both Linux and macOS; it looks great (font rendering) and is fast and customizable via a config file (which I manage with a config mng system). Ghostty is a passion project written in Zig, the author loved the community so much while working on it that he donated $300k to the Zig Foundation. #terminal #emulator
+
+ghostty.org
+
+

Go is not an easy programming language. Don't ...


+
+Go is not an easy programming language. Don't confuse easy with simple syntax. I'd agree to this. With the recent addition of Generics to the language I also feel that even the syntax stops being simple.. Also, simplicity is complex (especially under the hood how the language works - there are many mechanics you need to know if you really want to master the language). #golang
+
+www.arp242.net/go-easy.html
+
+

How will AI change software engineering (or has ...


+
+How will AI change software engineering (or has it already)? The bottom line is that less experienced engineers may have problems (accepting incomplete or incorrect programs, only reaching 70 percent solutions), while experienced engineers can leverage AI to boost their performance as they know how to fix the remaining 30 percent of the generated code. #ai #engineering #software
+
+newsletter.pragmaticengineer.com/p/how-ai-will-change-software-engineering
+
+

Eliminating toil - Toil is not always a bad ...


+
+Eliminating toil - Toil is not always a bad thing - some even enjoy toil - it is calming in small amounts - but it becomes toxic in large amounts - #SRE
+
+sre.google/sre-book/eliminating-toil/
+
+

Fun read. How about using the character ...


+
+Fun read. How about using the character sequence :-) as a statement separator in a programming language?
+
+ntietz.com/blog/researching-why-we-use-semicolons-as-statement-terminators/
+
+

Thats unexpected, you cant remove a NaN key ...


+
+Thats unexpected, you cant remove a NaN key from a map without clearing it! #golang via @wallabagapp
+
+unexpected-go.com/you-cant-remove-a-nan-key-from-a-map-without-clearing-it.html
+
+

Nice refresher for #shell #bash #zsh ...


+
+Nice refresher for #shell #bash #zsh redirection rules
+
+rednafi.com/misc/shell_redirection/
+
+

I think discussing action items in incident ...


+
+I think discussing action items in incident reviews is important. At least the obvious should be captured and noted down. It does not mean that the action items need to be fully refined in the review meeting; that would be out of scope, in my opinion.
+
+surfingcomplexity.blog/2024/09/28/why-..-..-action-items-during-incident-reviews/
+
+

At first, functional options add a bit of ...


+
+At first, functional options add a bit of boilerplate, but they turn out to be quite neat, especially when you have very long parameter lists that need to be made neat and tidy. #golang
+
+www.calhoun.io/using-functional-options-instead-of-method-chaining-in-go/
+
+

In the "Working with an SRE Interview" I have ...


+
+In the "Working with an SRE Interview" I have been askd about what it's like working with an SRE! We'd covered much more in depth, but we decided not to make it too long in the final version! #sre #interview
+
+foo.zone/gemfeed/2025-01-15-working-with-an-sre-interview.gmi (Gemini)
+foo.zone/gemfeed/2025-01-15-working-with-an-sre-interview.html
+
+

In the "Working with an SRE Interview" I have ...


+
+In the "Working with an SRE Interview" I have been askd about what it's like working with an SRE! We'd covered much more in depth, but we decided not to make it too long in the final version! #sre #interview
+
+foo.zone/gemfeed/2025-01-15-working-with-an-sre-interview.gmi (Gemini)
+foo.zone/gemfeed/2025-01-15-working-with-an-sre-interview.html
+
+

Small introduction to the #Android ...


+
+Small introduction to the #Android distribution called #GrapheneOS For myself, I am using a Pixel 7 Pro, which comes with "only" 5 years of support (not yet 7 years like the Pixel 8 and 9 series). I also wrote about GrapheneOS here once:
+
+dataswamp.org/~solene/2025-01-12-intro-to-grapheneos.html
+foo.zone/gemfeed/2023-01-23-why-grapheneos-rox.gmi (Gemini)
+foo.zone/gemfeed/2023-01-23-why-grapheneos-rox.html
+
+

Helix 2025.01 has been released. The completion ...


+
+Helix 2025.01 has been released. The completion of path names and the snippet functionality will be particularly useful for me. Overall, it's a great release. The release notes cover only some highlights, but there are many more changes in this version so also have a look at the Changelog! #HelixEditor
+
+helix-editor.com/news/release-25-01-highlights/
+
+

I found these are excellent examples of how ...


+
+I found these are excellent examples of how #OpenBSD's #relayd can be used.
+
+www.tumfatig.net/2023/using-openbsd-relayd8-as-an-application-layer-gateway/
+
+

LLMs for Ops? Summaries of logs, probabilities ...


+
+LLMs for Ops? Summaries of logs, probabilities about correctness, auto-generating Ansible, some uses cases are there. Wouldn't trust it fully, though.
+
+youtu.be/WodaffxVq-E?si=noY0egrfl5izCSQI
+
+

Enjoying an APC Power-UPS BX750MI in my ...


+
+Enjoying an APC Power-UPS BX750MI in my #homelab with #FreeBSD and apcupsd. I can easily use the UPS status to auto-shutdown a cluster of FreeBSD machines on a power cut. One FreeBSD machine acts as the apcupsd master, connected via USB to the APC, while the remaining machines read the status remotely via the apcupsd network port from the master. However, it won't work when the master is down. #APC #UPS
+
+

"Even in the projects where I'm the only ...


+
+"Even in the projects where I'm the only person, there are at least three people involved: past me, present me, and future me." - Quote from #software #programming
+
+liw.fi/40/#index1h1
+
+

Connecting an #UPS to my #FreeBSD cluster ...


+
+Connecting an #UPS to my #FreeBSD cluster in my #homelab, protecting it from power cuts!
+
+foo.zone/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi (Gemini)
+foo.zone/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html
+
+

So, the Co-founder and CTO of honeycomb.io and ...


+
+So, the Co-founder and CTO of honeycomb.io and author of the book Observability Engineering always hated observability. And Distinguished Software Engineer and The Pragmatic Engineer host can't pronounce the word Observability. :-) No, jokes aside, I liked this podcast episode of The Pragmatic Engineer: Observability: the present and future, with Charity Majors #sre #observability
+
+newsletter.pragmaticengineer.com/p/observability-the-present-and-future
+
+

February 2025


+
+

I don't know about you, but at work, I usually ...


+
+I don't know about you, but at work, I usually deal with complex setups involving thousands of servers and work in a complex hybrid microservices-based environment (cloud and on-prem), where homelabbing (as simple as described in my blog post) is really relaxing and recreative. So, I was homelabbing a bit again, securing my #FreeBSD cluster from power cuts. #UPS #recreative
+
+foo.zone/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi (Gemini)
+foo.zone/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html
+
+

Great proposal (got accepted by the Goteam) for ...


+
+Great proposal (got accepted by the Goteam) for safer file system open functions #golang
+
+github.com/golang/go/issues/67002
+
+

My Gemtexter has only 1320 LOC.... The Biggest ...


+
+My Gemtexter has only 1320 LOC.... The Biggest Shell Programs in the World are huuuge... #shell #sh
+
+github.com/oils-for-unix/oils/wiki/The-Biggest-Shell-Programs-in-the-World
+
+

Against /tmp - He is making a point #unix ...


+
+Against /tmp - He is making a point #unix #linux #bsd #filesystem via @wallabagapp
+
+dotat.at/@/2024-10-22-tmp.html
+
+

Random Weird Things Part 2: #blog ...


+
+Random Weird Things Part 2: #blog #computing
+
+foo.zone/gemfeed/2025-02-08-random-weird-things-ii.gmi (Gemini)
+foo.zone/gemfeed/2025-02-08-random-weird-things-ii.html
+
+

As a former #Pebble user and fan, thats ...


+
+As a former #Pebble user and fan, thats aweaome news. PebbleOS is now open source and there will aoon be a new watch. I don't know about you, but I will be the first getting one :-) #foss
+
+ericmigi.com/blog/why-were-bringing-pebble-back
+
+

I think I am slowly getting the point of Cue. ...


+
+I think I am slowly getting the point of Cue. For example, it can replace both a JSON file and a JSON Schema. Furthermore, you can convert it from and into different formats (Cue, JSON, YAML, Go data types, ...), and you can nicely embed this into a Go project as well. #cue #cuelang #golang #configuration
+
+cuelang.org
+
+

Jonathan's reflection of 10 years of ...


+
+Jonathan's reflection of 10 years of programming!
+
+jonathan-frere.com/posts/10-years-of-programming/
+
+

Really enjoyed reading this. Easily digestible ...


+
+Really enjoyed reading this. Easily digestible summary of what's new in Go 1.24. #golang
+
+antonz.org/go-1-24/
+
+