add part 6

16 files changed, 3564 insertions, 298 deletions

diff --git a/about/resources.gmi b/about/resources.gmi
index 108eb000..36438473 100644
--- a/about/resources.gmi
+++ b/about/resources.gmi
@@ -35,105 +35,105 @@ You won't find any links on this site because, over time, the links will break.
 
 In random order:
 
-* Modern Perl; Chromatic ; Onyx Neon Press
-* Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers
-* Object-Oriented Programming with ANSI-C; Axel-Tobias Schreiner
-* Effective awk programming; Arnold Robbins; O'Reilly
-* 97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly
+* Raku Recipes; J.J. Merelo; Apress
+* Polished Ruby Programming; Jeremy Evans; Packt Publishing
+* Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly
+* Systems Performance Tuning; Gian-Paolo D. Musumeci and others...; O'Reilly
+* Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson
+* Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers 
+* Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press
+* Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly
 * DNS and BIND; Cricket Liu; O'Reilly
-* Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly
-* 21st Century C: C Tips from the New School; Ben Klemens; O'Reilly
-* Data Science at the Command Line; Jeroen Janssens; O'Reilly
+* Terraform Cookbook; Mikael Krief; Packt Publishing
 * Concurrency in Go; Katherine Cox-Buday; O'Reilly
-* Perl New Features; Joshua McAdams, brian d foy; Perl School
-* The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton
-* Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf
 * DevOps And Site Reliability Engineering Handbook; Stephen Fleming; Audible
-* Pro Puppet; James Turnbull, Jeffrey McCune; Apress
+* Object-Oriented Programming with ANSI-C; Axel-Tobias Schreiner
 * Raku Fundamentals; Moritz Lenz; Apress
 * Ultimate Go Notebook; Bill Kennedy
+* The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton
+* 97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly
+* Effective Java; Joshua Bloch; Addison-Wesley Professional
 * Hands-on Infrastructure Monitoring with Prometheus; Joel Bastos, Pedro Araujo; Packt 
-* Leanring eBPF; Liz Rice; O'Reilly
-* Higher Order Perl; Mark Dominus; Morgan Kaufmann
-* Systems Performance Tuning; Gian-Paolo D. Musumeci and others...; O'Reilly
-* Polished Ruby Programming; Jeremy Evans; Packt Publishing
-* Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press
-* Systemprogrammierung in Go; Frank Müller; dpunkt
-* The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis; Audible
-* The Docker Book; James Turnbull; Kindle
-* The Kubernetes Book; Nigel Poulton; Unabridged Audiobook
+* 21st Century C: C Tips from the New School; Ben Klemens; O'Reilly
+* The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional
+* Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers
+* Site Reliability Engineering; How Google runs production systems; O'Reilly
+* Developing Games in Java; David Brackeen and others...; New Riders
+* Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press
+* Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf
+* Pro Puppet; James Turnbull, Jeffrey McCune; Apress
 * The Pragmatic Programmer; David Thomas; Addison-Wesley
-* Funktionale Programmierung; Peter Pepper; Springer
-* Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly
+* Java ist auch eine Insel; Christian Ullenboom; 
+* Systemprogrammierung in Go; Frank Müller; dpunkt
 * Clusterbau mit Linux-HA; Michael Schwartzkopff; O'Reilly
-* Terraform Cookbook; Mikael Krief; Packt Publishing
-* Site Reliability Engineering; How Google runs production systems; O'Reilly
+* 100 Go Mistakes and How to Avoid Them; Teiva Harsanyi; Manning Publications
+* The Kubernetes Book; Nigel Poulton; Unabridged Audiobook
 * C++ Programming Language; Bjarne Stroustrup;
+* Effective awk programming; Arnold Robbins; O'Reilly
+* The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis; Audible
+* Higher Order Perl; Mark Dominus; Morgan Kaufmann
+* Modern Perl; Chromatic ; Onyx Neon Press
+* Leanring eBPF; Liz Rice; O'Reilly
+* The Docker Book; James Turnbull; Kindle
 * The Practise of System and Network Administration; Thomas A. Limoncelli, Christina J. Hogan, Strata R. Chalup; Addison-Wesley Professional Pro Git; Scott Chacon, Ben Straub; Apress
-* 100 Go Mistakes and How to Avoid Them; Teiva Harsanyi; Manning Publications
-* Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly
-* Raku Recipes; J.J. Merelo; Apress
-* Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press
-* Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson
-* Effective Java; Joshua Bloch; Addison-Wesley Professional
-* Java ist auch eine Insel; Christian Ullenboom; 
-* Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers 
-* Developing Games in Java; David Brackeen and others...; New Riders
-* The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional
+* Data Science at the Command Line; Jeroen Janssens; O'Reilly
+* Perl New Features; Joshua McAdams, brian d foy; Perl School
+* Funktionale Programmierung; Peter Pepper; Springer
 * Amazon Web Services in Action; Michael Wittig and Andreas Wittig; Manning Publications
+* Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly
 
 ## Technical references
 
 I didn't read them from the beginning to the end, but I am using them to look up things. The books are in random order:
 
-* BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley
 * Implementing Service Level Objectives; Alex Hidalgo; O'Reilly
+* Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly
+* Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley
 * The Linux Programming Interface; Michael Kerrisk; No Starch Press 
 * Go: Design Patterns for Real-World Projects; Mat Ryer; Packt
-* Groovy Kurz & Gut; Joerg Staudemeier; O'Reilly
 * Relayd and Httpd Mastery; Michael W Lucas
-* Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley
-* Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly
+* Groovy Kurz & Gut; Joerg Staudemeier; O'Reilly
+* BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley
 
 ## Self-development and soft-skills books
 
 In random order:
 
-* So Good They Can't Ignore You; Cal Newport; Business Plus
+* The Good Enough Job; Simone Stolzoff; Ebury Edge
+* Ultralearning; Scott Young; Thorsons
+* Never Split the Difference; Chris Voss, Tahl Raz; Random House Business
+* The Off Switch; Mark Cropley; Virgin Books (RE-READ 1ST TIME)
 * Stop starting, start finishing; Arne Roock; Lean-Kanban University  
-* Psycho-Cybernetics; Maxwell Maltz; Perigee Books
-* Influence without Authority; A. Cohen, D. Bradford; Wiley
+* Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook
+* Coders at Work - Reflections on the craft of programming, Peter Seibel and Mitchell Dorian et al., Audiobook
 * The Complete Software Developer's Career Guide; John Sonmez; Unabridged Audiobook
-* Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion
+* 101 Essays that change the way you think; Brianna Wiest; Audiobook
 * Ultralearning; Anna Laurent; Self-published via Amazon
-* Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook
 * Deep Work; Cal Newport; Piatkus
-* Meditation for Mortals, Oliver Burkeman, Audiobook
-* The Bullet Journal Method; Ryder Carroll; Fourth Estate
-* Coders at Work - Reflections on the craft of programming, Peter Seibel and Mitchell Dorian et al., Audiobook
-* Atomic Habits; James Clear; Random House Business
-* The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd
-* Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly
 * Soft Skills; John Sommez; Manning Publications
-* Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press
-* Getting Things Done; David Allen
-* Buddah and Einstein walk into a Bar; Guy Joseph Ale, Claire Bloom; Blackstone Publishing
 * Eat That Frog; Brian Tracy
-* The 7 Habits Of Highly Effective People; Stephen R. Covey; Simon & Schuster UK
-* The Good Enough Job; Simone Stolzoff; Ebury Edge
-* Digital Minimalism; Cal Newport; Portofolio Penguin
-* Eat That Frog!; Brian Tracy; Hodder Paperbacks
-* The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books
-* The Joy of Missing Out; Christina Crook; New Society Publishers
+* Getting Things Done; David Allen
+* So Good They Can't Ignore You; Cal Newport; Business Plus
+* Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne
+* Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly
 * The Power of Now; Eckhard Tolle; Yellow Kite
-* The Off Switch; Mark Cropley; Virgin Books (RE-READ 1ST TIME)
+* Atomic Habits; James Clear; Random House Business
+* The Joy of Missing Out; Christina Crook; New Society Publishers
 * Slow Productivity; Cal Newport; Penguin Random House
+* Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press
+* The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd
+* Meditation for Mortals, Oliver Burkeman, Audiobook
+* Buddah and Einstein walk into a Bar; Guy Joseph Ale, Claire Bloom; Blackstone Publishing
+* Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion
+* Influence without Authority; A. Cohen, D. Bradford; Wiley
 * The Phoenix Project - A Novel About IT, DevOps, and Helping your Business Win; Gene Kim and Kevin Behr; Trade Select
+* Digital Minimalism; Cal Newport; Portofolio Penguin
+* Psycho-Cybernetics; Maxwell Maltz; Perigee Books
+* The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books
 * Solve for Happy; Mo Gawdat (RE-READ 1ST TIME)
-* Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne
-* Ultralearning; Scott Young; Thorsons
-* Never Split the Difference; Chris Voss, Tahl Raz; Random House Business
-* 101 Essays that change the way you think; Brianna Wiest; Audiobook
+* The 7 Habits Of Highly Effective People; Stephen R. Covey; Simon & Schuster UK
+* The Bullet Journal Method; Ryder Carroll; Fourth Estate
+* Eat That Frog!; Brian Tracy; Hodder Paperbacks
 
 => ../notes/index.gmi Here are notes of mine for some of the books
 
@@ -141,30 +141,30 @@ In random order:
 
 Some of these were in-person with exams; others were online learning lectures only. In random order:
 
-* Apache Tomcat Best Practises; 3-day on-site training
-* Linux Security and Isolation APIs Training; Michael Kerrisk; 3-day on-site training
-* Ultimate Go Programming; Bill Kennedy; O'Reilly Online
-* Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need)
-* AWS Immersion Day; Amazon; 1-day interactive online training 
-* F5 Loadbalancers Training; 2-day on-site training; F5, Inc. 
-* Protocol buffers; O'Reilly Online
-* Functional programming lecture; Remote University of Hagen
 * Structure and Interpretation of Computer Programs; Harold Abelson and more...; 
-* Developing IaC with Terraform (with Live Lessons); O'Reilly Online
 * The Ultimate Kubernetes Bootcamp; School of Devops; O'Reilly Online
+* Developing IaC with Terraform (with Live Lessons); O'Reilly Online
+* F5 Loadbalancers Training; 2-day on-site training; F5, Inc. 
+* Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need)
+* AWS Immersion Day; Amazon; 1-day interactive online training 
+* Ultimate Go Programming; Bill Kennedy; O'Reilly Online
 * Cloud Operations on AWS - Learn how to configure, deploy, maintain, and troubleshoot your AWS environments; 3-day online live training with labs; Amazon
-* Scripting Vim; Damian Conway; O'Reilly Online
-* Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online
+* Linux Security and Isolation APIs Training; Michael Kerrisk; 3-day on-site training
 * The Well-Grounded Rubyist Video Edition; David. A. Black; O'Reilly Online
 * MySQL Deep Dive Workshop; 2-day on-site training
+* Functional programming lecture; Remote University of Hagen
+* Scripting Vim; Damian Conway; O'Reilly Online
+* Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online
+* Apache Tomcat Best Practises; 3-day on-site training
+* Protocol buffers; O'Reilly Online
 
 ## Technical guides
 
 These are not whole books, but guides (smaller or larger) which I found very useful. in random order:
 
-* Raku Guide at https://raku.guide  
 * Advanced Bash-Scripting Guide 
 * How CPUs work at https://cpu.land
+* Raku Guide at https://raku.guide  
 
 ## Podcasts
 
@@ -172,57 +172,57 @@ These are not whole books, but guides (smaller or larger) which I found very use
 
 In random order:
 
-* BSD Now [BSD]
 * Modern Mentor
 * Hidden Brain
-* Backend Banter
 * The Pragmatic Engineer Podcast
 * Fork Around And Find Out
-* Fallthrough [Golang]
-* The Changelog Podcast(s)
 * Cup o' Go [Golang]
-* The ProdCast (Google SRE Podcast)
-* Maintainable
 * Deep Questions with Cal Newport
-* Pratical AI
+* BSD Now [BSD]
+* Backend Banter
 * Dev Interrupted
+* Maintainable
+* The Changelog Podcast(s)
+* Pratical AI
+* Fallthrough [Golang]
+* The ProdCast (Google SRE Podcast)
 
 ### Podcasts I liked
 
 I liked them but am not listening to them anymore. The podcasts have either "finished" (no more episodes) or I stopped listening to them due to time constraints or a shift in my interests.
 
+* CRE: Chaosradio Express [german]
 * Java Pub House
 * Ship It (predecessor of Fork Around And Find Out)
-* Go Time (predecessor of fallthrough)
 * FLOSS weekly
-* CRE: Chaosradio Express [german]
 * Modern Mentor
+* Go Time (predecessor of fallthrough)
 
 ## Newsletters I like
 
 This is a mix of tech and non-tech newsletters I am subscribed to. In random order:
 
 * The Pragmatic Engineer
-* Register Spill
-* Andreas Brandhorst Newsletter (Sci-Fi author)
-* The Imperfectionist
 * The Valuable Dev
-* Monospace Mentor
 * VK Newsletter
+* Andreas Brandhorst Newsletter (Sci-Fi author)
+* Monospace Mentor
+* Applied Go Weekly Newsletter
+* Register Spill
+* byteSizeGo
+* The Imperfectionist
 * Changelog News
-* Golang Weekly
 * Ruby Weekly
-* byteSizeGo
-* Applied Go Weekly Newsletter
+* Golang Weekly
 
 ## Magazines I like(d)
 
 This is a mix of tech I like(d). I may not be a current subscriber, but now and then, I buy an issue. In random order:
 
-* LWN (online only)
-* Linux Magazine
 * Linux User
 * freeX (not published anymore)
+* Linux Magazine
+* LWN (online only)
 
 # Formal education
 
diff --git a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi
index 67e7d8df..0d67f9ba 100644
--- a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi
+++ b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi
@@ -397,6 +397,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-)
 
 Other *BSD related posts are:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi
index 20a64ee1..b03610d3 100644
--- a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi
+++ b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.gmi
@@ -676,6 +676,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-)
 
 Other *BSD related posts are:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi
index 41c0d806..01cfe726 100644
--- a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi
+++ b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.gmi
@@ -51,6 +51,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-)
 
 Other *BSD related posts are:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi
index f04602be..3d02b234 100644
--- a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi
+++ b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.gmi
@@ -300,6 +300,7 @@ E-Mail your comments to `paul@nospam.buetow.org` :-)
 
 Other *BSD and KISS related posts are:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi
index 1716dba0..0d01ac9d 100644
--- a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi
+++ b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi
@@ -13,6 +13,7 @@ These are all the posts so far:
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
 
@@ -161,6 +162,7 @@ Read the next post of this series:
 
 Other *BSD-related posts:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi
index 8f362270..a2ceafb0 100644
--- a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi
+++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi
@@ -13,6 +13,7 @@ These are all the posts so far:
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
 
@@ -300,6 +301,7 @@ Read the next post of this series:
 
 Other *BSD-related posts:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi
index de156ab2..65c1637b 100644
--- a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi
+++ b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi
@@ -9,6 +9,7 @@ This is the third blog post about my f3s series for my self-hosting demands in m
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts (You are currently reading this)
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
 
@@ -363,6 +364,7 @@ Read the next post of this series:
 
 Other BSD related posts are:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts (You are currently reading this)
diff --git a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi
index 2a5416f9..e5c3a5ae 100644
--- a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi
+++ b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi
@@ -9,6 +9,7 @@ This is the fourth blog post about the f3s series for self-hosting demands in a
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs (You are currently reading this)
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
 
@@ -509,6 +510,7 @@ Read the next post of this series:
 
 Other *BSD-related posts:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs (You are currently reading this)
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi
index 77a06c55..f22ddc15 100644
--- a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi
+++ b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi
@@ -13,6 +13,7 @@ These are all the posts so far:
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network (You are currently reading this)
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 => ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
 
@@ -924,10 +925,13 @@ peer: 2htXdNcxzpI2FdPDJy4T4VGtm1wpMEQu1AkQHjNY6F8=
 
 Having a mesh network on our hosts is great for securing all the traffic between them for our future k3s setup. A self-managed WireGuard mesh network is better than Tailscale as it eliminates reliance on a third party and provides full control over the configuration. It reduces unnecessary abstraction and "magic," enabling easier debugging and ensuring full ownership of our network.
 
-I look forward to the next blog post in this series. We may start setting up k3s or take a first look at the NFS server (for persistent storage) side of things. I hope you liked all the posts so far in this series.
+Read the next post of this series:
+
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi f3s: Kubernetes with FreeBSD - Part 6: Storage
 
 Other *BSD-related posts:
 
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage
 => ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network (You are currently reading this)
 => ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
 => ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
diff --git a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi
new file mode 100644
index 00000000..bc1d7c00
--- /dev/null
+++ b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi
@@ -0,0 +1,1610 @@
+# f3s: Kubernetes with FreeBSD - Part 6: Storage
+
+> Published at 2025-07-13T16:44:29+03:00
+
+This is the sixth blog post about the f3s series for self-hosting demands in a home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution used on FreeBSD-based physical machines.
+
+=> ./2024-11-17-f3s-kubernetes-with-freebsd-part-1.gmi 2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage
+=> ./2024-12-03-f3s-kubernetes-with-freebsd-part-2.gmi 2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation
+=> ./2025-02-01-f3s-kubernetes-with-freebsd-part-3.gmi 2025-02-01 f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts
+=> ./2025-04-05-f3s-kubernetes-with-freebsd-part-4.gmi 2025-04-05 f3s: Kubernetes with FreeBSD - Part 4: Rocky Linux Bhyve VMs
+=> ./2025-05-11-f3s-kubernetes-with-freebsd-part-5.gmi 2025-05-11 f3s: Kubernetes with FreeBSD - Part 5: WireGuard mesh network
+=> ./2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi 2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage (You are currently reading this)
+
+=> ./f3s-kubernetes-with-freebsd-part-1/f3slogo.png f3s logo
+
+## Table of Contents
+
+* ⇢ f3s: Kubernetes with FreeBSD - Part 6: Storage
+* ⇢ ⇢ Introduction
+* ⇢ ⇢ Additional storage capacity
+* ⇢ ⇢ ZFS encryption keys
+* ⇢ ⇢ ⇢ UFS on USB keys
+* ⇢ ⇢ ⇢ Generating encryption keys
+* ⇢ ⇢ ⇢ Configuring `zdata` ZFS pool encryption
+* ⇢ ⇢ ⇢ Migrating Bhyve VMs to an encrypted `bhyve` ZFS volume
+* ⇢ ⇢ ZFS Replication with `zrepl`
+* ⇢ ⇢ ⇢ Understanding Replication Requirements
+* ⇢ ⇢ ⇢ Installing `zrepl`
+* ⇢ ⇢ ⇢ Configuring `zrepl` on `f1` (sink)
+* ⇢ ⇢ ⇢ Enabling and starting `zrepl` services
+* ⇢ ⇢ ⇢ Monitoring replication
+* ⇢ ⇢ ⇢ Verifying replication after reboot
+* ⇢ ⇢ ⇢ Understanding Failover Limitations and Design Decisions
+* ⇢ ⇢ ⇢ Mounting the NFS datasets
+* ⇢ ⇢ ⇢ Troubleshooting: Files not appearing in replication
+* ⇢ ⇢ ⇢ Configuring automatic key loading on boot
+* ⇢ ⇢ CARP (Common Address Redundancy Protocol)
+* ⇢ ⇢ ⇢ How CARP Works
+* ⇢ ⇢ ⇢ Configuring CARP
+* ⇢ ⇢ ⇢ CARP State Change Notifications
+* ⇢ ⇢ NFS Server Configuration
+* ⇢ ⇢ ⇢ Setting up NFS on `f0` (Primary)
+* ⇢ ⇢ ⇢ Configuring Stunnel for NFS Encryption with CARP Failover
+* ⇢ ⇢ ⇢ Creating a Certificate Authority for Client Authentication
+* ⇢ ⇢ ⇢ Install and Configure Stunnel on `f0`
+* ⇢ ⇢ ⇢ Setting up NFS on `f1` (Standby)
+* ⇢ ⇢ ⇢ CARP Control Script for Clean Failover
+* ⇢ ⇢ ⇢ CARP Management Script
+* ⇢ ⇢ ⇢ Automatic Failback After Reboot
+* ⇢ ⇢ Client Configuration for Stunnel
+* ⇢ ⇢ ⇢ Configuring Rocky Linux Clients (`r0`, `r1`, `r2`)
+* ⇢ ⇢ ⇢ Testing NFS Mount with Stunnel
+* ⇢ ⇢ ⇢ Testing CARP Failover with mounted clients and stale file handles:
+* ⇢ ⇢ ⇢ Complete Failover Test
+* ⇢ ⇢ Conclusion
+* ⇢ ⇢ Future Storage Explorations
+* ⇢ ⇢ ⇢ MinIO for S3-Compatible Object Storage
+* ⇢ ⇢ ⇢ MooseFS for Distributed High Availability
+
+## Introduction
+
+In the previous posts, we set up a FreeBSD-based Kubernetes cluster using k3s. While the base system works well, Kubernetes workloads often require persistent storage for databases, configuration files, and application data. Local storage on each node has significant limitations:
+
+* No data sharing: Pods (once we run Kubernetes) on different nodes can't access the same data
+* Pod mobility: If a pod moves to another node, it loses access to its data
+* No redundancy: Hardware failure means data loss
+
+This post implements a robust storage solution using:
+
+* CARP: For high availability with automatic IP failover
+* NFS over stunnel: For secure, encrypted network storage
+* ZFS: For data integrity, encryption, and efficient snapshots
+* `zrepl`: For continuous ZFS replication between nodes
+
+The result is a highly available, encrypted storage system that survives node failures while providing shared storage to all Kubernetes pods.
+
+Other than what was mentioned in the first post of this blog series, we aren't using HAST, but `zrepl` for data replication. Read more about it later in this blog post.
+
+## Additional storage capacity
+
+We add 1 TB of additional storage to each of the nodes (`f0`, `f1`, `f2`) in the form of an SSD drive. The Beelink mini PCs have enough space in the chassis for the extra space.
+
+=> ./f3s-kubernetes-with-freebsd-part-6/drives.jpg
+
+Upgrading the storage was as easy as unscrewing, plugging the drive in, and then screwing it back together again. The procedure was uneventful! We're using two different SSD models (Samsung 870 EVO and Crucial BX500) to avoid simultaneous failures from the same manufacturing batch.
+
+We then create the `zdata` ZFS pool on all three nodes:
+
+```sh
+paul@f0:~ % doas zpool create -m /data zdata /dev/ada1
+paul@f0:~ % zpool list
+NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
+zdata   928G  12.1M   928G        -         -     0%     0%  1.00x    ONLINE  -
+zroot   472G  29.0G   443G        -         -     0%     6%  1.00x    ONLINE  -
+
+paul@f0:/ % doas camcontrol devlist
+<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
+<Samsung SSD 870 EVO 1TB SVT03B6Q>  at scbus1 target 0 lun 0 (pass1,ada1)
+paul@f0:/ %
+```
+
+To verify that we have a different SSD on the second node (the third node has the same drive as the first):
+
+```sh
+paul@f1:/ % doas camcontrol devlist
+<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
+<CT1000BX500SSD1 M6CR072>          at scbus1 target 0 lun 0 (pass1,ada1)
+```
+
+## ZFS encryption keys
+
+ZFS native encryption requires encryption keys to unlock datasets. We need a secure method to store these keys that balances security with operational needs:
+
+* Security: Keys must not be stored on the same disks they encrypt
+* Availability: Keys must be available at boot for automatic mounting
+* Portability: Keys should be easily moved between systems for recovery
+
+Using USB flash drives as hardware key storage provides a convenient and elegant solution. The encrypted data is unreadable without physical access to the USB key, protecting against disk theft or improper disposal. In production environments, you may use enterprise key management systems; however, for a home lab, USB keys offer good security with minimal complexity.
+
+### UFS on USB keys
+
+We'll format the USB drives with UFS (Unix File System) rather than ZFS for simplicity. There is no need to use ZFS.
+
+Let's see the USB keys:
+
+=> ./f3s-kubernetes-with-freebsd-part-6/usbkeys1.jpg USB keys
+
+To verify that the USB key (flash disk) is there:
+
+```
+paul@f0:/ % doas camcontrol devlist
+<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
+<Samsung SSD 870 EVO 1TB SVT03B6Q>  at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07>          at scbus2 target 0 lun 0 (da0,pass2)
+paul@f0:/ %
+```
+
+Let's create the UFS file system and mount it (done on all three nodes `f0`, `f1` and `f2`):
+
+```sh
+paul@f0:/ % doas newfs /dev/da0
+/dev/da0: 15000.0MB (30720000 sectors) block size 32768, fragment size 4096
+        using 24 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
+        with soft updates
+super-block backups (for fsck_ffs -b #) at:
+ 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776,
+11524224, 12804672, 14085120, 15365568, 16646016, 17926464, 19206912,k 20487360,
+...
+
+paul@f0:/ % echo '/dev/da0 /keys ufs rw 0 2' | doas tee -a /etc/fstab
+/dev/da0 /keys ufs rw 0 2
+paul@f0:/ % doas mkdir /keys
+paul@f0:/ % doas mount /keys
+paul@f0:/ % df | grep keys
+/dev/da0             14877596       8  13687384     0%    /keys
+```
+
+=> ./f3s-kubernetes-with-freebsd-part-6/usbkeys2.jpg USB keys stuck in
+
+### Generating encryption keys
+
+The following keys will later be used to encrypt the ZFS file systems. They will be stored on all three nodes, serving as a backup in case one of the keys is lost or corrupted. When we later replicate encrypted ZFS volumes from one node to another, the keys must also be available on the destination node.
+
+```
+paul@f0:/keys % doas openssl rand -out /keys/f0.lan.buetow.org:bhyve.key 32