Update content for html

17 files changed, 1110 insertions, 1600 deletions

diff --git a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html
index 05f153a3..6ec2d003 100644
--- a/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html
+++ b/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html
@@ -414,6 +414,7 @@ Notice: Finished catalog run in 206.09 seconds
 <span>Other *BSD related posts are:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.html b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.html
index 71ccc187..cd045e63 100644
--- a/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.html
+++ b/gemfeed/2022-07-30-lets-encrypt-with-openbsd-and-rex.html
@@ -693,6 +693,7 @@ rex commons
 <span>Other *BSD related posts are:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.html b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.html
index f67c7110..a2b7eb99 100644
--- a/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.html
+++ b/gemfeed/2024-01-13-one-reason-why-i-love-openbsd.html
@@ -73,6 +73,7 @@ $ doas reboot <i><font color="silver"># Just in case, reboot one more time</font
 <span>Other *BSD related posts are:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html
index 3839324e..72dcfec3 100644
--- a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html
+++ b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html
@@ -332,6 +332,7 @@ http://www.gnu.org/software/src-highlite -->
 <span>Other *BSD and KISS related posts are:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.html b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.html
index a95ea52c..617ef9a6 100644
--- a/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.html
+++ b/gemfeed/2024-11-17-f3s-kubernetes-with-freebsd-part-1.html
@@ -29,6 +29,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -182,6 +183,7 @@
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.html b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.html
index e478754e..150f9db9 100644
--- a/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.html
+++ b/gemfeed/2024-12-03-f3s-kubernetes-with-freebsd-part-2.html
@@ -29,6 +29,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -518,6 +519,7 @@ Waking up e8:ff:1e:d7:1c:a0...
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html
index 7e9539f0..620eb55c 100644
--- a/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html
+++ b/gemfeed/2025-02-01-f3s-kubernetes-with-freebsd-part-3.html
@@ -25,6 +25,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -422,6 +423,7 @@ Jan 26 17:36:32 f2 apcupsd[2159]: apcupsd shutdown succeeded
 <span>Other BSD related posts are:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.html b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.html
index 77bd8bcd..1729d91c 100644
--- a/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.html
+++ b/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.html
@@ -25,6 +25,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -717,6 +718,7 @@ etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"} 408
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.html b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.html
index 6b4beb48..20b75e46 100644
--- a/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.html
+++ b/gemfeed/2025-05-11-f3s-kubernetes-with-freebsd-part-5.html
@@ -31,6 +31,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -1578,6 +1579,7 @@ earth$ curl https://ifconfig.me  <i><font color="silver"># Should show gateway's
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.html b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.html
index 1e60ff46..99e9577e 100644
--- a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.html
+++ b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.html
@@ -25,6 +25,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage (You are currently reading this)</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -2181,6 +2182,7 @@ http://www.gnu.org/software/src-highlite -->
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage (You are currently reading this)</a><br />
diff --git a/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.html b/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.html
index 7f13b083..3a8a18cf 100644
--- a/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.html
+++ b/gemfeed/2025-10-02-f3s-kubernetes-with-freebsd-part-7.html
@@ -25,6 +25,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments (You are currently reading this)</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -1494,6 +1495,7 @@ replicaset.apps/miniflux-server-85d7c64664     1         1         1       54d
 <span>Other *BSD-related posts:</span><br />
 <br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments (You are currently reading this)</a><br />
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
diff --git a/gemfeed/2025-12-07-f3s-kubernetes-with-freebsd-part-8.html b/gemfeed/2025-12-07-f3s-kubernetes-with-freebsd-part-8.html
index 3954f3dd..c864bb79 100644
--- a/gemfeed/2025-12-07-f3s-kubernetes-with-freebsd-part-8.html
+++ b/gemfeed/2025-12-07-f3s-kubernetes-with-freebsd-part-8.html
@@ -25,6 +25,7 @@
 <a class='textlink' href='./2025-07-14-f3s-kubernetes-with-freebsd-part-6.html'>2025-07-14 f3s: Kubernetes with FreeBSD - Part 6: Storage</a><br />
 <a class='textlink' href='./2025-10-02-f3s-kubernetes-with-freebsd-part-7.html'>2025-10-02 f3s: Kubernetes with FreeBSD - Part 7: k3s and first pod deployments</a><br />
 <a class='textlink' href='./2025-12-07-f3s-kubernetes-with-freebsd-part-8.html'>2025-12-07 f3s: Kubernetes with FreeBSD - Part 8: Observability (You are currently reading this)</a><br />
+<a class='textlink' href='./2025-12-14-f3s-kubernetes-with-freebsd-part-8b.html'>2025-12-14 f3s: Kubernetes with FreeBSD - Part 8b: Distributed Tracing with Tempo</a><br />
 <a class='textlink' href='./2026-04-02-f3s-kubernetes-with-freebsd-part-9.html'>2026-04-02 f3s: Kubernetes with FreeBSD - Part 9: GitOps with ArgoCD</a><br />
 <br />
 <a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br />
@@ -69,29 +70,6 @@
 <li>⇢ ⇢ <a href='#installing-node-exporter-on-openbsd'>Installing Node Exporter on OpenBSD</a></li>
 <li>⇢ ⇢ <a href='#adding-openbsd-hosts-to-prometheus'>Adding OpenBSD hosts to Prometheus</a></li>
 <li>⇢ ⇢ <a href='#openbsd-memory-metrics-compatibility'>OpenBSD memory metrics compatibility</a></li>
-<li>⇢ <a href='#distributed-tracing-with-grafana-tempo'>Distributed Tracing with Grafana Tempo</a></li>
-<li>⇢ ⇢ <a href='#why-distributed-tracing'>Why Distributed Tracing?</a></li>
-<li>⇢ ⇢ <a href='#deploying-grafana-tempo'>Deploying Grafana Tempo</a></li>
-<li>⇢ <a href='#-configuration-strategy'>⇢# Configuration Strategy</a></li>
-<li>⇢ <a href='#-tempo-deployment-files'>⇢# Tempo Deployment Files</a></li>
-<li>⇢ <a href='#-installation'>⇢# Installation</a></li>
-<li>⇢ ⇢ <a href='#configuring-grafana-alloy-for-trace-collection'>Configuring Grafana Alloy for Trace Collection</a></li>
-<li>⇢ <a href='#-otlp-receiver-configuration'>⇢# OTLP Receiver Configuration</a></li>
-<li>⇢ <a href='#-upgrade-alloy'>⇢# Upgrade Alloy</a></li>
-<li>⇢ ⇢ <a href='#demo-tracing-application'>Demo Tracing Application</a></li>
-<li>⇢ <a href='#-application-architecture'>⇢# Application Architecture</a></li>
-<li>⇢ ⇢ <a href='#visualizing-traces-in-grafana'>Visualizing Traces in Grafana</a></li>
-<li>⇢ <a href='#-accessing-traces'>⇢# Accessing Traces</a></li>
-<li>⇢ <a href='#-service-graph-visualization'>⇢# Service Graph Visualization</a></li>
-<li>⇢ ⇢ <a href='#correlation-between-observability-signals'>Correlation Between Observability Signals</a></li>
-<li>⇢ <a href='#-traces-to-logs'>⇢# Traces-to-Logs</a></li>
-<li>⇢ <a href='#-traces-to-metrics'>⇢# Traces-to-Metrics</a></li>
-<li>⇢ <a href='#-logs-to-traces'>⇢# Logs-to-Traces</a></li>
-<li>⇢ ⇢ <a href='#generating-traces-for-testing'>Generating Traces for Testing</a></li>
-<li>⇢ ⇢ <a href='#verifying-the-complete-pipeline'>Verifying the Complete Pipeline</a></li>
-<li>⇢ ⇢ <a href='#practical-example-viewing-a-distributed-trace'>Practical Example: Viewing a Distributed Trace</a></li>
-<li>⇢ ⇢ <a href='#storage-and-retention'>Storage and Retention</a></li>
-<li>⇢ ⇢ <a href='#configuration-files'>Configuration Files</a></li>
 <li>⇢ <a href='#summary'>Summary</a></li>
 </ul><br />
 <h2 style='display: inline' id='introduction'>Introduction</h2><br />
@@ -1129,674 +1107,29 @@ spec:
 <br />
 <span>After running <span class='inlinecode'>just upgrade</span>, the OpenBSD hosts appear in Prometheus targets and the Node Exporter dashboards.</span><br />
 <br />
-<h2 style='display: inline' id='distributed-tracing-with-grafana-tempo'>Distributed Tracing with Grafana Tempo</h2><br />
-<br />
-<span>After implementing logs (Loki) and metrics (Prometheus), the final pillar of observability is distributed tracing. Grafana Tempo provides distributed tracing capabilities that help understand request flows across microservices.</span><br />
-<br />
-<span>For a preview of what distributed tracing with Tempo looks like in Grafana, see the X-RAG blog post:</span><br />
-<br />
-<a class='textlink' href='./2025-12-24-x-rag-observability-hackathon.html'>X-RAG Observability Hackathon</a><br />
-<br />
-<h3 style='display: inline' id='why-distributed-tracing'>Why Distributed Tracing?</h3><br />
-<br />
-<span>In a microservices architecture, a single user request may traverse multiple services. Distributed tracing:</span><br />
-<br />
-<ul>
-<li>Tracks requests across service boundaries</li>
-<li>Identifies performance bottlenecks</li>
-<li>Visualizes service dependencies</li>
-<li>Correlates with logs and metrics</li>
-<li>Helps debug complex distributed systems</li>
-</ul><br />
-<h3 style='display: inline' id='deploying-grafana-tempo'>Deploying Grafana Tempo</h3><br />
-<br />
-<span>Tempo is deployed in monolithic mode, following the same pattern as Loki&#39;s SingleBinary deployment.</span><br />
-<br />
-<span>#### Configuration Strategy</span><br />
-<br />
-<span>**Deployment Mode:** Monolithic (all components in one process)</span><br />
-<ul>
-<li>Simpler operation than microservices mode</li>
-<li>Suitable for the cluster scale</li>
-<li>Consistent with Loki deployment pattern</li>
-</ul><br />
-<span>**Storage:** Filesystem backend using hostPath</span><br />
-<ul>
-<li>10Gi storage at /data/nfs/k3svolumes/tempo/data</li>
-<li>7-day retention (168h)</li>
-<li>Local storage is the only option for monolithic mode</li>
-</ul><br />
-<span>**OTLP Receivers:** Standard OpenTelemetry Protocol ports</span><br />
-<ul>
-<li>gRPC: 4317</li>
-<li>HTTP: 4318</li>
-<li>Bind to 0.0.0.0 to avoid Tempo 2.7+ localhost-only binding issue</li>
-</ul><br />
-<span>#### Tempo Deployment Files</span><br />
-<br />
-<span>Created in /home/paul/git/conf/f3s/tempo/:</span><br />
-<br />
-<span>**values.yaml** - Helm chart configuration:</span><br />
-<br />
-<pre>
-tempo:
-  retention: 168h
-  storage:
-    trace:
-      backend: local
-      local:
-        path: /var/tempo/traces
-      wal:
-        path: /var/tempo/wal
-  receivers:
-    otlp:
-      protocols:
-        grpc:
-          endpoint: 0.0.0.0:4317
-        http:
-          endpoint: 0.0.0.0:4318
-
-persistence:
-  enabled: true
-  size: 10Gi
-  storageClassName: ""
-
-resources:
-  limits:
-    cpu: 1000m
-    memory: 2Gi
-  requests:
-    cpu: 500m
-    memory: 1Gi
-</pre>
-<br />
-<span>**persistent-volumes.yaml** - Storage configuration:</span><br />
-<br />
-<pre>
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: tempo-data-pv
-spec:
-  capacity:
-    storage: 10Gi
-  accessModes:
-    - ReadWriteOnce
-  persistentVolumeReclaimPolicy: Retain
-  hostPath:
-    path: /data/nfs/k3svolumes/tempo/data
----
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: tempo-data-pvc
-  namespace: monitoring
-spec:
-  storageClassName: ""
-  accessModes:
-    - ReadWriteOnce
-  resources:
-    requests:
-      storage: 10Gi
-</pre>
-<br />
-<span>**Grafana Datasource Provisioning**</span><br />
-<br />
-<span>All Grafana datasources (Prometheus, Alertmanager, Loki, Tempo) are provisioned via a unified ConfigMap that is directly mounted to the Grafana pod. This approach ensures datasources are loaded on startup without requiring sidecar-based discovery.</span><br />
-<br />
-<span>In /home/paul/git/conf/f3s/prometheus/grafana-datasources-all.yaml:</span><br />
-<br />
-<pre>
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: grafana-datasources-all
-  namespace: monitoring
-data:
-  datasources.yaml: |
-    apiVersion: 1
-    datasources:
-      - name: Prometheus
-        type: prometheus
-        uid: prometheus
-        url: http://prometheus-kube-prometheus-prometheus.monitoring:9090/
-        access: proxy
-        isDefault: true
-      - name: Alertmanager
-        type: alertmanager
-        uid: alertmanager
-        url: http://prometheus-kube-prometheus-alertmanager.monitoring:9093/
-      - name: Loki
-        type: loki
-        uid: loki
-        url: http://loki.monitoring.svc.cluster.local:3100
-      - name: Tempo
-        type: tempo
-        uid: tempo
-        url: http://tempo.monitoring.svc.cluster.local:3200
-        jsonData:
-          tracesToLogsV2:
-            datasourceUid: loki
-            spanStartTimeShift: -1h
-            spanEndTimeShift: 1h
-          tracesToMetrics:
-            datasourceUid: prometheus
-          serviceMap:
-            datasourceUid: prometheus
-          nodeGraph:
-            enabled: true
-</pre>
-<br />
-<span>The kube-prometheus-stack Helm values (persistence-values.yaml) are configured to:</span><br />
-<ul>
-<li>Disable sidecar-based datasource provisioning</li>
-<li>Mount grafana-datasources-all ConfigMap directly to /etc/grafana/provisioning/datasources/</li>
-</ul><br />
-<span>This direct mounting approach is simpler and more reliable than sidecar-based discovery.</span><br />
-<br />
-<span>#### Installation</span><br />
-<br />
-<pre>
-cd /home/paul/git/conf/f3s/tempo
-just install
-</pre>
-<br />
-<span>Verify Tempo is running:</span><br />
-<br />
-<pre>
-kubectl get pods -n monitoring -l app.kubernetes.io/name=tempo
-kubectl exec -n monitoring &lt;tempo-pod&gt; -- wget -qO- http://localhost:3200/ready
-</pre>
-<br />
-<h3 style='display: inline' id='configuring-grafana-alloy-for-trace-collection'>Configuring Grafana Alloy for Trace Collection</h3><br />
-<br />
-<span>Updated /home/paul/git/conf/f3s/loki/alloy-values.yaml to add OTLP receivers for traces while maintaining existing log collection.</span><br />
-<br />
-<span>#### OTLP Receiver Configuration</span><br />
-<br />
-<span>Added to Alloy configuration after the log collection pipeline:</span><br />
-<br />
-<pre>
-// OTLP receiver for traces via gRPC and HTTP
-otelcol.receiver.otlp "default" {
-  grpc {
-    endpoint = "0.0.0.0:4317"
-  }
-  http {
-    endpoint = "0.0.0.0:4318"
-  }
-  output {
-    traces = [otelcol.processor.batch.default.input]
-  }
-}
-
-// Batch processor for efficient trace forwarding
-otelcol.processor.batch "default" {
-  timeout = "5s"
-  send_batch_size = 100
-  send_batch_max_size = 200
-  output {
-    traces = [otelcol.exporter.otlp.tempo.input]
-  }
-}
-
-// OTLP exporter to send traces to Tempo
-otelcol.exporter.otlp "tempo" {
-  client {
-    endpoint = "tempo.monitoring.svc.cluster.local:4317"
-    tls {
-      insecure = true
-    }
-    compression = "gzip"
-  }
-}
-</pre>
-<br />
-<span>The batch processor reduces network overhead by accumulating spans before forwarding to Tempo.</span><br />
-<br />
-<span>#### Upgrade Alloy</span><br />
-<br />
-<pre>
-cd /home/paul/git/conf/f3s/loki
-just upgrade
-</pre>
-<br />
-<span>Verify OTLP receivers are listening:</span><br />
-<br />
-<pre>
-kubectl logs -n monitoring -l app.kubernetes.io/name=alloy | grep -i "otlp.*receiver"
-kubectl exec -n monitoring &lt;alloy-pod&gt; -- netstat -ln | grep -E &#39;:(4317|4318)&#39;
-</pre>
-<br />
-<h3 style='display: inline' id='demo-tracing-application'>Demo Tracing Application</h3><br />
-<br />
-<span>Created a three-tier Python application to demonstrate distributed tracing in action.</span><br />
-<br />
-<span>#### Application Architecture</span><br />
-<br />
-<pre>
-User → Frontend (Flask:5000) → Middleware (Flask:5001) → Backend (Flask:5002)
-           ↓                          ↓                        ↓
-                    Alloy (OTLP:4317) → Tempo → Grafana
-</pre>
-<br />
-<span>Frontend Service:</span><br />
-<br />
-<ul>
-<li>Receives HTTP requests at /api/process</li>
-<li>Forwards to middleware service</li>
-<li>Creates parent span for the entire request</li>
-</ul><br />
-<span>Middleware Service:</span><br />
-<br />
-<ul>
-<li>Transforms data at /api/transform</li>
-<li>Calls backend service</li>
-<li>Creates child span linked to frontend</li>
-</ul><br />
-<span>Backend Service:</span><br />
-<br />
-<ul>
-<li>Returns data at /api/data</li>
-<li>Simulates database query (100ms sleep)</li>
-<li>Creates leaf span in the trace</li>
-</ul><br />
-<span>OpenTelemetry Instrumentation:</span><br />
-<br />
-<span>All services use Python OpenTelemetry libraries:</span><br />
-<br />
-<span>**Dependencies:**</span><br