# FreeBSD relayd Configuration for LAN Access to k3s Services This document describes how to configure FreeBSD relayd on the CARP VIP to provide LAN access to k3s services with TLS termination. ## Architecture ``` LAN Client ↓ 192.168.1.138:443 (CARP VIP - f0/f1) ↓ relayd (TLS termination) ↓ k3s Traefik NodePort :80 (r0/r1/r2) ↓ Service (Navidrome, etc.) ``` ## Current CARP Setup - **CARP VIP**: 192.168.1.138 (vhid 1) - **Master**: f0 (192.168.1.130) - advskew 0 - **Backup**: f1 (192.168.1.131) - advskew 100 - **Existing Service**: stunnel on port 2323 (NFS-over-TLS) ## Installation ### 1. Install relayd On both f0 and f1: ```bash doas pkg install relayd ``` ### 2. Configure relayd Create `/usr/local/etc/relayd.conf` on both f0 and f1: ``` # k3s nodes backend table table { 192.168.1.120 192.168.1.121 192.168.1.122 } # HTTP protocol (pass-through to Traefik) http protocol "lan_http" { # Pass all requests to backend pass request quick pass response quick } # HTTPS protocol with TLS termination http protocol "lan_https" { # TLS configuration tls keypair "f3s.lan.buetow.org" # Pass decrypted HTTP to Traefik pass request quick pass response quick } # HTTP relay (port 80) relay "lan_http" { listen on 192.168.1.138 port 80 protocol "lan_http" forward to port 80 check tcp } # HTTPS relay (port 443) with TLS relay "lan_https" { listen on 192.168.1.138 port 443 tls protocol "lan_https" forward to port 80 check tcp } ``` ### 3. Install TLS Certificates Certificates are generated by cert-manager in k3s and must be exported to FreeBSD. #### Export from k3s On a k3s node or workstation with kubectl access: ```bash cd /home/paul/git/conf/f3s/cert-manager just export-certs ``` This creates: - `/tmp/f3s-lan-cert.pem` - Certificate - `/tmp/f3s-lan-key.pem` - Private key #### Copy to FreeBSD Hosts ```bash # Copy to f0 scp /tmp/f3s-lan-cert.pem paul@192.168.1.130:/tmp/ scp /tmp/f3s-lan-key.pem paul@192.168.1.130:/tmp/ # Copy to f1 scp /tmp/f3s-lan-cert.pem paul@192.168.1.131:/tmp/ scp /tmp/f3s-lan-key.pem paul@192.168.1.131:/tmp/ ``` #### Install on FreeBSD On both f0 and f1: ```bash # Create certificate directory doas mkdir -p /usr/local/etc/ssl/relayd # Move certificates doas mv /tmp/f3s-lan-cert.pem /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.crt doas mv /tmp/f3s-lan-key.pem /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.key # Combine cert and key for relayd (relayd expects combined PEM) doas sh -c 'cat /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.crt /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.key > /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.pem' # Set permissions doas chmod 600 /usr/local/etc/ssl/relayd/* doas chown root:wheel /usr/local/etc/ssl/relayd/* ``` ### 4. Enable and Start relayd On both f0 and f1: ```bash # Enable relayd service doas sysrc relayd_enable=YES # Start relayd doas service relayd start # Check status doas service relayd status ``` ### 5. Verify relayd is Running ```bash # Check relayd process doas ps aux | grep relayd # Check listening ports on CARP VIP doas sockstat -4 -l | grep 192.168.1.138 ``` Expected output: ``` stunnel stunnel 1546 8 tcp4 192.168.1.138:2323 *:* relayd relayd 2101 3 tcp4 192.168.1.138:80 *:* relayd relayd 2101 4 tcp4 192.168.1.138:443 *:* ``` ## Testing ### Test HTTP Access ```bash curl -v http://navidrome.f3s.lan.buetow.org ``` ### Test HTTPS Access Without CA trust (expect certificate warning): ```bash curl -k https://navidrome.f3s.lan.buetow.org ``` With CA trust installed: ```bash curl https://navidrome.f3s.lan.buetow.org ``` ### Test CARP Failover 1. On f0 (current master), disable the interface: ```bash doas ifconfig re0 down ``` 2. Check CARP status on f1 (should become MASTER): ```bash ifconfig re0 | grep carp ``` 3. Test access (should still work): ```bash curl https://navidrome.f3s.lan.buetow.org ``` 4. Re-enable f0: ```bash doas ifconfig re0 up ``` ## Certificate Renewal Certificates from cert-manager are renewed automatically every 75 days (15 days before 90-day expiration). After renewal: 1. Export new certificates from k3s 2. Copy to f0 and f1 3. Reload relayd on both hosts: ```bash doas service relayd reload ``` ## Troubleshooting ### Check relayd Configuration ```bash doas relayd -n ``` ### View relayd Logs ```bash # Real-time logs doas tail -f /var/log/daemon.log | grep relayd # Recent logs doas grep relayd /var/log/daemon.log | tail -50 ``` ### Check Backend Health relayd performs TCP health checks on k3s nodes. View status: ```bash doas relayctl show summary doas relayctl show hosts ``` ### Connection Test from FreeBSD ```bash # Test Traefik on k3s nodes directly curl -H "Host: navidrome.f3s.lan.buetow.org" http://192.168.1.120 curl -H "Host: navidrome.f3s.lan.buetow.org" http://192.168.1.121 curl -H "Host: navidrome.f3s.lan.buetow.org" http://192.168.1.122 ``` ### Common Issues **relayd fails to start:** - Check configuration: `doas relayd -n` - Verify certificates exist and have correct permissions - Check `/var/log/daemon.log` for errors **Certificate errors:** - Ensure certificate and key are combined in one PEM file - Verify filename matches `tls keypair` directive in config - Check certificate validity: `openssl x509 -in /usr/local/etc/ssl/relayd/f3s.lan.buetow.org.pem -text -noout` **No response from services:** - Check k3s Traefik is running: `kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik` - Verify k3s nodes are reachable from FreeBSD - Check backend health: `doas relayctl show hosts` ## Security Considerations - Certificates are self-signed; clients must trust the CA - relayd runs as unprivileged user after binding to ports - Only ports 80 and 443 are exposed on CARP VIP for HTTP/HTTPS - Existing stunnel NFS service (port 2323) is unaffected ## Adding More Services To add LAN access to another service: 1. Create ingress in service's helm chart with `host: service.f3s.lan.buetow.org` 2. Add DNS entry: `192.168.1.138 service.f3s.lan.buetow.org` 3. No relayd changes needed - it forwards all traffic to Traefik ## References - [relayd(8) man page](https://man.freebsd.org/cgi/man.cgi?query=relayd) - [CARP Configuration on FreeBSD](https://docs.freebsd.org/en/books/handbook/advanced-networking/#carp) - [cert-manager Documentation](https://cert-manager.io/docs/)