summaryrefslogtreecommitdiff
path: root/prompts
diff options
context:
space:
mode:
Diffstat (limited to 'prompts')
-rw-r--r--prompts/skills/f3s/references/k3s-setup.md2
-rw-r--r--prompts/skills/f3s/references/k3s-setup/remote-access.md47
-rw-r--r--prompts/skills/f3s/references/remote-access.md20
-rw-r--r--prompts/skills/f3s/references/wireguard.md36
4 files changed, 99 insertions, 6 deletions
diff --git a/prompts/skills/f3s/references/k3s-setup.md b/prompts/skills/f3s/references/k3s-setup.md
index 952a4bf..3b94b88 100644
--- a/prompts/skills/f3s/references/k3s-setup.md
+++ b/prompts/skills/f3s/references/k3s-setup.md
@@ -7,6 +7,6 @@ over WireGuard.
## Sub-references
- [Install](k3s-setup/install.md) — bootstrap, kubeconfig, etcd/controller-manager metrics, built-in components, NFS PV pattern, ArgoCD, node IP summary, useful commands
-- [Remote access (off-LAN)](k3s-setup/remote-access.md) — reaching the cluster while roaming: jump via OpenBSD frontend (`ssh -A rex@fishfinger.buetow.org` → `ssh root@r0.wg0` → `kubectl`), one-shot commands, and SSH port-forward tunnel
+- [Remote access (off-LAN)](k3s-setup/remote-access.md) — reaching the cluster while roaming: **preferred** dedicated `wg0` kubectl context talking directly to `r0.wg0.wan.buetow.org:6443` over WireGuard (switch with `kubectl config use-context wg0`); fallback jump via OpenBSD frontend (`ssh -A rex@fishfinger.buetow.org` → `ssh root@r0.wg0` → `kubectl`), one-shot commands, and SSH port-forward tunnel
- [Ingress](k3s-setup/ingress.md) — OpenBSD `relayd` (internet) and FreeBSD `relayd` on CARP VIP (LAN), cert-manager wildcard, ingress pattern
- [Troubleshooting](k3s-setup/troubleshooting.md) — etcd Raft log corruption recovery; cluster-wide NFS outage pointer
diff --git a/prompts/skills/f3s/references/k3s-setup/remote-access.md b/prompts/skills/f3s/references/k3s-setup/remote-access.md
index 5da8c84..9837870 100644
--- a/prompts/skills/f3s/references/k3s-setup/remote-access.md
+++ b/prompts/skills/f3s/references/k3s-setup/remote-access.md
@@ -9,11 +9,50 @@ just times out:
Unable to connect to the server: dial tcp 192.168.1.120:6443: i/o timeout
```
-The WireGuard mesh IPs (`192.168.2.120` etc.) are also not directly reachable
-from a roaming laptop, because the laptop's `wg1` interface does not peer
-straight to the r-VMs.
+## Preferred method: a dedicated `wg0` kubectl context (direct over WireGuard)
+
+From **earth** the WireGuard mesh IPs **are** directly reachable
+(`192.168.2.120` = `r0.wg0.wan.buetow.org`), so `kubectl` can talk straight to
+the API server over `wg0` — no jump host needed. The k3s API cert already has
+the `r0.wg0.wan.buetow.org` / `r1.wg0…` / `r2.wg0…` SANs (see install.md
+`--tls-san`), so TLS verifies cleanly with no overrides.
+
+Add a second cluster + context to `~/.kube/config` alongside the LAN `default`
+one, reusing the same `certificate-authority-data` and `default` user:
+
+```yaml
+clusters:
+- cluster:
+ certificate-authority-data: <same as default>
+ server: https://r0.wg0.wan.buetow.org:6443 # 192.168.2.120
+ name: wg0
+contexts:
+- context:
+ cluster: wg0
+ namespace: services
+ user: default
+ name: wg0
+```
+
+Then switch by location:
+
+```sh
+kubectl config use-context default # on the f3s LAN (home)
+kubectl config use-context wg0 # on the road (WireGuard)
+kubectl --context=wg0 get nodes # one-off without switching
+```
+
+Notes:
+
+- `default` → `https://r0.lan.buetow.org:6443` (LAN, `192.168.1.120`); `wg0` →
+ `https://r0.wg0.wan.buetow.org:6443` (mesh, `192.168.2.120`).
+- For failover, point a `wg0` variant at `r1.wg0` / `r2.wg0` (all three are SANs
+ on the cert) if r0 is down.
+- This requires the laptop's WireGuard to actually route to the r-VM mesh IPs.
+ If a particular client's `wg` does **not** peer to the r-VMs, fall back to the
+ OpenBSD-frontend jump method below.
-## Working method: jump through an OpenBSD frontend
+## Fallback method: jump through an OpenBSD frontend
The OpenBSD internet gateways **fishfinger** and **blowfish** are reachable
from the public internet *and* sit on the WireGuard mesh, so they can reach the
diff --git a/prompts/skills/f3s/references/remote-access.md b/prompts/skills/f3s/references/remote-access.md
index c507671..984d5ce 100644
--- a/prompts/skills/f3s/references/remote-access.md
+++ b/prompts/skills/f3s/references/remote-access.md
@@ -5,6 +5,26 @@ unreachable directly. The f-hosts (`f0`–`f3`) and VMs have LAN IPs
(`192.168.1.x`) and WireGuard IPs (`192.168.2.x`), but a roaming laptop's
`wg0` only peers to the OpenBSD gateways — not directly to the homelab mesh.
+## Direct SSH from `earth` (when on the VPN)
+
+`earth` has been given a return route to the mesh: `192.168.2.200/32, fd42:beef:cafe:2::200/128`
+was added to the **fishfinger** peer's `AllowedIPs` on `f0`, `f1`, `f2`, `r0`, `r1`,
+`r2`, and `rocky`. So when earth's `wg0` is up, you can SSH directly — no ProxyJump
+needed:
+
+```sh
+ssh paul@f0.wg0 # also f1.wg0, f2.wg0
+ssh root@r0.wg0 # also r1.wg0, r2.wg0
+ssh root@rocky.wg0
+```
+
+This is earth-specific (other roaming clients like `pixel7pro` still need ProxyJump),
+returns go via fishfinger only, and it is currently a **manual, non-durable** change
+(reverted by a `wireguardmeshgenerator` regen; durability tracked as the generator
+`+reachableRoaming` task). See `wireguard.md` → "Direct SSH from a roaming client
+to mesh hosts" for details and constraints. The ProxyJump method below remains the
+general fallback and the way to reach hosts not yet patched (e.g. `pi0`/`pi1`, `f3`).
+
## Working method: jump through an OpenBSD gateway
**fishfinger** and **blowfish** are reachable from the public internet *and*
diff --git a/prompts/skills/f3s/references/wireguard.md b/prompts/skills/f3s/references/wireguard.md
index 2246578..126b86d 100644
--- a/prompts/skills/f3s/references/wireguard.md
+++ b/prompts/skills/f3s/references/wireguard.md
@@ -171,7 +171,23 @@ Endpoint = 46.23.94.99:56709
PersistentKeepalive = 25
```
-Roaming clients route all traffic (`0.0.0.0/0`) through gateways, only connect to blowfish/fishfinger, and cannot be directly reached by LAN hosts.
+Roaming clients route all traffic (`0.0.0.0/0`) through gateways and only **peer** to blowfish/fishfinger (they are not full-mesh members). By default they cannot be directly reached by mesh hosts, because no mesh host carries the roaming client's wg0 IP in any peer's `AllowedIPs` — so the mesh host has no return route and sends replies out its LAN interface, where they are lost.
+
+### Direct SSH from a roaming client to mesh hosts (earth enhancement)
+
+`earth` is an exception: its wg0 IP (`192.168.2.200/32`, `fd42:beef:cafe:2::200/128`) has been added to the **fishfinger** peer's `AllowedIPs` on `f0`, `f1`, `f2`, `r0`, `r1`, `r2`, and `rocky`, so those hosts route `192.168.2.200` back through `wg0` to fishfinger, which forwards to earth. This lets you SSH directly from earth over the VPN with no ProxyJump:
+
+```sh
+ssh paul@f0.wg0 # also f1.wg0, f2.wg0
+ssh root@r0.wg0 # also r1.wg0, r2.wg0
+ssh root@rocky.wg0
+```
+
+Constraints / notes:
+- earth still only **peers** to the gateways — reachability is via gateway forwarding plus a return route on the mesh side, not a direct peer relationship.
+- Returns go via **fishfinger only**, not blowfish. earth's `blowfish` peer ends up with `allowed ips: (none)` in the running config because both gateway peers are configured with `0.0.0.0/0, ::/0` and wg-quick can only install one default route (see the dual-`0.0.0.0/0` troubleshooting note below). Returns via blowfish would be dropped by earth.
+- `pi0`/`pi1` are not yet reachable directly from earth; they need the same `.200` addition to their fishfinger peer if desired.
+- This is currently a **manual, non-durable** change — it is reverted by any `wireguardmeshgenerator` regen because earth is in every infra host's `exclude_peers`. The durable version is tracked as generator `ask` tasks (`+reachableRoaming`); see the `wireguardmeshgenerator` project task list.
## /etc/hosts Entries for WireGuard
@@ -249,6 +265,23 @@ ipv4_with_mask = hosts[myself]['os'] == 'FreeBSD' ? "#{ipv4}/32" : ipv4
Note: `reload` only reconfigures peers/PSKs — it does not change the running interface address. A `restart` is needed to pick up the address change if the interface is already running.
+## Troubleshooting: Regenerated Keypair Breaks the Tunnel
+
+If a host's wg0 PrivateKey (and PSKs) are regenerated out-of-band (e.g. OS reinstall) but the peer configs on the other side are not updated, handshakes silently fail: `wg show` shows `0 B received` on the client and `endpoint: (none), rx=0` for that peer on the gateway, even though packets leave the client (confirmed with `tcpdump` on the wifi interface). WireGuard drops initiations it cannot authenticate and emits nothing, so the symptom is one-way traffic with no replies.
+
+All three must match on both sides:
+- the client's **PublicKey** as configured in the gateway's peer block,
+- the gateway's **PublicKey** as configured in the client's peer block (must equal `wg show wg0 public-key` on the gateway),
+- the per-pair **PresharedKey** (must be identical on both sides).
+
+When fixing this by hand, also update the `wireguardmeshgenerator` `keys/` directory (`keys/<host>/priv.key`, `pub.key`, `keys/psk/<sorted_pair>.key`) so a regen reproduces the live configs — otherwise the next `--generate`/`--install` reverts the fix. This is tracked as the generator `+credentials` task.
+
+## Troubleshooting: Dual `0.0.0.0/0` on Roaming Clients
+
+A roaming client config that gives `AllowedIPs = 0.0.0.0/0, ::/0` to **both** gateway peers (as the generator currently does for `gateway: true`) is only partially functional: wg-quick can install only one default route, so the second peer silently ends up with `allowed ips: (none)` in the running config and is **not** a real failover. On `earth`, `sudo wg show` shows fishfinger with `0.0.0.0/0, ::/0` and blowfish with `(none)`.
+
+Consequence: all return traffic to earth must go via fishfinger (the peer earth actually accepts traffic from). This is why the direct-SSH return route is added to the fishfinger peer only. A proper fix (single primary gateway with failover, or `Table = off` with policy routing) is tracked as the generator `+roamingFailover` task.
+
## Traffic Flows
| Flow | Purpose |
@@ -260,3 +293,4 @@ Note: `reload` only reconfigures peers/PSKs — it does not change the running i
| rN ↔ rM | k3s intra-cluster traffic |
| fN ↔ fM | zrepl storage replication |
| earth/pixel7pro ↔ gateways | Remote access (all traffic routed through VPN) |
+| earth ↔ fN/rN/rocky (via fishfinger) | Direct SSH from the VPN to mesh hosts (earth's IP added to the fishfinger peer AllowedIPs on those hosts; no ProxyJump needed) |