summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2026-05-06 10:00:26 +0300
committerPaul Buetow <paul@buetow.org>2026-05-06 10:00:26 +0300
commit60e00931aed47b7ce980575e37d43336bbb0914a (patch)
treec44f62fee141cf31e51db0fcaeef9eec9a691770
parentd78a2530da91b76625b71c2aeaf3293abc6c3a4b (diff)
fix
-rw-r--r--AGENTS.md13
-rw-r--r--Magefile.go19
-rw-r--r--README.md41
-rw-r--r--docs/parquet-querying.md2
-rw-r--r--docs/tutorial/TUTORIAL.md245
5 files changed, 36 insertions, 284 deletions
diff --git a/AGENTS.md b/AGENTS.md
index 3eceece..ac0b13e 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -15,16 +15,17 @@ make -C ../libbpfgo libbpfgo-static
If builds/tests fail with missing libbpf headers (for example `bpf/bpf.h` not found), rerun the commands above and then run `mage world`. Prefer Mage targets over raw `go test` for packages that import `libbpfgo`; Mage wires the required `CGO_CFLAGS`, `CGO_LDFLAGS`, and `LIBBPFGO` values.
```bash
-mage all # Build everything (BPF objects and Go binary)
+mage build # Build BPF object + Go binary (all is an alias)
mage buildDocker # Build ior inside a Rocky Linux 9 container (writes binary to repo root)
-mage test # Run all tests
+mage test # Run all tests
TEST_NAME=TestEventloop mage testWithName # Run specific test
-mage integrationTest # Build + run integration tests (default parallelism is capped)
-INTEGRATION_PARALLEL=1 mage integrationTest # Force serial integration tests
+mage integrationTest # Build + run integration tests in parallel (parallelism capped to NumCPU)
+mage integrationTestSerial # Build + run integration tests one at a time
mage generate # Generate code (required after modifying tracepoint definitions)
mage bench # Run benchmarks
mage prReview # Run PR review baseline: world + benchProf
mage clean # Clean build artifacts
+mage mrproper # Clean + remove generated outputs (*.zst, *.svg, *.prof, *.pdf, *.tmp…)
mage world # Clean + generate + test + build (recommended reset path)
mage demo # Regen docs/tutorial/ GIFs + screenshots (needs vhs+ttyd, sudo -v warmed)
TAPE=07-stream-live mage demoOne # Regen one demo tape only
@@ -33,7 +34,7 @@ mage installDemoTools # One-time: install vhs (go install) and ttyd (dnf)
## Demo Pipeline
-`docs/tutorial/` holds the reproducible TUI demo: 14 [VHS](https://github.com/charmbracelet/vhs) `.tape` files under `docs/tutorial/tapes/` drive every dashboard tab and headless mode, write GIFs and PNGs into `docs/tutorial/assets/`, and the resulting tutorial is `docs/tutorial/TUTORIAL.md`. Background workload is generated by `docs/tutorial/scripts/workload.sh`. `mage demo` is fully headless (no real terminal window) — safe to run in the background while editing code; the only foreground requirement is one `sudo -v` to pre-warm the sudo timestamp.
+`docs/tutorial/` holds the reproducible TUI demo: 14 [VHS](https://github.com/charmbracelet/vhs) `.tape` files under `docs/tutorial/tapes/` drive every dashboard tab and headless mode, write GIFs and PNGs into `docs/tutorial/assets/`, and the resulting tutorial is `docs/tutorial/tutorial.md`. Background workload is generated by `docs/tutorial/scripts/workload.sh`. `mage demo` is fully headless (no real terminal window) — safe to run in the background while editing code; the only foreground requirement is one `sudo -v` to pre-warm the sudo timestamp.
## Code Generation
@@ -71,7 +72,7 @@ Generator source code:
- **TUI trace flow** ingests events into the in-memory stats engine; it does **not** continuously write trace rows to disk.
- **File output in TUI** is explicit export only (`e`), writing `ior-stream-<timestamp>.csv` in the current directory from the current filtered stream snapshot.
- **Export toggle flag**: `-tuiExport=true|false` (default `true`) enables or disables TUI stream CSV export at runtime.
-- **Tab navigation** supports `tab/shift+tab`, numeric keys `1..6`, and directional keys `left/right` and `h/l`.
+- **Tab navigation** supports `tab/shift+tab`, numeric keys `1..7`, and directional keys `left/right` and `h/l`.
- **When export is disabled**, export key hints are hidden from dashboard help and `e` does not open export modal.
## Code Style
diff --git a/Magefile.go b/Magefile.go
index e675a8a..dcde50e 100644
--- a/Magefile.go
+++ b/Magefile.go
@@ -67,10 +67,9 @@ func GoBuildRace() error {
"-race", "-o", binaryName, "./cmd/ior/main.go")
}
-// All builds the BPF object and the Go binary.
+// All is an alias for Build (BPF object + Go binary).
func All() error {
- mg.SerialDeps(Build)
- return nil
+ return Build()
}
// BuildDocker builds the ior binary inside a Rocky Linux 9 Docker container
@@ -403,7 +402,8 @@ func Clean() error {
// Mrproper removes build artifacts and generated output files.
func Mrproper() error {
mg.SerialDeps(Clean)
- patterns := []string{"*.zst", "*.svg", "*profile", "*.pdf", "*.tmp", "palette.map"}
+ // *.prof covers flame-cpu.prof / flame-mem.prof written by benchFlameProf.
+ patterns := []string{"*.zst", "*.svg", "*profile", "*.prof", "*.pdf", "*.tmp", "palette.map"}
for _, pattern := range patterns {
if err := removeFilesByGlob(pattern); err != nil {
return err
@@ -435,21 +435,16 @@ func World() error {
}
// IntegrationTest builds everything and runs integration tests in parallel.
+// Set INTEGRATION_PARALLEL to tune `go test -parallel` (default: NumCPU, minimum 1).
func IntegrationTest() error {
return runIntegrationTests(true)
}
-// IntegrationTestSerial builds everything and runs integration tests serially.
+// IntegrationTestSerial builds everything and runs integration tests one at a time.
func IntegrationTestSerial() error {
return runIntegrationTests(false)
}
-// IntegrationTestParallel builds everything and runs integration tests in parallel.
-// Set INTEGRATION_PARALLEL to tune `go test -parallel` (default: NumCPU*2, minimum 1).
-func IntegrationTestParallel() error {
- return runIntegrationTests(true)
-}
-
func runIntegrationTests(parallel bool) error {
mg.SerialDeps(All)
if err := buildWorkloadBinary(); err != nil {
@@ -1215,7 +1210,7 @@ func Demo() error {
}
// DemoOne regenerates a single tape. Pass TAPE=07-stream-live (basename without
-// .tape) or TAPE=demo/tapes/07-stream-live.tape (full path).
+// .tape) or TAPE=docs/tutorial/tapes/07-stream-live.tape (full path).
func DemoOne() error {
mg.SerialDeps(Build)
if err := buildWorkloadBinary(); err != nil {
diff --git a/README.md b/README.md
index 739e681..01e07b6 100644
--- a/README.md
+++ b/README.md
@@ -12,52 +12,53 @@ This works only on Linux!
## Demo
-A short guided tour with animated GIFs of every major surface lives in [`docs/tutorial/TUTORIAL.md`](./docs/tutorial/TUTORIAL.md). Two teasers:
+A short guided tour with animated GIFs of every major surface lives in [`docs/tutorial/tutorial.md`](./docs/tutorial/tutorial.md). Two teasers:
+
+**Startup — the PID picker:** `sudo ./ior` opens a searchable process list. Navigate with arrow keys, filter by typing, and press `Enter` to start tracing. The dashboard appears immediately after.
<img src=docs/tutorial/assets/01-launch.gif width=720 alt="Cold start: PID picker, then the dashboard appears" />
+**Live flamegraph tab:** Once tracing, tab `1` shows a live flamegraph that rebuilds in real time as I/O events arrive. Bars grow and shift with the workload — this is the default landing tab.
+
<img src=docs/tutorial/assets/13-tui-flamegraph.gif width=720 alt="Live in-TUI flamegraph rebuilding from real workload" />
-The demo is fully reproducible: `mage installDemoTools` once, then `sudo -v && mage demo` regenerates every GIF and screenshot. See the [tutorial](./docs/tutorial/TUTORIAL.md) for the full walkthrough.
+The demo is fully reproducible: `mage installDemoTools` once, then `sudo -v && mage demo` regenerates every GIF and screenshot. See the [tutorial](./docs/tutorial/tutorial.md) for the full walkthrough.
## Requirements
-- Go 1.26 or newer (ior relies on cgo via libbpfgo).
-- Linux with a BTF-enabled kernel (`/sys/kernel/btf/vmlinux` present).
+- Docker (for the official build) **or** a Linux host with Go 1.26+, clang, and libbpfgo for native development builds.
+- Linux with a BTF-enabled kernel (`/sys/kernel/btf/vmlinux` present) to run `ior`.
## Build
-### Docker build (recommended — no toolchain setup required)
+The officially supported build method is Docker — no local Go, clang, or libbpfgo setup needed. Native builds are supported for contributors who want to iterate quickly without Docker.
-Builds the static `ior` binary inside a Rocky Linux 9 container and writes it
-to the repo root. Requires only Docker and a Linux host with tracefs and BTF:
+### Docker build (official)
+
+Builds a fully static `ior` binary inside a Rocky Linux 9 container and writes
+it to the repo root:
```shell
mage buildDocker
```
-On first run this takes ~15–20 minutes to build the image. Subsequent runs
-reuse the cached image and finish in under a minute. To skip the image build:
+First run takes ~15–20 minutes to build the image; subsequent runs reuse the
+cached image and finish in under a minute. To skip the image rebuild:
```shell
./scripts/build-with-docker.sh --run
```
-### Native build
+### Native build (development)
-`ior` links against a locally built `libbpfgo`. Clone it as a sibling of this
-repo and build the static archive once:
+For local development, `ior` links against a locally built `libbpfgo`. Clone it
+as a sibling of this repo and build the static archive once:
```shell
git clone https://github.com/aquasecurity/libbpfgo ../libbpfgo
git -C ../libbpfgo checkout v0.9.2-libbpf-1.5.1
git -C ../libbpfgo submodule update --init --recursive
make -C ../libbpfgo libbpfgo-static
-```
-
-Then build everything:
-
-```shell
mage world
```
@@ -77,8 +78,8 @@ explanation.
## TUI
Press **H** inside the dashboard to toggle the built-in help panel. Tabs are
-reachable with **tab/shift+tab** or number keys **1–6**. Full hotkey reference:
-[docs/tutorial/TUTORIAL.md](./docs/tutorial/TUTORIAL.md#hotkey-quick-reference).
+reachable with **tab/shift+tab** or number keys **1–7**. Full hotkey reference:
+[docs/tutorial/tutorial.md](./docs/tutorial/tutorial.md#hotkey-quick-reference).
## Recording Modes
@@ -92,4 +93,4 @@ reachable with **tab/shift+tab** or number keys **1–6**. Full hotkey reference
| Parquet recording | press `R` in TUI, or `-parquet <file>` | streaming Parquet file |
Full details and the `.ior.zst` vs Parquet trade-off:
-[docs/tutorial/TUTORIAL.md](./docs/tutorial/TUTORIAL.md#recording-for-offline-analysis).
+[docs/tutorial/tutorial.md](./docs/tutorial/tutorial.md#recording-for-offline-analysis).
diff --git a/docs/parquet-querying.md b/docs/parquet-querying.md
index e940f9c..5372325 100644
--- a/docs/parquet-querying.md
+++ b/docs/parquet-querying.md
@@ -1,6 +1,6 @@
# Querying ior Parquet recordings with ClickHouse
-ior can record I/O events to a Parquet file (see `--record` flag). This document explains
+ior can record I/O events to a Parquet file (press `R` in TUI, or use the `-parquet` flag in headless mode). This document explains
how to explore those recordings interactively using `clickhouse local`, which is bundled
inside the standard `clickhouse/clickhouse-server` Docker image — no server, no persistent
state, no installation needed beyond Docker.
diff --git a/docs/tutorial/TUTORIAL.md b/docs/tutorial/TUTORIAL.md
deleted file mode 100644
index e53013a..0000000
--- a/docs/tutorial/TUTORIAL.md
+++ /dev/null
@@ -1,245 +0,0 @@
-# I/O Riot NG: a guided tour
-
-This tutorial walks through every major surface of `ior` — the dashboard tabs, the live stream, recording, headless modes, and the in-TUI flamegraph — using short animated GIFs so you can *see* what the keys actually do.
-
-Every GIF in this document is regenerated from a [VHS](https://github.com/charmbracelet/vhs) tape under [`demo/tapes/`](./tapes). To rebuild them all, run `sudo -v && mage demo` (see [Regenerating the demo](#regenerating-the-demo)).
-
-## Contents
-
-1. [Installing ior](#installing-ior)
-2. [First launch: the PID picker](#first-launch-the-pid-picker)
-3. [Touring the dashboard tabs](#touring-the-dashboard-tabs)
- - [1 · Flamegraph (default landing tab)](#1--flamegraph-default-landing-tab)
- - [2 · Overview](#2--overview)
- - [3 · Syscalls](#3--syscalls)
- - [4 · Files](#4--files)
- - [5 · Processes](#5--processes)
- - [6 · Latency + Gaps](#6--latency--gaps)
- - [7 · Stream](#7--stream)
-4. [Mastering the Stream tab](#mastering-the-stream-tab)
- - [Pause + stacked filters](#pause--stacked-filters)
- - [Regex search](#regex-search)
- - [CSV export](#csv-export)
-5. [Choosing what to trace](#choosing-what-to-trace)
-6. [Recording for offline analysis](#recording-for-offline-analysis)
- - [TUI Parquet recording](#tui-parquet-recording)
- - [Headless modes](#headless-modes)
-7. [Regenerating the demo](#regenerating-the-demo)
-
-## Installing ior
-
-See the [main README](../../README.md) for full install steps. The quickest path from any Docker-capable Linux host:
-
-```shell
-git clone https://codeberg.org/snonux/ior ~/git/ior
-cd ~/git/ior
-mage buildDocker # builds inside a Rocky 9 container, ~15 min on first run
-```
-
-For a native build (libbpfgo must be cloned alongside the repo first — see the README):
-
-```shell
-mage all
-```
-
-ior needs `CAP_BPF`, so every invocation below uses `sudo`.
-
-The build dance only has to happen once: the resulting `ior` binary is fully statically linked and uses CO-RE, so the same binary runs on any BTF-enabled Linux kernel without recompilation. See the [Compile once, run everywhere](../../README.md#compile-once-run-everywhere) section for details.
-
-## First launch: the PID picker
-
-`sudo ./ior` starts with the **PID picker**. The cursor is on **All PIDs**, so pressing `Enter` traces the whole system. Type into the filter box to narrow the list by PID, comm, or cmdline; arrow keys move the selection.
-
-![Cold start: PID picker, then the dashboard appears](./assets/01-launch.gif)
-
-The same picker can be re-opened later from the dashboard with `p`.
-
-![PID picker default state](./assets/screenshot-pidpicker.png)
-
-## Touring the dashboard tabs
-
-The dashboard has seven tabs, addressable by number key. The default landing tab is **Flamegraph**. `tab` / `shift+tab` step forward / back.
-
-| Key | Tab | What it shows |
-|-----|-------------------|--------------------------------------------------------------------------|
-| `1` | Flamegraph (`Flm`)| Live FlameGraph of the configured stack (`comm`/`path`/`tracepoint`) |
-| `2` | Overview (`Ovr`) | Sparkline + top syscalls + top paths summary |
-| `3` | Syscalls (`Sys`) | Sortable per-syscall counters, latency, byte volume |
-| `4` | Files (`Fil`) | Per-path counters; `d` toggles directory grouping |
-| `5` | Processes (`Pro`) | Per-process / per-comm counters |
-| `6` | Latency (`Lat`) | Latency + inter-syscall gap histograms |
-| `7` | Stream (`Str`) | Live tail of individual traced events |
-
-### 1 · Flamegraph (default landing tab)
-
-The first thing you see after dismissing the PID picker is the **live flamegraph**. Bars grow as new events come in. `o` cycles the stack ordering (e.g. `comm/path/tracepoint` ↔ `comm/tracepoint/path`); `b` toggles the size metric (event count vs. duration vs. gap).
-
-![Live flamegraph rebuilding from real workload](./assets/13-tui-flamegraph.gif)
-
-### 2 · Overview
-
-Press `2`. The Overview tab is the at-a-glance view: a sparkline of recent event volume, the top syscalls, and the top paths.
-
-![Overview tab populating](./assets/02-overview-tab.gif)
-
-### 3 · Syscalls
-
-Press `3`. A sortable table of every traced syscall (count, average latency, total bytes). `j` / `k` (or arrow keys) scroll the rows; `←` / `→` move the selected column; `s` sorts by the selected column using its default direction; `S` reverses.
-
-![Syscalls table with sort + reverse-sort](./assets/03-syscalls-tab.gif)
-
-### 4 · Files
-
-Press `4`. Per-path counters. The most useful key here is `d`, which toggles **directory grouping** — paths roll up into their parent directory, which is essential when one process touches thousands of files in `/usr/share/...`.
-
-![Files tab toggling directory grouping](./assets/04-files-tab.gif)
-
-### 5 · Processes
-
-Press `5`. Per-process / per-comm view. `S` reverse-sorts; combine with `←` / `→` to pick a column.
-
-![Processes tab](./assets/05-processes-tab.gif)
-
-### 6 · Latency + Gaps
-
-Press `6`. Two histograms: syscall **latency** (how long the syscall ran) and the inter-syscall **gap** (idle time on the same thread between syscalls). The big-write workload running in the background spreads the latency distribution noticeably.
-
-![Latency + gap histograms](./assets/06-latency-gaps-tab.gif)
-
-### 7 · Stream
-
-Press `7`. A live tail of every traced event row — comm, PID, TID, syscall, file, FD, return value, bytes, latency, gap. This is the workhorse view; the next section explores it in depth.
-
-![Stream tab live-tailing rows](./assets/07-stream-live.gif)
-
-## Mastering the Stream tab
-
-Stream has two modes: **Live** (rows scroll past) and **Pause** (`space` toggles). Almost everything interesting happens in pause mode.
-
-### Pause + stacked filters
-
-In pause mode, navigate with `j`/`k` (rows) and `←` / `→` (columns). Pressing `Enter` on the selected cell **pushes a new filter onto a stack** and immediately re-filters the ring buffer. Filters are stackable, so you can drill down — first by `Comm`, then by `Syscall`, then by `File`. `Esc` pops the most recent filter (LIFO); keep hitting `Esc` to undo all the way back.
-
-![Pause, push two filters, undo with Esc](./assets/08-stream-pause-filter.gif)
-
-The filter is reflected in the bottom status line, and matches the same syntax you'd type by hand: `comm~bash`, `syscall~openat`, `latency>=100000`, etc.
-
-### Regex search
-
-`/` opens a forward regex prompt; `?` opens a backward one. `n` jumps to the next match in the same direction; `N` reverses. The search runs against every column on every row in the ring buffer and wraps at the end.
-
-![Regex search with /, n, n, then ?](./assets/09-stream-regex-search.gif)
-
-### CSV export
-
-Three keys, three flavours:
-
-- `e` — quick export of the **current TUI-filter snapshot** to `ior-stream-<timestamp>.csv` in the current working directory. Works from any tab, not just Stream.
-- `x` — quick export of the **paused stream view** specifically (preserves your filter stack).
-- `X` — same as `x`, but prompts for a filename first.
-- `E` — open the most recent stream-exported CSV in your `$EDITOR` (`hx` / `vi` fallback).
-
-![Press 'e', then ls the resulting CSV](./assets/10-stream-csv-export.gif)
-
-If you don't want CSV export at all, start ior with `-tuiExport=false`; the help footer hides the export keys and `e` becomes a no-op.
-
-## Choosing what to trace
-
-Three modal pickers reshape what the rest of the TUI sees:
-
-- `p` — **PID picker** (re-opens the launch picker).
-- `t` — **TID picker** for thread-level focus.
-- `o` — **Probes** dialog: enable / disable individual syscall tracepoints.
-
-![PID, TID, and probe pickers](./assets/11-pid-tid-probe.gif)
-
-Restricting to a single PID is also exposed as a CLI flag (`-pid <n>`), as is comm/path filtering (`-comm`, `-path`). Tracepoint subsetting on the command line uses `-tps <regex>` / `-tpsExclude <regex>`.
-
-## Recording for offline analysis
-
-ior has three persistence flows; each solves a different problem.
-
-| Flow | How | What you get |
-|------------------------|----------------------------------------------|---------------------------------------------------------|
-| TUI Parquet recording | `R` from the dashboard | streaming Parquet of every row that passes your filter |
-| Headless `.ior.zst` | `sudo ./ior -flamegraph -name <name>` | one aggregated native trace artifact (bandwidth-cheap) |
-| Headless Parquet | `sudo ./ior -parquet trace.parquet` | streaming Parquet, full firehose, no TUI |
-| Plain CSV | `sudo ./ior -plain` | one CSV row per event on stdout |
-
-### TUI Parquet recording
-
-Press `R` in the dashboard, accept the default filename (`ior-recording-<timestamp>.parquet`) with `Enter`, and rows start streaming to disk. The footer shows the active recording path (or the last error). Press `R` again to stop.
-
-![Start, run, and stop a parquet recording](./assets/12-parquet-recording.gif)
-
-The recorder follows your *current* TUI global filter — narrow with `p`/`t`/`o` first if you want a focused capture.
-
-### Headless modes
-
-For unattended captures or scripting, skip the TUI entirely. The demo runs all three back-to-back, capped with `-duration` so each terminates on its own.
-
-![Three headless flows in one tape](./assets/14-headless-modes.gif)
-
-`-flamegraph` writes one aggregated `.ior.zst` artifact at shutdown — ideal for `ior`'s native flamegraph and integration workflows. `-parquet` streams every row, so the file grows continuously. `-plain` is the lightest weight: CSV to stdout you can pipe into anything.
-
-## Regenerating the demo
-
-The whole asset pipeline is reproducible:
-
-```shell
-mage installDemoTools # one-time: VHS via go install + ttyd via dnf
-sudo -v # warm the sudo timestamp once
-mage demo # regen all 14 GIFs + screenshots (~10 min)
-```
-
-Or rebuild a single tape after editing it:
-
-```shell
-TAPE=07-stream-live mage demoOne
-```
-
-Tapes live in [`tapes/`](./tapes), the background workload that drives them is [`scripts/workload.sh`](./scripts/workload.sh), and the resulting assets land in [`assets/`](./assets). VHS records headlessly under `ttyd` + Chromium — no real terminal window opens, so `mage demo` is safe to run in the background while you keep working.
-
-## Hotkey Quick Reference
-
-### Global keys
-
-| Key | Action |
-|-----|--------|
-| `tab` / `shift+tab` | next / previous tab |
-| `1`–`6` (`7` = alias for `6`) | jump to tab by number |
-| `H` | toggle bottom help panel |
-| `e` | export filtered stream snapshot to CSV |
-| `R` | start / stop Parquet recording |
-| `p` | re-open PID picker |
-| `t` | open TID picker |
-| `o` | open probe selection dialog |
-| `r` | refresh dashboard snapshot |
-| `q` / `ctrl+c` | quit |
-
-### Tab-specific keys (`2:Syscalls`, `3:Files`, `4:Processes`)
-
-| Key | Action |
-|-----|--------|
-| `s` | sort by selected column (default direction) |
-| `S` | reverse-sort by selected column |
-| `j`/`k` or `↑`/`↓` | scroll list |
-| `d` (Files only) | toggle directory grouping |
-
-### Stream tab (`7:Stream`)
-
-| Key | Action |
-|-----|--------|
-| `space` | toggle live / pause mode |
-| `g` / `G` | jump to top / tail |
-| `j`/`k` or `↑`/`↓` | move row (pause) / scroll (live) |
-| `←`/`→` or `h`/`l` | move selected column (pause only) |
-| `enter` | push cell value as filter (pause) |
-| `esc` | pop most recent filter (LIFO) |
-| `c` | clear all stream filters |
-| `f` | open advanced filter modal |
-| `/` / `?` | regex search forward / backward |
-| `n` / `N` | next / previous search match |
-| `x` | quick CSV export of paused view |
-| `X` | CSV export with filename prompt |
-| `E` | open last CSV export in `$EDITOR` |