diff options
| -rw-r--r-- | gemfeed/2026-05-08-unveiling-ior-ng-part-1.html | 20 | ||||
| -rw-r--r-- | gemfeed/2026-05-11-unveiling-ior-ng-part-2.html | 29 | ||||
| -rw-r--r-- | gemfeed/2026-05-17-unveiling-ior-ng-part-3.html (renamed from gemfeed/DRAFT-unveiling-ior-ng-part-3.html) | 47 | ||||
| -rw-r--r-- | gemfeed/DRAFT-unveiling-ior-ng-part-2.html | 248 | ||||
| -rw-r--r-- | gemfeed/atom.xml | 416 | ||||
| -rw-r--r-- | gemfeed/index.html | 5 | ||||
| -rw-r--r-- | index.html | 7 | ||||
| -rw-r--r-- | uptime-stats.html | 277 |
8 files changed, 577 insertions, 472 deletions
diff --git a/gemfeed/2026-05-08-unveiling-ior-ng-part-1.html b/gemfeed/2026-05-08-unveiling-ior-ng-part-1.html index 980b9768..a4917493 100644 --- a/gemfeed/2026-05-08-unveiling-ior-ng-part-1.html +++ b/gemfeed/2026-05-08-unveiling-ior-ng-part-1.html @@ -2,7 +2,7 @@ <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>Unveiling I/O Riot NG — Part 1: a guided tour</title> +<title>Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</title> <link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> <link rel="stylesheet" href="../style.css" /> <link rel="stylesheet" href="style-override.css" /> @@ -11,7 +11,7 @@ <p class="header"> <a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/2026-05-08-unveiling-ior-ng-part-1.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/2026-05-08-unveiling-ior-ng-part-1.gmi">Gemini</a> | <a href="https://snonux.foo">Microblog</a> | <a href="https://irregular.ninja">Street photography</a> </p> -<h1 style='display: inline' id='unveiling-io-riot-ng--part-1-a-guided-tour'>Unveiling I/O Riot NG — Part 1: a guided tour</h1><br /> +<h1 style='display: inline' id='unveiling-io-riot-ng-v100--part-1-a-guided-tour'>Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</h1><br /> <br /> <span class='quote'>Published at 2026-05-07T09:46:29+03:00</span><br /> <br /> @@ -23,6 +23,7 @@ <br /> <a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG — Part 1: a guided tour (You are currently reading this)</a><br /> <a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a><br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG — Part 3: under the hood</a><br /> <br /> <a class='textlink' href='https://codeberg.org/snonux/ior'>I/O Riot NG on Codeberg</a><br /> <a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>the original I/O Riot post (2018)</a><br /> @@ -30,7 +31,7 @@ <h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> <br /> <ul> -<li><a href='#unveiling-io-riot-ng--part-1-a-guided-tour'>Unveiling I/O Riot NG — Part 1: a guided tour</a></li> +<li><a href='#unveiling-io-riot-ng-v100--part-1-a-guided-tour'>Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</a></li> <li>⇢ <a href='#what-it-does'>What it does</a></li> <li>⇢ <a href='#first-launch'>First launch</a></li> <li>⇢ <a href='#the-seven-tabs-in-30-seconds-each'>The seven tabs, in 30 seconds each</a></li> @@ -43,6 +44,7 @@ <li>⇢ <a href='#the-stream-tab-is-the-good-one'>The Stream tab is the good one</a></li> <li>⇢ <a href='#filtering-more-thoroughly'>Filtering, more thoroughly</a></li> <li>⇢ <a href='#recording'>Recording</a></li> +<li>⇢ <a href='#what-s-new-in-v110'>What's new in v1.1.0</a></li> <li>⇢ <a href='#what-s-still-missing'>What's still missing</a></li> </ul><br /> <h2 style='display: inline' id='what-it-does'>What it does</h2><br /> @@ -218,6 +220,17 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Once a parquet file is on disk, point any SQL-over-parquet tool at it — Part 3 walks through ClickHouse Local, with real query output against a 30-second capture.</span><br /> <br /> +<h2 style='display: inline' id='what-s-new-in-v110'>What's new in v1.1.0</h2><br /> +<br /> +<span>A handful of TUI additions landed in <span class='inlinecode'>v1.1.0</span> after this post was originally written against <span class='inlinecode'>1.0.0</span>. Nothing in the tour above became wrong, but a few keystrokes do more than they used to:</span><br /> +<br /> +<ul> +<li>Three-way metric cycle on the flamegraph. <span class='inlinecode'>b</span> now toggles count → bytes → duration; the new "duration" mode weights bars by total syscall latency, which answers "where is the wall-clock time going?" in one keystroke. The same metric is wired through the bubble, treemap, and icicle views.</li> +<li>Auto-reset timer for the live aggregates. <span class='inlinecode'>-resetTimer=<dur></span> (default <span class='inlinecode'>30s</span>, <span class='inlinecode'>0</span> disables) sets the cadence at launch; the <span class='inlinecode'>I</span> hotkey cycles <span class='inlinecode'>off → 10s → 30s → 60s → 2m → 5m → off</span> while ior is running, and the dashboard chrome shows the remaining countdown. Same effect as hitting <span class='inlinecode'>r</span> on a schedule — keeps the live trie and stats engine bounded on long traces without you remembering to do it.</li> +<li>In-place global filter swap. Pushing or popping the global filter (the Enter-on-a-cell trick, the PID/TID/probe pickers, <span class='inlinecode'>ESC</span> to pop) no longer detaches and reattaches every BPF tracepoint, so the "Attaching tracepoints..." overlay that used to flash for several seconds on busy I/O boxes is gone. Filter changes are now instant.</li> +<li>Flame graph TUI keeps up under heavy load. Per-tick snapshot refresh runs on a background goroutine, navigation walks a precomputed ancestry index, and <span class='inlinecode'>View()</span> output is memoized. Keystrokes (pause, zoom, navigate, search) land within one frame even when the live trie is ingesting thousands of events per tick.</li> +<li><span class='inlinecode'>-tui-fast-refresh=<dur></span> (default <span class='inlinecode'>250ms</span>, <span class='inlinecode'>0</span> disables) makes the flamegraph and stream tabs' high-frequency refresh cadence configurable, in case you want a lighter feel on a slow terminal or a punchier one on a busy workload.</li> +</ul><br /> <h2 style='display: inline' id='what-s-still-missing'>What's still missing</h2><br /> <br /> <ul> @@ -240,6 +253,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Other related posts are:</span><br /> <br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG — Part 3: under the hood</a><br /> <a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a><br /> <a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG — Part 1: a guided tour (You are currently reading this)</a><br /> <a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>2018-06-01 Realistic load testing with I/O Riot for Linux</a><br /> diff --git a/gemfeed/2026-05-11-unveiling-ior-ng-part-2.html b/gemfeed/2026-05-11-unveiling-ior-ng-part-2.html index 23d289ca..9f761689 100644 --- a/gemfeed/2026-05-11-unveiling-ior-ng-part-2.html +++ b/gemfeed/2026-05-11-unveiling-ior-ng-part-2.html @@ -2,7 +2,7 @@ <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</title> +<title>Unveiling I/O Riot NG v1.0.0 — Part 2: install and compile once, run everywhere</title> <link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> <link rel="stylesheet" href="../style.css" /> <link rel="stylesheet" href="style-override.css" /> @@ -11,31 +11,33 @@ <p class="header"> <a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/2026-05-11-unveiling-ior-ng-part-2.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/2026-05-11-unveiling-ior-ng-part-2.gmi">Gemini</a> | <a href="https://snonux.foo">Microblog</a> | <a href="https://irregular.ninja">Street photography</a> </p> -<h1 style='display: inline' id='unveiling-io-riot-ng--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</h1><br /> +<h1 style='display: inline' id='unveiling-io-riot-ng-v100--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG v1.0.0 — Part 2: install and compile once, run everywhere</h1><br /> <br /> <span class='quote'>Published at 2026-05-10T22:53:35+03:00</span><br /> <br /> <span>This is Part 2 of three. Part 1 is the demo-driven tour: what ior looks like, how the dashboard tabs work, how filtering and recording behave. This part is about the installation for Rocky Linux 8 and 9 and, more interestingly, why you only have to do that dance on a single machine: the resulting binary is portable to every other Linux box thanks to CO-RE (Compile Once, Run Everywhere) plus full static linking. Part 3 is the under-the-hood companion (per-event schema, async-syscall caveats, the syscall-coverage probe generator, and post-mortem SQL on the parquet output).</span><br /> <br /> -<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG — Part 1: a guided tour</a><br /> +<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</a><br /> <a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere (You are currently reading this)</a><br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG v1.0.0 — Part 3: under the hood</a><br /> <br /> <a href='./unveiling-ior-ng/00-hero-flamegraph.png'><img alt='Live flamegraph' title='Live flamegraph' src='./unveiling-ior-ng/00-hero-flamegraph.png' /></a><br /> <br /> <h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> <br /> <ul> -<li><a href='#unveiling-io-riot-ng--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a></li> +<li><a href='#unveiling-io-riot-ng-v100--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG v1.0.0 — Part 2: install and compile once, run everywhere</a></li> <li>⇢ <a href='#installing-ior'>Installing ior</a></li> <li>⇢ ⇢ <a href='#why-native-installation-is-a-mess'>Why native installation is a mess</a></li> <li>⇢ ⇢ <a href='#what-the-docker-build-is-actually-doing'>What the Docker build is actually doing</a></li> <li>⇢ <a href='#a-short-detour-ebpf-and-libbpfgo'>A short detour: eBPF and libbpfgo</a></li> -<li>⇢ <a href='#co-re--the-part-that-makes-the-binary-actually-portable'>CO-RE — the part that makes the binary actually portable</a></li> +<li>⇢ <a href='#co-re--the-part-that-makes-the-bpf-binary-actually-portable'>CO-RE — the part that makes the BPF binary actually portable</a></li> <li>⇢ ⇢ <a href='#static-linking'>Static linking</a></li> <li>⇢ ⇢ <a href='#go-programs-are-statically-linked-by-default'>Go programs are statically linked by default</a></li> <li>⇢ ⇢ <a href='#cgo-programs-are-not-statically-linked-by-default'><span class='inlinecode'>cgo</span> programs are not statically linked by default.</a></li> <li>⇢ ⇢ <a href='#co-re'>CO-RE</a></li> <li>⇢ <a href='#a-note-on-cgo-overhead'>A note on cgo overhead</a></li> +<li>⇢ <a href='#what-s-new-in-v110'>What's new in v1.1.0</a></li> <li>⇢ <a href='#if-you-want-to-go-deeper'>If you want to go deeper</a></li> </ul><br /> <h2 style='display: inline' id='installing-ior'>Installing ior</h2><br /> @@ -171,7 +173,7 @@ sudo ./ior -plain -duration <font color="#000000">5</font> <br /> <span>I went with libbpfgo specifically because it's a wrapper, not a reimplementation.</span><br /> <br /> -<h2 style='display: inline' id='co-re--the-part-that-makes-the-binary-actually-portable'>CO-RE — the part that makes the binary actually portable</h2><br /> +<h2 style='display: inline' id='co-re--the-part-that-makes-the-bpf-binary-actually-portable'>CO-RE — the part that makes the BPF binary actually portable</h2><br /> <br /> <span>The headline fact about <span class='inlinecode'>ior</span>'s deployment story: build it once on one box, then <span class='inlinecode'>scp ior other-host:/usr/local/bin/</span> to anywhere else and it just runs. No recompile per kernel, no kernel-debuginfo dance, no DKMS hooks. Two mechanisms make that work, and they reinforce each other.</span><br /> <br /> @@ -263,6 +265,14 @@ Old I/O Riot (Systemtap) New ior (libbpf + CO-RE) <br /> <span>Go 1.26, the current release at the time of writing (early May 2026), is the one that finally took a serious bite out of cgo's per-call cost. The runtime can elide a chunk of the bookkeeping for calls that don't need it. Real-world wins depend heavily on the workload, but the rough direction is that cgo now feels closer to "an unusually expensive function call" than to "a context switch", which is the right mental model for almost everyone touching a C library from Go. The shorter version: cgo overhead used to be a real footgun for ports that called into C in the inner loop. With Go 1.26 it's a footnote unless you're doing many millions of small calls per second, in which case batching across the boundary still fixes it.</span><br /> <br /> +<h2 style='display: inline' id='what-s-new-in-v110'>What's new in v1.1.0</h2><br /> +<br /> +<span>Two changes since the <span class='inlinecode'>1.0.0</span> release this post was originally written against touch the install / portability story directly:</span><br /> +<br /> +<ul> +<li><span class='inlinecode'>mage buildDockerEl8</span> is a new build target that produces a sibling binary called <span class='inlinecode'>ior.el8</span>, built inside a Rocky Linux 8 container against its older glibc. Drop it on RHEL/Rocky/Alma 8 hosts where the default Rocky 9-built <span class='inlinecode'>ior</span> would refuse to start with a <span class='inlinecode'>version 'GLIBC_2.34' not found</span> complaint. CO-RE still means one binary across kernel versions; the el8 split is purely about pinning the userspace libc floor lower for older fleets. The Dockerfile mirrors the same source-build dance described above (libelf.a from elfutils, libzstd.a from upstream, Go 1.26 from go.dev), just on top of a Rocky 8 base.</li> +<li>Probe attach is now tolerant of missing tracepoints. Older kernels that don't expose every tracepoint v1.1.0 knows about (because the syscall didn't exist yet on that kernel, or the tracepoint name was renamed under it) log a one-line warning per missing probe and continue, instead of aborting startup. Same static binary, more kernels it actually attaches on, fewer surprises when you <span class='inlinecode'>scp</span> it to a host that's a few major versions behind your build box. Pairs naturally with the el8 build above: a Rocky 8 host running a 4.18 kernel will silently skip the tracepoints that arrived in 5.x and keep tracing the ones that exist.</li> +</ul><br /> <h2 style='display: inline' id='if-you-want-to-go-deeper'>If you want to go deeper</h2><br /> <br /> <span>If any of this sounds interesting and you want to learn how to write your own BPF programs, two books are the standard recommendations and both well worth the time:</span><br /> @@ -273,12 +283,17 @@ Old I/O Riot (Systemtap) New ior (libbpf + CO-RE) </ul><br /> <span>Between the two, Rice teaches you the moving parts and Gregg teaches you what to do with them.</span><br /> <br /> +<span>Read the next post of the series:</span><br /> +<br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>Unveiling I/O Riot NG — Part 3: under the hood</a><br /> +<br /> <span>E-Mail your comments to <span class='inlinecode'>paul@nospam.buetow.org</span> :-)</span><br /> <br /> <span>Other related posts are:</span><br /> <br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG v1.0.0 — Part 3: under the hood</a><br /> <a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere (You are currently reading this)</a><br /> -<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG — Part 1: a guided tour</a><br /> +<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</a><br /> <a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>2018-06-01 Realistic load testing with I/O Riot for Linux</a><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> diff --git a/gemfeed/DRAFT-unveiling-ior-ng-part-3.html b/gemfeed/2026-05-17-unveiling-ior-ng-part-3.html index e38aafd6..0aace800 100644 --- a/gemfeed/DRAFT-unveiling-ior-ng-part-3.html +++ b/gemfeed/2026-05-17-unveiling-ior-ng-part-3.html @@ -2,43 +2,40 @@ <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>Unveiling I/O Riot NG — Part 3: under the hood</title> +<title>Unveiling I/O Riot NG v1.0.0 — Part 3: under the hood</title> <link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> <link rel="stylesheet" href="../style.css" /> <link rel="stylesheet" href="style-override.css" /> </head> <body> <p class="header"> -<a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/DRAFT-unveiling-ior-ng-part-3.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/DRAFT-unveiling-ior-ng-part-3.gmi">Gemini</a> | <a href="https://snonux.foo">Microblog</a> | <a href="https://irregular.ninja">Street photography</a> +<a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/2026-05-17-unveiling-ior-ng-part-3.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/2026-05-17-unveiling-ior-ng-part-3.gmi">Gemini</a> | <a href="https://snonux.foo">Microblog</a> | <a href="https://irregular.ninja">Street photography</a> </p> -<h1 style='display: inline' id='unveiling-io-riot-ng--part-3-under-the-hood'>Unveiling I/O Riot NG — Part 3: under the hood</h1><br /> +<h1 style='display: inline' id='unveiling-io-riot-ng-v100--part-3-under-the-hood'>Unveiling I/O Riot NG v1.0.0 — Part 3: under the hood</h1><br /> <br /> -<span class='quote'>Draft — not in the gemfeed yet. Promote with the usual rename + index dance.</span><br /> +<span class='quote'>Published at 2026-05-16T18:00:00+03:00</span><br /> <br /> <span>This is the third and final post in the series. Part 1 is the demo-driven tour: what ior looks like, how the dashboard tabs work, how the live flamegraph reads, how filtering and recording behave. Part 2 covers the install dance for Rocky Linux 9 and the "compile once, run everywhere" portability story (eBPF, CO-RE, libbpfgo, static linking). This one is the part you read when you've got ior running and want to know what's actually in the data: the per-event schema, the safeguard that keeps syscall coverage current as new kernels ship, the integration test harness that proves it stays current, async-syscall caveats, and what to do with the parquet output once it's on disk.</span><br /> <br /> -<span>If you haven't read Part 1, it's not a hard prerequisite, but the screenshots and key bindings referenced here all live there. Part 2 is also independent of this one; you can read them in either order.</span><br /> +<a href='./unveiling-ior-ng/00-hero-flamegraph.png'><img alt='Live flamegraph' title='Live flamegraph' src='./unveiling-ior-ng/00-hero-flamegraph.png' /></a><br /> <br /> -<a class='textlink' href='./DRAFT-unveiling-ior-ng-part-1.html'>Part 1: a guided tour</a><br /> -<a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>Part 2: install + compile once, run everywhere</a><br /> <a class='textlink' href='https://codeberg.org/snonux/ior'>I/O Riot NG on Codeberg</a><br /> -<a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>the original I/O Riot post (2018)</a><br /> <br /> -<a href='./unveiling-ior-ng/00-logo.png'><img alt='I/O Riot NG logo' title='I/O Riot NG logo' src='./unveiling-ior-ng/00-logo.png' /></a><br /> -<br /> -<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG — Part 1: a guided tour</a><br /> +<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</a><br /> <a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a><br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG — Part 3: under the hood (You are currently reading this)</a><br /> <br /> <h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> <br /> <ul> -<li><a href='#unveiling-io-riot-ng--part-3-under-the-hood'>Unveiling I/O Riot NG — Part 3: under the hood</a></li> +<li><a href='#unveiling-io-riot-ng-v100--part-3-under-the-hood'>Unveiling I/O Riot NG v1.0.0 — Part 3: under the hood</a></li> <li>⇢ <a href='#what-ior-actually-captures-per-event'>What ior actually captures per event</a></li> <li>⇢ ⇢ <a href='#async-syscalls-and-what-latency-means-for-them'>Async syscalls and what "latency" means for them</a></li> <li>⇢ <a href='#keeping-up-with-new-syscalls'>Keeping up with new syscalls</a></li> <li>⇢ ⇢ <a href='#the-integration-test-harness-keeps-the-classifier-honest'>The integration test harness keeps the classifier honest</a></li> <li>⇢ <a href='#querying-a-parquet-trace-with-clickhouse'>Querying a parquet trace with ClickHouse</a></li> <li>⇢ <a href='#asking-an-ai-to-do-the-reading-for-you'>Asking an AI to do the reading for you</a></li> +<li>⇢ <a href='#what-s-new-in-v110'>What's new in v1.1.0</a></li> <li>⇢ <a href='#wrapping-up'>Wrapping up</a></li> </ul><br /> <h2 style='display: inline' id='what-ior-actually-captures-per-event'>What ior actually captures per event</h2><br /> @@ -90,12 +87,14 @@ <br /> <span><span class='inlinecode'>mage integrationTest</span> builds both binaries and runs the suite in parallel up to <span class='inlinecode'>INTEGRATION_PARALLEL</span> (default <span class='inlinecode'>NumCPU * 2</span>). <span class='inlinecode'>mage integrationTestSerial</span> does the same one at a time, which is the right knob when triaging a flake. They need root because of <span class='inlinecode'>CAP_BPF</span>, and they self-skip when not root.</span><br /> <br /> -<span>What this buys, in practice: when CO-RE field offsets shift under me, when libbpfgo bumps a major version, when a new kernel quietly changes which syscalls bookkeep <span class='inlinecode'>bytes</span> at submission vs. completion, the suite's the thing that goes red first. The codegen safeguard tells me "the kernel surface changed". The integration harness tells me "and here's specifically what ior is now getting wrong about it". Together that's a much shorter feedback loop than the old "wait for the next time I notice the flamegraph looks weird" workflow.</span><br /> +<span>What this buys, in practice: when CO-RE field offsets shift under me, when libbpfgo bumps a major version, when a new kernel quietly changes which syscalls bookkeep <span class='inlinecode'>bytes</span> at submission vs. completion, the suite's the thing that goes red first (not that I believe the kernel would introduce such a breaking change, so maybe this is a bad example). The codegen safeguard tells me "the kernel surface changed". The integration harness tells me "and here's specifically what ior is now getting wrong about it". That's much better than the old "wait for the next time I notice the flamegraph looks weird" workflow.</span><br /> <br /> <h2 style='display: inline' id='querying-a-parquet-trace-with-clickhouse'>Querying a parquet trace with ClickHouse</h2><br /> <br /> <span>The schema is flat and stable: <span class='inlinecode'>seq, time_ns, gap_ns, latency_ns, comm, pid, tid, syscall, fd, ret, bytes, file, is_error, filter_epoch</span>. ClickHouse Local reads parquet directly without a server, which makes it a perfect post-mortem tool — point it at the file and run SQL:</span><br /> <br /> +<a class='textlink' href='https://clickhouse.com/docs/operations/utilities/clickhouse-local'>ClickHouse Local — single-binary SQL over Parquet/CSV, no server needed</a><br /> +<br /> <!-- Generator: GNU source-highlight 3.1.9 by Lorenzo Bettini http://www.lorenzobettini.it @@ -163,6 +162,8 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Same trick works in DuckDB (<span class='inlinecode'>duckdb -c "SELECT ... FROM 'trace.parquet'"</span>), pandas, polars, anything that reads Parquet. The point of streaming Parquet rather than ior's native <span class='inlinecode'>.ior.zst</span> format is exactly this: once it's on disk, you're in the standard data-tools ecosystem.</span><br /> <br /> +<a class='textlink' href='https://duckdb.org/'>DuckDB — single-binary embedded SQL, also reads Parquet directly</a><br /> +<br /> <h2 style='display: inline' id='asking-an-ai-to-do-the-reading-for-you'>Asking an AI to do the reading for you</h2><br /> <br /> <span>Parquet is great if you already have an angle of attack. Sometimes you don't. You just want to know "what's hammering this box right now, and is any of it interesting?" That's where pasting a chunk of trace into an LLM and asking it to look turns out to be a surprisingly effective first pass. ior's <span class='inlinecode'>-plain</span> mode emits CSV, which is exactly the format AI assistants are best at chewing through:</span><br /> @@ -231,16 +232,28 @@ NOTHING SUSPICIOUS — no unexpected processes, no surprise paths outside /proc, <li>The model is reading text, not running SQL. It will round, it will sometimes miscount the long tail, and it cannot tell you a true p99 from a 250k-row sample without writing code. Treat the output as a lead-generator: it points you at suspects, then you confirm with ClickHouse on the parquet file.</li> <li>For deeper questions ("what changed between these two traces?", "which pids dominate during the 12:34 spike?") an agentic assistant that can run shell commands does much better. It'll write the awk/clickhouse query itself, run it, and feed the result back into its own analysis.</li> </ul><br /> -<span>The combination that's worked best for me in practice: capture parquet with <span class='inlinecode'>-parquet</span>, capture a parallel <span class='inlinecode'>-plain</span> CSV slice for the AI to read, ask the AI for a triage pass, then drill into the suspects with ClickHouse on the parquet file. Triage and ground-truth, in that order.</span><br /> +<span>The combination that's worked best for me in practice: capture parquet with <span class='inlinecode'>-parquet</span>, also capture a <span class='inlinecode'>-plain</span> CSV slice for the AI to read, ask the AI for a triage pass, then drill into the suspects with ClickHouse on the parquet file. Triage and ground-truth, in that order.</span><br /> +<br /> +<h2 style='display: inline' id='what-s-new-in-v110'>What's new in v1.1.0</h2><br /> +<br /> +<span>One v1.1.0 change is directly relevant to the syscall-coverage story above: probe attach is now tolerant of missing tracepoints. The codegen safeguard still flags new arrivals at build time, but when the resulting binary lands on a kernel that lacks one of its handlers (because the syscall is newer than the host kernel, or the tracepoint name was renamed under it), ior logs a one-line warning per missing probe and keeps attaching the rest, instead of failing startup with a hard error. Codegen keeps the list honest going forward; the runtime change makes the binary forgiving going backward. The new <span class='inlinecode'>ior.el8</span> build (Part 2's <span class='inlinecode'>mage buildDockerEl8</span> target, for RHEL/Rocky/Alma 8 hosts on 4.18 kernels) leans on this directly — it ships handlers for tracepoints that only exist on newer kernels, and now just skips them at attach instead of aborting.</span><br /> <br /> <h2 style='display: inline' id='wrapping-up'>Wrapping up</h2><br /> <br /> <span>That's the bottom of the stack. For the dashboard surface (what ior looks like, how the seven tabs behave, how filtering and recording work in practice) Part 1 is the demo-driven tour with all the GIFs. For the install dance and the why-the-binary-is-portable story (eBPF, CO-RE, static linking), Part 2 is the install + portability companion.</span><br /> <br /> -<a class='textlink' href='./DRAFT-unveiling-ior-ng-part-1.html'>Part 1: a guided tour</a><br /> -<a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>Part 2: install + compile once, run everywhere</a><br /> <a class='textlink' href='https://codeberg.org/snonux/ior'>Source on Codeberg</a><br /> -<a class='textlink' href='https://codeberg.org/snonux/ior/src/branch/main/docs/tutorial/tutorial.md'>The full in-repo tutorial</a><br /> +<br /> +<span>E-Mail your comments to <span class='inlinecode'>paul@nospam.buetow.org</span> :-)</span><br /> +<br /> +<span>Other related posts are:</span><br /> +<br /> +<a class='textlink' href='./2026-05-17-unveiling-ior-ng-part-3.html'>2026-05-17 Unveiling I/O Riot NG — Part 3: under the hood (You are currently reading this)</a><br /> +<a class='textlink' href='./2026-05-11-unveiling-ior-ng-part-2.html'>2026-05-11 Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a><br /> +<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG v1.0.0 — Part 1: a guided tour</a><br /> +<a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>2018-06-01 Realistic load testing with I/O Riot for Linux</a><br /> +<br /> +<a class='textlink' href='../'>Back to the main site</a><br /> <p class="footer"> Generated with <a href="https://codeberg.org/snonux/gemtexter">Gemtexter 3.0.1-develop</a> | served by <a href="https://www.OpenBSD.org">OpenBSD</a>/<a href="https://man.openbsd.org/relayd.8">relayd(8)</a>+<a href="https://man.openbsd.org/httpd.8">httpd(8)</a> | diff --git a/gemfeed/DRAFT-unveiling-ior-ng-part-2.html b/gemfeed/DRAFT-unveiling-ior-ng-part-2.html deleted file mode 100644 index fb8928b6..00000000 --- a/gemfeed/DRAFT-unveiling-ior-ng-part-2.html +++ /dev/null @@ -1,248 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</title> -<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> -<link rel="stylesheet" href="../style.css" /> -<link rel="stylesheet" href="style-override.css" /> -</head> -<body> -<p class="header"> -<a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/DRAFT-unveiling-ior-ng-part-2.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/DRAFT-unveiling-ior-ng-part-2.gmi">Gemini</a> | <a href="https://snonux.foo">Microblog</a> | <a href="https://irregular.ninja">Street photography</a> -</p> -<h1 style='display: inline' id='unveiling-io-riot-ng--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</h1><br /> -<br /> -<span class='quote'>Draft — not in the gemfeed yet. Promote with the usual rename + index dance.</span><br /> -<br /> -<span>This is Part 2 of three. Part 1 is the demo-driven tour: what ior looks like, how the dashboard tabs work, how filtering and recording behave. This part is about the install dance for Rocky Linux 9 (with one annoying kernel-backport caveat) and, more interestingly, why you only have to do that dance on a single machine: the resulting binary is portable to every other Linux box thanks to CO-RE (Compile Once, Run Everywhere) plus full static linking. Part 3 is the under-the-hood companion (per-event schema, async-syscall caveats, the syscall-coverage probe generator, and post-mortem SQL on the parquet output).</span><br /> -<br /> -<span>If you came here for the dashboard tour, that's Part 1. If you want to know how the data pipeline is shaped, that's Part 3. This one is for the moment between "I want to try this" and "OK, it's running on the box I care about."</span><br /> -<br /> -<a class='textlink' href='./DRAFT-unveiling-ior-ng-part-1.html'>Part 1: a guided tour</a><br /> -<a class='textlink' href='./DRAFT-unveiling-ior-ng-part-3.html'>Part 3: under the hood (schema, probe generator, ClickHouse)</a><br /> -<a class='textlink' href='https://codeberg.org/snonux/ior'>I/O Riot NG on Codeberg</a><br /> -<a class='textlink' href='./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html'>the original I/O Riot post (2018)</a><br /> -<br /> -<a href='./unveiling-ior-ng/00-logo.png'><img alt='I/O Riot NG logo' title='I/O Riot NG logo' src='./unveiling-ior-ng/00-logo.png' /></a><br /> -<br /> -<a class='textlink' href='./2026-05-08-unveiling-ior-ng-part-1.html'>2026-05-08 Unveiling I/O Riot NG 1.0.0 — Part 1: a guided tour</a><br /> -<br /> -<h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> -<br /> -<ul> -<li><a href='#unveiling-io-riot-ng--part-2-install-and-compile-once-run-everywhere'>Unveiling I/O Riot NG — Part 2: install and compile once, run everywhere</a></li> -<li>⇢ <a href='#installing-ior'>Installing ior</a></li> -<li>⇢ ⇢ <a href='#why-native-installation-is-a-mess'>Why native installation is a mess</a></li> -<li>⇢ ⇢ <a href='#what-the-docker-build-is-actually-doing'>What the Docker build is actually doing</a></li> -<li>⇢ <a href='#a-short-detour-ebpf-and-libbpfgo'>A short detour: eBPF and libbpfgo</a></li> -<li>⇢ <a href='#co-re--the-part-that-makes-the-binary-actually-portable'>CO-RE — the part that makes the binary actually portable</a></li> -<li>⇢ <a href='#a-note-on-cgo-overhead'>A note on cgo overhead</a></li> -<li>⇢ <a href='#if-you-want-to-go-deeper'>If you want to go deeper</a></li> -<li>⇢ <a href='#wrapping-up'>Wrapping up</a></li> -</ul><br /> -<h2 style='display: inline' id='installing-ior'>Installing ior</h2><br /> -<br /> -<span>The short answer: use Docker. One command, no toolchain setup, works from any Docker-capable Linux host with BTF available:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>git clone https://codeberg.org/snonux/ior ~/git/ior -cd ~/git/ior -mage buildDocker -</pre> -<br /> -<span>First run builds a Rocky Linux 9 builder image (~15–20 minutes). Subsequent runs reuse the cached image and finish in under a minute. The resulting static binary lands at <span class='inlinecode'>./ior</span>.</span><br /> -<br /> -<span>That's the officially supported install path, and it's the right one for anyone who just wants to run ior without living in its build system.</span><br /> -<br /> -<h3 style='display: inline' id='why-native-installation-is-a-mess'>Why native installation is a mess</h3><br /> -<br /> -<span>If you're curious why Docker became the answer, the native install on Rocky Linux 9 illustrates the problem well. Three separate things bite you before you even get to <span class='inlinecode'>mage build</span>:</span><br /> -<br /> -<span>Rocky 9 ships neither <span class='inlinecode'>libelf.a</span> nor <span class='inlinecode'>libzstd.a</span>. There are no <span class='inlinecode'>*-static</span> subpackages for either, only the dynamic <span class='inlinecode'>.so</span> files. Both have to be compiled from source. <span class='inlinecode'>libelf</span> from the elfutils source RPM, <span class='inlinecode'>libzstd</span> from the upstream GitHub release tarball.</span><br /> -<br /> -<span>Rocky 9 also only ships Go 1.25.x, but ior requires 1.26+. So Go itself has to be installed from go.dev in parallel with the library builds.</span><br /> -<br /> -<span>And there's a kernel quirk that used to make this section much longer. Pre-fix, ior would happily load on a stock 5.14 RHEL kernel and then die on the very first tracepoint attach with <span class='inlinecode'>BPF_LINK_CREATE</span>/<span class='inlinecode'>BPF_PERF_EVENT</span> returning <span class='inlinecode'>EACCES</span>, even as root, with SELinux permissive, with every BPF-related sysctl wide open. The cause is that RHEL 9 carries an <span class='inlinecode'>rt</span>-tree backport that adds <span class='inlinecode'>preempt_lazy_count</span> to <span class='inlinecode'>struct trace_entry</span>. That widens the BTF-emitted alias <span class='inlinecode'>trace_event_raw_sys_enter</span>/<span class='inlinecode'>_exit</span> by 8 bytes and shifts the <span class='inlinecode'>args</span>/<span class='inlinecode'>ret</span> offsets, but the actual context the kernel hands the BPF program is still <span class='inlinecode'>struct syscall_trace_enter</span>/<span class='inlinecode'>_exit</span>, where the offsets did not move. Programs written against <span class='inlinecode'>trace_event_raw_sys_*</span> (the conventional choice; bcc, libbpf-tools, and ior all used to do this) end up reading past <span class='inlinecode'>max_ctx_offset</span>, so the verifier rejects the attach. The fix (also what bcc shipped in [PR #4920](https://github.com/iovisor/bcc/pull/4920) and what inspektor-gadget did) is to type the BPF context as <span class='inlinecode'>syscall_trace_enter</span>/<span class='inlinecode'>_exit</span> directly. ior now generates its handlers that way, and stock 5.14 RHEL/Rocky/Alma works without an ElRepo kernel.</span><br /> -<br /> -<h3 style='display: inline' id='what-the-docker-build-is-actually-doing'>What the Docker build is actually doing</h3><br /> -<br /> -<span>The Dockerfile encodes exactly the same steps that a native install on Rocky 9 would require. Here is the full sequence so you have a mental model of what's inside the image, and so you could reproduce it on a bare host if you ever needed to:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><i><font color="silver"># 1) Enable repos and install build dependencies. CRB ships zlib-static / glibc-static.</font></i> -sudo dnf config-manager --set-enabled crb -sudo dnf install -y epel-release -sudo dnf install -y gcc clang bpftool elfutils-libelf-devel zlib-static \ - glibc-static libzstd-devel git make cmake wget rpmdevtools strace bpftrace -sudo dnf builddep -y elfutils - -<i><font color="silver"># 2) Install Go 1.26 from go.dev. Rocky 9 ships only Go 1.25.x, ior needs 1.26+.</font></i> -cd /tmp -wget -q https://go.dev/dl/go<font color="#000000">1.26</font>.<font color="#000000">2</font>.linux-amd<font color="#000000">64</font>.tar.gz -sudo tar -C /usr/local -xf go1.<font color="#000000">26.2</font>.linux-amd<font color="#000000">64</font>.tar.gz -echo <font color="#808080">'export PATH=/usr/local/go/bin:$HOME/go/bin:$PATH'</font> | sudo tee /etc/profile.d/go.sh -<b><u><font color="#000000">source</font></u></b> /etc/profile.d/go.sh - -<i><font color="silver"># 3) Build libelf.a from the elfutils source RPM.</font></i> -mkdir -p ~/src && cd ~ -dnf download --source elfutils-libelf -rpm -ivh elfutils-*.src.rpm -tar -C ~/src -xjf rpmbuild/SOURCES/elfutils-*.tar.bz2 -cd ~/src/elfutils-* -./configure --enable-deterministic-archives --disable-debuginfod --disable-libdebuginfod -make -C lib -j$(nproc) -make -C libelf -j$(nproc) -sudo cp -v libelf/libelf.a /usr/lib<font color="#000000">64</font>/ - -<i><font color="silver"># 4) Build libzstd.a from upstream (libzstd-devel doesn't ship the static archive).</font></i> -cd /tmp -wget -q https://github.com/facebook/zstd/releases/download/v<font color="#000000">1.5</font>.<font color="#000000">5</font>/zstd-<font color="#000000">1.5</font>.<font color="#000000">5</font>.tar.gz -tar xzf zstd-<font color="#000000">1.5</font>.<font color="#000000">5</font>.tar.gz -make -C zstd-<font color="#000000">1.5</font>.<font color="#000000">5</font>/lib -j$(nproc) libzstd.a -sudo cp -v zstd-<font color="#000000">1.5</font>.<font color="#000000">5</font>/lib/libzstd.a /usr/lib<font color="#000000">64</font>/ - -<i><font color="silver"># 5) Clone ior + libbpfgo, pin libbpfgo, build the static libbpf archive, install mage.</font></i> |