diff options
| author | Paul Buetow <paul@buetow.org> | 2026-02-22 23:54:26 +0200 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2026-02-22 23:54:26 +0200 |
| commit | 55f33883838336c70d483779b0435c6e781b615e (patch) | |
| tree | f26a8b2a72a28483675fe87ee3ea03d35bd39862 | |
| parent | 1666ba49ef9e5b61e14d1a32d2f6e2380064105e (diff) | |
some stuff
| -rw-r--r-- | FIXES.md | 5 | ||||
| -rw-r--r-- | IDEAS.md | 36 | ||||
| -rw-r--r-- | INTEGRATIONTESTS-PLAN.md | 146 | ||||
| -rw-r--r-- | Magefile.go | 2 | ||||
| -rw-r--r-- | integrationtests/link_test.go | 2 |
5 files changed, 3 insertions, 188 deletions
diff --git a/FIXES.md b/FIXES.md deleted file mode 100644 index 0bd8700..0000000 --- a/FIXES.md +++ /dev/null @@ -1,5 +0,0 @@ -## Plan -1. Prevent `openat2` file tracking from panicking on unknown flags by allowing `-1` (unknown) flags in userland and covering it with a focused test. -2. Decide whether to enable `name_to_handle_at`/`open_by_handle_at` tracing or remove the correlation logic if the tracepoints stay excluded. -3. Add `close_range` handling that removes all fds in the range from the fd table. -4. Improve `io_uring_*` attribution by correlating setup/enter operations with returned fds or otherwise marking them as non-file-backed operations. diff --git a/IDEAS.md b/IDEAS.md deleted file mode 100644 index 61b1134..0000000 --- a/IDEAS.md +++ /dev/null @@ -1,36 +0,0 @@ -# Ideas - -## FlameGraphs - -More ideas - -``` -user;cmd_name;pid;tid;syscall_name count -user;cmd_name;pid;tid;syscall_name bytes - -user:cmd_name;pid:tid;prev_syscall_name->syscall_name;duration IN DIFFERENT COLOR -user:cmd_name;pid:tid;prev_syscall_name->syscall_name;fd;duration IN DIFFERENT COLOR -user;cmd_name;pid;tid;syscall_name duration -user;cmd_name;pid;tid;syscall_name;fd duration - -user;cmd_name;pid;tid;PATHMATCH;syscall_name duration -user;cmd_name;pid;tid;OTHER;syscall_name duration -``` - -`pathdecoded`? Maybe: - -* By directory -* By mountpoint -* By device - - -Consider: - -* File base path or mount point or device name -* Filename? -* Time spent between syscalls? - -## Other - -* More ways to transfer file descriptors between processes: pidfd_getfd https://biriukov.dev/docs/fd-pipe-session-terminal/1-file-descriptor-and-open-file-description/ -* Trace for ALL syscalls and only count the count and times .... thats for another mode diff --git a/INTEGRATIONTESTS-PLAN.md b/INTEGRATIONTESTS-PLAN.md deleted file mode 100644 index e9e9986..0000000 --- a/INTEGRATIONTESTS-PLAN.md +++ /dev/null @@ -1,146 +0,0 @@ -# Integration Tests Plan - -> Individual implementation tasks are tracked in **Taskwarrior** (`task project:ior +integrationtests`). - -## Overview - -End-to-end integration tests that verify ior correctly captures real I/O syscalls from a -known workload process via BPF tracepoints. A standalone Go binary performs deterministic -I/O operations, ior traces it by PID, and the test harness asserts the captured `.ior.zst` -output matches expectations. - -## Architecture - -``` - ┌─────────────────────────────────────────────────────────┐ - │ Go Test Harness (*_test.go) │ - │ │ - │ 1. Start ioworkload --scenario=X │ - │ 2. Read PID from workload's stdout (line 1) │ - │ 3. Start ior -pid=PID -flamegraph -duration=N │ - │ 4. Workload sleeps 2s, then performs I/O, exits │ - │ 5. ior finishes, produces .ior.zst │ - │ 6. Parse .ior.zst → assert expected events present │ - └─────────────────────────────────────────────────────────┘ - │ │ - ▼ ▼ - ┌──────────────┐ ┌──────────────────┐ - │ ioworkload │ │ ior │ - │ (separate │ │ (BPF tracing │ - │ binary) │ │ -pid=WORKLOAD) │ - │ │ │ │ - │ prints PID │ │ writes .ior.zst │ - │ sleeps 2s │ └──────────────────┘ - │ does I/O │ - │ exits │ - └──────────────┘ -``` - -## Directory Layout - -``` -integrationtests/ -├── cmd/ -│ └── ioworkload/ -│ └── main.go # Standalone I/O workload binary -├── harness.go # Test orchestration (start ior + workload, collect output) -├── parse.go # Parse .ior.zst into assertable TestResult -├── expectations.go # ExpectedEvent type & assertion helpers -├── open_test.go # open, openat, creat, open_by_handle_at -├── readwrite_test.go # read, write, pread64, pwrite64, readv, writev -├── close_test.go # close, close_range -├── dup_test.go # dup, dup2, dup3 -├── fcntl_test.go # fcntl (F_DUPFD, F_SETFL, F_DUPFD_CLOEXEC) -├── rename_test.go # rename, renameat, renameat2 -├── link_test.go # link, linkat, symlink, symlinkat, readlink -├── unlink_test.go # unlink, unlinkat, rmdir -├── dir_test.go # mkdir, mkdirat, chdir, getdents -├── stat_test.go # stat, fstat, lstat, statx, access, faccessat -├── sync_test.go # fsync, fdatasync, sync, sync_file_range -├── truncate_test.go # truncate, ftruncate -├── iouring_test.go # io_uring_setup, io_uring_enter, io_uring_register -└── README.md -``` - -## Components - -### 1. I/O Workload Binary (`cmd/ioworkload/main.go`) - -A standalone Go binary that: -- Accepts `--scenario=<name>` flag (e.g. `open-basic`, `dup-dup3-cloexec`) -- Prints its PID to stdout on line 1 -- Sleeps 2 seconds (gives harness time to start ior with PID filter) -- Performs deterministic, known I/O operations in a temp directory -- Cleans up and exits with code 0 - -### 2. Test Harness (`harness.go`) - -```go -type TestHarness struct { - IorBinary string // path to built ior binary - WorkloadBinary string // path to built ioworkload binary - BpfObject string // path to ior.bpf.o - OutputDir string // temp dir for .ior.zst output -} - -func (h *TestHarness) Run(scenario string, duration int) (*TestResult, error) -``` - -`Run()` sequence: -1. Start `ioworkload --scenario=<name>` -2. Read PID from workload stdout (line 1) -3. Start `ior -pid=<PID> -flamegraph -name=<scenario> -duration=<N>` -4. Workload's 2s sleep expires, it performs I/O, then exits -5. ior finishes (duration expires or SIGTERM), writes `.ior.zst` -6. Parse `.ior.zst` into `TestResult` - -Requires root/CAP_BPF. Tests skip with `t.Skip("requires root for BPF")` if not root. - -### 3. Parser (`parse.go`) - -Reuses existing `flamegraph.newIorDataFromFile()` and `iorData.iter()` to deserialize -`.ior.zst` into an assertable `TestResult` struct containing all captured events. - -### 4. Assertions (`expectations.go`) - -```go -type ExpectedEvent struct { - PathContains string // substring match on file path - Tracepoint string // e.g. "sys_enter_openat" - Comm string // e.g. "ioworkload" - MinCount uint64 // minimum occurrences -} - -func AssertEventsPresent(t *testing.T, result *TestResult, expected []ExpectedEvent) -func AssertNoUnexpectedComm(t *testing.T, result *TestResult, expectedComm string) -``` - -### 5. Test Files (per syscall family) - -Each `*_test.go` defines scenarios and expectations for its syscall family. Example: - -```go -// open_test.go -func TestOpenBasic(t *testing.T) { runScenario(t, "open-basic", ...) } -func TestOpenCreat(t *testing.T) { runScenario(t, "open-creat", ...) } -func TestOpenByHandleAt(t *testing.T) { runScenario(t, "open-by-handle-at", ...) } -``` - -### 6. Mage Target (`Magefile.go`) - -```go -func IntegrationTest() error { - mg.SerialDeps(All) - // build ioworkload - // sudo go test ./integrationtests/... -v -failfast -count=1 -} -``` - -## Key Design Decisions - -- **Separate binary**: syscalls come from a real process with a distinct PID, matching production -- **`.ior.zst` as verification format**: reuses existing serialization, avoids parsing noisy stdout -- **`-pid` filter**: isolates workload I/O from system noise -- **2s sleep**: simple timing-based coordination, no synchronization pipes needed -- **One `*_test.go` per syscall family**: scales to many scenarios without monolithic test files -- **Standard `go test`**: no custom runner; `t.Skip` for non-root environments diff --git a/Magefile.go b/Magefile.go index 86e5958..1c2e793 100644 --- a/Magefile.go +++ b/Magefile.go @@ -107,6 +107,7 @@ func TestWithName() error { "./integrationtests/...", "-run", "^"+testName+"$", "-failfast", + "-timeout=30m", "-count=1", "-json", ) @@ -274,6 +275,7 @@ func IntegrationTest() error { return runGoTestWithProgress(env, "./integrationtests/...", "-failfast", + "-timeout=30m", "-count=1", "-json", ) diff --git a/integrationtests/link_test.go b/integrationtests/link_test.go index 4f92f6e..df76be4 100644 --- a/integrationtests/link_test.go +++ b/integrationtests/link_test.go @@ -61,7 +61,7 @@ func TestLinkReadlinkat(t *testing.T) { func TestLinkEnoent(t *testing.T) { runScenario(t, "link-enoent", []ExpectedEvent{ { - PathContains: "link-enoent-missing.txt", + PathContains: "link-enoent-dst.txt", Tracepoint: "enter_link", Comm: "ioworkload", MinCount: 1, |
