diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/SERVERLESS_LARGE_FILES_ISSUE.md | 61 | ||||
| -rw-r--r-- | docs/turbo-vs-normal-benchmark-20260718-run1.csv | 127 | ||||
| -rw-r--r-- | docs/turbo-vs-normal-benchmark-20260718-run2.csv | 127 | ||||
| -rw-r--r-- | docs/turbo-vs-normal-benchmark-20260718.md | 126 |
4 files changed, 441 insertions, 0 deletions
diff --git a/docs/SERVERLESS_LARGE_FILES_ISSUE.md b/docs/SERVERLESS_LARGE_FILES_ISSUE.md new file mode 100644 index 0000000..3ff4b5f --- /dev/null +++ b/docs/SERVERLESS_LARGE_FILES_ISSUE.md @@ -0,0 +1,61 @@ +# Serverless Mode Large File Issue + +## Summary +While the serverless mode deadlock has been partially resolved, files larger than approximately 10KB still experience timeouts in serverless mode. + +## Current Status +- ✅ Files up to 10KB work correctly +- ❌ Files larger than 100KB timeout +- ❌ The 72MB test_data.log used in profiling examples still hangs + +## Technical Details +The current fix uses a channel-based approach to prevent deadlocks: +- Separate goroutines for reading from client/server handlers +- Buffered channels (100 slots) for data transfer +- 32KB buffer size for read operations + +However, this approach still has limitations with larger files, possibly due to: +1. Channel buffer exhaustion +2. Synchronization issues between read/write operations +3. EOF handling complexities +4. Memory pressure from buffering large amounts of data + +## Workaround +For profiling large files, avoid serverless mode by specifying a dummy server: +```bash +./dcat -profile -profiledir profiles -plain -cfg none -servers dummy test_data.log +``` + +## Proposed Solutions + +### Short-term +1. Increase channel buffer sizes dynamically based on file size +2. Implement backpressure handling +3. Add proper flow control between readers and writers + +### Long-term +1. Redesign serverless mode to avoid bidirectional copying +2. Implement a proper streaming architecture +3. Consider using io.Pipe with proper goroutine management +4. Add file size detection and automatic mode switching + +## Testing +Use the test_serverless.go script to verify fixes: +```go +// Test different file sizes +sizes := []struct { + name string + size int +}{ + {"tiny", 100}, // ✅ Works + {"small", 1024}, // ✅ Works + {"medium", 10240}, // ✅ Works + {"large", 102400}, // ❌ Timeouts + {"xlarge", 1048576}, // ❌ Timeouts +} +``` + +## Impact +- Profiling benchmarks work for small to medium test files +- Large file profiling requires non-serverless mode +- Integration tests may need adjustment if they use large files in serverless mode
\ No newline at end of file diff --git a/docs/turbo-vs-normal-benchmark-20260718-run1.csv b/docs/turbo-vs-normal-benchmark-20260718-run1.csv new file mode 100644 index 0000000..f5af6d2 --- /dev/null +++ b/docs/turbo-vs-normal-benchmark-20260718-run1.csv @@ -0,0 +1,127 @@ +scenario,mode,iter,seconds,note +sl_dcat_100mb,turbo,1,0.092, +sl_dcat_100mb,noturbo,1,1.310, +sl_dcat_100mb,turbo,2,0.094, +sl_dcat_100mb,noturbo,2,1.268, +sl_dcat_100mb,turbo,3,0.092, +sl_dcat_100mb,noturbo,3,1.282, +sl_dcat_100mb,turbo,4,0.093, +sl_dcat_100mb,noturbo,4,1.286, +sl_dcat_100mb,turbo,5,0.100, +sl_dcat_100mb,noturbo,5,1.314, +sl_dcat_100mb,turbo,6,0.111, +sl_dcat_100mb,noturbo,6,1.896, +sl_dcat_100mb,turbo,7,0.114, +sl_dcat_100mb,noturbo,7,1.854, +sl_dcat_1gb,turbo,1,0.979, +sl_dcat_1gb,noturbo,1,18.027, +sl_dcat_1gb,turbo,2,0.991, +sl_dcat_1gb,noturbo,2,17.776, +sl_dcat_1gb,turbo,3,1.018, +sl_dcat_1gb,noturbo,3,17.335, +sl_dgrep_low_100mb,turbo,1,0.104, +sl_dgrep_low_100mb,noturbo,1,2.103, +sl_dgrep_low_100mb,turbo,2,0.116, +sl_dgrep_low_100mb,noturbo,2,2.127, +sl_dgrep_low_100mb,turbo,3,0.107, +sl_dgrep_low_100mb,noturbo,3,2.102, +sl_dgrep_low_100mb,turbo,4,0.104, +sl_dgrep_low_100mb,noturbo,4,2.104, +sl_dgrep_low_100mb,turbo,5,0.104, +sl_dgrep_low_100mb,noturbo,5,2.125, +sl_dgrep_low_100mb,turbo,6,0.111, +sl_dgrep_low_100mb,noturbo,6,2.091, +sl_dgrep_low_100mb,turbo,7,0.106, +sl_dgrep_low_100mb,noturbo,7,2.129, +sl_dgrep_high_100mb,turbo,1,0.095, +sl_dgrep_high_100mb,noturbo,1,2.212, +sl_dgrep_high_100mb,turbo,2,0.099, +sl_dgrep_high_100mb,noturbo,2,2.198, +sl_dgrep_high_100mb,turbo,3,0.097, +sl_dgrep_high_100mb,noturbo,3,2.178, +sl_dgrep_high_100mb,turbo,4,0.098, +sl_dgrep_high_100mb,noturbo,4,2.181, +sl_dgrep_high_100mb,turbo,5,0.098, +sl_dgrep_high_100mb,noturbo,5,2.184, +sl_dgrep_high_100mb,turbo,6,0.098, +sl_dgrep_high_100mb,noturbo,6,2.194, +sl_dgrep_high_100mb,turbo,7,0.099, +sl_dgrep_high_100mb,noturbo,7,2.170, +sl_dmap_agg_100mb,turbo,1,2.969, +sl_dmap_agg_100mb,noturbo,1,3.107, +sl_dmap_agg_100mb,turbo,2,3.072, +sl_dmap_agg_100mb,noturbo,2,3.022, +sl_dmap_agg_100mb,turbo,3,3.002, +sl_dmap_agg_100mb,noturbo,3,4.415, +sl_dmap_agg_100mb,turbo,4,7.546, +sl_dmap_agg_100mb,noturbo,4,7.415, +sl_dmap_agg_100mb,turbo,5,6.090, +sl_dmap_agg_100mb,noturbo,5,3.279, +sv_dcat_100mb,turbo,1,1.242, +sv_dcat_100mb,noturbo,1,8.246, +sv_dcat_100mb,turbo,2,1.293, +sv_dcat_100mb,noturbo,2,12.903, +sv_dcat_100mb,turbo,3,2.157, +sv_dcat_100mb,noturbo,3,16.865, +sv_dcat_100mb,turbo,4,1.962, +sv_dcat_100mb,noturbo,4,14.526, +sv_dcat_100mb,turbo,5,2.162, +sv_dcat_100mb,noturbo,5,15.263, +sv_dcat_100mb,turbo,6,1.943, +sv_dcat_100mb,noturbo,6,18.553, +sv_dcat_100mb,turbo,7,2.176, +sv_dcat_100mb,noturbo,7,18.176, +sv_dgrep_low_100mb,turbo,1,0.778, +sv_dgrep_low_100mb,noturbo,1,4.850, +sv_dgrep_low_100mb,turbo,2,0.764, +sv_dgrep_low_100mb,noturbo,2,4.513, +sv_dgrep_low_100mb,turbo,3,0.826, +sv_dgrep_low_100mb,noturbo,3,5.549, +sv_dgrep_low_100mb,turbo,4,0.783, +sv_dgrep_low_100mb,noturbo,4,4.310, +sv_dgrep_low_100mb,turbo,5,0.846, +sv_dgrep_low_100mb,noturbo,5,5.752, +sv_dgrep_low_100mb,turbo,6,0.873, +sv_dgrep_low_100mb,noturbo,6,5.524, +sv_dgrep_low_100mb,turbo,7,0.781, +sv_dgrep_low_100mb,noturbo,7,6.254, +sv_dgrep_high_100mb,turbo,1,0.976, +sv_dgrep_high_100mb,noturbo,1,5.826, +sv_dgrep_high_100mb,turbo,2,0.852, +sv_dgrep_high_100mb,noturbo,2,6.800, +sv_dgrep_high_100mb,turbo,3,0.909, +sv_dgrep_high_100mb,noturbo,3,5.802, +sv_dgrep_high_100mb,turbo,4,1.063, +sv_dgrep_high_100mb,noturbo,4,6.199, +sv_dgrep_high_100mb,turbo,5,0.842, +sv_dgrep_high_100mb,noturbo,5,6.405, +sv_dgrep_high_100mb,turbo,6,0.967, +sv_dgrep_high_100mb,noturbo,6,7.096, +sv_dgrep_high_100mb,turbo,7,0.962, +sv_dgrep_high_100mb,noturbo,7,6.095, +sv_dmap_count_100mb,turbo,1,2.529, +sv_dmap_count_100mb,noturbo,1,6.376, +sv_dmap_count_100mb,turbo,2,2.818, +sv_dmap_count_100mb,noturbo,2,6.334, +sv_dmap_count_100mb,turbo,3,2.442, +sv_dmap_count_100mb,noturbo,3,5.477, +sv_dmap_count_100mb,turbo,4,2.219, +sv_dmap_count_100mb,noturbo,4,5.969, +sv_dmap_count_100mb,turbo,5,2.364, +sv_dmap_count_100mb,noturbo,5,6.113, +sv_dmap_agg_100mb,turbo,1,2.729, +sv_dmap_agg_100mb,noturbo,1,5.318, +sv_dmap_agg_100mb,turbo,2,2.653, +sv_dmap_agg_100mb,noturbo,2,5.486, +sv_dmap_agg_100mb,turbo,3,2.782, +sv_dmap_agg_100mb,noturbo,3,5.710, +sv_dmap_agg_100mb,turbo,4,2.717, +sv_dmap_agg_100mb,noturbo,4,5.150, +sv_dmap_agg_100mb,turbo,5,2.767, +sv_dmap_agg_100mb,noturbo,5,5.133, +sv_dtail_follow_10mb,turbo,1,0.240,ok delivered=100000/100000 +sv_dtail_follow_10mb,turbo,2,0.239,ok delivered=100000/100000 +sv_dtail_follow_10mb,turbo,3,0.241,ok delivered=100000/100000 +sv_dtail_follow_10mb,noturbo,1,0.448,ok delivered=46564/100000 +sv_dtail_follow_10mb,noturbo,2,98.235,timeout_gt90s delivered=41949/100000 +sv_dtail_follow_10mb,noturbo,3,98.494,timeout_gt90s delivered=43146/100000 diff --git a/docs/turbo-vs-normal-benchmark-20260718-run2.csv b/docs/turbo-vs-normal-benchmark-20260718-run2.csv new file mode 100644 index 0000000..1718773 --- /dev/null +++ b/docs/turbo-vs-normal-benchmark-20260718-run2.csv @@ -0,0 +1,127 @@ +scenario,mode,iter,seconds,note +sl_dcat_100mb,turbo,1,0.088, +sl_dcat_100mb,noturbo,1,1.266, +sl_dcat_100mb,turbo,2,0.090, +sl_dcat_100mb,noturbo,2,1.256, +sl_dcat_100mb,turbo,3,0.094, +sl_dcat_100mb,noturbo,3,1.322, +sl_dcat_100mb,turbo,4,0.094, +sl_dcat_100mb,noturbo,4,1.322, +sl_dcat_100mb,turbo,5,0.101, +sl_dcat_100mb,noturbo,5,1.333, +sl_dcat_100mb,turbo,6,0.094, +sl_dcat_100mb,noturbo,6,1.439, +sl_dcat_100mb,turbo,7,0.098, +sl_dcat_100mb,noturbo,7,1.687, +sl_dcat_1gb,turbo,1,0.977, +sl_dcat_1gb,noturbo,1,18.337, +sl_dcat_1gb,turbo,2,1.034, +sl_dcat_1gb,noturbo,2,17.723, +sl_dcat_1gb,turbo,3,1.069, +sl_dcat_1gb,noturbo,3,17.912, +sl_dgrep_low_100mb,turbo,1,0.110, +sl_dgrep_low_100mb,noturbo,1,2.289, +sl_dgrep_low_100mb,turbo,2,0.113, +sl_dgrep_low_100mb,noturbo,2,2.309, +sl_dgrep_low_100mb,turbo,3,0.111, +sl_dgrep_low_100mb,noturbo,3,2.249, +sl_dgrep_low_100mb,turbo,4,0.111, +sl_dgrep_low_100mb,noturbo,4,2.236, +sl_dgrep_low_100mb,turbo,5,0.115, +sl_dgrep_low_100mb,noturbo,5,2.319, +sl_dgrep_low_100mb,turbo,6,0.108, +sl_dgrep_low_100mb,noturbo,6,2.195, +sl_dgrep_low_100mb,turbo,7,0.116, +sl_dgrep_low_100mb,noturbo,7,2.336, +sl_dgrep_high_100mb,turbo,1,0.108, +sl_dgrep_high_100mb,noturbo,1,2.374, +sl_dgrep_high_100mb,turbo,2,0.102, +sl_dgrep_high_100mb,noturbo,2,2.338, +sl_dgrep_high_100mb,turbo,3,0.104, +sl_dgrep_high_100mb,noturbo,3,2.367, +sl_dgrep_high_100mb,turbo,4,0.103, +sl_dgrep_high_100mb,noturbo,4,2.299, +sl_dgrep_high_100mb,turbo,5,0.101, +sl_dgrep_high_100mb,noturbo,5,2.321, +sl_dgrep_high_100mb,turbo,6,0.107, +sl_dgrep_high_100mb,noturbo,6,2.367, +sl_dgrep_high_100mb,turbo,7,0.106, +sl_dgrep_high_100mb,noturbo,7,2.342, +sl_dmap_agg_100mb,turbo,1,3.205, +sl_dmap_agg_100mb,noturbo,1,3.183, +sl_dmap_agg_100mb,turbo,2,2.999, +sl_dmap_agg_100mb,noturbo,2,2.861, +sl_dmap_agg_100mb,turbo,3,3.378, +sl_dmap_agg_100mb,noturbo,3,3.102, +sl_dmap_agg_100mb,turbo,4,2.800, +sl_dmap_agg_100mb,noturbo,4,3.068, +sl_dmap_agg_100mb,turbo,5,2.959, +sl_dmap_agg_100mb,noturbo,5,3.067, +sv_dcat_100mb,turbo,1,1.284, +sv_dcat_100mb,noturbo,1,8.378, +sv_dcat_100mb,turbo,2,1.249, +sv_dcat_100mb,noturbo,2,8.216, +sv_dcat_100mb,turbo,3,1.251, +sv_dcat_100mb,noturbo,3,8.301, +sv_dcat_100mb,turbo,4,1.245, +sv_dcat_100mb,noturbo,4,8.231, +sv_dcat_100mb,turbo,5,1.256, +sv_dcat_100mb,noturbo,5,8.274, +sv_dcat_100mb,turbo,6,1.253, +sv_dcat_100mb,noturbo,6,8.259, +sv_dcat_100mb,turbo,7,1.268, +sv_dcat_100mb,noturbo,7,8.274, +sv_dgrep_low_100mb,turbo,1,0.720, +sv_dgrep_low_100mb,noturbo,1,2.044, +sv_dgrep_low_100mb,turbo,2,0.725, +sv_dgrep_low_100mb,noturbo,2,2.062, +sv_dgrep_low_100mb,turbo,3,0.725, +sv_dgrep_low_100mb,noturbo,3,2.070, +sv_dgrep_low_100mb,turbo,4,0.720, +sv_dgrep_low_100mb,noturbo,4,2.184, +sv_dgrep_low_100mb,turbo,5,0.726, +sv_dgrep_low_100mb,noturbo,5,2.261, +sv_dgrep_low_100mb,turbo,6,0.727, +sv_dgrep_low_100mb,noturbo,6,2.200, +sv_dgrep_low_100mb,turbo,7,0.720, +sv_dgrep_low_100mb,noturbo,7,2.176, +sv_dgrep_high_100mb,turbo,1,0.769, +sv_dgrep_high_100mb,noturbo,1,2.923, +sv_dgrep_high_100mb,turbo,2,0.753, +sv_dgrep_high_100mb,noturbo,2,2.956, +sv_dgrep_high_100mb,turbo,3,0.759, +sv_dgrep_high_100mb,noturbo,3,2.754, +sv_dgrep_high_100mb,turbo,4,0.763, +sv_dgrep_high_100mb,noturbo,4,2.783, +sv_dgrep_high_100mb,turbo,5,0.764, +sv_dgrep_high_100mb,noturbo,5,2.872, +sv_dgrep_high_100mb,turbo,6,0.761, +sv_dgrep_high_100mb,noturbo,6,2.893, +sv_dgrep_high_100mb,turbo,7,0.775, +sv_dgrep_high_100mb,noturbo,7,2.914, +sv_dmap_count_100mb,turbo,1,1.556, +sv_dmap_count_100mb,noturbo,1,4.056, +sv_dmap_count_100mb,turbo,2,1.543, +sv_dmap_count_100mb,noturbo,2,3.739, +sv_dmap_count_100mb,turbo,3,1.491, +sv_dmap_count_100mb,noturbo,3,3.841, +sv_dmap_count_100mb,turbo,4,1.463, +sv_dmap_count_100mb,noturbo,4,3.926, +sv_dmap_count_100mb,turbo,5,1.451, +sv_dmap_count_100mb,noturbo,5,3.837, +sv_dmap_agg_100mb,turbo,1,1.678, +sv_dmap_agg_100mb,noturbo,1,2.843, +sv_dmap_agg_100mb,turbo,2,1.692, +sv_dmap_agg_100mb,noturbo,2,2.667, +sv_dmap_agg_100mb,turbo,3,1.713, +sv_dmap_agg_100mb,noturbo,3,2.638, +sv_dmap_agg_100mb,turbo,4,1.691, +sv_dmap_agg_100mb,noturbo,4,2.694, +sv_dmap_agg_100mb,turbo,5,1.728, +sv_dmap_agg_100mb,noturbo,5,2.619, +sv_dtail_follow_10mb,turbo,1,0.115,ok delivered=100000/100000 +sv_dtail_follow_10mb,turbo,2,0.115,ok delivered=100000/100000 +sv_dtail_follow_10mb,turbo,3,0.114,ok delivered=100000/100000 +sv_dtail_follow_10mb,noturbo,1,95.740,timeout_gt90s delivered=39564/100000 +sv_dtail_follow_10mb,noturbo,2,95.685,timeout_gt90s delivered=38747/100000 +sv_dtail_follow_10mb,noturbo,3,95.723,timeout_gt90s delivered=39253/100000 diff --git a/docs/turbo-vs-normal-benchmark-20260718.md b/docs/turbo-vs-normal-benchmark-20260718.md new file mode 100644 index 0000000..3d86fa9 --- /dev/null +++ b/docs/turbo-vs-normal-benchmark-20260718.md @@ -0,0 +1,126 @@ +# Turbo vs Normal (Non-Turbo) Benchmark Report — 2026-07-18 + +**Question:** After the recent server/turbo-path work (in particular task `ws0`, +which gave `TurboAggregate` an input-exhausted `FinishInput` signal), is turbo +mode still faster than the normal (channel-based) path — and does the +server-mode `dmap` deadlock the previous report flagged still exist? + +**Answer: Yes — turbo remains faster in every scenario where it applies, and the +result is reproducible across two independent full runs. The server-mode `dmap` +turbo deadlock documented in the 2026-07-10 report is FIXED: turbo `dmap` in +server mode now completes and is ~1.6×–2.6× faster than non-turbo. Server-mode +`dcat`, previously transport-bound and neutral, is now ~6–8× faster on the turbo +path.** + +## Methodology + +- Binaries rebuilt from clean at HEAD `1a079d9` (`make clean && make build`). +- Turbo is default-on; non-turbo runs set `DTAIL_TURBOBOOST_DISABLE=yes` on the + process that runs the server handlers (the client itself in serverless mode, + `dserver` in server mode). +- Serverless scenarios: client binaries run directly against local files. +- Server scenarios: two `dserver` instances on localhost (one turbo, one + non-turbo, `--cfg none`, real SSH auth via a dedicated RSA key + `CacheDir` + authorized_keys) — *not* `DTAIL_INTEGRATION_TEST_RUN_MODE`, because that mode + force-disables turbo (`internal/config/initializer.go`) and would compare + non-turbo against non-turbo. +- Turbo/non-turbo iterations alternate A/B to spread thermal/load drift (dcat, + dgrep, dmap scenarios); the dtail follow scenario runs all turbo iterations + first, then all non-turbo. 1 warmup + 5–7 measured iterations per + scenario/mode; medians reported. +- Wall-clock timing, client stdout to `/dev/null` (except dtail follow). +- Datasets: `benchmarks/testdata/medium.log` (100 MB, 1.22 M lines), + `large.log` (1 GB), and a generated 100 MB `MAPREDUCE:STATS` file (800 k + lines, 50 hostnames). dgrep patterns: `"user999 "` (low hit rate, ~0.07 %) and + `"ERROR"` (high hit rate, ~19 %). +- dtail follow: client follows an empty file over SSH; after the stream is + established (probe line round-trip), a 10 MB / 100 k-line burst plus end + marker is appended; time from append to marker arrival at the client is + measured, and delivered-line completeness is counted. +- The benchmark was run twice end to end (run 1 and run 2) to check + reproducibility. Raw per-iteration data is committed alongside this report: + `turbo-vs-normal-benchmark-20260718-run1.csv` and `...-run2.csv`. Harness: + `benchmarks/turbo_vs_normal_bench.sh`. +- Machine: 11th Gen Intel i7-1185G7 @ 3.00 GHz (8 threads), Fedora Linux 44, + Go 1.26.4. Caveat: laptop; absolute numbers are noisy (thermal/background + load), so the **turbo/non-turbo ratios are the meaningful signal**, not the + absolute seconds. This is visible in the server-mode rows, where run 2's + non-turbo times are lower than run 1's (lighter machine load), which + compresses those ratios without any change to the turbo path. +- Non-turbo serverless runs go through an `env` wrapper that turbo runs do not + (~1 ms extra process spawn). Negligible at these magnitudes, and it biases + against turbo, not for it. + +## Results (median wall time, seconds) + +### Serverless mode + +| Scenario | Turbo (r1 / r2) | Non-Turbo (r1 / r2) | Speedup (r1 / r2) | Verdict | +|------------------------------|-----------------|---------------------|-------------------|---------| +| dcat 100 MB | 0.094 / 0.094 | 1.310 / 1.322 | 13.9× / 14.1× | turbo much faster | +| dcat 1 GB | 0.991 / 1.034 | 17.776 / 17.912 | 17.9× / 17.3× | turbo much faster | +| dgrep 100 MB, low hit rate | 0.106 / 0.111 | 2.104 / 2.289 | 19.8× / 20.6× | turbo much faster | +| dgrep 100 MB, high hit rate | 0.098 / 0.104 | 2.184 / 2.342 | 22.3× / 22.5× | turbo much faster | +| dmap agg 100 MB (control) | 3.072 / 2.999 | 3.279 / 3.068 | 1.07× / 1.02× | expected: turbo N/A serverless | + +Serverless ratios are extremely stable between the two runs (dcat ~14–18×, +dgrep ~20–22×). Serverless `dmap` shows parity because turbo is a server-side +optimization; client-side aggregation does not use it (as documented). + +### Server mode (localhost SSH) + +| Scenario | Turbo (r1 / r2) | Non-Turbo (r1 / r2) | Speedup (r1 / r2) | Verdict | +|------------------------------|-----------------|---------------------|-------------------|---------| +| dcat 100 MB | 1.962 / 1.253 | 15.263 / 8.274 | 7.8× / 6.6× | turbo much faster | +| dgrep 100 MB, low hit rate | 0.783 / 0.725 | 5.524 / 2.176 | 7.1× / 3.0× | turbo faster | +| dgrep 100 MB, high hit rate | 0.962 / 0.763 | 6.199 / 2.893 | 6.4× / 3.8× | turbo faster | +| dmap count, 100 MB stats | 2.442 / 1.491 | 6.113 / 3.841 | 2.5× / 2.6× | **turbo faster — now completes (was deadlock)** | +| dmap group-by-agg, 100 MB | 2.729 / 1.692 | 5.318 / 2.667 | 1.9× / 1.6× | **turbo faster — now completes (was deadlock)** | +| dtail follow 10 MB burst | 0.240 / 0.115, **100 % delivered** | timeout, ~39–47 % delivered* | n/a | turbo strictly better | + +\* Non-turbo tail drops lines when the consumer lags (documented behavior, +`internal/io/line/line.go`). Across both runs the non-turbo follow delivered +only ~39 000–47 000 of 100 000 lines and generally never received the end +marker, timing out at >90 s. Turbo delivered all 100 000 lines in ~0.1–0.24 s in +every iteration. + +## Per-tool verdicts + +- **dcat**: turbo a big win serverless (~14–18×). In server mode it is now + ~6–8× faster (previously transport-bound and neutral) — the turbo network + writer path (64 KB buffering, `bytesWritten` accounting) pays off over SSH. +- **dgrep**: turbo a big win everywhere (serverless ~20–22×; server ~3–7×, + larger under heavier load). +- **dtail**: turbo is strictly better — complete delivery in ~0.1 s vs lossy + (~40 %) delivery that times out on the normal path. +- **dmap (serverless)**: turbo not applicable; parity confirmed. +- **dmap (server mode)**: **now works.** Turbo aggregation completes and is + ~1.6×–2.6× faster than non-turbo. See the "what changed" note below. + +## What changed since the 2026-07-10 report + +The previous report (`benchmarks/turbo_vs_normal_report_20260710.md`) found +turbo dramatically faster everywhere it applied **but** uncovered a functional +deadlock: server-mode `dmap` with turbo enabled (the default) never terminated +the client session, because nothing signaled "input exhausted" to +`TurboAggregate` — the map command stayed active, so the shutdown coordinator's +`activeCommands == 0` finalization never ran. + +Task `ws0` fixed this by giving `TurboAggregate` a `FinishInput()` that performs +the final flush + serialize + `done.Shutdown()` once input is exhausted, +analogous to the regular aggregate's channel-close path. This benchmark confirms +the fix end to end: server-mode turbo `dmap` (both `count group by` and the +multi-aggregation query) completes cleanly in every iteration of both runs and +is faster than the non-turbo path. No hangs, no `timeout`/`rc124` rows. + +## Reproducing + +```bash +make clean && make build +bash benchmarks/turbo_vs_normal_bench.sh /tmp/dtail_turbo_bench +# medians: +awk -F, 'NR>1 && $4!="" {k=$1","$2; v[k]=v[k]" "$4} + END{for(x in v){n=split(v[x],a," ");asort(a); + m=(n%2)?a[(n+1)/2]:(a[n/2]+a[n/2+1])/2;printf "%s,%.3f\n",x,m}}' \ + /tmp/dtail_turbo_bench/results.csv | sort +``` |