1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
|
# Turbo vs Normal (Non-Turbo) Benchmark Report — 2026-07-18
**Question:** After the recent server/turbo-path work (in particular task `ws0`,
which gave `TurboAggregate` an input-exhausted `FinishInput` signal), is turbo
mode still faster than the normal (channel-based) path — and does the
server-mode `dmap` deadlock the previous report flagged still exist?
**Answer: Yes — turbo remains faster in every scenario where it applies, and the
result is reproducible across two independent full runs. The server-mode `dmap`
turbo deadlock documented in the 2026-07-10 report is FIXED: turbo `dmap` in
server mode now completes and is ~1.6×–2.6× faster than non-turbo. Server-mode
`dcat`, previously transport-bound and neutral, is now ~6–8× faster on the turbo
path.**
## Methodology
- Binaries rebuilt from clean at HEAD `1a079d9` (`make clean && make build`).
- Turbo is default-on; non-turbo runs set `DTAIL_TURBOBOOST_DISABLE=yes` on the
process that runs the server handlers (the client itself in serverless mode,
`dserver` in server mode).
- Serverless scenarios: client binaries run directly against local files.
- Server scenarios: two `dserver` instances on localhost (one turbo, one
non-turbo, `--cfg none`, real SSH auth via a dedicated RSA key + `CacheDir`
authorized_keys) — *not* `DTAIL_INTEGRATION_TEST_RUN_MODE`, because that mode
force-disables turbo (`internal/config/initializer.go`) and would compare
non-turbo against non-turbo.
- Turbo/non-turbo iterations alternate A/B to spread thermal/load drift (dcat,
dgrep, dmap scenarios); the dtail follow scenario runs all turbo iterations
first, then all non-turbo. 1 warmup + 5–7 measured iterations per
scenario/mode; medians reported.
- Wall-clock timing, client stdout to `/dev/null` (except dtail follow).
- Datasets: `benchmarks/testdata/medium.log` (100 MB, 1.22 M lines),
`large.log` (1 GB), and a generated 100 MB `MAPREDUCE:STATS` file (800 k
lines, 50 hostnames). dgrep patterns: `"user999 "` (low hit rate, ~0.07 %) and
`"ERROR"` (high hit rate, ~19 %).
- dtail follow: client follows an empty file over SSH; after the stream is
established (probe line round-trip), a 10 MB / 100 k-line burst plus end
marker is appended; time from append to marker arrival at the client is
measured, and delivered-line completeness is counted.
- The benchmark was run twice end to end (run 1 and run 2) to check
reproducibility. Raw per-iteration data is committed alongside this report:
`turbo-vs-normal-benchmark-20260718-run1.csv` and `...-run2.csv`. Harness:
`benchmarks/turbo_vs_normal_bench.sh`.
- Machine: 11th Gen Intel i7-1185G7 @ 3.00 GHz (8 threads), Fedora Linux 44,
Go 1.26.4. Caveat: laptop; absolute numbers are noisy (thermal/background
load), so the **turbo/non-turbo ratios are the meaningful signal**, not the
absolute seconds. This is visible in the server-mode rows, where run 2's
non-turbo times are lower than run 1's (lighter machine load), which
compresses those ratios without any change to the turbo path.
- Non-turbo serverless runs go through an `env` wrapper that turbo runs do not
(~1 ms extra process spawn). Negligible at these magnitudes, and it biases
against turbo, not for it.
## Results (median wall time, seconds)
### Serverless mode
| Scenario | Turbo (r1 / r2) | Non-Turbo (r1 / r2) | Speedup (r1 / r2) | Verdict |
|------------------------------|-----------------|---------------------|-------------------|---------|
| dcat 100 MB | 0.094 / 0.094 | 1.310 / 1.322 | 13.9× / 14.1× | turbo much faster |
| dcat 1 GB | 0.991 / 1.034 | 17.776 / 17.912 | 17.9× / 17.3× | turbo much faster |
| dgrep 100 MB, low hit rate | 0.106 / 0.111 | 2.104 / 2.289 | 19.8× / 20.6× | turbo much faster |
| dgrep 100 MB, high hit rate | 0.098 / 0.104 | 2.184 / 2.342 | 22.3× / 22.5× | turbo much faster |
| dmap agg 100 MB (control) | 3.072 / 2.999 | 3.279 / 3.068 | 1.07× / 1.02× | expected: turbo N/A serverless |
Serverless ratios are extremely stable between the two runs (dcat ~14–18×,
dgrep ~20–22×). Serverless `dmap` shows parity because turbo is a server-side
optimization; client-side aggregation does not use it (as documented).
### Server mode (localhost SSH)
| Scenario | Turbo (r1 / r2) | Non-Turbo (r1 / r2) | Speedup (r1 / r2) | Verdict |
|------------------------------|-----------------|---------------------|-------------------|---------|
| dcat 100 MB | 1.962 / 1.253 | 15.263 / 8.274 | 7.8× / 6.6× | turbo much faster |
| dgrep 100 MB, low hit rate | 0.783 / 0.725 | 5.524 / 2.176 | 7.1× / 3.0× | turbo faster |
| dgrep 100 MB, high hit rate | 0.962 / 0.763 | 6.199 / 2.893 | 6.4× / 3.8× | turbo faster |
| dmap count, 100 MB stats | 2.442 / 1.491 | 6.113 / 3.841 | 2.5× / 2.6× | **turbo faster — now completes (was deadlock)** |
| dmap group-by-agg, 100 MB | 2.729 / 1.692 | 5.318 / 2.667 | 1.9× / 1.6× | **turbo faster — now completes (was deadlock)** |
| dtail follow 10 MB burst | 0.240 / 0.115, **100 % delivered** | timeout, ~39–47 % delivered* | n/a | turbo strictly better |
\* Non-turbo tail drops lines when the consumer lags (documented behavior,
`internal/io/line/line.go`). Across both runs the non-turbo follow delivered
only ~39 000–47 000 of 100 000 lines and generally never received the end
marker, timing out at >90 s. Turbo delivered all 100 000 lines in ~0.1–0.24 s in
every iteration.
## Per-tool verdicts
- **dcat**: turbo a big win serverless (~14–18×). In server mode it is now
~6–8× faster (previously transport-bound and neutral) — the turbo network
writer path (64 KB buffering, `bytesWritten` accounting) pays off over SSH.
- **dgrep**: turbo a big win everywhere (serverless ~20–22×; server ~3–7×,
larger under heavier load).
- **dtail**: turbo is strictly better — complete delivery in ~0.1 s vs lossy
(~40 %) delivery that times out on the normal path.
- **dmap (serverless)**: turbo not applicable; parity confirmed.
- **dmap (server mode)**: **now works.** Turbo aggregation completes and is
~1.6×–2.6× faster than non-turbo. See the "what changed" note below.
## What changed since the 2026-07-10 report
The previous report (`benchmarks/turbo_vs_normal_report_20260710.md`) found
turbo dramatically faster everywhere it applied **but** uncovered a functional
deadlock: server-mode `dmap` with turbo enabled (the default) never terminated
the client session, because nothing signaled "input exhausted" to
`TurboAggregate` — the map command stayed active, so the shutdown coordinator's
`activeCommands == 0` finalization never ran.
Task `ws0` fixed this by giving `TurboAggregate` a `FinishInput()` that performs
the final flush + serialize + `done.Shutdown()` once input is exhausted,
analogous to the regular aggregate's channel-close path. This benchmark confirms
the fix end to end: server-mode turbo `dmap` (both `count group by` and the
multi-aggregation query) completes cleanly in every iteration of both runs and
is faster than the non-turbo path. No hangs, no `timeout`/`rc124` rows.
## Reproducing
```bash
make clean && make build
bash benchmarks/turbo_vs_normal_bench.sh /tmp/dtail_turbo_bench
# medians:
awk -F, 'NR>1 && $4!="" {k=$1","$2; v[k]=v[k]" "$4}
END{for(x in v){n=split(v[x],a," ");asort(a);
m=(n%2)?a[(n+1)/2]:(a[n/2]+a[n/2+1])/2;printf "%s,%.3f\n",x,m}}' \
/tmp/dtail_turbo_bench/results.csv | sort
```
|
