From 17bf7e042496a4afcbf6ee7a583378adb3ec502d Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Wed, 22 Jul 2026 23:52:43 +0300 Subject: =?UTF-8?q?build:=20DTail=20fork=20=E2=80=94=20benchmarks,=20PGO?= =?UTF-8?q?=20tooling,=20Makefile,=20docker?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Squashed development of the build/benchmark/profiling tooling: - Turbo-vs-normal and per-tool benchmark harnesses under benchmarks/ (with dated baseline and result artifacts preserved), plus MapReduce/SSH/network suites. - Profile-Guided Optimization workflow (internal/tools/pgo lives with the code): Makefile targets (make pgo / pgo-quick / build-pgo), profile generation with non-zero-sample and representativeness guards. - Makefile targets for build, test (unit + integration), lint, vet, benchmarking and profiling; docker config updates. - .gitignore hardened against pgo profiles and accidental integration-test debug/output dumps. Co-Authored-By: Claude Opus 4.8 --- .../baselines/benchmark_comparison_report.md | 75 ++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 benchmarks/baselines/benchmark_comparison_report.md (limited to 'benchmarks/baselines/benchmark_comparison_report.md') diff --git a/benchmarks/baselines/benchmark_comparison_report.md b/benchmarks/baselines/benchmark_comparison_report.md new file mode 100644 index 0000000..89ce05a --- /dev/null +++ b/benchmarks/baselines/benchmark_comparison_report.md @@ -0,0 +1,75 @@ +# Benchmark Comparison Report: v4.3.0 vs Turbo-Enabled + +## Summary + +This report compares the performance of DTail v4.3.0 (baseline) with the current version that has turbo boost mode enabled by default. + +## Performance Improvements + +### DCat Operations +- **10MB file**: + - v4.3.0: 9.363 MB/sec + - Turbo: 246.8 MB/sec + - **Improvement: 2,535% (26.3x faster)** + +### DGrep Operations (10MB file) +- **1% hit rate**: + - v4.3.0: 25.38 MB/sec + - Turbo: 363.9 MB/sec + - **Improvement: 1,334% (14.3x faster)** + +- **10% hit rate**: + - v4.3.0: 22.81 MB/sec + - Turbo: 342.6 MB/sec + - **Improvement: 1,402% (15.0x faster)** + +- **50% hit rate**: + - v4.3.0: 16.14 MB/sec + - Turbo: 265.1 MB/sec + - **Improvement: 1,543% (16.4x faster)** + +- **90% hit rate**: + - v4.3.0: 10.99 MB/sec + - Turbo: 210.0 MB/sec + - **Improvement: 1,811% (19.1x faster)** + +### DMap Operations (10MB file) +- **Count query**: + - v4.3.0: 17.09 MB/sec + - Turbo: 21.77 MB/sec + - **Improvement: 27.4%** + +- **Sum/Avg query**: + - v4.3.0: 13.54 MB/sec + - Turbo: 21.05 MB/sec + - **Improvement: 55.5%** + +- **Min/Max query**: + - v4.3.0: 17.46 MB/sec + - Turbo: 21.80 MB/sec + - **Improvement: 24.9%** + +- **Multi-field query**: + - v4.3.0: 21.85 MB/sec + - Turbo: 21.32 MB/sec + - **Slight decrease: -2.4%** (within margin of error) + +## Key Findings + +1. **Massive improvements in DCat and DGrep**: The turbo boost mode shows extraordinary performance gains for file reading (DCat) and searching (DGrep) operations, with improvements ranging from 14x to 26x faster. + +2. **Moderate improvements in DMap**: MapReduce operations show more modest but still significant improvements of 25-55% for most query types. + +3. **Consistent performance across hit rates**: DGrep performance improvements scale well across different hit rates, with even better improvements at higher hit rates. + +## Technical Details + +The turbo boost mode achieves these improvements through: +- Direct writing bypassing channels for cat/grep/tail operations +- Direct line processing without channels for MapReduce in server mode +- Batch processing to reduce lock contention +- Memory pooling to reduce garbage collection pressure + +## Recommendation + +The turbo boost mode delivers exceptional performance improvements and should remain enabled by default. The performance gains are substantial enough to justify any potential trade-offs in code complexity. \ No newline at end of file -- cgit v1.2.3