diff options
| author | Paul Buetow <paul@buetow.org> | 2025-07-04 15:35:16 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-07-04 15:35:16 +0300 |
| commit | d37f32deb6cd6a575cc169adf1a1c1fba44e53d9 (patch) | |
| tree | aaf5f6abc90066892a6a23cb619969ddd4ef5574 /Makefile | |
| parent | 1249f9ec51b1355ca17f73244dcbe0acc5556516 (diff) | |
feat: add Profile-Guided Optimization (PGO) support
- Add comprehensive PGO module in internal/tools/pgo/
- Integrate PGO into dtail-tools command with full CLI support
- Add Makefile targets for PGO workflow:
- make pgo: Full PGO workflow
- make pgo-quick: Quick PGO with smaller datasets
- make pgo-generate: Generate profiles only
- make build-pgo: Build with existing profiles
- make install-pgo: Install optimized binaries
- Add convenience functions to data generator for PGO
- Document PGO workflow in CLAUDE.md
Performance improvements observed:
- DCat: 3.8-7.0% additional improvement over turbo mode
- DGrep: Up to 19% improvement for low hit rates
- DMap: Variable impact, up to 64% for min_max on large files
Benchmarks show total performance gains (pre-turbo → turbo+PGO):
- DCat: 14-21x faster
- DGrep: 9-15x faster
- DMap: 9-29% faster
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Diffstat (limited to 'Makefile')
| -rw-r--r-- | Makefile | 76 |
1 files changed, 75 insertions, 1 deletions
@@ -6,7 +6,8 @@ ifdef DTAIL_USE_PROPRIETARY GO_TAGS+=proprietary endif all: build -build: dserver dcat dgrep dmap dtail dtailhealth +build: dserver dcat dgrep dmap dtail dtailhealth dtail-tools +build-pgo: pgo-build-binaries dserver: ${GO} build ${GO_FLAGS} -tags '${GO_TAGS}' -o dserver ./cmd/dserver/main.go dcat: @@ -135,3 +136,76 @@ profile-help: @echo "" .PHONY: profile-all profile-quick profile-dmap profile-list profile-analyze profile-web profile-clean profile-help + +## Profile-Guided Optimization targets +pgo: build dtail-tools + @echo "Running Profile-Guided Optimization for all commands..." + ./dtail-tools pgo + +pgo-quick: build dtail-tools + @echo "Running quick PGO with smaller datasets..." + ./dtail-tools pgo -datasize 100000 -iterations 2 + +pgo-commands: build dtail-tools + @if [ -z "${COMMANDS}" ]; then \ + echo "Usage: make pgo-commands COMMANDS='dcat dgrep'"; \ + exit 1; \ + fi + ./dtail-tools pgo ${COMMANDS} + +pgo-clean: + @echo "Cleaning PGO artifacts..." + rm -rf pgo-profiles pgo-build + +pgo-help: + @echo "DTail PGO (Profile-Guided Optimization) Targets:" + @echo "" + @echo " make pgo - Run PGO for all commands (full optimization)" + @echo " make pgo-quick - Quick PGO with smaller datasets" + @echo " make pgo-commands - PGO for specific commands" + @echo " Example: make pgo-commands COMMANDS='dcat dgrep'" + @echo " make pgo-clean - Remove PGO artifacts" + @echo "" + @echo "After running PGO, optimized binaries will be in pgo-build/" + @echo "" + +# Build PGO-optimized binaries without running benchmarks +# This assumes PGO profiles already exist in pgo-profiles/ +pgo-build-binaries: dtail-tools + @if [ ! -d "pgo-profiles" ]; then \ + echo "Error: pgo-profiles directory not found."; \ + echo "Run 'make pgo' first to generate profiles, or 'make pgo-generate' to only generate profiles."; \ + exit 1; \ + fi + @echo "Building PGO-optimized binaries using existing profiles..." + @mkdir -p pgo-build + @for cmd in dcat dgrep dmap dtail dserver; do \ + profile="pgo-profiles/$$cmd.pprof"; \ + if [ -f "$$profile" ]; then \ + echo "Building $$cmd with PGO..."; \ + ${GO} build ${GO_FLAGS} -tags '${GO_TAGS}' -pgo=$$profile -o pgo-build/$$cmd ./cmd/$$cmd/main.go; \ + else \ + echo "Warning: Profile $$profile not found, building without PGO..."; \ + ${GO} build ${GO_FLAGS} -tags '${GO_TAGS}' -o pgo-build/$$cmd ./cmd/$$cmd/main.go; \ + fi \ + done + @echo "PGO-optimized binaries built in pgo-build/" + +# Generate PGO profiles without building optimized binaries +pgo-generate: build dtail-tools + @echo "Generating PGO profiles..." + ./dtail-tools pgo -profileonly + @echo "PGO profiles generated in pgo-profiles/" + +# Install PGO-optimized binaries to system +install-pgo: pgo-build-binaries + @echo "Installing PGO-optimized binaries..." + @for cmd in dcat dgrep dmap dtail dserver; do \ + if [ -f "pgo-build/$$cmd" ]; then \ + echo "Installing $$cmd..."; \ + cp pgo-build/$$cmd ${GOPATH}/bin/$$cmd || sudo cp pgo-build/$$cmd /usr/local/bin/$$cmd; \ + fi \ + done + @echo "PGO-optimized binaries installed" + +.PHONY: pgo pgo-quick pgo-commands pgo-clean pgo-help pgo-build-binaries pgo-generate install-pgo |
