| Age | Commit message (Collapse) | Author |
|---|
|
- Moved profile_benchmarks.sh, profile_dmap.sh, and profile_quick.sh
to the profiling/ directory where they belong
- Updated Makefile targets to reference new locations
- Fixed profile_dmap.sh to remove outfile clauses since they're not
needed for profiling and were preventing proper execution
- Updated .gitignore to exclude generated files in profiling/
This better separates benchmarking (performance comparison) from
profiling (performance analysis).
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Added || true to kill and wait commands in run_profile_dmap() to prevent
script failures when processes exit with non-zero codes after receiving
SIGINT. This is expected behavior when interrupting dmap.
Note: There appears to be a separate issue where dtail commands (dcat,
dgrep, dmap) hang when run in serverless mode with file arguments. This
needs further investigation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
dmap is designed to run continuously and report MapReduce results at
intervals, which caused it to hang during profiling. Fixed by:
- Added run_profile_dmap() function that runs dmap in background
- Sends SIGINT after 3 seconds to cleanly exit dmap
- Updated all dmap profiling calls to use the new function
- Applied fix to both profile_benchmarks.sh and profile_dmap.sh
This ensures dmap can be profiled successfully without timing out.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Fixed multiple issues preventing dmap from being profiled correctly:
- Updated profile_dmap.sh to use DTail default log format
- Fixed MapReduce queries to use correct field syntax
- Reduced file sizes and run counts for faster profiling
- Added proper command echoing to all data generation steps
Optimizations:
- Reduced PROFILE_RUNS from 3 to 1
- Reduced test data sizes (1MB/10MB instead of 10MB/100MB/1GB)
- Commented out medium/large file tests for faster runs
- Reduced dmap test data from 1000/10000 to 100/1000 lines
The profiling framework now successfully profiles all three commands
(dcat, dgrep, dmap) with reasonable execution times.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Created a comprehensive profiling framework for dtail commands (dcat, dgrep, dmap)
to analyze CPU usage and memory allocations. The framework now prints all executed
commands to stdout for full transparency.
Key features:
- Integrated Go profiling (CPU, memory, allocations) into all three commands
- Created profile.sh bash script for analyzing pprof profiles
- Added multiple Makefile targets for different profiling scenarios
- Automated profiling scripts with command echoing
- Support for different data sizes (quick, normal, full)
- Special handling for dmap MapReduce format
All profiling commands are now echoed to stdout before execution, making it
easy to understand what the framework is doing and reproduce commands manually.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
