| Age | Commit message (Collapse) | Author |
|---|
|
- Fix serverless mode extra blank lines by removing DTail 3 backward
compatibility fallthrough for '\n' character
- Fix empty line handling in client message processing
- Update integration test framework to inherit environment variables,
allowing turbo boost testing
- Clean up debug logging code
Note: dcat1d.txt test fails because DTail adds newline to files without
trailing newlines - this is a protocol limitation where newlines are
stripped during transmission and re-added by the client.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Replace DTAIL_CHANNELLESS_GREP and DTAIL_OPTIMIZED_READER with single flag
- Rename documentation to TURBOBOOST_OPTIMIZATION.md
- Fix channel-less adapter to use blocking sends (prevent data loss)
- Update logging messages to reference "turbo boost" mode
The DTAIL_TURBOBOOST_ENABLE variable now controls all performance
optimizations and can be extended to other commands in the future.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Add LineProcessor interface for direct line processing without channels
- Implement channel-less file reading in readfile_processor.go
- Add optimized reader with 256KB buffering for efficient I/O
- Create GrepLineProcessor for direct writing without intermediate channels
- Fix serverless mode hanging due to stdin pipe detection
- Fix base64 decoding bug (was counting characters instead of arguments)
- Fix message output formatting by adding proper newline handling
Performance improvements:
- Channel-based: 9.00s → Channel-less: 3.42s (62% faster on 100MB files)
- Removed channel synchronization overhead and context switching
- Reduced memory allocations with buffer pooling
Environment variables:
- DTAIL_CHANNELLESS_GREP=yes - Enable channel-less implementation
- DTAIL_OPTIMIZED_READER=yes - Use optimized buffered reader
Known limitation: Inverted grep with context (--invert with --before/--after)
not fully implemented in channel-less mode.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Added detailed README.md for internal/profiling package documenting:
- Architecture and core components
- Usage instructions for command-line and programmatic access
- Profile output formats and analysis techniques
- Best practices and troubleshooting guides
- Integration with CI/CD pipelines
- Removed dprofile binary (obsolete, replaced by built-in profiling)
- Cleaned up go.mod to remove unused pprof dependency
The profiling package is now fully documented to help developers
understand and utilize DTail's performance analysis capabilities.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
The WHERE clause in dmap queries uses 'eq' operator for string equality,
not '=' or '=='. This was causing the CSV profiling query to fail with
a parsing error.
Fixed query:
- Changed from: where $status = "success"
- Changed to: where $status eq "success"
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
This major refactoring replaces all bash-based profiling and benchmarking
scripts with a unified Go tool (dtail-tools) that provides:
- Better cross-platform compatibility
- Improved error handling and reliability
- Structured data generation for test files
- Consistent command-line interface
- Easier maintenance and extensibility
Key changes:
- Created dtail-tools command with profile and benchmark subcommands
- Implemented common utilities for data generation and file operations
- Updated Makefile to use the new Go-based tools
- Maintained backward compatibility with existing make targets
- Fixed ParseSize to handle single-letter suffixes (10M, 1G, etc.)
The new tool supports all previous functionality:
- profile-quick, profile-all, profile-dmap
- benchmark creation, comparison, and management
- Test data generation with multiple formats
- Profile analysis and listing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Move main package files to benchmarks/cmd/ to fix test failures
- Update CLAUDE.md with comprehensive benchmarking and profiling instructions
- Fix unused imports in serverless.go
- Remove experimental buffered pipe/copy implementations
- Remove outdated documentation files
All integration tests now pass successfully.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Implement channel-based bidirectional copying in serverless connector
to prevent deadlocks that occur with io.Copy when processing large files.
Changes:
- Replace direct io.Copy with channel-based approach in serverless.go
- Add bufferedpipe and bufferedcopy utilities (for future use)
- Add tests to verify deadlock prevention
- Fix dmap profiling example to use absolute paths
The fix successfully handles files up to ~10KB in serverless mode.
Larger files still experience issues and will be addressed in a
follow-up fix.
Fixes profiling hang issue when using -cfg none without servers.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Created a comprehensive profiling framework for dtail commands (dcat, dgrep, dmap)
to analyze CPU usage and memory allocations. The framework now prints all executed
commands to stdout for full transparency.
Key features:
- Integrated Go profiling (CPU, memory, allocations) into all three commands
- Created profile.sh bash script for analyzing pprof profiles
- Added multiple Makefile targets for different profiling scenarios
- Automated profiling scripts with command echoing
- Support for different data sizes (quick, normal, full)
- Special handling for dmap MapReduce format
All profiling commands are now echoed to stdout before execution, making it
easy to understand what the framework is doing and reproduce commands manually.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Fixed fmt.Errorf calls in integrationtests/fileutils.go
- Fixed fmt.Errorf calls in integrationtests-old/fileutils.go
- Fixed fmt.Errorf calls in internal/color/color.go
- Fixed t.Errorf call in internal/mapr/logformat/default_test.go
All tests now pass with Go 1.24's stricter format string requirements.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Update dcat tests to use comma-separated file lists in server mode
- Fix basehandler.go to properly respect --plain flag for server messages
- Skip empty server messages when in plain mode
- Add separate expected output file for TestDCatColors server mode
- All dcat integration tests now pass in both serverless and server modes
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
added the syntax to the mapr query
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
