| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
- Add SafeAggregateSet wrapper with mutex protection for concurrent access
- Implement TurboAggregate for direct line processing without channels
- Fix race conditions in turbo mode MapReduce aggregation
- Add proper synchronization for batch processing completion
- Update shutdown sequence to ensure all data is serialized
- Add integration test configuration for high-concurrency scenarios
The turbo mode now correctly handles MapReduce queries with significant
performance improvements while maintaining data integrity and preventing
race conditions during concurrent aggregation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Add TurboModeEnable setting to server configuration with environment variable override.
The DTAIL_TURBOBOOST_ENABLE environment variable takes precedence over config file setting.
Turbo mode is automatically disabled for MapReduce operations to prevent data accuracy issues.
- Add TurboModeEnable boolean to ServerConfig struct
- Update config initializer to check environment variable for backward compatibility
- Replace direct env var checks with config.Server.TurboModeEnable throughout codebase
- Enable turbo mode in example config file (dtail.json.example)
- Add property to JSON schema with descriptive documentation
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Add mutex protection to prevent race conditions in nextLine()
- Implement synchronous channel put-back in turbo mode when possible
- Add timeout mechanism to prevent goroutine leaks
- Increase NextLinesCh buffer size to 1000 for better concurrency handling
- Document known limitation with turbo mode and high-concurrency MapReduce
These changes ensure TestDMap3 passes consistently without turbo mode.
With turbo mode, extreme concurrency (100+ files) may still have issues
due to the fundamental mismatch between turbo mode's speed and the
aggregate's channel rotation design. Workarounds are documented.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
|
|
|
|
|
|
packages for logging
|
|
|
|
|
|
|
|
|
|
logger package rewrite as dlog
|
|
|
|
|
|
|
|
|
|
useful for log file monitorig for example. breaking protocol change which allows to mapreduce aggreate messages containing the default field separator |. add of more unit tests. add logformat mapreduce query keyword. add set mapreduce clause support and support to evaluate built-in functions such as md5sum() and maskdigits().
|
|
|
|
log lines containing a certain string, such as "ERROR"
|
|
|
|
|
|
|
