| Age | Commit message (Collapse) | Author |
|---|
|
- stdout logger: release mutex while waiting on pause resume so prompt
callbacks can log (fixes hang after trusting new hosts; known_hosts
was written but Resume never ran).
- known hosts callback: stop borrowing the SSH dial throttle channel
(could block or interact badly with parallel handshakes).
- host key path: use errors.Is(..., fs.ErrNotExist) for RootedPath.Stat
wrapped errors; stat errors now fail fast instead of mis-read.
- public key path: same ErrNotExist check for authorized_keys miss.
- Build: optional DTAIL_NO_ZSTD=yes / nozstd tag for CGO-free builds;
split zstd readers into tagged files.
- Docs/examples: firewalld note for port 2222, log prune timer+script,
SSHBindAddress note, dserver unit disabled-by-default comment;
firewalld helper script example.
- Regression test for stdout pause/mutex behavior.
Made-with: Cursor
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Task: 4e6d7744-3f5c-4880-9e5d-368ece96470d
|
|
|
|
Modernize network dialing to use Go's context-aware patterns for better
cancellation support and connection reliability.
Changes:
- Update Go version from 1.24 to 1.25 in go.mod
- Replace ssh.Dial with net.Dialer.DialContext + ssh.NewClientConn
for SSH client connections in serverconnection.go
- Add TCP KeepAlive (30s) for SSH connection health monitoring
- Implement context-aware dialing for SSH agent connections in ssh.go
- Improve error messages to distinguish dial vs SSH handshake failures
- Update AGENTS.md with integration test requirements
Benefits:
- Context cancellation now properly affects connection establishment
- TCP KeepAlive prevents silent connection failures
- Better integration with Go's cancellation patterns
- Improved reliability for distributed systems
All integration tests pass with race detection enabled.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
This commit adds two major features and fixes:
1. SSH Agent Key Selection:
- Add --agentKeyIndex flag to select specific SSH agent key (0-based)
- Solves "too many authentication failures" with multiple SSH keys
- Default -1 uses all keys (backwards compatible)
- Available in dtail, dcat, dgrep, dmap commands
2. MapReduce Outfile Fixes:
- CSV files now written at every interval, not just on exit
- Proper signal handling (SIGTERM/SIGINT) with graceful shutdown
- 5-second grace period for cleanup before force exit
- Fixes issue where outfile remained as .tmp during execution
Usage:
dtail --servers host --agentKeyIndex 0 --query '...' outfile results.csv
This is particularly useful with YubiKey/hardware tokens where many
keys are loaded in the SSH agent, and for monitoring MapReduce results
in real-time as they're computed.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
- Add explicit interface satisfaction checks (var _ Interface = (*Type)(nil)) for compile-time verification:
- TurboWriter implementations (DirectTurboWriter, TurboChannelWriter)
- Processor implementations (GrepLineProcessor, ChannellessLineProcessor)
- Parser implementations (genericParser, csvParser, genericKVParser, custom parsers, mimecastParser)
- Logger implementations (file, stdout)
- Handler implementations (ServerHandler, ClientHandler)
- Connector implementations (Serverless, ServerConnection)
- SSH callback implementations (KnownHostsCallback)
- Improve error handling with context wrapping (%w):
- SSH operations: GeneratePrivateRSAKey, Agent
- Query parsing: Query.parse
- SSH client connections: dial, session, handle methods
- Fix receiver consistency:
- Convert Query.String() from value to pointer receiver
- Convert Outfile.String() from value to pointer receiver
- Convert all KnownHostsCallback methods to pointer receivers
- Convert mapCommand.Start() to pointer receiver
- Reorganize file structure for better clarity:
- internal/io/dlog/dlog.go: Move type definition before public functions
- internal/mapr/token.go: Reorganize helper functions after public ones
- Add documentation comments:
- Query.String() method
- Outfile.String() method
- Regex.String() method
- Improve config variable documentation
All unit tests and integration tests pass.
Amp-Thread-ID: https://ampcode.com/threads/T-019c0b08-0eeb-705d-a1f7-31bb764b659a
Co-authored-by: Amp <amp@ampcode.com>
|
|
In serverless mode (when dcat runs locally), data is written directly to stdout
and doesn't need network transmission delays. This fix eliminates the 500ms+
exit delay by skipping unnecessary sleep calls when running in serverless mode.
Changes:
- Skip 500ms wait in readFiles() when serverless
- Skip 50ms wait in readWithTurboProcessor() when serverless
- Skip aggregate serialization waits when serverless
- Fix turbo benchmark test compilation errors
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Fix serverless MapReduce to pass options with map command for proper mode detection
- Prevent raw lines from being sent to client during MapReduce operations
- Only use turbo mode for cat/grep/tail when no aggregate is present
- Fix race conditions in TurboAggregate with proper synchronization
- Add SafeAggregateSet wrapper for thread-safe operations
- Fix parser selection to use correct parser names
- Add comprehensive unit tests for turbo aggregate functionality
This ensures MapReduce operations in both turbo and non-turbo modes
produce identical results and fixes serverless mode processing.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Fix serverless mode extra blank lines by removing DTail 3 backward
compatibility fallthrough for '\n' character
- Fix empty line handling in client message processing
- Update integration test framework to inherit environment variables,
allowing turbo boost testing
- Clean up debug logging code
Note: dcat1d.txt test fails because DTail adds newline to files without
trailing newlines - this is a protocol limitation where newlines are
stripped during transmission and re-added by the client.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Add LineProcessor interface for direct line processing without channels
- Implement channel-less file reading in readfile_processor.go
- Add optimized reader with 256KB buffering for efficient I/O
- Create GrepLineProcessor for direct writing without intermediate channels
- Fix serverless mode hanging due to stdin pipe detection
- Fix base64 decoding bug (was counting characters instead of arguments)
- Fix message output formatting by adding proper newline handling
Performance improvements:
- Channel-based: 9.00s → Channel-less: 3.42s (62% faster on 100MB files)
- Removed channel synchronization overhead and context switching
- Reduced memory allocations with buffer pooling
Environment variables:
- DTAIL_CHANNELLESS_GREP=yes - Enable channel-less implementation
- DTAIL_OPTIMIZED_READER=yes - Use optimized buffered reader
Known limitation: Inverted grep with context (--invert with --before/--after)
not fully implemented in channel-less mode.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Move main package files to benchmarks/cmd/ to fix test failures
- Update CLAUDE.md with comprehensive benchmarking and profiling instructions
- Fix unused imports in serverless.go
- Remove experimental buffered pipe/copy implementations
- Remove outdated documentation files
All integration tests now pass successfully.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Implement channel-based bidirectional copying in serverless connector
to prevent deadlocks that occur with io.Copy when processing large files.
Changes:
- Replace direct io.Copy with channel-based approach in serverless.go
- Add bufferedpipe and bufferedcopy utilities (for future use)
- Add tests to verify deadlock prevention
- Fix dmap profiling example to use absolute paths
The fix successfully handles files up to ~10KB in serverless mode.
Larger files still experience issues and will be addressed in a
follow-up fix.
Fixes profiling hang issue when using -cfg none without servers.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
connect to it via a DTail client 4.x
|
|
more tests
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
