diff options
| author | Paul Buetow <paul@buetow.org> | 2025-06-29 21:55:36 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-06-29 21:55:36 +0300 |
| commit | a688faabdd2f1ddca1e28744eb1efe11a23f29d3 (patch) | |
| tree | 7edc0d361b22816e769bc2f2f1dde30b5048da45 /integrationtests/testhelpers.go | |
| parent | 2140ed9dcbd180cd5e810eaabd2f3c2fbce55a57 (diff) | |
fix: improve aggregate channel switching for MapReduce operations
- Add mutex protection to prevent race conditions in nextLine()
- Implement synchronous channel put-back in turbo mode when possible
- Add timeout mechanism to prevent goroutine leaks
- Increase NextLinesCh buffer size to 1000 for better concurrency handling
- Document known limitation with turbo mode and high-concurrency MapReduce
These changes ensure TestDMap3 passes consistently without turbo mode.
With turbo mode, extreme concurrency (100+ files) may still have issues
due to the fundamental mismatch between turbo mode's speed and the
aggregate's channel rotation design. Workarounds are documented.
Co-Authored-By: Claude <noreply@anthropic.com>
Diffstat (limited to 'integrationtests/testhelpers.go')
0 files changed, 0 insertions, 0 deletions