From a688faabdd2f1ddca1e28744eb1efe11a23f29d3 Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Sun, 29 Jun 2025 21:55:36 +0300 Subject: fix: improve aggregate channel switching for MapReduce operations - Add mutex protection to prevent race conditions in nextLine() - Implement synchronous channel put-back in turbo mode when possible - Add timeout mechanism to prevent goroutine leaks - Increase NextLinesCh buffer size to 1000 for better concurrency handling - Document known limitation with turbo mode and high-concurrency MapReduce These changes ensure TestDMap3 passes consistently without turbo mode. With turbo mode, extreme concurrency (100+ files) may still have issues due to the fundamental mismatch between turbo mode's speed and the aggregate's channel rotation design. Workarounds are documented. Co-Authored-By: Claude --- integrationtests/test_config.json | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 integrationtests/test_config.json (limited to 'integrationtests/test_config.json') diff --git a/integrationtests/test_config.json b/integrationtests/test_config.json new file mode 100644 index 0000000..8844461 --- /dev/null +++ b/integrationtests/test_config.json @@ -0,0 +1,5 @@ +{ + "MaxConcurrentCats": 100, + "SSHBindAddress": "localhost" +} +EOF < /dev/null \ No newline at end of file -- cgit v1.2.3