Skip to content

Commit 3befc3e

Browse files
committed
Seems like the pool is self-exhausting all the resources? Is it due to _worker_task_orchestrator loop being too intensive?
1 parent 02246f5 commit 3befc3e

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/crawlee/_autoscaling/autoscaled_pool.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ class AutoscaledPool:
4646
_AUTOSCALE_INTERVAL = timedelta(seconds=10)
4747
"""Interval at which the autoscaled pool adjusts the desired concurrency based on the latest system status."""
4848

49-
_LOGGING_INTERVAL = timedelta(minutes=1)
49+
_LOGGING_INTERVAL = timedelta(seconds=0.5)
5050
"""Interval at which the autoscaled pool logs its current state."""
5151

5252
_DESIRED_CONCURRENCY_RATIO = 0.9

0 commit comments

Comments
 (0)