self.worker.idle.wait() has race-conditions #6
Description
In my setup (using python3) the idle event doesn't seem to work. There seems to be a race condition in the event handling. Example:
execute_some_task.delay()
self.worker.idle.wait()
# something else
This doesn't work for me because self.worker.idle.wait() is executed before the idle event is cleared and thus passes immediately. That's because the events are not handled in the same thread as the call to delay(), thus the order in which the call the wait() and the call to idle.clear() are executed is undefined.
There's actually is a simple way to make those events work: use a simple (lock-protected) counter for the currently running tasks. Increment it on signal.after_task_sent and decrement it on signal.task_postrun.
signal.after_task_sent is guaranteed to run in the same thread as the call to delay.