Skip to content

Commit 7e66063

Browse files
authored
[Minor Bug] Fix a bug in the blocking algorithm of the generate thread. (#174)
Co-authored-by: Lihao Ran <ranlihao@google.com>
1 parent a2be72d commit 7e66063

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

jetstream/core/orchestrator.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -685,7 +685,8 @@ def _generate_thread(self, idx: int):
685685
# For interleaved mode, we also blocks when prefill backlog
686686
# is not empty or there are transfer work to do.
687687
block |= not self._prefill_backlog.empty()
688-
block |= not self._transfer_backlogs[idx].empty()
688+
for transfer_backlog in self._transfer_backlogs:
689+
block |= not transfer_backlog.empty()
689690
try:
690691
new_request = my_generate_backlog.get(block=block, timeout=1.0)
691692
if new_request is None:

0 commit comments

Comments
 (0)