Skip to content

Commit 080ae7e

Browse files
hiworldwzjwangzaijun
andauthored
fix prefill classed mode. (#1134)
Co-authored-by: wangzaijun <wangzaijun@sensetime.com>
1 parent a41492e commit 080ae7e

File tree

2 files changed

+4
-0
lines changed
  • lightllm/server/router/model_infer/mode_backend

2 files changed

+4
-0
lines changed

lightllm/server/router/model_infer/mode_backend/chunked_prefill/impl.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ def __init__(self) -> None:
4646
else:
4747
self.prefill = self.prefill_normal
4848
self.decode = self.decode_normal
49+
50+
self.classed_req_strict_prefill = False
4951
return
5052

5153
def infer_loop(self):

lightllm/server/router/model_infer/mode_backend/dp_backend/impl.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,8 @@ def __init__(self) -> None:
5858
self.decode = self.decode_overlap
5959
else:
6060
self.decode = self.decode_normal
61+
62+
self.classed_req_strict_prefill = False
6163
return
6264

6365
def infer_loop(self):

0 commit comments

Comments
 (0)