Commit 576b1be

committed

Implementing speculative decoding for batch_size > 1; applicable if target and draft model have the same tokenizer.

1 parent 47e795b commit 576b1beCopy full SHA for 576b1be

4 files changed

+449

-112

lines changed

+449

-112

lines changed

Comments

(0)