We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent d5a9b75 commit 04d35acCopy full SHA for 04d35ac
benchmarks/yaml/eb45-32k-wint4-ep4-tp4.yaml
@@ -0,0 +1,7 @@
1
+num_gpu_blocks_override: 1024
2
+max_model_len: 8192
3
+max_num_seqs: 64
4
+data_parallel_size: 4
5
+tensor_parallel_size: 1
6
+enable_expert_parallel: True
7
+quantization: wint4
0 commit comments