We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 7109710 commit e51bd31Copy full SHA for e51bd31
README.md
@@ -92,13 +92,12 @@ pip install "git+https://github.com/bigcode-project/bigcodebench.git#egg=bigcode
92
We use the greedy decoding as an example to show how to evaluate the generated code samples via remote API.
93
94
```bash
95
+# greedy decoding by default
96
bigcodebench.evaluate \
97
--model meta-llama/Meta-Llama-3.1-8B-Instruct \
98
--split [complete|instruct] \
99
--subset [full|hard] \
- --backend [vllm|openai|anthropic|google|mistral|hf] \
100
- --tp [TENSOR_PARALLEL_SIZE] \
101
- --greedy
+ --backend [vllm|openai|anthropic|google|mistral|hf]
102
```
103
104
- All the resulted files will be stored in a folder named `bcb_results`.
0 commit comments