We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent f6fc695 commit edcf5c6Copy full SHA for edcf5c6
run.sh
@@ -33,8 +33,5 @@ bigcodebench.sanitize --samples $FILE_HEADER.jsonl --calibrate
33
# Check if the ground truth works on your machine
34
bigcodebench.evaluate --subset $SUBSET --samples $FILE_HEADER-sanitized-calibrated.jsonl
35
36
-# If the ground truth does not work, you can skip it:
37
-bigcodebench.evaluate --subset $SUBSET --samples $FILE_HEADER-sanitized-calibrated.jsonl --no-gt
38
-
39
# If the execution is slow:
40
bigcodebench.evaluate --subset $SUBSET --samples $FILE_HEADER-sanitized-calibrated.jsonl --parallel 32
0 commit comments