Skip to content

Commit e05de32

Browse files
committed
feat: print ou rest for check gt only
1 parent f8b30d2 commit e05de32

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

bigcodebench/evaluate.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,10 @@ def evaluate(flags):
140140
else:
141141

142142
if flags.check_gt_only:
143+
if gt_pass_rate > 0.95:
144+
cprint(f"Groundtruth pass rate: {gt_pass_rate:.3f}", "green")
145+
else:
146+
cprint(f"Groundtruth pass rate: {gt_pass_rate:.3f}\nPlease be cautious!", "red")
143147
return
144148

145149
results = {

0 commit comments

Comments
 (0)