How to use an Anthropic model for evals without unsetting OPENAI_API_KEY?

[holdenmatt]

I wanted to use an Anthropic Claude model for all my evals, but a mix of openai/anthropic models for my generation tasks.

But I couldn't find a way to do this.

If I use Claude for both the generation and eval models it works using my local ANTHROPIC_API_KEY, but only if I don't have an OPENAI_API_KEY set. But if I unset this, then I can't use openai for generation tasks or elsewhere in my app.

Not sure what the best solution is, but it seems odd that you'd need to unset OPENAI_API_KEY to use a different eval model?

[ankrgyl]

Currently the only way to use an anthropic model + autoevals is:

• Use your BRAINTRUST_API_KEY
• Set up an Anthropic API key in Braintrust
• Specify the model string you'd like to use

This will use our proxy and exchange your BRAINTRUST_API_KEY for the anthropic one behind the scenes.

[holdenmatt]

I'd be fine with that.

But I couldn't get the proxy to work as long as I had an OPENAI_API_KEY defined in .env.local. Is there a way to use the proxy without deleting that env var?