Skip to content

Conversation

@delner
Copy link
Collaborator

@delner delner commented Dec 18, 2025

Instrument all OpenAI clients with one call

Purpose & Benefits

In the pursuit of an easier setup experience, where before you needed to instrument each individual OpenAI client, this pull request makes it possible to instrument all OpenAI clients with a single call:

# Instrument OpenAI
Braintrust.init
Braintrust::Contrib.instrument!(:openai)

It also supports instrumenting individual clients*, and has backwards compatibility with the wrap API:

# Both calls are equivalent
# New API
Braintrust::Contrib.instrument!(:openai, target: client)
# Old API still works (deprecated)
Braintrust::Trace::OpenAI.wrap(client)

(*) NOTE: Instrumenting both global and client-level is not recommended: although there are some safeguards to check instrumentation state, mixing these can result in double instrumentation (especially instrumenting globally after instrumenting a specific client.) This is due to limitations in Ruby's patching mechanisms.

Design & Key Principles

Architecture (at a glance)

  Braintrust::Contrib
  └── openai/
      ├── instrumentation/      # Contains instrumentation modules
      |       ├── chat.rb       # Chat completions API
      |       └── responses.rb  # Responses API
      ├── integration.rb        # OpenAI integration schema
      └── patcher.rb            # OpenAI patcher

Key Design Decisions

  • Leverage new Integration API framework (which will help us further improve auto-instrumentation later)
  • Retain backwards compatibility with the old wrap API (for now)
  • Check for potential conflicts with ruby-openai (which both use require 'openai')
  • Isolate instrumentation into small modules which utilize prepend to patch.

@delner delner requested review from clutchski and realark December 18, 2025 05:43
@delner delner self-assigned this Dec 18, 2025
@delner delner added the enhancement New feature or request label Dec 18, 2025
@delner delner force-pushed the auto_instrument/02-openai-integration branch from 963650f to 43168f3 Compare December 19, 2025 00:43
@delner delner force-pushed the auto_instrument/02-openai-integration branch 2 times, most recently from 05a972c to c8b96e1 Compare December 19, 2025 01:04
@delner delner force-pushed the auto_instrument/02-openai-integration branch from c8b96e1 to 104d072 Compare December 19, 2025 02:17
@delner
Copy link
Collaborator Author

delner commented Dec 19, 2025

Closing this while I reformulate PRs.

@delner delner closed this Dec 19, 2025
@delner delner deleted the auto_instrument/02-openai-integration branch December 19, 2025 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants