This library provides an architecture for building LLM-based applications.
Specifically it wraps underlying Chat interfaces for LLMs providing an application architecture that can be useful for building agents. The focus is to provide first-class support for tools and complex workflows.
This can be thought of as a native-go approach to Python frameworks such as langchain.
The library uses the official OpenAI Go SDK for OpenAI API integration.
p := openaichat.New("my-api-key", "gpt-5-mini-2025-08-07")
ts := tools.New()
a := agent.New(p,
agent.WithFilter(agent.LimitMessagesFilter(5)),
// Next add functions because they are one of the rare cases where we
// intercept and directly handle completions.
tools.WithTools(ts),
)
a.Add(agent.RoleSystem, "You are a helpful assistant.")
a.Add(agent.RoleUser, "Are you alive?")
// Allow a single call to the underlying provider.
r, err := a.Step(context.Background())
if err != nil {
log.Fatalf("error from Agent: %v", err)
}
c, err := r.Content(context.Background())
if err != nil {
log.Fatalf("error from Message: %v", err)
}
fmt.Println(c)Agent is fundamentally a data model for a dialog. It contains a sequence of
messages between entities (including system, user, assistant, and
function).
When invoked, these messages are passed to a CompletionFunc which is
ultimately a call to an LLM provider such as OpenAI. But through the use of
Middleware can also provide significant flexibility and control.
The design for Agent is inspired by net/http.
Most features are implemented as middleware around the underlying provider.
type CompletionFunc func(context.Context, []*Message, []ToolDef) (*Message, error)All providers need to implement this signature to issue completion requests to the underlying LLM.
"Middleware" is implemented by wrapping the provider in layers, like an onion.
The provider is the interface for the underlying LLM. It is responsible for sending the available messages and functions to the LLM. The provided example providers also make use of the middleware pattern for implementing providing specific functionality.
For example, logging for the openaichat provider is configured like:
slog.SetDefault(slog.New(&slogor.Handler{
Mutex: new(sync.Mutex),
Level: slog.LevelDebug,
TimeFormat: time.Stamp,
}))
p := openaichat.New("my-api-key", "gpt-4",
openaichat.WithMiddleware(openaichat.Logger(slog.Default())),
)While the implementation itself composes like an onion:
func Logger(l *slog.Logger) MiddlewareFunc {
return func(ctx context.Context, params openai.ChatCompletionNewParams, next CreateCompletionFn) (
*openai.ChatCompletion, error) {
st := time.Now()
resp, err := next(ctx, params)
if err != nil {
l.LogAttrs(ctx, slog.LevelError, "failed executing completion", slog.String("error", err.Error()))
return resp, err
}
l.LogAttrs(ctx, slog.LevelDebug, "executed completion",
slog.Duration("elapsed", time.Since(st)),
slog.Int("prompt_tokens", int(resp.Usage.PromptTokens)),
slog.Int("completion_tokens", int(resp.Usage.CompletionTokens)),
slog.String("finish_reason", string(resp.Choices[0].FinishReason)),
)
return resp, err
}
}Filters allow for breaking the 1-1 relationship between Agent messages and what is sent to the underlying provider.
LLMs have limited Context windows, so this provides helpful functionality for having greater control over the context without throwing away potentially valuable information.
A simple example of a filter might limit how many messages to send:
func LimitMessagesFilter(max int) FilterFunc {
return func(ctx context.Context, msgs []*Message) ([]*Message, error) {
if len(msgs) > max {
return msgs[len(msgs)-max:], nil
}
return msgs, nil
}
}Filters are configured as options when creating the agent:
a := agent.New(c, agent.WithFilter(LimitMessagesFilter(5)))Checks are the response side of filters: it's a convenient way to intercept responses from the provider chain.
func hasSecret(ctx context.Context, msg *agent.Message) error {
c, _ := msg.Content(ctx)
if strings.Contains(c, "secret") {
return errors.New("has a secret")
}
return nil
}Checks are configured as options when creating the agent:
a := agent.New(c, agent.WithCheck(hasSecret))A core capability for building an Agent is providing tools. Tools allow the agent the perform actions autonomously.
func hello() (string, error) {
return "Hello World", nil
}
// Empty parameters
params := map[string]any{
"type": "object",
"properties": map[string]any{},
}
ts := tools.New()
ts.Add("hello", "receive a welcome message", params, hello)
a := agent.New(c, tools.WithTools(ts))Behind the scenes, WithTools middleware will intercept tool invocations and
run the provided function as an agent step.
Tool middleware provides a way to add cross-cutting functionality around tool execution, such as logging, timing, error handling, or metrics collection. This follows the same onion-style middleware pattern as the rest of the framework.
// Create tools with middleware using the options pattern
ts := tools.New(
tools.WithMiddleware(tools.ToolLogger(slog.Default())),
)
ts.Add("hello", "receive a welcome message", params, hello)
a := agent.New(c, tools.WithTools(ts))The ToolLogger middleware is provided for logging tool calls with timing information:
import "log/slog"
ts := tools.New(
tools.WithMiddleware(tools.ToolLogger(slog.Default())),
)You can create custom middleware to add any functionality around tool execution:
// Custom middleware that adds timing attributes
timingMiddleware := func(next tools.ToolInvokeFunc) tools.ToolInvokeFunc {
return func(ctx context.Context, call *agent.ToolCall) (*agent.Message, error) {
start := time.Now()
msg, err := next(ctx, call)
if msg != nil {
msg.SetAttr("elapsed", time.Since(start).String())
}
return msg, err
}
}
ts := tools.New(tools.WithMiddleware(timingMiddleware))Middleware composes in onion layers - the first registered middleware wraps the outermost layer. Multiple middleware can be added:
ts := tools.New(
tools.WithMiddleware(loggingMiddleware),
tools.WithMiddleware(metricsMiddleware),
tools.WithMiddleware(tracingMiddleware),
)Tool middleware works transparently with both Tool and AttributesTool types.
The library supports real-time streaming of responses from the LLM. This allows you to receive and process content as it's generated, rather than waiting for the complete response.
// Define a callback function to handle streaming deltas
printDeltas := func(ctx context.Context, delta openaichat.MessageDelta) {
if delta.Content != "" {
fmt.Printf("Content: %s", delta.Content)
}
if delta.ToolCallID != "" {
fmt.Printf("Tool Call: %s(%s)\n", delta.ToolCallName, delta.ToolCallArguments)
}
}
// Configure the provider with streaming
p := openaichat.New(apiKey, "gpt-5-mini-2025-08-07",
openaichat.WithMessageDeltaFunc(printDeltas))
a := agent.New(p)
a.Add(agent.RoleSystem, "You are a helpful assistant.")
a.Add(agent.RoleUser, "Tell me a story.")
// The deltaFunc will be called for each chunk as it arrives
r, err := a.Step(context.Background())The MessageDelta contains:
Content: Incremental text contentToolCallID,ToolCallName,ToolCallArguments: Tool call information as it streams
Streaming works transparently with all middleware - the final response is still a complete Message object that your application logic can use normally.
The library includes support for OpenAI's Responses API, which provides access to the model's reasoning process. This allows you to see how the model thinks through problems step-by-step.
// Create a Responses API provider with reasoning enabled
p := openairesponses.New(apiKey, "gpt-4o-2024-08-06",
openairesponses.WithReasoningEffort("medium"), // Set reasoning effort level
openairesponses.WithReasoningSummary("concise"), // Get concise reasoning summaries
)
a := agent.New(p)
a.Add(agent.RoleSystem, "You are a helpful assistant that shows your reasoning.")
a.Add(agent.RoleUser, "Explain why the sky is blue.")
resp, err := a.Step(context.Background())
if err != nil {
log.Fatalf("error: %v", err)
}
// Access the response content
content, _ := resp.Content(context.Background())
fmt.Println("Response:", content)
// Access the reasoning if available
if resp.ReasoningContent != "" {
fmt.Println("Reasoning:", resp.ReasoningContent)
}
if resp.ReasoningEncryptedContent != "" {
fmt.Println("Encrypted reasoning available (length:", len(resp.ReasoningEncryptedContent), ")")
}
for i, summary := range resp.ReasoningSummaries {
fmt.Printf("Summary %d: %s\n", i+1, summary)
}WithReasoningEffort(effort): Set reasoning effort level. Accepted values:"minimal","low","medium","high". Reducing effort results in faster responses and fewer tokens used on reasoning.WithReasoningSummary(summary): Set reasoning summary level. Accepted values:"auto","concise","detailed". Provides a summary of the reasoning performed by the model.
The Message struct includes optional reasoning fields:
type Message struct {
// ... other fields ...
// Reasoning support (optional, primarily for assistant messages)
ReasoningContent string // Plain reasoning text
ReasoningEncryptedContent string // Encrypted reasoning blob
ReasoningSummaries []string // Human-readable summaries
}Reasoning is preserved through message copying and middleware processing, allowing you to build complex workflows while maintaining access to the model's thought process.
Note: The Responses API provider is fully functional and uses OpenAI's official Go SDK v2 with complete Responses API support including streaming.
See example
Messages can include image data:
imgData, _ := os.ReadFile("camera.jpg")
m := agent.NewImageMessage(agent.RoleUser, "please explain", "camera.jpg", imgData)
a.AddMessage(m)See example
An Agent Set allows an LLM to start a dialog with another LLM. It exposes two new tools for your primary agent to call:
agent_start- starts an agentagent_stop- stops an agent
as := agentset.New()
as.Add("eyes", EyesAgentStartFunc())
ts := tools.New()
ts.AddTools(as.Tools())
a := agent.New(c, agentset.WithAgentSet(as))This is an example of advanced control flow that is supported by the design of Agent. The implementation of AgentSet required no modifications to Agent core.
Each message may contain a set of attributes that are not directly used by the LLM, but can provide important contextual clues for other parts of the application.
m := agent.NewContentMessage(agent.RoleUser, "content...")
m.SetAttr("summary", getSummary())A special case of attributes with no content are Tags:
m := agent.NewContentMessage(agent.RoleUser, "content...")
m.Tag("important")This works well with filters.
The API for retrieving the content of a message is designed to support more than simply returning a string.
content, err := m.Content(context.Background())
if err != nil {
fmt.Fprintf(os.Stderr, "error getting message content: %v\n", err)
os.Exit(1)
}Messages support running functions to dynamically generate content.
func generateSystem(ctx context.Context) (string, error) {
currentDate := time.Now().Format("2006-01-02")
content := fmt.Sprintf("System Message - Date: %s", currentDate)
return content, nil
}
msg := agent.NewDynamicMessage(agent.RoleSystem, generateSystem)
a.AddMessage(msg)Agents are designed to operate in "steps" by calling:
msg, err := a.Step(context.Background())A step is generally a single message sent to the LLM and the response returned. Middleware such as Tools can intercept those steps and run other actions instead.
For prototypes, tools or other simple use cases, it may be helpful to run Step automatically until some end condition is detected. There are some built-in helpers to support this.
RunUntil provides some sugar for running steps until either the context is
canceled or a provided function returns true.
count := 0
func runTwice(ctx context.Context) bool {
count++
return count == 2
}
err := agent.RunUntil(context.Background(), a, runTwice)One common usecase is to run an Agent until the Assistant responds to the user
(rather than run tools). This can be easily configured with the StopOnReply
check function.
a := agent.New(c, agent.WithCheck(agent.StopOnReply))
err := Run(context.Background(), a)These patterns may not be appropriate for production applications where full control over the stopping of an agent is likely desired. These patterns should be adapted as needed.