Skip to content

Handling non-OpenAI APIs #490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
annie444 wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Handling non-OpenAI APIs #490

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
aef49c3
Non-OpenAI APIs that implement a simmilar interface don't always incl…
annie444 Nov 21, 2025
2e58310
The LM Studio API doesn't always output token information
annie444 Nov 21, 2025
9914679
Merge branch 'main' into main
annie444 Nov 25, 2025
fc08fd4
Move the ResponseUsage struct to match upstream
annie444 Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions async-openai/src/types/responses/response.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1440,6 +1440,7 @@ pub struct ResponseLogProb {
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
pub struct OutputTextContent {
/// The annotations of the text output.
#[serde(default)]
pub annotations: Vec<Annotation>,
pub logprobs: Option<Vec<LogProb>>,
/// The text output from the model.
Expand Down
7 changes: 4 additions & 3 deletions async-openai/src/types/shared/response_usage.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,21 @@
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
pub struct InputTokenDetails {
/// The number of tokens that were retrieved from the cache.
/// [More on prompt caching](https://platform.openai.com/docs/guides/prompt-caching).
pub cached_tokens: u32,
}

#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
pub struct OutputTokenDetails {
/// The number of reasoning tokens.
pub reasoning_tokens: u32,
}

/// Usage statistics for a response.
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq)]
#[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Default)]
#[serde(default)]
pub struct ResponseUsage {
/// The number of input tokens.
pub input_tokens: u32,
Expand Down