Skip to content

Surface Responses output_text logprobs on the text content - #1105

Open
PratikDhanave (PratikDhanave) wants to merge 4 commits into
microsoft:mainfrom
PratikDhanaveFork:fix/openai-responses-output-text-logprobs
Open

PratikDhanave (PratikDhanave) wants to merge 4 commits into
microsoft:mainfrom
PratikDhanaveFork:fix/openai-responses-output-text-logprobs

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

The Responses path built a TextContent from an output_text part (non-streaming responsesProcessResponse, and the streaming output_item.done handler) but never read output_text.logprobs. So a caller that requested logprobs via raw params (Include: ["message.output_text.logprobs"], TopLogprobs) never received them.

The Python client preserves them on the text content's additional_properties["logprobs"] on both parse paths.

Change

  • Non-streaming: populate the TextContent AdditionalProperties with the output_text logprobs when present.
  • Streaming: the output_item.done handler previously emitted a text content only when annotations were present (the text itself already arrived via deltas); it now also emits when logprobs are present and carries them, mirroring the annotations handling. Key uses the PascalCase convention already used elsewhere in the provider.

Test

  • TestResponsesOutputTextLogprobsSurfaced_NonStreaming: a canned response whose output_text carries a logprobs array now surfaces Logprobs on the TextContent AdditionalProperties. Fails before the change (nil), passes after.

The Responses path built a TextContent from an output_text part but never
read its logprobs, so a caller requesting logprobs (Include:
message.output_text.logprobs) never received them. The Python client stores
them on the text content's additional_properties.

Populate the TextContent's AdditionalProperties with the output_text
logprobs when present, on both the non-streaming path and the streaming
output_item.done path (which now emits an annotations/logprobs-only content
the same way it already does for annotations).
Copilot AI lite review requested due to automatic review settings September 18, 2026 15:41
@github-actions github-actions Bot added area:provider Changes files in the provider area area:provider/openai Changes files in the provider / openai area size:medium At most 100 changed lines across at most 5 files labels Sep 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Streaming logprobs can be lost during response collection, and the new streaming path lacks regression coverage.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates Responses API handling to expose output_text logprobs for non-streaming and streaming responses.

Changes:

  • Propagates logprobs to TextContent.AdditionalProperties.
  • Emits streaming metadata for logprobs.
  • Adds non-streaming regression coverage.
File summaries
File Summary
provider/openaiprovider/responses.go Propagates output text logprobs.
provider/openaiprovider/responses_test.go Tests non-streaming logprobs exposure.
Review details

Suppressed comments (1)

provider/openaiprovider/responses.go:1602

  • When logprobs are present without annotations, this creates an empty TextContent immediately after the delta text. ResponseStream.Collect() then coalesces adjacent text contents and copies AdditionalProperties only from the first item (message/content.go:1175-1179), so the returned collected response loses these logprobs. Preserve the metadata through coalescing (for example, merge the content metadata) or otherwise make the logprobs update survive Collect(); add a streaming regression test for the logprobs-only case.
						populateAnnotations(outputText.Annotations, annotatedContent)
						if len(outputText.Logprobs) > 0 {
							annotatedContent.AdditionalProperties = map[string]any{"Logprobs": outputText.Logprobs}
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +1590 to +1602
if c, ok := c.AsAny().(responses.ResponseOutputText); ok && (len(c.Annotations) > 0 || len(c.Logprobs) > 0) {
hasMetadata = true
break
}
}

if hasAnnotations {
if hasMetadata {
annotatedContent := &message.TextContent{}
for _, c := range item.Content {
if outputText, ok := c.AsAny().(responses.ResponseOutputText); ok {
populateAnnotations(outputText.Annotations, annotatedContent)
if len(outputText.Logprobs) > 0 {
annotatedContent.AdditionalProperties = map[string]any{"Logprobs": outputText.Logprobs}
@github-actions github-actions Bot added kind:code Changes production behavior or code kind:tests Changes tests, fixtures, or test infrastructure labels Sep 18, 2026
@github-actions

This comment has been minimized.

Add a streaming test asserting the output_item.done path surfaces
output_text logprobs on the emitted text content, not just the non-streaming
path.
@github-actions github-actions Bot added size:large At most 300 changed lines across at most 10 files kind:dependencies Changes dependencies or manifests and removed size:medium At most 100 changed lines across at most 5 files labels Sep 21, 2026
@github-actions

This comment has been minimized.

…s-output-text-logprobs

# Conflicts:
#	go.mod
#	go.sum
@github-actions github-actions Bot added size:medium At most 100 changed lines across at most 5 files public-api-change Pull Request changes public APIs and removed size:large At most 300 changed lines across at most 10 files kind:dependencies Changes dependencies or manifests labels Oct 3, 2026
@github-actions

This comment has been minimized.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Firewall blocked 4 domains

The following domains were blocked by the firewall during workflow execution:

  • proxy.golang.org
  • raw.githubusercontent.com
  • storage.googleapis.com
  • sum.golang.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"
    - "raw.githubusercontent.com"
    - "storage.googleapis.com"
    - "sum.golang.org"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent for #1105 · copilot · auto · 273.7 AIC · ⌖ 6.27 AIC · ⊞ 13.9K

@github-actions github-actions Bot removed the public-api-change Pull Request changes public APIs label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

API consistency review

Result Scope
unable to determine user-visible behavior

Changed Go contract

provider/openaiprovider/responses.go: Responses output_text logprobs are now surfaced on message.TextContent.AdditionalProperties["Logprobs"], in both non-streaming (responsesProcessResponse) and streaming (output_item.done, which now also emits a text content when logprobs are present). No exported identifiers changed.

Upstream evidence reviewed

No upstream source could be reviewed. GitHub code search was rate-limited (429, retry after about 12 minutes). python/packages/openai/agent_framework_openai/_chat_client.py is about 244 KB, which is too large to fetch in full here. The .NET OpenAI Responses adapter (OpenAIResponsesChatClient.cs) was not checked either.

Assessment

The PR description says Python stores logprobs under additional_properties["logprobs"] on both parse paths. I could not confirm this. I also could not check the key naming (Logprobs against logprobs), whether .NET MEAI surfaces logprobs, or how upstream streaming emits them. Maintainers should check these points manually. No parity finding is reported. The public-api-change label was removed because no exported Go APIs changed.

Generated by Go API Consistency Review Agent · copilot · auto · 20.9 AIC · ⌖ 0.618 AIC · ⊞ 5.1K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider/openai Changes files in the provider / openai area area:provider Changes files in the provider area kind:code Changes production behavior or code kind:tests Changes tests, fixtures, or test infrastructure size:medium At most 100 changed lines across at most 5 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants