Skip to content

feat(search-events): teach the logs query agent regex message search - #1406

Draft
JoshuaKGoldberg wants to merge 8 commits into
joshgoldberg/logs-1013-add-sentry-mcp-evals-for-regex-log-searchfrom
joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search
Draft

feat(search-events): teach the logs query agent regex message search#1406
JoshuaKGoldberg wants to merge 8 commits into
joshgoldberg/logs-1013-add-sentry-mcp-evals-for-regex-log-searchfrom
joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search

Conversation

@JoshuaKGoldberg

@JoshuaKGoldberg JoshuaKGoldberg commented Oct 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Teaches the embedded search agent to write key://pattern// regex log filters when wildcards can't express a request, with logs few-shot examples to match.

See #1405 for evals around this.

Closes LOGS-1014.

@linear-code

linear-code Bot commented Oct 5, 2026

Copy link
Copy Markdown

LOGS-1014

@JoshuaKGoldberg JoshuaKGoldberg changed the title feat(search-events): Teach the logs query agent regex message search feat(search-events): teach the logs query agent regex message search Oct 5, 2026
@JoshuaKGoldberg
JoshuaKGoldberg force-pushed the joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search branch from ee5eae6 to 79f6832 Compare October 5, 2026 15:56
@JoshuaKGoldberg
JoshuaKGoldberg changed the base branch from main to joshgoldberg/logs-1013-add-sentry-mcp-evals-for-regex-log-search October 5, 2026 15:56
@JoshuaKGoldberg
JoshuaKGoldberg added this pull request to stack #1411 October 5, 2026 15:58
@JoshuaKGoldberg
JoshuaKGoldberg force-pushed the joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search branch from f5c77bf to 5bbb4b0 Compare October 5, 2026 18:17
@JoshuaKGoldberg
JoshuaKGoldberg force-pushed the joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search branch from 8693a55 to 37b5f88 Compare October 5, 2026 19:27
@JoshuaKGoldberg
JoshuaKGoldberg force-pushed the joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search branch from 37b5f88 to 7edd98c Compare October 5, 2026 19:38
JoshuaKGoldberg and others added 6 commits October 5, 2026 16:13
@JoshuaKGoldberg @claude
feat(search-events): Teach the logs query agent regex message search
f8b655a 
Add logs text-matching guidance to the embedded query-translation agent
prompt so it uses `key://pattern//` regex filters when wildcards cannot
express the request, and add regex few-shot examples for the logs dataset.

Refs LOGS-1014
Co-Authored-By: Claude Code <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Keep the regex prompt from favoring regex over wi…
5b2c766 
…ldcards

Teach prefix, suffix, and wildcard-list matching so plain alternatives
and anchors don't reach for a regex, keep exact value sets as key:[a,b],
and swap examples that overlapped the regex evals or needed no regex.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Point the logs agent at attributes before message…
49bb616 
… regex

A value with its own attribute, like a response status, should filter
that attribute rather than a regex over message text.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Teach regex for ignoring case and drop the trace-…
229b8c5 
…id example

Wildcards are case sensitive, so ignoring case is a regex trigger; the
trace id few-shot is replaced since trace has its own attribute, and
array attributes are noted as regex-capable.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Say regex works on string attributes only
b07e21f 
Array regex needs the tags[name,array][*] key form, so naming arrays
without it invites a plain key regex that silently matches nothing.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Ask for short regex patterns near the 64-characte…
01ddebc 
…r limit

An unneeded \b or (?i) pushed a UUID pattern past Sentry's limit in an
eval run, so the length rule now says to leave them out unless needed.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg
JoshuaKGoldberg force-pushed the joshgoldberg/logs-1014-teach-sentry-mcps-search-agent-regex-log-search branch from 7edd98c to 01ddebc Compare October 5, 2026 20:13
@JoshuaKGoldberg @claude
fix(search-events): Forbid regex filters outside the logs dataset
a737557 
The agent still wrote key://pattern// for errors in some eval runs, so
the rule now names the other datasets and says to use wildcards there.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JoshuaKGoldberg @claude
fix(search-events): Prefer (?i) over mixed-case classes to fit the re…
0d0b8c9 
…gex limit

An eval run spelled a UUID with [0-9a-fA-F], landing at 75 characters, so
the length rule now says to match either case with (?i) and a lowercase
class, which fits a UUID in 64.

Refs LOGS-1014
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mr-danya pushed a commit to mr-danya/sentry-mcp that referenced this pull request Oct 6, 2026
@betegon
fix(init): require explicit zero workflow exit (#1451)
a43dcda 
## Summary

Make `sentry init` accept a terminal workflow result as successful only
when it explicitly reports `status: "success"` and `result.exitCode: 0`.

Previously, a successful response with no `result` or no `exitCode`
passed the truthiness check and could print the success handoff with
process exit 0. The CLI now treats that as malformed, surfaces a
specific `WizardError`, and exits nonzero. Existing nonzero
workflow-to-CLI exit mappings are unchanged.

This makes the process exit status a reliable black-box signal for the
init smoke harness.

## Compatibility

This intentionally stops accepting terminal success payloads that omit
`exitCode`. The current wizard success output already returns `exitCode:
0`.

The draft [getsentry#1406](getsentry/cli#1406) also
touches `handleFinalResult`; merge order will require rebasing whichever
PR lands second.

## Test plan

- focused wizard runner tests: 74 passed
- CLI typecheck
- Biome on the three changed files
- `git diff --check`
- native `darwin-arm64` binary build
- real Express init against the local candidate API: exit 0
- invalid OpenRouter credential: CLI exit 61 and smoke harness exit 1

## Rollout

The paired API-owned smoke is
[getsentry/cli-init-api#249](getsentry/cli-init-api#249).
Once this lands, bump its pinned CLI SHA so malformed successful
workflow responses cannot produce a false-green smoke result.
mr-danya pushed a commit to mr-danya/sentry-mcp that referenced this pull request Oct 6, 2026
@betegon
fix(init): require explicit zero workflow exit (#1451)
904f1c6 
## Summary

Make `sentry init` accept a terminal workflow result as successful only
when it explicitly reports `status: "success"` and `result.exitCode: 0`.

Previously, a successful response with no `result` or no `exitCode`
passed the truthiness check and could print the success handoff with
process exit 0. The CLI now treats that as malformed, surfaces a
specific `WizardError`, and exits nonzero. Existing nonzero
workflow-to-CLI exit mappings are unchanged.

This makes the process exit status a reliable black-box signal for the
init smoke harness.

## Compatibility

This intentionally stops accepting terminal success payloads that omit
`exitCode`. The current wizard success output already returns `exitCode:
0`.

The draft [getsentry#1406](getsentry/cli#1406) also
touches `handleFinalResult`; merge order will require rebasing whichever
PR lands second.

## Test plan

- focused wizard runner tests: 74 passed
- CLI typecheck
- Biome on the three changed files
- `git diff --check`
- native `darwin-arm64` binary build
- real Express init against the local candidate API: exit 0
- invalid OpenRouter credential: CLI exit 61 and smoke harness exit 1

## Rollout

The paired API-owned smoke is
[getsentry/cli-init-api#249](getsentry/cli-init-api#249).
Once this lands, bump its pinned CLI SHA so malformed successful
workflow responses cannot produce a false-green smoke result.
mr-danya pushed a commit to mr-danya/sentry-mcp that referenced this pull request Oct 6, 2026
@betegon
fix(init): require explicit zero workflow exit (#1451)
e806f7d 
## Summary

Make `sentry init` accept a terminal workflow result as successful only
when it explicitly reports `status: "success"` and `result.exitCode: 0`.

Previously, a successful response with no `result` or no `exitCode`
passed the truthiness check and could print the success handoff with
process exit 0. The CLI now treats that as malformed, surfaces a
specific `WizardError`, and exits nonzero. Existing nonzero
workflow-to-CLI exit mappings are unchanged.

This makes the process exit status a reliable black-box signal for the
init smoke harness.

## Compatibility

This intentionally stops accepting terminal success payloads that omit
`exitCode`. The current wizard success output already returns `exitCode:
0`.

The draft [getsentry#1406](getsentry/cli#1406) also
touches `handleFinalResult`; merge order will require rebasing whichever
PR lands second.

## Test plan

- focused wizard runner tests: 74 passed
- CLI typecheck
- Biome on the three changed files
- `git diff --check`
- native `darwin-arm64` binary build
- real Express init against the local candidate API: exit 0
- invalid OpenRouter credential: CLI exit 61 and smoke harness exit 1

## Rollout

The paired API-owned smoke is
[getsentry/cli-init-api#249](getsentry/cli-init-api#249).
Once this lands, bump its pinned CLI SHA so malformed successful
workflow responses cannot produce a false-green smoke result.

This branch was successfully deployed

1 active deployment
Actions — 0d0b8c93 Deployed Oct 5, 2026 by JoshuaKGoldberg via eval #1229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@JoshuaKGoldberg