Skip to content

feat(dgw): analyze TRP recordings with AI - #2040

Closed
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 8 commits into
feat/recording-ai-analysisfrom
feat/trp-decode-complete
Closed

feat(dgw): analyze TRP recordings with AI#2040
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 8 commits into
feat/recording-ai-analysisfrom
feat/trp-decode-complete

Conversation

@irvingoujAtDevolution

@irvingoujAtDevolution irvingouj@Devolutions (irvingoujAtDevolution) commented Oct 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Adds TRP recordings to AI analysis (the PR below only reads .cast and fails a .trp session with "unsupported recording type: trp").

  • terminal-streamer: a finished .trp recording can now be decoded as a stream of asciicast lines (trp_decoder::AsciicastLines), one packet at a time. It shares the packet decoding with the live stream decoder, which would otherwise wait forever at the end of a finished file.
  • The AI crate's recording analysis feeds .trp sessions through it, so memory stays bounded by the chunk size, same as .cast.

Kept on top of the stack on purpose: it's an extra format, not the core of the feature.

Tested: terminal-streamer 6/6 (incl. "complete decoder = live decoder" on the same bytes, truncated tail, malformed input), devolutions-gateway-ai 81 + 27 (a .trp session decoded, a large one streamed into bounded chunks), devolutions-gateway lib 211, tests/ai_analysis.rs 8/8.

🤖 Generated with Claude Code

@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) removed this pull request from stack #2045 October 6, 2026 21:26
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) changed the base branch from feat/gateway-db to feat/recording-ai-analysis October 6, 2026 21:27
@irvingoujAtDevolution irvingouj@Devolutions (irvingoujAtDevolution) changed the title feat(terminal-streamer): decode a complete TRP recording feat(dgw): analyze TRP recordings with AI Oct 6, 2026
This was referenced Oct 6, 2026
Open
Open
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) force-pushed the feat/trp-decode-complete branch 2 times, most recently from 152ae3e to a807ea1 Compare October 7, 2026 17:39
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) removed this pull request from stack #2046 October 7, 2026 20:36
Closed
@irvingoujAtDevolution @claude
feat(dgw): analyze a recording in the AI crate
c2dc1c7 
AiClient::analyze_recording takes the parsed session manifest and its path and does the rest: it reads the terminal recordings as a stream into transcript chunks, asks the AI about each chunk (a truncated answer is asked again in two halves), saves each answer as a checkpoint, and merges them into a Session Recording Log next to the manifest. Its working files live in .ai-analysis/ next to the manifest, so analysing again after a failure only asks about the chunks left; files made with another model or prompt are not reused. Only asciicast recordings are read for now.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
refactor(dgw): give the gateway state the provisioner task store
ed7ee48 
The Gateway state now holds the provisioner task store, on its own
gateway.db connection, so the work it records can reach it. Building
the mock state opens an in-memory database, so `DgwState::mock` is now
async.

The job queue attempt limit is now the named constant `MAX_ATTEMPTS`,
still 5, so a job can refer to it.

No behavior change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
feat(dgw): run AI analysis as a provisioner task
7d7a9bd 
Gateway can now describe what the user did in a recorded terminal
session with an AI provider, and add the result to the session as a
new log. Each request is a Task recorded in gateway.db (kind
`recording.ai-analysis`), run as a job on the existing job queue. The
analysis itself is the AI crate's `analyze_recording`; Gateway gives it
the session manifest once the recording is finished, then adds the log
it returns to the session through the recording manager.

A repeated request with the same Task ID gets its Task back, even
while the session records again. A new request is refused while the
session is recording, or when it has no recording.

The API key and the base URL stay in memory only, until the Task ends
or reaches its deadline. After a restart the job finds no key and
fails its Task. Neither the key nor the URL appears in errors, logs or
stored results: the AI client now drops the URL from its transport
errors.

Rate limits, server and network errors, and attempts running longer
than 30 minutes, are retried by the job queue; the Task fails once its
attempts are used up, or on any other error. A timed-out attempt is
cancelled and awaited, and analyses of one session run one at a time,
so two attempts never share the working files the AI crate keeps next
to the manifest. Those files are removed when the Task ends, or when a
job finds its Task ended.

Starting an AI analysis is an internal function for now; the HTTP
endpoint comes next.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
feat(dgw): add endpoints to start an AI analysis and read a task
e4a5e19 
The provisioner can now start an AI analysis of a recorded session with
`POST /jet/jrec/{session_id}/ai-analysis` (scope
`gateway.recording.analyze`), and follow any Task with
`GET /jet/tasks/{id}` (scope `gateway.tasks.read`).

The request body holds the Task ID chosen by the caller, the AI provider
and model, an optional base URL and output token limit, and the API key.
A new Task answers 202, the same request again answers 200 with the same
Task. Gateway answers 409 with a code when the Task ID is used for other
parameters, when the session still records, or when another AI analysis
of the session is not finished; the last one names the Task ID. A
session without recording answers 404, invalid settings or an unknown
field answer 400.

The API key and the base URL stay in memory only: they are never logged,
returned or written to gateway.db.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
feat(dotnet-utils): add the AI analysis and task read scopes
d400975 
DVLS can now sign scope tokens for `gateway.recording.analyze`, to start
an AI analysis of a recorded session, and `gateway.tasks.read`, to read
a Task.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
chore(openapi): regenerate clients for AI analysis
6083262 
Adds `POST /jet/jrec/{session_id}/ai-analysis`, `GET /jet/tasks/{id}`,
their schemas and the `gateway.recording.analyze` and
`gateway.tasks.read` scopes to gateway-api.yaml, and regenerates the
dotnet and TypeScript clients and the docs. Only the AI analysis output
is kept; the existing version and content-type drift of the committed
clients is left as is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
feat(terminal-streamer): decode a complete TRP recording
8373a0b 
A finished recording can now be read as asciicast lines without
tailing it. The lines come out one packet at a time, so memory use
does not grow with the recording. The packet decoding is shared with
the live stream decoder.

Events written before the terminal setup are still held until the
setup arrives, but no more than 1 MiB of them: past that, the header
goes out with the terminal size known so far, so a recording that
never sends its setup is not held whole in memory.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution @claude
feat(dgw): analyze TRP recordings with AI
09841cf 
AI analysis only read asciicast recordings so far and refused TRP ones
as unsupported. The AI crate now reads TRP sessions through the
streaming TRP decoder, so their memory use stays bounded by the chunk
size like asciicast.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) removed this pull request from stack #2049 October 7, 2026 22:19
@irvingoujAtDevolution
irvingouj@Devolutions (irvingoujAtDevolution) removed this pull request from stack #2051 October 7, 2026 22:20
@irvingoujAtDevolution

irvingouj@Devolutions (irvingoujAtDevolution) commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #2043: the TRP decoder (feat(terminal-streamer): decode a complete TRP recording) and TRP support in the AI crate are both there now, so nothing is left here. Closing; the stack is #2052: #2039 -> #2041 -> #2043 -> #2044.

Note

LLM-assisted content (no human feedback).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@irvingoujAtDevolution