Agent becomes unresponsive during interview and final report is INCOMPLETE

[xuntla]

Description

During the interview, Jim occasionally took an unusually long time to respond. Near the end of the interview, Jim asked a follow-up question, but after I tried to clarify the question, he stopped responding for more than three minutes.

I eventually had to manually end the interview. The final report was marked as INCOMPLETE, and no scores were generated.

Steps to Reproduce

1. Launch the prebuilt Windows version of CodeTrial.
2. Open http://127.0.0.1:3000/ in Google Chrome.
3. Start a coding interview for Gift Card Bundle Pair.
4. Complete the REACTO process and ask Jim for clarification about a follow-up question.
5. Wait when Jim stops responding, then manually end the interview.
6. Open the interview report.

Expected Behavior

Jim should respond within a reasonable amount of time and allow the interview to continue normally.

After the interview ends, CodeTrial should generate a complete report with evaluation scores.

Actual Behavior

Jim stopped responding during the interview. There were also several unusually long response delays earlier in the session.

The logs showed Gemini-related failures and a transport restart:

interim review skipped: HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)

Gemini requested a transport restart in 50s; room=interview-5v1zf3xj
Gemini session resumed; the interview continues where it left off

After the restart, response latency increased significantly:

timing: 21.35s from the candidate finishing to the reply starting
timing: 38.90s from the candidate finishing to the reply starting
timing: 9.34s from the candidate finishing to the reply starting

Jim eventually stopped responding completely, so I manually ended the interview.

After the interview ended, report generation failed repeatedly with 503 Service Unavailable:

gemini report transport_failed call=1 backoff_s=1 error=HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)
gemini report transport_failed call=2 backoff_s=1 error=HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)
gemini report transport_failed call=3 backoff_s=1 error=HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)
gemini report transport_failed call=4 backoff_s=1 error=HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)

codetrial report_incomplete reason=Rust LiveKit runner ended (face_missing_severe) but Gemini report generation failed for model gemini-3.1-flash-lite on two-sum-ii-input-array-is-sorted. Final editor state: 607 bytes of cpp. 
Error: HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent)

The final report was marked as INCOMPLETE, with no scores generated:

The automatic evaluation could not be completed: Rust LiveKit runner ended (face_missing_severe) 
but Gemini report generation failed for model gemini-3.1-flash-lite on two-sum-ii-input-array-is-sorted. 
Final editor state: 607 bytes of cpp. Error: HTTP status server error (503 Service Unavailable) for url (https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent). 
Your session ran end-to-end, but no scores were produced, so nothing here is an assessment of your work. 
Check the agent logs and GOOGLE_API_KEY, then try again.

Environment

• OS: Windows 10
• Browser: Google Chrome 153.0.8010.53
• Access method: http://127.0.0.1:3000/
• Git revision: 2c9e151

[sidney-Hung]

Hi! I also got this issue.

I use curl to test gemini api.

$ curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-flash-lite:generateContent?key={$API_KEY}" \
-H 'Content-Type: application/json' \
-d '{"contents": [{"parts":[{"text": "ping"}]}]}'

{
  "error": {
    "code": 503,
    "message": "This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.",
    "status": "UNAVAILABLE"
  }
}

Looks like it curse by the gemini api.

When I check gemini api dashboard the Success Rate almost 0%

[jserv]

Provide an initial analysis and a list of follow-up actions. Approach the task as a CS student capable of identifying and overcoming engineering challenges, rather than as an end user.

[itisJoshuaTseng]

I encountered a similar interviewer-silence issue on a newer CodeTrial build,
but with one notable difference: report generation was still successful in my
case.

Environment

• OS: macOS
• Browser: Google Chrome
• Binary: codetrial-aarch64-apple-darwin
• Git revision: 4874d8d
• Access method: http://127.0.0.1:3000/
• Interview duration: 30 minutes

Observed behavior

I observed this behavior in at least two separate interview sessions.

In one session, after I completed the implementation and all 7 test cases
passed, I finished my explanation and explicitly said:

"Um yes. I'm done."

Jim did not provide a follow-up response.

However, the UI still showed:

• Listening
• Live-synced to Jim

I also observed another session where the response was significantly delayed
and the eventual response appeared to lose the immediate conversational
context.

Unlike the original report in this issue, at least one of my affected sessions
still generated a complete scored report successfully.

Initial analysis

This suggests that the interviewer becoming unresponsive may not necessarily
be caused by the same failure as the final-report 503 errors.

There may be at least two separate failure paths:

1. the live Gemini/interviewer turn becomes delayed or stalled during the
   interview; and
2. the Gemini Flash request used for final report generation fails separately.

Since the UI continued to show Listening and Live-synced to Jim, the client
also did not clearly indicate whether the interviewer was waiting normally,
processing a response, or recovering from a transport/model failure.

I do not yet have enough logs to determine whether the stalled live response
was caused by Gemini Live, LiveKit transport, or CodeTrial's conversation state.

Follow-up actions

I plan to:

1. Reproduce the issue again using the latest revision.
2. Record the exact interview timestamp when the candidate finishes speaking
   and when Jim begins responding.
3. Capture the corresponding terminal logs, especially any Gemini restart,
   timeout, 503, or reconnect messages.
4. Check whether the report succeeds or fails independently of the live
   interviewer stall.
5. Compare the behavior before and after any Gemini transport restart.

One possible UX improvement would be to distinguish states such as
Listening, Waiting for interviewer, and Reconnecting, so a stalled
interviewer response is visible instead of appearing as a normal listening
state.

[jserv]

This thread now carries two problems that have separate fixes, and it is worth naming which one lives here.

The interviewer going silent mid-session is this issue, and #119 was already closed against it. The report coming back INCOMPLETE after a Gemini 503 is #97, which has three independent reproductions including a deterministic one on 4874d8d, and asks for something this thread does not: retrying a 5xx and regenerating a report from a session that already completed. Commit 2287c45 improved the message a candidate sees when a report is lost and made the retries back off exponentially, but it does not save a completed session for a later attempt, so #97 is still open work.

Please keep 503-during-report-generation reports on #97 and silence-during-the-interview reports here. #64 may be this same silence and is waiting on a log to confirm it.

[jserv]

Carrying over two observations from #64, which is being closed as a duplicate of this issue.

On revision d591c18, @RWong04 reports that after the interviewer went silent, ending the session and starting a new one produced a session that also opened without the greeting, and that normal behavior returned after about 15 minutes with no local change. Recordings of a silent session and of the language-switch workaround are in that thread.

Both fit a provider-side window rather than per-session state, which is consistent with the 503 evidence already here.

[jserv]

See if PR #202 resolves this issue.