Skip to content

Let the candidate hide the worked examples - #174

Merged
jserv merged 3 commits into
sysprog21:mainfrom
ascodeasice:hide-worked-examples
Sep 30, 2026
Merged

jserv merged 3 commits into
sysprog21:mainfrom
ascodeasice:hide-worked-examples

Conversation

@ascodeasice

@ascodeasice ascodeasice commented Sep 29, 2026 •

Copy link
Copy Markdown

What this does

Adds a "Hide the worked examples" checkbox to the preflight. The REACTO Example step asks the candidate for an ordinary and a boundary case, but the Problem tab already shows one or two examples, so the step can be passed by reading them aloud. When the box is ticked, the Problem tab shows the brief without the example blocks, so the candidate has to come up with the cases. The choice is remembered between visits, the same way as the Meet presentation checkbox. Without it, the page is unchanged.

Also fixes: the preflight could not scroll

The preflight card could not scroll when it was taller than the window, so on a laptop screen its top and the Start interview button were cut off with no way to reach them. This was already the case on main; the new checkbox makes the card taller and the problem easier to hit. The overlay now scrolls and keeps its top in view; a card that fits stays centered as before. This is a separate commit.

Not included

Jim is still told the examples are on screen, and "Add a case" does not yet show its input format when they are hidden. The first changes the live prompt, which needs a new prompt contract version, so it is left for a follow-up.

Validation

scripts/test.sh passes locally with Playwright Chromium installed (713 browser tests, none skipped); cargo-audit, ruff and shellcheck were not installed. A new browser test drives the real page: tick, reload, untick, and blocked storage. The scroll fix was checked by hand at 545 px and 1091 px window heights.

Closes #173
Closes #178


Summary by cubic

Adds a "Hide the worked examples" checkbox to the preflight so the Problem tab shows the brief without the worked cases, making the REACTO Example step require candidates to propose their own cases rather than read what's on screen. The choice is remembered between visits, like the Meet presentation checkbox, and also clears the "Add a case" input placeholder, which is a worked case in its own right. The topology replica brief no longer points at the example blocks.

The hide choice now reaches the interviewer: the page sends hideExamples with the token request, the server carries it through the participant metadata into the live prompt, which says no examples are on screen and offers a small example only after the candidate has tried or is stuck. A session that never ticks the box mints the same metadata and gets the live prompt unchanged. The live prompt moves to version 15 and the bundle to 23.

Bug Fixes

  • The preflight overlay now scrolls when the card is taller than the window, with a fallback centering for browsers without safe center support, so the top and the Start button are reachable on laptop screens.

Written for commit 2fdbadd. Summary will update on new commits.

Review in cubic

Comment thread web/styles.css
Comment thread web/render.js
Comment thread web/interview.js
Comment thread web/interview.html
@jserv

jserv commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

@ascodeasice Your body ends with Refs #173, which closes nothing on merge, and the separate scroll commit resolves #178 without naming it at all, so as it stands both issues would need closing by hand; please end the body with Closes #173 and Closes #178, and if acceptance criterion 4 in #173 is not yet met ("Add a case" still showing the input format once the examples are hidden) say so there and keep Refs #173 instead.

@ascodeasice

Copy link
Copy Markdown
Author

I've pushed three commits according to the review

  • 7631f12 declares plain center before safe center, so a browser without safe keeps the old centering.
  • 74c5a8a rewords the topology replica brief so it states the mesh format itself instead of pointing at the examples. I checked every brief in the bank; no other one refers to the example blocks.
  • e8c0878 hides the "Add a case" placeholder along with the examples, since it was the judge's first input and so a worked case of its own.

The body now ends with Closes #178 and Refs #173, because criteria 3 and 4 in #173 are not met yet. How I plan to meet them:

Criterion 3, Jim not told the examples are on screen

The page sends the choice to the server with the other interview options. When the examples are hidden, Jim's instructions say so instead of saying the screen shows one or two worked examples.

A few hints and clarifications also point at the examples, such as "Look at the air cells on the right edge in the example". I would reword those so they make sense either way, rather than keep a second version for hidden examples.

This changes the live prompt, so it comes with a prompt contract version bump. Would you rather have this in this PR, or as a follow-up once this one is in?

Criterion 4, the input format in "Add a case"

I see two ways to go, and would like your view on which:

  • Drop the criterion from Let candidates propose their own examples instead of reading existing ones #173. A candidate who hides the examples has chosen to work the way a live interview runs, where the input format is something you ask about. The starter code already shows the signature, and "Add a case" names the expected type when an input does not match. The weak spot is node inputs such as a graph or a linked list, whose JSON form the starter code does not show.
  • Show the format without a case. When the examples are hidden, the placeholder shows the parameters and their types from the judge, such as [nums: integer[], target: integer], with one line for each node shape on how it is written. No value from any case appears.

If you prefer the first, I will edit #173 to match.

@jserv

jserv commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

I see two ways to go, and would like your view on which:

  • Drop the criterion from Let candidates propose their own examples instead of reading existing ones #173. A candidate who hides the examples has chosen to work the way a live interview runs, where the input format is something you ask about. The starter code already shows the signature, and "Add a case" names the expected type when an input does not match. The weak spot is node inputs such as a graph or a linked list, whose JSON form the starter code does not show.
  • Show the format without a case. When the examples are hidden, the placeholder shows the parameters and their types from the judge, such as [nums: integer[], target: integer], with one line for each node shape on how it is written. No value from any case appears.

If you prefer the first, I will edit #173 to match.

Go ahead, minimizing scope for an issue and making collaboration easier.

@ascodeasice

Copy link
Copy Markdown
Author

Pushed 9eec154 for criterion 3 and removed criterion 4 from #173.

When the candidate ticks the box, the page sends hideExamples: true with its /api/token request. The server copies it into the candidate's LiveKit participant metadata, and the agent reads it there when it builds Jim's prompt. With the flag set, the prompt no longer says the screen shows one or two worked examples. It says no examples are on the candidate's screen, and tells Jim to ask the candidate for an ordinary case and a boundary case before giving an example of his own. A candidate who hides the examples wants to practice coming up with cases, so Jim should not give them one first. This follows the Example step, which already tells Jim not to choose the examples for the candidate; with the flag set, he gives one only after they have tried or are stuck. Without the flag, the prompt is unchanged.

Earlier I suggested rewording the hints and clarifications in the problem bank. I did not, because #173 keeps the bank out of scope. Instead, with the flag set, the prompt also tells Jim that when a hint or clarification mentions an example, he should use a case the candidate proposed or a small case of his own, and never point the candidate at the screen.

I tried both modes in live sessions. With the examples shown, I asked Jim for an example, and he told me the examples were on my screen. With them hidden, he said there were none on my screen and asked me to propose my own cases first.

With the examples hidden, I also tried a hint and a clarification that mention an example, on two problems:

  • sealed-air-pocket-backfill: the first hint reads "Look at the air cells on the right edge in the example". Jim said "If you look at an air cell on the very edge of the map..." and did not point me at the screen.
  • terrain-mask-tiling, clarification: I asked "How is the result shown in the examples?", the same question as one of its stored clarifications. Jim explained the format without pointing me at the screen.

Jim never sees the problem's examples in either mode, because they are not in the prompt. That is why he guessed at them in #182. When the fix for #182 puts them into the prompt, it can check hideExamples and leave them out for a candidate who hid them.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase the current branch onto the upstream default branch and rework the series into functionally minimal commits, folding similar ones and enforcing the project's commit message rules.

@ascodeasice

Copy link
Copy Markdown
Author

Rebased onto main and folded the commits into three by function. I also rewrote the messages to say what changed and why, without the how, as the rules ask:

  • 7333b5e makes the preflight card scroll when it is taller than the window
  • b86390a lets the candidate hide the worked examples and the "Add a case" placeholder with a preflight checkbox, and rewords the one problem brief that pointed at the examples.
  • 9cb056d tells Jim, when hideExamples is set, not to point the candidate at an example and to ask for their own cases first. Keep misrecognized speech out of the assessment #187 took bundle 21 while this was open, so this one is bundle 22 with live prompt 14.

Also, I've updated my PR description from Refs #173 into Closes #173 since all the criteria are met now

@ColtenOuO ColtenOuO left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the CI error report and resolve the issue.

@jserv
jserv requested a review from ColtenOuO September 30, 2026 19:53
The preflight card is taller than a laptop window once every step is
shown, and the overlay pushed it past both edges, where neither the
wheel nor the keyboard could reach the Start button. The overlay now
scrolls and keeps the card's top in view, and a card that fits stays
centered as before, including in a browser that does not understand
safe centering.
The Example step asks the candidate for an ordinary and a boundary case
while the Problem tab already shows one or two worked ones, so the step
can be passed by reading them aloud. A preflight checkbox, stored like
the Meet presentation choice, now leaves them off the page, along with
the Add a case placeholder, which is the judge's first input and so a
worked case of its own. The one brief that pointed at the examples, the
topology replica's, now states the mesh format itself.

@ColtenOuO ColtenOuO left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase and resolve the conflict.

Hiding the examples changed only the page, so the live prompt still
told Jim the screen showed one or two worked examples, and a hint or
clarification that mentions an example sent the candidate looking for
one. The choice now reaches the prompt, which says none are on screen,
has Jim put any example a hint or clarification mentions as a case of
the candidate's or his own, and asks for the candidate's cases before
he offers one. A session that does not hide them gets the prompt
unchanged, and the problem bank stays out of scope as sysprog21#173 sets it. The
live prompt moves to 15 and the bundle to 23.

@ColtenOuO ColtenOuO left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Great work!

@ColtenOuO
ColtenOuO requested a review from jserv September 30, 2026 21:13
@jserv
jserv merged commit 417b2c2 into sysprog21:main Sep 30, 2026
6 checks passed
@jserv

jserv commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Thank @ascodeasice for contributing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot click "Start interview" button when zoomed in (modal unscrollable) Let candidates propose their own examples instead of reading existing ones

3 participants