feat(speech-to-text): add igc-speech-to-text component and providers

[rkaraivanov]

Description

Add a button component that turns speech into text through a SpeechToTextProvider. The built-in WebSpeechProvider wraps the Web Speech API and expands region-less language tags for Edge. The extras entry point ships WebSocketSpeechToTextProvider, which streams MediaRecorder audio to a WebSocket endpoint over a JSON + binary protocol documented in src/extras/speech-to-text-websocket-protocol.md.

Recognized text arrives through igcInterim and igcResult, sessions end with igcEnd, and errors use Web Speech API compatible codes. Firefox keeps the API behind media.webspeech.recognition.enable, so the component renders disabled there by default.

Type of Change

• New feature (non-breaking change that adds functionality)

Checklist

• My code follows the project's coding standards
• I have tested my changes locally

[Copilot]

🟡 Changes recommended

Critical lifecycle and WebSocket robustness issues, along with moderate API behavior issues, remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds an igc-speech-to-text component with Web Speech and WebSocket providers, transcript events, accessibility support, theming, tests, and documentation.

Changes:

• Adds component lifecycle, events, localization, styling, registration, and exports.
• Adds Web Speech and WebSocket providers with protocol documentation.
• Adds stories, tests, microphone icon, and changelog entry.

File summaries

File	Description
stories/speech-to-text.stories.ts	Adds stories; CustomContent controls for locale, maxAlternatives, and silenceTimeout are currently inert (nit, 2 votes).
src/internals/testing/helpers.spec.ts	Adds a rejection assertion helper.
src/internals/i18n/EN/speech-to-text.resources.ts	Adds English resource strings.
src/internals/definitions/defineAllComponents.ts	Registers the component globally.
src/index.ts	Exports component APIs and types.
src/extras/speech-to-text-websocket-provider.ts	Implements WebSocket audio streaming; lacks activity callbacks (moderate, 2 votes), is unsafe when navigator is unavailable (critical, 2 votes), and accepts malformed result types (critical, 1 vote).
src/extras/speech-to-text-websocket-provider.spec.ts	Tests WebSocket provider behavior.
src/extras/speech-to-text-websocket-protocol.md	Documents the WebSocket protocol.
src/extras/index.ts	Exports the WebSocket provider.
src/components/speech-to-text/types.ts	Defines public types; the error union omits bad-grammar and phrases-not-supported (moderate, 2 votes).
src/components/speech-to-text/themes/themes.ts	Adds theme configuration.
src/components/speech-to-text/themes/speech-to-text.base.scss	Adds component styles and animation.
src/components/speech-to-text/themes/shared/speech-to-text.common.scss	Adds shared theme variables.
src/components/speech-to-text/testing.spec.ts	Adds provider listener test utilities.
src/components/speech-to-text/speech-to-text.ts	Implements the component; contains lifecycle re-entrancy issues affecting igcStart and igcEnd (critical, 1 vote each), abort handling while stopping (moderate, 2 votes), runtime silenceTimeout changes (moderate, 2 votes), and Escape handling while starting (moderate, 1 vote).
src/components/speech-to-text/speech-to-text.spec.ts	Tests lifecycle, accessibility, interaction, and results.
src/components/speech-to-text/providers/web-speech.ts	Wraps the Web Speech API.
src/components/speech-to-text/providers/web-speech.spec.ts	Tests the Web Speech provider.
src/components/speech-to-text/provider-error.ts	Defines provider error handling.
src/components/icon/internal-icons-lib.ts	Adds the microphone icon.
src/components/icon/icon-references.ts	Registers the microphone icon alias.
CHANGELOG.md	Documents the new feature.

Review details

Suppressed comments (1)

src/components/speech-to-text/speech-to-text.ts:225

• Escape is advertised as aborting the session, and _end() explicitly supports aborting a pending starting session, but this predicate only enables the binding in listening. While permission or provider startup is pending, pressing Escape is ignored and the session cannot be canceled from the keyboard. Allow the binding for starting as well while still skipping idle/stopping.

      skip: () => this._state !== 'listening',

• Files reviewed: 22/22 changed files
• Comments generated: 9
• Review effort level: Lite

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Unresolved session lifecycle, accessibility, WebSocket buffering, payload-validation, and theme integration findings remain.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (4)

src/components/speech-to-text/speech-to-text.ts:490

• The rendered aria-label is updated on the igc-icon-button host, but that child does not observe native aria-label changes; its internal <button> is rendered from this.ariaLabel only during the child's own Lit update (src/components/button/button-base.ts:244-257). After the first render, the accessible name can therefore remain "Start voice input" while the control is listening (and can remain the unsupported label after a provider becomes available). Please add reactive ARIA-label forwarding/update to the button or use a reactive slotted label, and cover the listening state in the accessibility test.

        aria-label=${label}

src/components/speech-to-text/speech-to-text.ts:443

• _resetSession() emits igcStateChange synchronously. If a handler starts a new session from that event, _session points at the new session when _emitError runs, so the failed old start's error is emitted and announced during the new session. Emit the captured error before resetting, or otherwise re-check the session after the state-change callback, so the error remains associated with the failed start.

      if (this._session === session) {
        const { code, message } = SpeechToTextProviderError.from(error);
        this._resetSession();
        this._emitError({ code, message });

src/components/speech-to-text/themes/themes.ts:5

• This theme scaffold is incomplete compared with the repository's established component pattern: all has no shared/light/dark theme output, and there are no themes/light or themes/dark _themes.scss files. Existing components such as src/components/tooltip/themes/themes.ts:4-53 aggregate generated light and dark theme styles; leaving this empty means the new component cannot participate in per-theme overrides or the normal theme build. Add the light/dark theme scaffolding and aggregate its generated output before merging.

export const all: Themes = { light: {}, dark: {} };

src/extras/speech-to-text-websocket-provider.ts:357

• The wire payload is parsed without runtime validation, and this expression forwards any non-null value as SpeechToTextErrorEventArgs.message. A server sending { message: 42 } therefore emits an error detail whose message is not a string, violating the public event contract and potentially breaking consumers that call string methods. Accept only strings and use the fallback otherwise.

          message: message.message ?? 'The speech service reported an error.',

• Files reviewed: 22/22 changed files
• Comments generated: 4
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Transition-state handling, WebSocket close/buffer handling, customizable i18n resources, and protocol documentation need updates.

Review details

Suppressed comments (5)

src/components/speech-to-text/speech-to-text.ts:499

• The trigger remains enabled and is labeled “Stop voice input” while the component is starting or stopping, but toggle() only handles the listening state, so activation is an enabled no-op for keyboard and screen-reader users. Disable the trigger during these transition states (or make toggle() abort starting) so the exposed action matches the behavior.

        ?disabled=${this.disabled || !supported}

src/extras/speech-to-text-websocket-protocol.md:165

• The provider accepts not-supported through isSpeechToTextErrorCode and forwards it, but this public protocol table omits that valid SpeechToTextErrorCode. A server author following the table cannot discover the accepted value; document the code here (or remove it from the implementation's accepted server codes).

Error codes, mirroring the Web Speech API:

| Code                     | Meaning |
| ------------------------ | ------- |
| `no-speech`              | No speech was detected. |

src/extras/speech-to-text-websocket-provider.ts:231

• An unexpected server-side close is treated as a clean end here, so the listener receives onEnd() without any network error and the component reports a provider/manual completion. The protocol requires the server to send an end message and says a failed connection is reported as network; teardown already aborts this signal before intentionally calling close(), so this handler can fail the session on an unexpected close.

      socket.addEventListener('close', () => this._finish(), { signal });

src/extras/speech-to-text-websocket-provider.ts:247

• This check only considers the buffer size before the current chunk is sent, so a non-empty chunk can push the socket past the configured maximum (for example, a 16-byte chunk is sent with maxBufferedAmount: 8 while bufferedAmount is 0). Compare the projected amount before calling send() so the documented limit is actually enforced.

          if (socket.bufferedAmount > maxBufferedAmount) {

src/internals/i18n/EN/speech-to-text.resources.ts:6

• These resource properties are required even though this interface is the generic type passed to I18nMixin; consequently, a consumer cannot override just one string via resourceStrings without supplying all six. The i18n controller merges custom values over defaults (src/internals/mixins/i18n.ts:29-33), so make the interface members optional, as the existing component resource interfaces do, while the resolved getter can remain required.

  /** The label of the button while idle. */
  speechToTextStart: string;

• Files reviewed: 24/24 changed files
• Comments generated: 0 new
• Review effort level: Lite

[ivanvpetrov]

This standalone implementation of the speech-to-text functionality is a strong improvement over the earlier Chat-coupled approach in #1893.

It is reusable beyond Chat (input, text-area, etc.), avoids a SignalR dependency, and still support both the browser Web Speech API and custom backend providers through the WebSocket.

One tradeoff is that it has to be manually integrated in a chart component in order to be used there, which is normal and not a problem.

The only thing it lacks, compared with the earlier implementation, is the ability to auto-submit upon silence in the chat component. It's currently not possible, because the default submit operation is private _sendMessage(). For this to work there should be a small publc sendMessage() method in the IgcChatComponent

@rkaraivanov Since I'll be closing my previous PR about the speech-to-text. Do you want any assistance on this one, besides reviewing it?

[rkaraivanov]

@ivanvpetrov
A review would be really helpful but maybe you can take a build of the PR and try the web socket provider with a backend service. Aside from testing correct functionality I am interested if the protocol description is comprehensive enough.