Skip to content

🐛 fix(backend): escape output the stdout encoding cannot represent - #330

Merged
gaborbernat merged 4 commits into
tox-dev:mainfrom
nk-batman21:fix/backend-unencodable-output
Oct 28, 2026
Merged

gaborbernat merged 4 commits into
tox-dev:mainfrom
nk-batman21:fix/backend-unencodable-output

Conversation

@nk-batman21

@nk-batman21 nk-batman21 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

On Windows, SubprocessFrontend and python -m pyproject_api failed when a path the backend prints contains a character outside the ANSI code page, for example a project in a Cyrillic folder on an English-locale machine. Python writes pipes in the ANSI code page on Windows, so the backend's progress messages raised UnicodeEncodeError and the build stopped with a missing response file:

UnicodeEncodeError: 'charmap' codec can't encode character 'ệ' in position 119: character maps to <undefined>

🐛 The backend now writes such characters to standard output as backslash escapes, the same error handler Python already uses for standard error. This is the encoding-side counterpart of #311, which made the frontend tolerate bytes it cannot decode. tox is not affected because it sets PYTHONIOENCODING=utf-8 for the processes it runs.

The setting applies only when the backend runs as its own process, so callers that import the backend module in-process keep their stream configuration.

nk-batman21 and others added 2 commits October 24, 2026 03:30
The backend prints its module and the hook arguments, which include
project and output paths. Before Python 3.15 a Windows pipe uses the
ANSI code page, so a path such as a Cyrillic folder name made print()
raise UnicodeEncodeError and the build failed. Use backslashreplace for
stdout, as stderr already does.
ty needs the suppression because typeshed declares sys.stdout as TextIO,
which has no reconfigure(). Inline the single-use frontend in the test.
@gaborbernat gaborbernat changed the title fix(backend): escape output the stdout encoding cannot represent 🐛 fix(backend): escape output the stdout encoding cannot represent Oct 27, 2026
@gaborbernat gaborbernat added the bug Something isn't working label Oct 27, 2026
The fix lives in the backend module, so its test belongs in the backend
test file, importing the frontend from the public package.
@gaborbernat
gaborbernat enabled auto-merge (squash) October 28, 2026 01:12
@gaborbernat
gaborbernat disabled auto-merge October 28, 2026 01:29

@gaborbernat gaborbernat left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaborbernat
gaborbernat merged commit c42c1b1 into tox-dev:main Oct 28, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants