fix(python): keep ChatMessageContent.to_prompt a str when encoding is set

[fei (feiiiiii5)]

What this fixes

ChatMessageContent.to_prompt() is annotated -> str but returns bytes for any message that carries an encoding.

root = self.to_element()
return ElementTree.tostring(root, encoding=self.encoding or "unicode", short_empty_elements=False)

encoding is a real field on the content, so the or never protects it: setting encoding="utf-8" makes tostring take its bytes branch. Measured on ca40aa72:

encoding='utf-8'  ->  type=bytes  value=b'<message role="user" encoding="utf-8">...'
encoding='ascii'  ->  b'<?xml ...?>\n<message ...'   # XML declaration too

This is user-visible rather than merely untidy: jinja2_system_helpers._message_to_prompt and its handlebars twin render a context variable via str(context.to_prompt()), so the prompt the model receives literally contains b'<message role="user" ...>' — bytes repr plus quote noise — whenever the message had an encoding set.

Every other tostring call site in the same package already hardcodes encoding="unicode": chat_history.py:285, :317, :362, chat_message_content.py:275, handlebars_system_helpers.py:37, :55 and jinja2_system_helpers.py:29. This one line was the outlier, so the change aligns it rather than introducing a new convention.

The message's own encoding is still preserved: it is an attribute on the element and round-trips through to_element/from_element; it describes the text content, not how we serialize a prompt string.

Change

One line, plus a docstring note, and a parametrized regression test over utf-8, ascii and latin-1 asserting the prompt is a str, has no XML declaration, keeps the non-ASCII text, and re-parses to the same content.

How this was verified

cd /private/tmp/sk/tree/python, PYTHONPATH=., the repo's own test file tests/unit/contents/test_chat_message_content.py.

Base control — semantic_kernel/contents/chat_message_content.py restored from ca40aa7226531d28a721d0ca0e451d0aaf86dafc with git diff -- <that file> printing nothing (so the run measures upstream code with the new tests kept), sha re-echoed:

FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[utf8]
FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[ascii]
FAILED tests/unit/contents/test_chat_message_content.py::test_cmc_to_prompt_with_encoding[latin1]
3 failed, 43 passed

With the patch applied: 46 passed.

Enclosing directory, both revisions, same command (pytest tests/unit/contents): the test_chat_history.py::test_to_from_file / ::test_chat_history_serialize collection errors appear identically at base and at head — they come from optional extras missing in my environment, not from this diff. I did not run the integration or dotnet suites.

ruff check on the two files reports only a pre-existing D finding at chat_message_content.py:201, present at base too; ruff format --check is clean. (mypy was not installed in this environment, so the CI type gate is unverified here.)

Dedup: searched issues and PRs (all states) for to_prompt, message_to_prompt and encoding unicode prompt; the only hits are merged history (#13738, #5550). The nearby data_uri MIME-subtype defect I found while investigating is already claimed by open #14304, so it is deliberately not part of this PR.

[Copilot]

Copilot review overview

🟢 Approval recommended

The fix is focused, tested across supported encodings, and has no unresolved blocking issues.

Review effort: Lite
Findings: None

What changed in this PR

Fixes ChatMessageContent.to_prompt() so it always returns a string when an encoding is set.

Changes:

• Forces Unicode XML serialization.
• Documents encoding behavior.
• Adds regression tests for UTF-8, ASCII, and Latin-1.

File	Description
python/​semantic_kernel/​contents/​chat_message_content.py	Ensures prompt serialization returns str.
python/​tests/​unit/​contents/​test_chat_message_content.py	Adds encoding-specific regression coverage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.