fix(mem_os): strip only anchored markdown fences in clean_json_response (#2420)

[RerankerGuo]

Summary

clean_json_response used a global response.replace("```json", "").replace("```", ""), which removed every triple-backtick sequence — including ones inside JSON string values. Any LLM response whose payload contained a code snippet came out silently corrupted before json.loads.

Root cause

src/memos/mem_os/utils/format_utils.py:1416 (pre-fix):

return response.replace("```json", "").replace("```", "").strip()

Reproducer from the issue:

clean_json_response("```json\n{\"text\": \"```py\\ncode```\"}\n```")
# before:  "{\"text\": \"py\\ncode\"}"      ← fences stripped from the value
# after:   "{\"text\": \"```py\\ncode```\"}"

Both api/handlers/chat_handler.py and api/handlers/suggestion_handler.py pipe responses through this function, so the bug surface was the entire chat-metadata and suggestion pipeline.

Fix

Replace the global string replace with two anchored regex substitutions that only strip the outer fences:

• Opening: re.compile(r"^```[^\n`]*\n?") — covers the optional info string (json or otherwise) and trailing newline
• Closing: re.compile(r"\n?```\s*$") — at the very end only

Interior ``` sequences inside a JSON string value are preserved verbatim. As a side effect the function now correctly handles responses that have only an opening or only a closing fence (the original implementation would leave those stranded mid-string).

Tests

Added four cases to tests/mem_os/test_format_utils.py:

1. test_clean_json_response_preserves_internal_fenced_code — the issue reproducer (red against unpatched main)
2. test_clean_json_response_preserves_unclosed_fence_in_value — a value containing a single unclosed fence
3. test_clean_json_response_handles_only_opening_fence — best-effort handling when the closing fence is missing
4. test_clean_json_response_handles_only_closing_fence — best-effort handling when the opening fence is missing

Existing tests (single backticks in JSON content, plain JSON, complex JSON, etc.) all continue to pass — test_clean_json_response_preserves_internal_backticks was a good seed for the new tests.

Verification

PYTHONPATH=src pytest tests/mem_os/test_format_utils.py -v
============================== 12 passed in 3.09s ==============================

Ruff: ruff format --check and ruff check both clean on touched files (the 2 pre-existing findings were auto-fixed by ruff check --fix during this PR — both were E501 line-too-long in clean_json_response itself, not introduced by this change).

Out of scope

• No new dependencies
• No public API / OpenAPI contract changes
• No DB schema, migration, or graph model changes

Closes #2420

[Memtensor-AI]

🤖 Open Code Review

Target: PR #2489
Task: 53eaa9e04a315a2e
Base: dev-v2.0.36
Head: fix/issue-2420-clean-json-fence

🔍 OpenCodeReview found 1 issue(s) in this PR.

---

1. src/memos/mem_os/utils/format_utils.py (L14)

The opening-fence pattern can swallow the entire payload when the fence is not followed by a newline, because the info-string class [^\n]*` is greedy and will happily match JSON content up to the next backtick.

Example: clean_json_response('```{"a": 1}```')

• _JSON_FENCE_OPEN_RE matches from the start of the string through {"a": 1} (only \n and a backtick stop the class), leaving ```
• _JSON_FENCE_CLOSE_RE then removes that remainder
• Result: '' (empty string), so the downstream json.loads fails on nothing.

The same happens with '```json {"key": "value"}```' (space instead of newline). The pre-change implementation (response.replace("```json", "").replace("```", "")) returned the correct JSON for both inputs, so this is a regression — models do emit short, single-line fenced JSON. Restricting the info string to an actual info string (and keeping \n? optional so a truncated fence is still stripped) avoids consuming payload characters.

💡 Suggested Change

Before:

        _JSON_FENCE_OPEN_RE = re.compile(r"^```[^\n`]*\n?")

After:

_JSON_FENCE_OPEN_RE = re.compile(r"^```[ \t]*[A-Za-z0-9_+-]*[ \t]*\n?")

Generated by cloud-assistant via Open Code Review.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (12/12 executed). memos_python_core/changed-repo-python: 12/12. Duration: 13s

Branch: fix/issue-2420-clean-json-fence