Skip to content

Preserve remote post languages from contentMap - #659

Merged
dahlia merged 2 commits into
fedify-dev:0.9-maintenancefrom
dahlia:bugfix/language-tags
Oct 5, 2026
Merged

dahlia merged 2 commits into
fedify-dev:0.9-maintenancefrom
dahlia:bugfix/language-tags

Conversation

@dahlia

@dahlia dahlia commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Fixes #647.

Mastodon and Hollo send content together with contentMap. After JSON-LD expansion, the content property holds both values, with the plain string first. persistPost() read only the singular object.content, so it never saw the LanguageString and stored null as every remote post's language.

getLanguage() in src/federation/post.ts scans contents, then summaries, for the first language-tagged value. It skips und because a default @language in the sender's @context tags the plain content too, which would otherwise hide the language from contentMap. A missing locale.language counts as und as well, since Node.js 24 returns undefined for new Intl.Locale("und").language.

The tests in src/federation/post.test.ts use Note.fromJsonLd() to get the value order that triggers the bug. A Note built with the constructor keeps whatever order you give it.

The second commit upgrades Fedify to 2.2.16, which normalizes Mastodon's Cantonese tag zh-YUE to yue (fedify-dev/fedify#1229). zh-YUE is valid BCP 47, but Intl.Locale rejects it, so earlier versions dropped the value. I kept the upgrade in its own commit so it can be reviewed or reverted separately.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed remote posts losing language information when they include both plain and language-mapped content.
    • Improved language detection to fall back to summary languages and retain normalized language tags, including tags such as zh-YUE.
  • Maintenance
    • Updated the Fedify runtime and linting tools.

dahlia added 2 commits October 5, 2026 19:13
Mastodon and Hollo send content together with contentMap, so after
JSON-LD expansion the content property holds both a plain string and a
language-tagged one.  persistPost() only looked at the singular
object.content and object.summary accessors, which return the plain
string, so the language of such posts was always stored as null.

The language is now taken from the first LanguageString found among
all content values, then all summary values.  Values tagged as
undetermined (und), which a context-level default @language such as
Pleroma's produces, are skipped so that they don't shadow the real
language in contentMap.

Fixes fedify-dev#647

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-fable-5-1
Fedify 2.2.16 normalizes BCP 47 extended language subtags before
handing language tags to Intl.Locale.  Mastodon tags Cantonese posts
as zh-YUE, which is a valid BCP 47 tag but not a valid Unicode locale
identifier, so Fedify used to drop such language-tagged strings and
Hollo stored no language for those posts.  They are now kept with the
canonical tag yue.

fedify-dev#647
fedify-dev/fedify#1229

Assisted-by: Claude Code:claude-opus-5-5
@dahlia dahlia self-assigned this Oct 5, 2026
@dahlia dahlia added the bug Something isn't working label Oct 5, 2026
@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (1)
AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1e41858f-3d64-40d6-9efa-71c548211f1a
📥 Commits

Reviewing files that changed from the base of the PR and between f2dca7f and 2160706.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (4)
  • CHANGES.md
  • package.json
  • src/federation/post.test.ts
  • src/federation/post.ts

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


📝 Walkthrough

Walkthrough

Remote post persistence now selects language metadata from language-tagged content or summaries. The change also upgrades Fedify dependencies to 2.2.16 and adds tests for language-map selection and language-tag canonicalization.

Changes

Remote post language preservation

Layer / File(s) Summary
Language extraction and persistence
package.json, src/federation/post.ts, src/federation/post.test.ts, CHANGES.md
Fedify dependencies are upgraded to 2.2.16. getLanguage selects the first language-tagged value from content, then summaries, while skipping missing language tags and und. persistPost uses that result. Tests cover content maps, summary maps, and zh-YUE canonicalization. The changelog records the changes.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 21607

The language-selection change is implemented as intended, and no current user-facing failure is established. A test for conflicting content and summary languages would be a useful non-blocking improvement.

Architecture Summary

Architecture risk: 🟡 Medium · up to 21607

The change affects 3 systems.

Changed systems: src, CHANGES.md, package.json

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — src (service) was modified; 2 changed files map to changed impact.
  • observed — CHANGES.md (service) was modified; 1 changed file maps to changed impact.
  • observed — package.json (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in CHANGES.md: Adds a changelog entry for preserving the language of remote posts received with both content and contentMap.
  • observed — Modified behavior in CHANGES.md: Adds a changelog entry for the Fedify 2.2.16 upgrade and its handling of extended language subtags, including canonicalizing tags such as zh-YUE to yue instead of dropping them.
  • observed — Modified behavior in CHANGES.md: Adds reference links for the language-tag issue and related Hollo issue and pull request.
  • observed — Modified behavior in package.json: The listed @fedify runtime dependencies, excluding the unchanged Markdown plugins, now use ~2.2.16 instead of ~2.2.15.

Reliability and maintainability

  • inferred — Risk-relevant change factors for src: blast_radius_1; direct_dependents_1
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. (2 skipped: 2… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: preserving remote post languages from contentMap.
Linked Issues check ✅ Passed Issue [#647] requires Hollo to preserve language metadata on incoming Hollo and Mastodon posts, including cases beyond zh-YUE. getLanguage() searches content and summary values for a tagged langua…
Out of Scope Changes check ✅ Passed The post-language extraction, related tests, changelog entry, and Fedify upgrade all support issue [#647]. The upgrade addresses the issue’s reported zh-YUE case. No unrelated changes are identified…
Full details: Docstring Coverage

Explanation

Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dahlia
dahlia merged commit e5eda56 into fedify-dev:0.9-maintenance Oct 5, 2026
4 checks passed
@dahlia dahlia linked an issue Oct 5, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Post language metadata is not correctly federated to Hollo

1 participant