Skip to content

Index existing messages in merge_template/3 instead of a linear find per message - #443

Open
mstroeck wants to merge 1 commit into
elixir-gettext:mainfrom
mstroeck:indexed-merge-template
Open

mstroeck wants to merge 1 commit into
elixir-gettext:mainfrom
mstroeck:indexed-merge-template

Conversation

@mstroeck

@mstroeck mstroeck commented Oct 7, 2026 •

Copy link
Copy Markdown

Extraction got very slow on our catalogue - merge_template/3 calls Expo.Messages.find/2 once per message, and find/2 walks the whole list, so the merge is O(n²).

On our default.pot (~25.9k msgids) merge_template/3 alone takes 224.7s. With this change it takes 0.307s (~730x), with byte-identical output.

What changed:

  • build a map of the existing messages by key once, then look up instead of calling find/2 per message
  • first match wins, same as find/2, so duplicate keys behave exactly as before
  • ~8 lines, no API change

How I checked it:

  • gettext's own test suite passes (173 tests)
  • byte-identical output on our real catalogue
  • synthetic changed catalogues across all sort_by_msgid modes

Happy to add a benchmark or a regression test to the repo if you'd like one.

Thanks!

merge_template/3 called Expo.Messages.find/2 once per message, and find/2
walks the whole list, so merging was quadratic in the catalogue size.
Look messages up by key instead. The first message for a key wins, as
with find/2, so the output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@whatyouhide

Copy link
Copy Markdown
Contributor

@mstroeck lovely find, can we add a regression test with two new messages that have the same key but different references and extracted comments? Just to make sure the map behavior doesn't break that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants