[blarghmatey]

Description

_wait_for_meili_task polls Meilisearch until a task finishes. Meilisearch applies tasks one at a time in the order they were enqueued, across every index on the instance, so a one-document write waits behind whatever is ahead of it in the queue. Several library handlers in handlers.py run their search task with .apply(), so that wait happens inside the Studio request, and when the queue is backed up the author gets a 504 for a change that was saved. #39072 bounded this for library block upserts with SYNC_INDEX_WAIT_TIMEOUT (3s). The other inline writes still wait with no deadline: block deletes, library create and update, and collection and container writes and deletes.

This adds a MEILISEARCH_WRITE_WAIT_TIMEOUT setting (seconds, default None). The three helpers used for incremental document writes (_update_index_docs, _delete_index_doc, _delete_documents) pass it to the timeout that #39072 added to _wait_for_meili_task, unless the caller passes its own wait_timeout, so the block upserts keep SYNC_INDEX_WAIT_TIMEOUT. When the timeout runs out the helper logs and returns; the write stays in Meilisearch's queue and is applied when Meilisearch reaches it. Index creation, rebuilds and settings updates call _wait_for_meili_task directly and keep waiting until done.

The default keeps current behavior. The limit is per write, not per request. A library change with fewer than five affected entities runs send_change_events_for_modified_entities inline, and each entity's handler makes its own write, so an operator has to keep the timeout times that count under the gateway timeout (the setting's description says so). I went with a per-write limit because a shared per-request budget needs request-scoped state threaded through the Celery tasks, and the per-write version is enough to stop the 504s.

The cost of setting it is the same one #39072 accepts for block upserts: a write that is not waited for is not visible to the Authoring MFE's refetch until Meilisearch applies it, and a write that later fails is logged by Meilisearch but not raised. The helpers are shared with the .delay() paths, so in a worker that also means autoretry_for does not see a failure that happens after the timeout. An operator who doesn't want either keeps the default.

The index split in #39103 does not remove this wait. It separates documents, but studio_content and studio_library_content share one Meilisearch task queue ("Enqueued tasks are processed one at a time, following the order in which they were requested", and a batch only groups tasks on the same index, per https://www.meilisearch.com/docs/learn/async/asynchronous_operations), so library writes still queue behind course writes.

This is narrower than the async API discussed in #38993, which would remove the polling altogether. It bounds the existing behavior until that lands. It does not overlap #39042, which bounds Celery retries and time limits but cannot bound the eager path, since soft_time_limit is not enforced under .apply().

Roles affected: Course Author (no 504 on the library writes above when the setting is used), Operator (new optional setting).

Supporting information

What we measured on courses.learn.mit.edu (Meilisearch 1.53.1, single instance, after #39103):

• studio_content holds 1,588,735 documents and studio_library_content 1,226.
• On 2026-10-14 between 14:50 and 15:00 UTC, four POST /api/libraries/v2/<lib>/blocks/ requests got a 504 from our CDN at 60.0s. The traces for two of them show the CMS returned 200 after 140.4s and 141.9s, and 140.37s of the first was the eager upsert_library_block_index_doc span.
• meilisearch_task_queue_latency_seconds was 76-201s for that whole window and peaked at 439s at 15:12.
• Over the 7 days to 2026-10-15, 4 of 18 component creates on that instance returned 504.

Those 504s were component creates, which #39072 now covers. The other inline writes wait on the same queue; I have not measured them separately.

Testing instructions

TestDocumentWriteWaitTimeout covers the three write helpers passing the setting through, an explicit wait_timeout taking precedence over it, and None when unset. test_api.py and test_handlers.py pass locally (59 tests).

To reproduce against a running Studio, the queue has to be backed up, not unreachable:

1. Enable Meilisearch for Studio and set MEILISEARCH_WRITE_WAIT_TIMEOUT = 5.
2. Enqueue a large batch against the course index (e.g. run ./manage.py cms reindex_studio on a course-heavy instance), so the queue is busy for minutes.
3. Delete a component in a V2 library while that runs. The request returns in roughly 5s and the CMS logs Stopped waiting for Meilisearch task <uid> after 5.0s. Without the setting, the same request blocks until the batch ahead of it finishes.
4. Once the queue drains, the component is gone from the library listing.

Deadline

None.

https://claude.ai/code/session_01JxJWMHVXsuXaChbeEpKT8t

[openedx-webhooks]

Thanks for the pull request, @blarghmatey!

This repository is currently maintained by @openedx/wg-maintenance-openedx-platform-oncall.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[mphilbrick211]

@blarghmatey flagging some conflicts have popped up!