Skip to content

fix(content-drive): user-cache sizing + warm-up to cut per-row DB lookups (#37186) - #37394

Open
ihoffmann-dot wants to merge 6 commits into
issue-37186-content-drive-user-cache-sizingfrom
issue-37186-content-drive-user-cache-sizing-impl
Open

fix(content-drive): user-cache sizing + warm-up to cut per-row DB lookups (#37186)#37394
ihoffmann-dot wants to merge 6 commits into
issue-37186-content-drive-user-cache-sizingfrom
issue-37186-content-drive-user-cache-sizing-impl

Conversation

@ihoffmann-dot

@ihoffmann-dot ihoffmann-dot commented Sep 4, 2026

Copy link
Copy Markdown
Member

Summary

  • Raises cache.userdotcmscache.size from the effectively-1000 default to 4000 (FR-002), sized against a reference ~1,828 active-user base.
  • Adds a warm-up pass (BrowserAPIImpl#warmUpUserCache) that resolves each distinct modUser/owner on a listing page once, ahead of parallel row hydration, cutting redundant per-row UserAPI#loadUserById calls (User Story 1).
  • Dedupes the mod-user-name lookup already done once per row (DefaultTransformStrategy#resolveModUserName, User Story 2).
  • Adds an orphan-user fallback (FR-004a): a listing row whose modUser/lockedBy user was deleted no longer fails the whole page — it now degrades to N/A for that row only.
  • Adds a @VisibleForTesting DB-lookup counter (UserFactoryImpl) used only by the new integration test to assert warm-up actually reduces lookups to one per distinct author.

Test plan

  • ./mvnw test -pl :dotcms-core -Dtest=UserFactoryImplTest,BrowserAPIImplTest,UserCacheConfigTest,DefaultTransformStrategyTest
  • just test-integration-ide
  • ./mvnw verify -pl :dotcms-integration -Dcoreit.test.skip=false -Dit.test=BrowserAPITest
  • Confirm no System.out/System.getProperty/System.getenv introduced (checked via diff at implementation time)

Branched off the approved spec branch per this repo's Spec-Kit flow (spec.md-only in PR1, not merged to main yet).

🤖 Generated with Claude Code

This PR fixes: #37186

…ydration (#37186)

Adds a sequential warm-up pass in BrowserAPIImpl that resolves every distinct
modUser/owner id a page needs before hydrateContentletsInParallel fans rows
into concurrent chunks, eliminating the thundering-herd race on
UserFactoryImpl#loadUserById. Also wraps the two unguarded loadUserById calls
in DefaultTransformStrategy#addVersionProperties (modUser, locked-by) so an
orphaned user id degrades one row instead of failing the whole listing.
…n thrashing (#37186)

FR-002: cache.userdotcmscache.size was effectively 1000 (commented out, falling
back to the shared cache.default.size), which thrashes continuously on the
1,828-active-user reference instance. Line 519 (email region) is left
untouched — it's a dead region due to a key-mismatch bug in UserCacheImpl,
so raising it would only reserve memory nothing reads back.
…nsform (#37186)

User Story 2 (optional, kept per developer decision): addVersionProperties
now reuses the modUserName addAuditProperties already resolved for the same
row instead of calling loadUserById a second time for the same id. Once
FR-001's warm-up makes the id cache-warm, the repeat call was already a
cache hit, not a DB round trip -- this trims call-count/CPU overhead, and
matters on its own for any caller that doesn't warm up first.
@claude

claude Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Claude finished @ihoffmann-dot's task in 3m 0s —— View job


Code Review — PR #37394 (content-drive user-cache sizing + warm-up)

  • Read the PR diff
  • Analyze main code changes (BrowserAPIImpl, UserFactoryImpl, DefaultTransformStrategy, config)
  • Analyze test changes
  • Post review

I traced the full transform/hydration path and verified the cache region key, the dedup logic, and the warm-up placement against the surrounding code.

New Issues

  • 🟡 Medium: dotcms-integration/src/test/java/com/dotcms/browser/BrowserAPITest.java:~2680alter table user_ disable trigger all is table-level DDL that affects every session against the shared DB, not just this test's connection. For the brief window between disable and re-enable, FK enforcement on user_ is off process-wide. It's correctly wrapped in try/finally, but the enable only runs after the delete; if the integration suite is ever run with any parallel/concurrent DB activity, a sibling test touching user_ during that window could bypass constraints.

    • Assumption: dotcms-integration runs tests serially in a single JVM against one DB (which is the current setup), so this is not a live problem today.
    • What to verify: that no integration test parallelism is enabled for this module now or planned. If it is, scope the delete differently (e.g. SET session_replication_role = replica on this connection only, or delete via a path that doesn't trip the parent-side trigger).
  • 🟡 Medium: dotCMS/src/main/java/com/dotmarketing/business/UserFactoryImpl.java:118 / dotcms-integration/.../BrowserAPITest.java:2600dbLookupCount is a process-wide static incremented on every loadUserById cache miss JVM-wide. The integration assertion getDbLookupCountForTesting() == expectedIds.size() (exactly 3) assumes nothing else in the process resolves a not-yet-cached user between reset() and the assertion. getPaginatedContents runs parallel hydration chunks and other listing machinery on background threads; any incidental cold-user resolution (or a concurrently-running test) would push the count above 3 and fail.

    • Assumption: in the current serial integration run, the only cold resolutions during this call are the 3 seeded authors (system/other users are already warm from AbstractDataGen's static init), so it passes deterministically.
    • What to verify: the test is stable across repeated CI runs. If it flakes, assert >= expectedIds.size() on the cold pass isn't enough — consider assertEquals(0, ...) on the warm second pass (already present) as the primary signal and relax the cold-pass exact count.

Notes (non-blocking, no action required)

  • DefaultTransformStrategy.resolveModUserName correctly reuses the value addAuditProperties writes under MOD_USER_NAME_KEY. I confirmed the ordering: transform() runs addCommonProperties (→ addAuditProperties) before addVersionProperties, both resolve the same modUser id, so the reused value is identical to what the old unconditional loadUserById produced. The only behavioral change is the intended FR-004a degrade-to-N/A instead of throwing NoSuchUserException — consistent with addAuditProperties' existing Try.getOrNull pattern. ✅
  • Config key is correct: UserCacheImpl.primaryGroup = "UserDotCMSCache" → property cache.userdotcmscache.size. The "dead email region" comment is accurate (add keys emailGroup by email address, get reads it by primaryGroup + key, so entries are never read back). ✅
  • warmUpUserCache per-id failure handling (Try.run().onFailureLogger.debug) correctly isolates a single orphan from aborting the warm-up, leaving the per-row FR-004a fallback to handle the row later. ✅
  • collectWarmUpUserIds is pure/side-effect-free, null-safe via UtilMethods.isSet, and deterministically ordered (LinkedHashSet). ✅
  • No System.out/getProperty/getenv; Logger/Config/APILocator conventions followed; @VisibleForTesting public-vs-package visibility is justified by the cross-module integration test. ✅

No blocking issues. The two medium items are test-robustness concerns that only bite if integration test concurrency changes — worth a glance but they don't block merge.

· issue-37186-content-drive-user-cache-sizing-impl

…erAPITest (#37186)

- ContentletDataGen#user(User) is a no-op on the no-categories persist
  path (it always uses the static system user as the acting user, per
  ContentletDataGen.java:317-326), so the warm-up test's per-author
  assignment and the orphan test's modUser were both silently ignored.
  Reassign modUser/owner directly via SQL after creation instead.
- The orphan test's raw delete of the deleted user's row was blocked by
  two FKs: users_cms_roles.fkusers_cms_roles2 (role assignment) and
  contentlet.fk_user_contentlet, now correctly referenced once modUser
  is actually set. Delete the role assignment first, and disable/
  restore user_'s triggers (where Postgres registers the parent-side RI
  check for a DELETE) around the user delete.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant