Skip to content

RHINENG-26122: remove advisories_account_data - #2345

Open
Dugowitch wants to merge 17 commits into
RedHatInsights:masterfrom
Dugowitch:rm-aad
Open

Dugowitch wants to merge 17 commits into
RedHatInsights:masterfrom
Dugowitch:rm-aad

Conversation

@Dugowitch

@Dugowitch Dugowitch commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Remove advisory_account_data and rely on the workspace-scoped account_advisory aggregation path.

Enhancements:

  • Remove the legacy advisory_account_data cache and consolidate advisory aggregation on the workspace-scoped account_advisory table.
  • Move advisory count ownership to the aggregator component and assign it dedicated database access.
  • Remove legacy advisory cache refresh, backfill, cleanup, recovery, notification, and cache-validation functionality.
  • Simplify advisory notification payloads and eliminate evaluator-side advisory notification publishing.
  • Update database schema, migrations, deployment configuration, monitoring, documentation, and tests for the new advisory aggregation model.

Deployment:

  • Update aggregator and database migration deployments to use the dedicated aggregator database credentials and remove obsolete advisory maintenance jobs.

Documentation:

  • Update architecture, database, and migration documentation to remove references to advisory_account_data.

Tests:

  • Update tests and fixtures to validate account_advisory-based aggregation without the legacy advisory cache.

@sourcery-ai

sourcery-ai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Schema and migrations

Drop advisory_account_data and its cache-maintenance objects, and grant the aggregator ownership of account_advisory aggregation through migrations and the baseline schema.

  • database_admin/migrations/171_drop_advisory_account_data.down.sql L1-176
  • database_admin/migrations/171_drop_advisory_account_data.up.sql L1-24
  • database_admin/migrations/172_add_aggregator_role.down.sql L1-11
  • database_admin/migrations/172_add_aggregator_role.up.sql L1-31
  • database_admin/schema/create_schema.sql L401-420, L729-733, L960-964 (+8 small edits)
  • dev/test_data.sql L9 (removed lines), L205
  • database_admin/schema/repair_system_advisories_0.sql L12 (removed lines)

Data structures

Remove the legacy advisory-account model, notification context types, and obsolete platform-event notification-suppression field.

  • base/mqueue/platform_event.go L78-82, L102-111, L116-125 (+2 small edits)
  • base/notification/notification.go L4-68 (5 small edits)
  • base/models/models.go L291 (removed lines)

API

Remove evaluator-owned notification publication and legacy advisory-cache updates, and narrow cache invalidation to package data.

  • evaluator/evaluate.go L700-705 (+5 small edits)
  • evaluator/evaluate_advisories.go L8-344 (5 small edits)
  • evaluator/notifications.go (deleted)

Uses of the API

Update aggregator event processing and notification payloads to use the account-advisory refresh path without drift checks or legacy notification context.

  • aggregator/notifications_test.go L54-55, L70 (removed lines)
  • aggregator/drift_check.go (deleted)
  • aggregator/events.go L99 (removed lines)

Implementation

Remove database helpers and task implementations that exist only for legacy advisory-cache validation, refresh, backfill, cleanup, and recovery.

  • base/database/database.go L43 (removed lines)
  • base/database/testing.go L4-261 (7 small edits)
  • tasks/caches/backfill_account_advisory.go (deleted)
  • tasks/caches/caches.go L9 (removed lines), L13 (removed lines)
  • tasks/caches/refresh_advisory_caches.go (deleted)
  • tasks/cleaning/clean_advisory_account_data.go (deleted)
  • tasks/system_advisories_0_recovery/recovery.go (deleted)
  • tasks/vmaas_sync/vmaas_sync.go L12 (removed lines), L116 (removed lines)
  • turnpike/controllers/admin.go L4-153 (4 small edits)

UI

Remove the legacy advisory-account-data cache metric and its dashboard panel.

  • manager/middlewares/prometheus.go L46 (removed lines), L56
  • dashboards/app-sre/grafana-dashboard-insights-patchman-engine-general.configmap.yaml L2658 (removed lines)

Configuration

Configure and deploy a dedicated aggregator database user, remove obsolete cache and recovery jobs, and update migration locking and credentials.

  • conf/aggregator_common.env L1-2
  • conf/database_admin.env L8
  • database_admin/schema/create_users.sql L8
  • database_admin/update.go L17, L260
  • deploy/clowdapp.yaml L465-1149 (10 small edits)
  • main.go L16 (removed lines), L75 (removed lines), L81 (removed lines)
  • manager/config/config.go L17 (removed lines)
  • manager/controllers/advisories.go L218 (removed lines), L228 (removed lines), L234 (removed lines)
  • manager/routes/routes.go L104 (removed lines)
  • tasks/config.go L11 (removed lines), L26 (removed lines), L41 (removed lines)

Documentation

Revise architecture, database, and migration guidance to describe the workspace-scoped aggregation model and remove legacy-cache workflows.

  • AGENTS.md L60 (removed lines), L66
  • docs/md/architecture.md L32-34, L53-54
  • docs/md/database.md L9
  • docs/md/major-migration-runbook.md L156

Tests

Update tests and fixtures for aggregator-owned account-advisory aggregation, and remove assertions and helpers tied to the legacy cache and recovery behavior.

  • listener/common_test.go L36-37
  • tasks/caches/refresh_account_advisory_caches_test.go L22-23, L61
  • tasks/system_culling/system_culling_test.go L45-73, L118-128, L159-171 (+6 small edits)
  • tasks/vmaas_sync/metrics_db_test.go L20-21
  • aggregator/drift_check_test.go (deleted)
  • aggregator/events_test.go L69 (removed lines)
  • base/mqueue/platform_event_test.go L14 (removed lines), L39 (removed lines), L61 (removed lines)
  • evaluator/advisory_update_test.go L156 (removed lines), L160 (removed lines), L185 (removed lines)
  • evaluator/evaluate_advisories_test.go L151 (removed lines), L237 (removed lines)
  • evaluator/evaluate_test.go L72-153 (6 small edits)
  • evaluator/inventory_views_test.go L72 (removed lines), L77 (removed lines)
  • evaluator/notifications_test.go (deleted)
  • evaluator/template_advisory_e2e_test.go L69 (removed lines)
  • tasks/caches/refresh_advisory_caches_test.go (deleted)
  • tasks/system_advisories_0_recovery/recovery_test.go (deleted)

Lock files and vendored code

Remove the generated OpenAPI entries for the deleted cache-validation and cleanup endpoints.

  • docs/admin/openapi.json L19 (removed lines)

Other changes

Files not grouped.

  • tasks/cleaning/clean_unused_data.go L48 (removed lines), L53 (removed lines)

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov-commenter

codecov-commenter commented Sep 24, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 68.16%. Comparing base (4b0be4c) to head (ab20869).

Files with missing lines Patch % Lines
manager/middlewares/prometheus.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2345      +/-   ##
==========================================
+ Coverage   66.82%   68.16%   +1.33%     
==========================================
  Files         157      151       -6     
  Lines        9080     8606     -474     
==========================================
- Hits         6068     5866     -202     
+ Misses       3010     2738     -272     
  Partials        2        2              
Flag Coverage Δ
unittests 68.16% <93.33%> (+1.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Dugowitch
Dugowitch marked this pull request as ready for review September 25, 2026 12:40
@Dugowitch
Dugowitch requested a review from a team as a code owner September 25, 2026 12:40

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="base/mqueue/platform_event.go" line_range="79" />
<code_context>
-	if size <= 0 {
-		size = BatchSize
-	}
+func batchSize(grouped map[int][]uuid.UUID) int {
+	// compute how many batches we will create
 	var batches = 0
 	for _, ev := range grouped {
-		batches += (len(ev) + size - 1) / size
+		batches += len(ev)/BatchSize + 1
 	}
 	return batches
</code_context>
<issue_to_address>
**nitpick (performance):** `batchSize` returns one extra batch whenever an account has an exact multiple of `BatchSize` systems; for example, 500 systems produces 2 instead of 1. `WriteEvents` still emits only the correct number of events, but it allocates an unnecessarily large `PlatformEvents` backing array for every such account, increasing memory usage during large event batches.

**Triggers:** When an account's inventory count is an exact multiple of `BatchSize`.

**Suggested fix:** Use `(len(ev) + BatchSize - 1) / BatchSize`, matching the previous calculation, and return zero for empty slices.

```suggestion
		batches += (len(ev) + BatchSize - 1) / BatchSize
```
</issue_to_address>

Sourcery assessment

Needs a human reviewer. The migration drops the production advisory_account_data table and its contents, while the down migration only recreates an empty table, so reverting cannot restore the deleted data. It also changes database roles and permissions, making an incorrect cutover potentially affect both data availability and component access.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread base/mqueue/platform_event.go Resolved
@Dugowitch

Copy link
Copy Markdown
Contributor Author

The PR is ok to review, but wait with merging, notified backport needs to be triggered first

MichaelMraka
MichaelMraka previously approved these changes Sep 29, 2026
@Dugowitch
Dugowitch marked this pull request as draft October 1, 2026 07:48
@swadeley

Copy link
Copy Markdown
Contributor

Hi, please rebase.

@MichaelMraka

MichaelMraka commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

I've found 2 more unused variables
DisableCachedCounts
skipNAccountsRefresh

MichaelMraka
MichaelMraka previously approved these changes Nov 15, 2026
@Dugowitch

Copy link
Copy Markdown
Contributor Author

Removed the two variables and rebased, but Konflux seems to be temporarily out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants