synchronization: log per-event partial delivery errors from PublishBatch#22998
Draft
pkcll wants to merge 1 commit into
Draft
synchronization: log per-event partial delivery errors from PublishBatch#22998pkcll wants to merge 1 commit into
pkcll wants to merge 1 commit into
Conversation
Contributor
|
✅ No conflicts with other open PRs targeting |
Contributor
|
I see you updated files related to
|
pkcll
force-pushed
the
chipingress-partial-delivery-cl-nodes
branch
2 times, most recently
from
55ced63 to
32d3c31
Compare
- chip_ingress_batch_worker.Send was discarding the PublishBatch response.
- Now inspects resp.GetResults(): per-event errors are grouped by error
code and emitted as one WARN per batch ("dropped", "by_code").
- Adds chip_ingress_partial_delivery_dropped Prometheus counter with
telemetry_type and error_code labels so dashboards and alerts can track
rejection types without relying on logs.
Jira: INFOPLAT-13901
pkcll
force-pushed
the
chipingress-partial-delivery-cl-nodes
branch
from
32d3c31 to
094609f
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
chip_ingress_batch_worker.Sendwas discarding thePublishBatchresponse (_, err := ...).respand iteratesresp.GetResults(): any result with a non-nilErroris logged at WARN withevent_id,error_code,reason,telemType, andcontractID, and counted inTelemetryClientMessagesDropped.PublishBatchcall sites and surface partial-delivery errors.Jira: INFOPLAT-13901 | Epic: INFOPLAT-7177
Depends on chainlink-common PR #2210 (beholder partial delivery logging) merging first, then a
go.modbump before this PR is ready to merge.Changes
core/services/synchronization/chip_ingress_batch_worker.go: captureresp, iterateresp.GetResults()for per-event errors.core/services/synchronization/chip_ingress_batch_worker_test.go:TestChipIngressBatchWorker_Send_PartialDelivery— verifies partial delivery code path runs without panic and logs the WARN.Test plan
go test ./core/services/synchronization/... -count=1passesTestChipIngressBatchWorker_Send_PartialDeliverypasses (verifies WARN log withevent_id,error_code,reason)