Fix tier2/live-backfiller crash, race, leaks and cursor-resolver cancellation

[sduchesneau]

Fixes

Five confirmed bugs in the service layer (plus one small contained fix in pipeline/resolve.go):

1. tier2: panic on nil Modules (service/tier2.go)
   The nil-Modules guard in ProcessRange built and logged an InvalidArgument error but was missing the return, falling through to a nil pointer dereference on request.Modules.Modules. Added the return.

2. tier2: concurrent map iteration and write in GetExecutionPlan (service/tier2.go)
   The cleanup goroutine iterates existingExecOuts on ctx.Done() while the main loop is still populating the map — a fatal runtime error that kills the whole tier2 process. Both sides now synchronize on a mutex; exec output readers are still closed on cancellation.

3. tier2: reversed errors.Is arguments (service/tier2.go)
   errors.Is(context.Canceled, streamErr) never matches wrapped cancellations, so context.Cause(ctx) (e.g. errShuttingDown) was lost and the wrong gRPC code was returned. Arguments swapped.

4. live back filler: connection and goroutine leaks (service/live_back_filler.go)

   • requestBackProcessing leaked the tier2 client connection on both error returns (cleanup defer was registered after them) and closed it twice on success. closeFunc/CloseSend are now deferred right after their resources are acquired.
   • RequestBackProcessing blocked forever sending on the unbuffered jobResult channel once Start had returned on ctx.Done(). The send now selects on ctx.Done(), and Start buffers the channel (capacity 1).

5. cursor resolver: cancellation converted into undo-to-LIB signal (pipeline/resolve.go)
   Fixes Context canceled (shutting down) may cause UNDO signal to be sent #399 residual: on cancellation, <-src.Terminated() can win the select over <-ctx.Done() and the error fallback then emitted an undo-to-LIB signal instead of propagating the context error. The Terminated branch now checks ctx.Err() and returns it.

Tests

All new tests were verified to fail against the unfixed code:

• TestProcessRange_NilModules — panicked (nil deref) before, returns InvalidArgument after.
• TestGetExecutionPlan_CancelledContextNoMapRace — WARNING: DATA RACE under -race before, clean after.
• TestRequestBackProcessing_ConnectionClosed — closeFunc called 0 times on error paths / 2 times on success before; exactly once on every path after.
• TestRequestBackProcessing_NoGoroutineLeakOnCancel — blocked >5s before, returns immediately after.
• TestCursorResolver_CancellationDoesNotEmitUndo — got junction=#100 (aa) (undo signal) before, always context.Canceled after.

go test -race ./service/... ./pipeline/... passes.

🤖 Generated with Claude Code

[sduchesneau]

🔍 Vulnerabilities of ghcr.io/streamingfast/substreams:a470c69

📦 Image Reference ghcr.io/streamingfast/substreams:a470c69

digest	sha256:12b9b685519fd198dbf9e2777031a032de421fe64d0a6369532049634902eb25
vulnerabilities
platform	linux/amd64
size	108 MB
packages	353

📦 Base Image ubuntu:24.04

also known as	noble noble-20260610
digest	sha256:52df9b1ee71626e0088f7d400d5c6b5f7bb916f8f0c82b474289a4ece6cf3faf
vulnerabilities