pdpv0: pruning `message_waits_eth` to handle stuck messages

[ZenGround0]

See discussion here: #937 (comment).

[rjan90]

Posting int Zen´s comment from the 937 PR here:

Yeah as you'd expect there's nothing I can see preventing massive hangs. The problem of infinitely stuck pending transactions should be solved at the level of the message_waits_eth subsystem. One proposal would be to include a new status "dropped" alongside "pending" and "confirmed" that we periodically assign to transactions that have been hanging for a long time. Then the settlement watcher task can handle submitting retries for these hanging messages.

I would hold off on doing this until we observe or otherwise suspect the frequent existance of large scale mpool hanging as a significant issue. The fact that the PoRep message_waits table doesn't deal with the stuck-in-mpool case more directly makes me suspicious that lower down in the message sending system we have strategies to avoid this that generally work. But since we are tracking by message cids across the board I suspect that we don't fully have premium / fee cap increase logic yet.

I will bump the milestone here back to M4.2: Mainnet GA to allow more time to observe or suspect frequent existence of large scale mpool hanging). If we do not observe any such cases before GA, we will postpone this to after GA

[rvagg]

Discussed with Zen: simpler first step is to just prune entries >1 month old (or some amount) based on block number. This addresses table growth and implicitly handles ancient stuck messages. We thought that we'd at leasta prepare a PR for it for discussion, I don't recall if there was a critical reason this table didn't have a clean-up.

The more sophisticated "dropped" status + retry logic can wait until we observe it's actually needed.

[TippyFlitsUK]

Experiencing this on mainnet. A single stuck tx in message_waits_eth has generated over 11K error log lines in the last 24 hours. Manually marking it as failed to stop the noise. +1 for the "dropped" status proposal.

[BigLep]

2026-04-09 conversation:

• we have refocused this to be about pruning old messages
• We have put it post-GA unless it comes up again before then.
• We discussed about doing this in a migration, but agreed this should be "another cleanup task" (look for other db cleanups) that we can model from.