Add opt-in handling for cache provider errors (1.4.0)

[skunkworker]

Summary

This PR brings the cache error handler from #16 forward onto 1.3.0.

#16 ("Handle cache failures", 2023) added :handle_cache_error and :cache_error_proc. It got approval, and it went out as the internal release 0.3.0.rc2. Nobody merged it, and 1.0.0 to 1.3.0 started from master without it. alfred still pins 0.3.0.rc2 for this handler, so it cannot move to 1.3.0 without a change in behavior.

# config/initializers/active_remote_cached.rb (alfred today)
ActiveRemote::Cached.default_options(
  :handle_cache_error => true,
  :cache_error_proc => lambda { |e| Rails.logger.error(e); Buttress.exception_notifier&.notify(e) }
)

Changes

• Cache#read, #write, #delete, #exist? and #fetch rescue a StandardError from the cache provider. When default_options[:handle_cache_error] is true, the error goes to :cache_error_proc and the call acts as a cache miss. When it is not true, the error goes to the caller, as in 1.3.0.
• An error from the fetch block (the RPC call, or RemoteRecordNotFound from a bang finder) always goes to the caller. It never goes to :cache_error_proc, and it never goes into the cache.
• delete, exist?, read and write share one private failsafe helper for the rescue. With nested caching, the nested cache and the cache provider each get their own failsafe, so an error in one does not skip the other.
• If :cache_error_proc raises, the library writes a warning to stderr, and the call continues.
• A provider error whose cause chain holds the block error counts as the block error. It goes to the caller and is not reported as a cache error.
• The fetch block runs at most once. If the provider fails after the block ran (for example, on the write), fetch returns the value from that run and does not call the RPC again.
• fetch keeps the provider fetch. Handle cache failures #16 changed it to read then write, which turned off :race_condition_ttl. This PR does not.
• The two error options go to the provider no more. Other options pass through, as in 1.3.0. This PR does not bring the option whitelist from Handle cache failures #16.
• README: a "Cache errors" section, and an "Upgrading to 1.4.0" section.
• VERSION is 1.4.0.

Design rule

A cache outage must not stop calls to the RPC services. With :handle_cache_error on, each finder call goes to the remote service while the cache is down. This increases the load on that service, and that is the expected trade-off.

Change in behavior for all apps

Both options are off by default. One change applies to every app, with or without the handler: the cleanup delete in fetch (for a nil or empty value) no longer raises. Before, a failed delete raised and the caller lost the value from the remote service. Now fetch returns the value, and the nil or empty entry stays until its TTL ends. The README "Upgrading to 1.4.0" section records this.

Evidence

• 22 new specs (cache methods, finder level, fetch block errors not cached, one block run, options not passed to the provider, no-block fetch, nested cache errors, proc errors, wrapped block errors, cleanup delete errors). The 7 code review specs fail on the earlier commit (6 of them) or prove the block path under nested caching.

• Full suite: 213 examples, 0 failures on MRI 3.4.9 with active_remote 6.1, 7.1 and 8.0, on JRuby 10.0.6.0, and on JRuby 9.4.14.0. RuboCop: 19 files, no offenses.

• Probe with an RPC error (activesupport 7.1.6): the error goes to the caller, and the cache has 0 entries, with the handler off and on.

• Probe with a cache entry that fails to deserialize:

  default:  TypeError goes to the caller
  handled:  fetch returns the RPC value, block calls=1, cache_error_proc got [TypeError]
  

After merge

• Close Handle cache failures #16 with a link to this PR.
• alfred: remove the 0.3.0.rc2 pin and the internal source block. Keep the initializer.

🤖 Generated with Claude Code