IGNITE-29211 Renew TLS certificates automatically before expiry

docs/_docs/monitoring-metrics/new-metrics.adoc

@@ -432,6 +432,20 @@ Register name: `communication.tcp.{node_consistent_id}`
 |sentMessagesToNode | long | Total number of messages sent by current node to the given node.
 |===
 
+== Certificate Renewal
+
+Register name: `ssl`
+
+[cols="2,1,3",opts="header"]
+|===
+|Name | Type | Description
+|ConsecutiveRenewalFailures | long | Sum of consecutive automatic renewal failures across the node's SSL factories. Zero when renewal is disabled.
+|ManualReloadAttempts | long | Factory reload attempts started by the node for manual commands, including errors. Excludes startup loads.
+|AutomaticRenewalAttempts | long | Automatic factory renewal calls, including errors and calls that find renewal is no longer due.
+|===
+
+See link:security/ssl-tls#automatic-certificate-renewal[Automatic Certificate Renewal] for configuration and recovery.
+
 == Communication Connection Pool
 
 Register name: `communication.tcp.connectionPool`

docs/_docs/security/ssl-tls.adoc

@@ -176,18 +176,113 @@ The command asks for confirmation; use `--yes` to skip it.
 . Check the command result and the reload logs of any failed nodes, then verify a new connection with the replacement certificates.
 Established connections continue using their existing TLS context.
 
+`SslContextFactory` checks that every certificate in each private-key chain is valid whenever it loads a key store: at node startup, on reload and during automatic renewal.
+A node with expired or not yet valid certificates does not start, and a reload of such certificates keeps the current ones.
+Client applications and `control.sh` that use `SslContextFactory` apply the same check to their own key stores.
+Replacement certificates may expire earlier than the current ones or have the same expiry date.
+You can also update the trust store without replacing the node's certificate.
+
 If a factory fails to reload, it keeps its previous context, and that node stops reloading its remaining factories.
 Successful reloads remain in effect; other nodes continue independently.
 Check the affected node's log for error details.
 Correct the failed store and run the command again.
-To restore the previous certificates, restore the backed-up files and reload again while peers still trust those certificates.
+To restore the previous certificates, restore the backed-up files and reload again.
+The restored certificates must still be valid and trusted by peers.
 
 Custom factories must extend javadoc:org.apache.ignite.ssl.AbstractSslContextFactory[].
 They must reread key and trust data from their sources on reload and supply the updated context for new connections.
 If any factory on a node has another type, the command reports it as unsupported and reloads nothing on that node.
 For these factories, follow the same trust-first sequence and restart affected nodes one at a time instead of running reload.
 If you configured `trustManagers` directly, the command reuses them instead of reloading a trust store.
 
+== Automatic Certificate Renewal
+
+Automatic renewal lets nodes load replacement certificates without a reload command.
+You must supply the replacements: Ignite reads them from the configured source and does not issue certificates.
+Renewal covers the same TCP transports as <<Upgrading Certificates,manual reload>>.
+It does not update the HTTP REST endpoint or external client applications.
+
+=== Enable Renewal
+
+Enable renewal on each configured `SslContextFactory` before starting the node:
+
+[source,java]
+----
+factory.setRenewalEnabled(true);
+----
+
+Changing this setting requires a node restart.
+The key store must contain a private key, and every private key must have an X.509 certificate chain.
+A chain contains the key's certificate and the certificates of its issuers.
+With renewal enabled, a store that does not meet these requirements prevents node startup.
+
+Before replacing key stores, prepare peers' trust stores as described in <<Upgrading Certificates>>.
+Once every affected peer trusts the replacement certificates, put the replacement key stores in the configured source.
+Keep the configured paths, store types and passwords unchanged.
+Ignite loads the replacements automatically when renewal is due; no reload command is needed for this step.
+
+=== When Renewal Runs
+
+A chain expires when any certificate in it expires, including an issuing certificate.
+Ignite selects the chain that expires first and starts renewal attempts with 15% of its lifetime remaining.
+If this point has already passed when the node starts, Ignite schedules an attempt immediately.
+See link:{javadoc_base_url}/org/apache/ignite/ssl/SslContextFactory.html#getRenewalTime()[getRenewalTime(), window=_blank] for the calculation.
+
+Automatic renewal applies the same certificate checks as <<Upgrading Certificates,manual reload>>.
+If the loaded certificates are still within their renewal window, Ignite records a failed renewal attempt even though it installed the valid material.
+This can happen when the certificates have not changed, only the trust store changed, or the replacements are already within their renewal window.
+The loaded context remains in use while Ignite retries with the intervals below.
+
+After a manual reload, Ignite schedules the next automatic attempt from the loaded certificates.
+If their renewal window has already started, the attempt runs immediately.
+
+New connections use the accepted certificates; established connections remain open.
+To confirm a replacement, open a new TLS connection to each affected endpoint and compare its peer certificate with the replacement.
+
+=== Check Failures and Recover
+
+If loading or validation fails, Ignite keeps the current certificates and retries, even after they expire.
+Keeping expired certificates does not extend their validity, so new connections may fail until a valid replacement is available.
+Retry intervals grow from 1–1.5 minutes to 60–90 minutes.
+The base delay doubles after each failure, with a random addition of up to 50%.
+
+Check the node log for the next attempt, affected transports and the reason for a failure.
+The `ssl.ConsecutiveRenewalFailures` metric sums consecutive failed automatic attempts across the node's factories.
+Each factory's count resets after a successful manual reload or an automatic attempt after which renewal is no longer due.
+
+`ssl.ManualReloadAttempts` and `ssl.AutomaticRenewalAttempts` count the node's calls to reload each factory.
+They include errors and automatic attempts that find renewal is no longer due.
+A factory shared by several transports is counted once per attempt.
+Startup loading and direct application calls to the factory are not counted.
+
+To recover without waiting for the next retry:
+
+. Correct the replacement stores and run link:tools/control-script#reloading-tls-certificates[`control.sh --ssl reload`].
+. Check the command result and verify a new connection with the replacement certificates.
+
+*What a successful command means.*
+The result `reloaded` means that the node loaded the stores and accepted their certificate dates.
+The command also reports `reloaded` when the files have not changed.
+Checking certificate dates does not establish whether peers trust the certificates or whether they have been revoked.
+Verify a new TLS connection after a replacement.
+
+*What happens to automatic renewal.*
+If enabled, automatic renewal continues after manual reload, as described in <<When Renewal Runs>>.
+A failed automatic attempt does not undo the manual reload.
+
+*If the command cannot connect.*
+First check the command's certificates and trust settings.
+If the problem is on the node, put a valid replacement in its source and wait for an automatic retry.
+To load corrected stores without the command or waiting for a retry, restart the affected nodes.
+
+=== Custom Certificate Sources
+
+For automatic renewal, custom sources require a subclass of javadoc:org.apache.ignite.ssl.SslContextFactory[].
+Factories extending only `AbstractSslContextFactory` support manual reload only.
+The certificate checks described above belong to `SslContextFactory`.
+Factories extending only the base class retain their own loading behavior.
+Limit request duration in custom sources: a blocked load can delay other renewals, manual reload and node shutdown.
+
 == SslContextFactory Properties
 
 `SslContextFactory` supports the following properties:
@@ -216,6 +311,10 @@ If you configured `trustManagers` directly, the command reuses them instead of r
 |The protocol for secure transport. https://docs.oracle.com/en/java/javase/11/docs/specs/security/standard-names.html#sslcontext-algorithms[Supported algorithms,window=_blank].
 |`TLS`
 
+|`renewalEnabled`
+|Enables automatic certificate renewal. See <<Automatic Certificate Renewal>>.
+|`false`
+
 |`trustStoreFilePath`
 |The path to the trust store file.
 |`N/A`

modules/commons/src/main/java/org/apache/ignite/ssl/AbstractSslContextFactory.java

@@ -137,6 +137,8 @@ private SSLContext createSslContext() throws SSLException {
 
             ctx.init(keyMgrs, trustMgrs, null);
 
+            acceptKeyManagers(keyMgrs);
+
             return ctx;
         }
         catch (NoSuchAlgorithmException e) {
@@ -147,6 +149,17 @@ private SSLContext createSslContext() throws SSLException {
         }
     }
 
+    /**
+     * Accepts the key managers of a new SSL context before the context replaces the current one.
+     * Does nothing by default.
+     *
+     * @param keyMgrs Key managers created by {@link #createKeyManagers()} for the new context.
+     * @throws SSLException If the new context must not replace the current one.
+     */
+    void acceptKeyManagers(KeyManager[] keyMgrs) throws SSLException {
+        // No-op.
+    }
+
     /**
      * @param param Value.
      * @param name Name.