JGroups AWS – aws.S3_PING

Projects JGroups AWS provides an implementation of aws.S3_PING discovery protocol using AWS S3 buckets as cluster information store. It is based on the original code written by Tobias Sarnowski at Zalando.

It uses the official AWS SDK and does not implement the HTTP protocol on its own. The benefit is a more stable connection as well as usage of IAM server profiles and AWS standardized credential distribution.

Maven Coordinates

<dependency>
    <groupId>org.jgroups.aws</groupId>
    <artifactId>jgroups-aws</artifactId>
    <version>4.0.2.Final</version>
</dependency>

Protocol Configuration

This library implements a JGroups discovery protocol which replaces protocols like MPING or TCPPING.

<aws.S3_PING region_name="us-east-1a"
             bucket_name="jgroups-s3-test"/>

Configurations Properties

Attribute Name System Property Environment variable	Default	Description
region_name System property: jgroups.aws.s3.region_name Environment variable: JGROUPS_AWS_S3_REGION_NAME	required	The AWS region with which to communicate.
bucket_name System property: jgroups.aws.s3.bucket_name Environment variable: JGROUPS_AWS_S3_BUCKET_NAME	required	The AWS S3 bucket name to use.
bucket_prefix System property: jgroups.aws.s3.bucket_prefix Environment variable: JGROUPS_AWS_S3_BUCKET_PREFIX		The prefix to prefix all AWS S3 paths with, e.g. jgroups/.
check_if_bucket_exists System property: jgroups.aws.s3.check_if_bucket_exists Environment variable: JGROUPS_AWS_S3_CHECK_IF_BUCKET_EXISTS	true	Whether to check if the bucket exists in AWS S3 and create a new one if it does not exist yet.
endpoint System property: jgroups.aws.s3.endpoint Environment variable: JGROUPS_AWS_S3_ENDPOINT		The AWS endpoint with which to communicate.
path_style_access_enabled System property: jgroups.aws.s3.path_style_access_enabled Environment variable: JGROUPS_AWS_S3_PATH_STYLE_ACCESS_ENABLED	false	Forces the AWS S3 client to use path-style addressing for buckets.
kms_key_id System property: jgroups.aws.s3.kms_key_id Environment variable: JGROUPS_AWS_S3_KMS_KEY_ID		KMS key to use for enabling KMS server-side encryption (SSE-KMS) for AWS S3.
acl_grant_bucket_owner_full_control System property: jgroups.aws.s3.acl_grant_bucket_owner_full_control Environment variable: JGROUPS_AWS_S3_ACL_GRANT_BUCKET_OWNER_FULL_CONTROL	false	Whether to grant the bucket owner full control over the bucket on each update. This is useful in multi-region deployments where each region exists in its own AWS account.
System property: s3ping.magic_number	790	The protocol automatically registers itself to JGroups with the magic number 790. This can be overwritten by setting the system property to different number.

Note	The cluster name is used as part of the S3 object key, therefore the S3 object key naming restrictions also apply to the JGroups cluster name when using this protocol.

Note	This protocol implementation extends FILE_PING and inherits its configuration, however, since version 4.0.1.Final, the register_shutdown_hook is set to false by default.

Note	System properties are supported since JGroups AWS release version 3.0.1.Final. To use properties in the previous version, the property must be explicitly defined in the configuration itself, e.g. <aws.S3_PING region_name="${jgroups.aws.s3.region_name}" …​ />.

Note

Be mindful of how JGroups resolves values at runtime when using system properties or environment variables. For example, consider the configuration <aws.S3_PING bucket_name="${my.bucket.property:myBucketDefaultValue}" …​ />. JGroups will first attempt to resolve the my.bucket.property property. If it is not defined, the default value myBucketDefaultValue will be used. If no default value is specified, JGroups will then attempt to resolve jgroups.aws.s3.bucket_name system property, followed by the environment variable JGROUPS_AWS_S3_BUCKET_NAME.

Example Stack

<!--
Based on default tcp.xml with discovery replaced with aws.S3_PING.
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups-5.5.xsd">
    <include file="${transport-config:tcp-default.xml}"/>
    <aws.S3_PING/>
    <MERGE3 min_interval="10s"
            max_interval="30s"/>
    <FD_SOCK2/>
    <FD_ALL3 timeout="40s"
             interval="5s"/>
    <VERIFY_SUSPECT2 timeout="1.5s"/>
    <BARRIER/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE desired_avg_gossip="50s"
                   max_bytes="4M"/>
    <pbcast.GMS print_local_addr="true"
                join_timeout="2s"/>
    <MFC max_credits="10M"
         min_threshold="0.4"/>
    <UFC max_credits="10M"
         min_threshold="0.4"/>
    <FRAG2 frag_size="60K"/>
    <pbcast.STATE_TRANSFER/>
</config>

Credentials

The protocol does not provide any properties for configuring AWS credentials. Instead, credentials are resolved using the AWS SDK default credentials provider chain, which looks for credentials in the following locations, in order:

1. Java system properties (aws.accessKeyId, aws.secretAccessKey and aws.sessionToken)

2. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN)

3. Web identity token (e.g. IAM roles for service accounts on Amazon EKS)

4. Shared credentials and config files (~/.aws/credentials and ~/.aws/config)

5. Amazon ECS container credentials

6. Amazon EC2 instance profile credentials (IAM role attached to the instance)

When running on AWS infrastructure, using IAM roles is recommended over static credentials.

For the full and up-to-date description of the provider chain and all supported options, refer to the official documentation: AWS SDK for Java 2.x – Default credentials provider chain.

AWS SDK Version Policy

The AWS SDK for Java 2.x publishes new patch releases almost daily, which this project does not track with its own releases. Instead, the project continuously consumes the latest AWS SDK minor version, and each release is built and tested against the latest version available at the time of release. This version is the endorsed version for that release.

Users are encouraged to update the AWS SDK independently to newer versions as needed (e.g. to pick up fixes or address security vulnerabilities) without waiting for a new release of this project. To override the AWS SDK version, import the AWS SDK BOM in your project’s dependency management:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>bom</artifactId>
            <version>${aws.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Note	AWS SDK minor version updates can change default behavior, which can affect third-party S3-compatible storage configured via the endpoint property. When reporting issues, please include the AWS SDK version in use.

Testing

Running the automated tests requires having AWS credentials setup with appropriate permissions along with setting the region name and a bucket name.

declare -x AWS_ACCESS_KEY_ID="..."
declare -x AWS_SECRET_ACCESS_KEY="..."
declare -x JGROUPS_AWS_S3_REGION_NAME="eu-central-1"
declare -x JGROUPS_AWS_S3_BUCKET_NAME="jgroups"
./mvnw verify

If the required AWS credentials are not specified, testing with AWS S3 will be skipped (uses org.junit.jupiter.api.Assumptions).

In case credentials are not provided and running on Linux, tests will be run against mock containerized S3 instance. These require a functioning podman or Docker environment.

Reporting Issues

Project JGroups AWS uses GitHub Issues for tracking:

https://github.com/jgroups-extras/jgroups-aws/issues

Support Matrix

Version/Branch	JGroups Version	AWS SDK Version	Base JDK Version
4.x/main	5.5.x	2.x	17
3.1.x	5.4.x	2.x	11
3.0.x	5.2.x – 5.3.x	2.x	11
2.x	5.2.x	1.x	11
1.x	4.2.x	1.x	8