From a75bb26697cd62b2ac7bc0222e7bf0f0db6a2a8a Mon Sep 17 00:00:00 2001 From: David Pilato Date: Mon, 31 Aug 2026 15:48:00 +0200 Subject: [PATCH] Bump Elasticsearch test images and cover current behavior on latest Use **7.17.29**, **8.19.20** and **9.5.2**, with LATEST aliases pointing at 9.x so the next major is a pointer change. Exercise TLS, auth, heap and Docker Hub on the latest image; keep 7.x, OSS 7.10.2 and non-semantic :latest tag cases only where behavior actually differs. Restructure and rename Elasticsearch and Kibana tests accordingly, add AssertJ descriptions where a failure would otherwise be ambiguous, and rewrite the module docs so HTTPS on latest is the primary example. Align the test REST client and custom heap JVM options with those versions. --- docs/modules/elasticsearch.md | 54 +- modules/elasticsearch/build.gradle | 2 +- .../elasticsearch/KibanaContainer.java | 4 +- .../ElasticsearchContainerTest.java | 602 ++++++++---------- .../elasticsearch/KibanaContainerTest.java | 392 ++++++------ .../resources/test-custom-memory-jvm.options | 4 +- 6 files changed, 525 insertions(+), 533 deletions(-) diff --git a/docs/modules/elasticsearch.md b/docs/modules/elasticsearch.md index 5817a21de02..d7b8b551f86 100644 --- a/docs/modules/elasticsearch.md +++ b/docs/modules/elasticsearch.md @@ -7,27 +7,53 @@ Note that it's based on the [official Docker image](https://www.elastic.co/guide ## Usage example -You can start an elasticsearch container instance from any Java application by using: +In the following examples, we will be using the following versions: -[HttpClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientContainer7 -[HttpClient with Elasticsearch 8](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientContainer8 -[HttpClient with Elasticsearch 8 and SSL disabled](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientContainerNoSSL8 -[TransportClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:transportClientContainer +[Version 9 (recommended)](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:version_9 +[Version 8 (maintained)](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:version_8 +[Version 7 (deprecated)](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:version_7 +From Elasticsearch 8 onwards, security and HTTPS are enabled by default. You can start a container and talk to it +with the REST client as follows: -Note that if you are still using the [TransportClient](https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html) -(not recommended as it is deprecated), the default cluster name is set to `docker-cluster` so you need to change `cluster.name` setting -or set `client.transport.ignore_cluster_name` to `true`. + +[HttpClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientLatest + + +### Disable TLS + +HTTPS can be turned off if you do not need it: + + +[HttpClient with TLS disabled](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientTlsDisabled + + +### Elasticsearch 7 (deprecated) + +Elasticsearch 7 listens on HTTP and does not enable security unless you opt in with `withPassword()`. + + +[HttpClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientV7 +[HttpClient with security enabled](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientV7Secured + + +The [TransportClient](https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html) +has been removed in Elasticsearch 8. It can still be used against a 7.x container. The default cluster name is +`docker-cluster`, so you need to change the `cluster.name` setting or set `client.transport.ignore_cluster_name` to `true`. + + +[TransportClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:transportClientV7 + -## Secure your Elasticsearch cluster +### OSS distribution -The default distribution of Elasticsearch comes with the basic license which contains security feature. -You can turn on security by providing a password: +The last OSS image is `elasticsearch-oss:7.10.2`. It does not include features under the Elastic License, and +`withPassword()` is rejected: -[HttpClient](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:httpClientSecuredContainer +[OSS image](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java) inside_block:ossContainer ## Kibana container @@ -40,7 +66,7 @@ Kibana requires a connection to Elasticsearch and `KibanaContainer` supports two In managed mode, `KibanaContainer` automatically connects to an `ElasticsearchContainer`: -[Kibana with Elasticsearch](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java) inside_block:managedModeCanStartAndReachElasticsearchInSameExplicitNetwork +[Kibana with Elasticsearch](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java) inside_block:managedModeReachesElasticsearchOnSharedNetwork When using managed mode with explicit networks, both containers must share the same `Network` instance. @@ -51,7 +77,7 @@ Alternatively, you can omit the network configuration entirely, and `KibanaConta In external mode, `KibanaContainer` connects to an external Elasticsearch instance via URL and using provided credentials: -[Kibana with external Elasticsearch](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java) inside_block:externalModeCanWorkWithUsernamePassword +[Kibana with external Elasticsearch](../../modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java) inside_block:externalModeReachesElasticsearchWithUsernamePassword For external mode with HTTPS, use `withElasticsearchCaCertificate()` to provide the CA certificate. diff --git a/modules/elasticsearch/build.gradle b/modules/elasticsearch/build.gradle index 7f57916e991..3134e201e02 100644 --- a/modules/elasticsearch/build.gradle +++ b/modules/elasticsearch/build.gradle @@ -3,6 +3,6 @@ description = "Testcontainers :: elasticsearch" dependencies { api project(':testcontainers') - testImplementation "org.elasticsearch.client:elasticsearch-rest-client:9.4.3" + testImplementation "org.elasticsearch.client:elasticsearch-rest-client:9.5.2" testImplementation "org.elasticsearch.client:transport:7.17.29" } diff --git a/modules/elasticsearch/src/main/java/org/testcontainers/elasticsearch/KibanaContainer.java b/modules/elasticsearch/src/main/java/org/testcontainers/elasticsearch/KibanaContainer.java index ee8ce714854..7dd94728561 100644 --- a/modules/elasticsearch/src/main/java/org/testcontainers/elasticsearch/KibanaContainer.java +++ b/modules/elasticsearch/src/main/java/org/testcontainers/elasticsearch/KibanaContainer.java @@ -33,9 +33,9 @@ * Supports two modes: * *

*/ diff --git a/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java b/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java index 52cd78b4a34..a63b5455738 100644 --- a/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java +++ b/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/ElasticsearchContainerTest.java @@ -37,13 +37,42 @@ class ElasticsearchContainerTest { /** - * Elasticsearch version which should be used for the Tests + * Elasticsearch 7.x version to use in tests. */ - private static final String ELASTICSEARCH_VERSION = "7.9.2"; + // version_7 { + static final String ELASTICSEARCH_VERSION_7 = "7.17.29"; + static final DockerImageName ELASTICSEARCH_IMAGE_7 = DockerImageName.parse( + "docker.elastic.co/elasticsearch/elasticsearch") + .withTag(ELASTICSEARCH_VERSION_7); + // } - private static final DockerImageName ELASTICSEARCH_IMAGE = DockerImageName - .parse("docker.elastic.co/elasticsearch/elasticsearch") - .withTag(ELASTICSEARCH_VERSION); + /** + * Elasticsearch 8.x version to use in tests. + */ + // version_8 { + static final String ELASTICSEARCH_VERSION_8 = "8.19.20"; + static final DockerImageName ELASTICSEARCH_IMAGE_8 = DockerImageName.parse( + "docker.elastic.co/elasticsearch/elasticsearch") + .withTag(ELASTICSEARCH_VERSION_8); + // } + + /** + * Elasticsearch 9.x version to use in tests. + */ + // version_9 { + static final String ELASTICSEARCH_VERSION_9 = "9.5.2"; + static final DockerImageName ELASTICSEARCH_IMAGE_9 = DockerImageName.parse( + "docker.elastic.co/elasticsearch/elasticsearch") + .withTag(ELASTICSEARCH_VERSION_9); + // } + + /** + * Latest Elasticsearch version exercised by default in these tests. + * Point this at the next major when adding coverage for it. + */ + static final String ELASTICSEARCH_VERSION_LATEST = ELASTICSEARCH_VERSION_9; + + static final DockerImageName ELASTICSEARCH_IMAGE_LATEST = ELASTICSEARCH_IMAGE_9; /** * Elasticsearch default username, when secured @@ -51,7 +80,7 @@ class ElasticsearchContainerTest { private static final String ELASTICSEARCH_USERNAME = "elastic"; /** - * From 6.8, we can optionally activate security with a default password. + * Default password used by ElasticsearchContainer for versions >= 8, and by withPassword() in these tests. */ private static final String ELASTICSEARCH_PASSWORD = "changeme"; @@ -71,121 +100,13 @@ public void stopRestClient() throws IOException { } } - @SuppressWarnings("deprecation") // Using deprecated constructor for verification of backwards compatibility - @Test - @Deprecated // We will remove this test in the future - void elasticsearchDeprecatedCtorTest() throws IOException { - // Create the elasticsearch container. - try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE).withEnv("foo", "bar") // dummy env for compiler checking correct generics usage - ) { - // Start the container. This step might take some time... - container.start(); - - // Do whatever you want with the rest client ... - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains(ELASTICSEARCH_VERSION); - - // The default image is running with the features under Elastic License - response = getClient(container).performRequest(new Request("GET", "/_xpack/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - // For now we test that we have the monitoring feature available - assertThat(EntityUtils.toString(response.getEntity())).contains("monitoring"); - } - } - - @Test - void elasticsearchDefaultTest() throws IOException { - // Create the elasticsearch container. - try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE).withEnv("foo", "bar") // dummy env for compiler checking correct generics usage - ) { - // Start the container. This step might take some time... - container.start(); - - // Do whatever you want with the rest client ... - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains(ELASTICSEARCH_VERSION); - - // The default image is running with the features under Elastic License - response = getClient(container).performRequest(new Request("GET", "/_xpack/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - // For now we test that we have the monitoring feature available - assertThat(EntityUtils.toString(response.getEntity())).contains("monitoring"); - } - } - - @Test - void elasticsearchSecuredTest() throws IOException { - try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE) - .withPassword(ELASTICSEARCH_PASSWORD) - ) { - container.start(); - - // The cluster should be secured so it must fail when we try to access / without credentials - assertThat(catchThrowable(() -> getAnonymousClient(container).performRequest(new Request("GET", "/")))) - .as("We should not be able to access / URI with an anonymous client.") - .isInstanceOf(ResponseException.class); - - // But it should work when we try to access / with the proper login and password - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains(ELASTICSEARCH_VERSION); - } - } - - @Test - void elasticsearchVersion() throws IOException { - try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)) { - container.start(); - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - String responseAsString = EntityUtils.toString(response.getEntity()); - assertThat(responseAsString).contains(ELASTICSEARCH_VERSION); - } - } + // Latest (default coverage) @Test - void elasticsearchVersion83() throws IOException { - try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.3.0" - ) - ) { - container.start(); - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains("8.3.0"); - } - } - - @Test - void elasticsearchOssImage() throws IOException { - try ( - // ossContainer { - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2" - ) - // } - ) { - container.start(); - Response response = getClient(container).performRequest(new Request("GET", "/")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - // The OSS image does not have any feature under Elastic License - assertThat(catchThrowable(() -> getClient(container).performRequest(new Request("GET", "/_xpack/")))) - .as("We should not have /_xpack endpoint with an OSS License") - .isInstanceOf(ResponseException.class); - } - } - - @Test - void restClientClusterHealth() throws IOException { - // httpClientContainer7 { + void latestStartsWithTlsAndPassword() throws IOException { + // httpClientLatest { // Create the elasticsearch container. - try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)) { + try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST)) { // Start the container. This step might take some time... container.start(); @@ -198,47 +119,10 @@ void restClientClusterHealth() throws IOException { client = RestClient - .builder(HttpHost.create(container.getHttpHostAddress())) - .setHttpClientConfigCallback(httpClientBuilder -> { - return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider); - }) - .build(); - - Response response = client.performRequest(new Request("GET", "/_cluster/health")); - // }} - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); - // httpClientContainer7 {{ - } - // } - } - - @Test - void restClientClusterHealthElasticsearch8() throws IOException { - // httpClientContainer8 { - // Create the elasticsearch container. - try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) - ) { - // Start the container. This step might take some time... - container.start(); - - // Do whatever you want with the rest client ... - final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); - credentialsProvider.setCredentials( - AuthScope.ANY, - new UsernamePasswordCredentials(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD) - ); - - client = - RestClient - // use HTTPS for Elasticsearch 8 .builder(HttpHost.create("https://" + container.getHttpHostAddress())) .setHttpClientConfigCallback(httpClientBuilder -> { httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider); - // SSL is activated by default in Elasticsearch 8 + // SSL is activated by default in Elasticsearch 8+ httpClientBuilder.setSSLContext(container.createSslContextFromCa()); return httpClientBuilder; }) @@ -246,21 +130,36 @@ void restClientClusterHealthElasticsearch8() throws IOException { Response response = client.performRequest(new Request("GET", "/_cluster/health")); // }} - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); + assertThat(response.getStatusLine().getStatusCode()).as("cluster health is available").isEqualTo(200); assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); - // httpClientContainer8 {{ + assertThat(container.getHttpScheme()).as("HTTP API uses HTTPS by default").isEqualTo("https"); + + response = client.performRequest(new Request("GET", "/")); + assertThat(response.getStatusLine().getStatusCode()).as("root endpoint is available").isEqualTo(200); + assertThat(EntityUtils.toString(response.getEntity())) + .as("reported version matches the latest image") + .contains(ELASTICSEARCH_VERSION_LATEST); + + response = client.performRequest(new Request("GET", "/_xpack/")); + assertThat(response.getStatusLine().getStatusCode()).as("xpack API is available").isEqualTo(200); + assertThat(EntityUtils.toString(response.getEntity())) + .as("Elastic licensed features are available") + .contains("monitoring"); + + assertThat(catchThrowable(() -> getAnonymousClient(container).performRequest(new Request("GET", "/")))) + .as("anonymous requests are rejected") + .isInstanceOf(ResponseException.class); + // httpClientLatest {{ } // } } @Test - void restClientClusterHealthElasticsearch8WithoutSSL() throws IOException { - // httpClientContainerNoSSL8 { + void latestCanDisableTls() throws IOException { + // httpClientTlsDisabled { // Create the elasticsearch container. try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) // disable SSL .withEnv("xpack.security.transport.ssl.enabled", "false") .withEnv("xpack.security.http.ssl.enabled", "false") @@ -285,169 +184,180 @@ void restClientClusterHealthElasticsearch8WithoutSSL() throws IOException { Response response = client.performRequest(new Request("GET", "/_cluster/health")); // }} - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); + assertThat(response.getStatusLine().getStatusCode()).as("cluster health is available").isEqualTo(200); assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); - // httpClientContainerNoSSL8 {{ + assertThat(container.getHttpScheme()).as("HTTP API uses HTTP when TLS is disabled").isEqualTo("http"); + // httpClientTlsDisabled {{ } // } } @Test - void restClientSecuredClusterHealth() throws IOException { - // httpClientSecuredContainer { - // Create the elasticsearch container. + void latestRejectsMismatchedCa() throws Exception { + final MountableFile mountableFile = MountableFile.forClasspathResource("http_ca.crt"); + String caPath = "/tmp/http_ca.crt"; try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE) - // With a password - .withPassword(ELASTICSEARCH_PASSWORD) + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withCopyToContainer(mountableFile, caPath) + .withCertPath(caPath) ) { - // Start the container. This step might take some time... container.start(); - // Create the secured client. - final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); - credentialsProvider.setCredentials( - AuthScope.ANY, - new UsernamePasswordCredentials(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD) - ); + assertThat(catchThrowable(() -> getClusterHealth(container))) + .as("TLS handshake fails when the configured CA does not match the cluster") + .isInstanceOf(SSLHandshakeException.class); + } + } - client = - RestClient - .builder(HttpHost.create(container.getHttpHostAddress())) - .setHttpClientConfigCallback(httpClientBuilder -> { - return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider); - }) - .build(); + @Test + void latestHonorsCustomHttpsWaitStrategy() throws Exception { + final HttpWaitStrategy httpsWaitStrategy = Wait + .forHttps("/") + .forPort(9200) + .forStatusCode(200) + .withBasicCredentials(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD) + // trusting self-signed certificate + .allowInsecure(); - Response response = client.performRequest(new Request("GET", "/_cluster/health")); - // }} - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); - // httpClientSecuredContainer {{ + try ( + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .waitingFor(httpsWaitStrategy) + ) { + container.start(); + + assertClusterHealthResponse(container); } - // } } - @SuppressWarnings("deprecation") // The TransportClient will be removed in Elasticsearch 8. @Test - void transportClientClusterHealth() { - // transportClientContainer { - // Create the elasticsearch container. - try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)) { - // Start the container. This step might take some time... + void latestWorksWithDockerHubImage() throws Exception { + try ( + ElasticsearchContainer container = new ElasticsearchContainer( + "elasticsearch:" + ELASTICSEARCH_VERSION_LATEST + ) + ) { container.start(); - // Do whatever you want with the transport client - TransportAddress transportAddress = new TransportAddress(container.getTcpHost()); - String expectedClusterName = "docker-cluster"; - Settings settings = Settings.builder().put("cluster.name", expectedClusterName).build(); - try ( - TransportClient transportClient = new PreBuiltTransportClient(settings) - .addTransportAddress(transportAddress) - ) { - ClusterHealthResponse healths = transportClient.admin().cluster().prepareHealth().get(); - String clusterName = healths.getClusterName(); - // }}} - assertThat(clusterName).isEqualTo(expectedClusterName); - // transportClientContainer {{{ - } + assertClusterHealthResponse(container); } - // } } @Test - void incompatibleSettingsTest() { - // The OSS image can not use security feature - assertThat( - catchThrowable(() -> { - new ElasticsearchContainer("docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2") - .withPassword("foo"); - }) - ) - .as("We should not be able to activate security with an OSS License") - .isInstanceOf(IllegalArgumentException.class); + void latestDefaultHeapIsTwoGb() throws Exception { + // Default is 2g, see https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html + long defaultHeapSize = 2_147_483_648L; + + try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST)) { + container.start(); + assertElasticsearchContainerHasHeapSize(container, defaultHeapSize); + } } @Test - void testDockerHubElasticsearch8ImageSecureByDefault() throws Exception { - try (ElasticsearchContainer container = new ElasticsearchContainer("elasticsearch:8.1.2")) { + void latestHeapCanBeSetViaEnv() throws Exception { + String customHeapSize = "1500m"; + try ( + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withEnv("ES_JAVA_OPTS", String.format("-Xms%s -Xmx%s", customHeapSize, customHeapSize)) + ) { container.start(); - - assertClusterHealthResponse(container); + assertElasticsearchContainerHasHeapSize(container, 1_572_864_000L); } } @Test - void testElasticsearch8SecureByDefaultCustomCaCertFails() throws Exception { - final MountableFile mountableFile = MountableFile.forClasspathResource("http_ca.crt"); - String caPath = "/tmp/http_ca.crt"; + void latestHeapCanBeSetViaJvmOptionsFile() throws Exception { try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) - .withCopyToContainer(mountableFile, caPath) - .withCertPath(caPath) + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withClasspathResourceMapping( + "test-custom-memory-jvm.options", + "/usr/share/elasticsearch/config/jvm.options.d/a-user-defined-jvm.options", + BindMode.READ_ONLY + ); ) { container.start(); - - // this is expected, as a different cert is used for creating the SSL context - assertThat(catchThrowable(() -> getClusterHealth(container))) - .as( - "PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors" - ) - .isInstanceOf(SSLHandshakeException.class); + assertElasticsearchContainerHasHeapSize(container, 1_572_864_000L); } } + // Elasticsearch 8 (maintained) + @Test - void testElasticsearch8SecureByDefaultHttpWaitStrategy() throws Exception { - final HttpWaitStrategy httpsWaitStrategy = Wait - .forHttps("/") - .forPort(9200) - .forStatusCode(200) - .withBasicCredentials(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD) - // trusting self-signed certificate - .allowInsecure(); + void v8StartsWithDefaults() throws IOException { + try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_8)) { + container.start(); + Response response = getClient(container).performRequest(new Request("GET", "/")); + assertThat(response.getStatusLine().getStatusCode()).as("root endpoint is available").isEqualTo(200); + assertThat(EntityUtils.toString(response.getEntity())) + .as("reported version matches the 8.x image") + .contains(ELASTICSEARCH_VERSION_8); + assertThat(container.getHttpScheme()).as("HTTP API uses HTTPS by default").isEqualTo("https"); + } + } - try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) - .waitingFor(httpsWaitStrategy) - ) { + // Elasticsearch 7 (deprecated) + + @Test + void v7UsesHttpWithoutSecurityByDefault() throws IOException { + // httpClientV7 { + // Create the elasticsearch container. + try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_7)) { // Start the container. This step might take some time... container.start(); - assertClusterHealthResponse(container); + client = RestClient.builder(HttpHost.create(container.getHttpHostAddress())).build(); + + Response response = client.performRequest(new Request("GET", "/_cluster/health")); + // }} + assertThat(response.getStatusLine().getStatusCode()).as("cluster health is available").isEqualTo(200); + assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); + assertThat(container.getHttpScheme()).as("HTTP API uses HTTP by default on 7.x").isEqualTo("http"); + // httpClientV7 {{ } + // } } @Test - void testElasticsearch8SecureByDefaultFailsSilentlyOnLatestImages() throws Exception { - // this test exists for custom images by users that use the `latest` tag - // even though the version might be older than version 8 - // this tags an old 7.x version as :latest - tagImage("docker.elastic.co/elasticsearch/elasticsearch:7.9.2", "elasticsearch-tc-older-release", "latest"); - DockerImageName image = DockerImageName - .parse("elasticsearch-tc-older-release:latest") - .asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch"); - - try (ElasticsearchContainer container = new ElasticsearchContainer(image)) { + void v7EnablesSecurityWithPassword() throws IOException { + // httpClientV7Secured { + // Create the elasticsearch container. + try ( + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_7) + // With a password + .withPassword(ELASTICSEARCH_PASSWORD) + ) { + // Start the container. This step might take some time... container.start(); - Response response = getClient(container).performRequest(new Request("GET", "/_cluster/health")); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); + // Create the secured client. + final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); + credentialsProvider.setCredentials( + AuthScope.ANY, + new UsernamePasswordCredentials(ELASTICSEARCH_USERNAME, ELASTICSEARCH_PASSWORD) + ); + + client = + RestClient + .builder(HttpHost.create(container.getHttpHostAddress())) + .setHttpClientConfigCallback(httpClientBuilder -> { + return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider); + }) + .build(); + + Response response = client.performRequest(new Request("GET", "/_cluster/health")); + // }} + assertThat(response.getStatusLine().getStatusCode()).as("cluster health is available").isEqualTo(200); assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); + // httpClientV7Secured {{ } + // } } @Test - void testElasticsearch7CanHaveSecurityEnabledAndUseSslContext() throws Exception { + void v7SupportsCustomTlsCertificates() throws Exception { String customizedCertPath = "/usr/share/elasticsearch/config/certs/http_ca_customized.crt"; try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:7.17.15" - ) + ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_7) .withPassword(ElasticsearchContainer.ELASTICSEARCH_DEFAULT_PASSWORD) .withEnv("xpack.security.enabled", "true") .withEnv("xpack.security.http.ssl.enabled", "true") @@ -484,43 +394,90 @@ void testElasticsearch7CanHaveSecurityEnabledAndUseSslContext() throws Exception } } + @SuppressWarnings("deprecation") // The TransportClient will be removed in Elasticsearch 8. @Test - void testElasticsearchDefaultMaxHeapSize() throws Exception { - long defaultHeapSize = 2147483648L; - - try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)) { + void v7TransportClientCanQueryClusterHealth() { + // transportClientV7 { + // Create the elasticsearch container. + try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_7)) { + // Start the container. This step might take some time... container.start(); - assertElasticsearchContainerHasHeapSize(container, defaultHeapSize); + + // Do whatever you want with the transport client + TransportAddress transportAddress = new TransportAddress(container.getTcpHost()); + String expectedClusterName = "docker-cluster"; + Settings settings = Settings.builder().put("cluster.name", expectedClusterName).build(); + try ( + TransportClient transportClient = new PreBuiltTransportClient(settings) + .addTransportAddress(transportAddress) + ) { + ClusterHealthResponse healths = transportClient.admin().cluster().prepareHealth().get(); + String clusterName = healths.getClusterName(); + // }}} + assertThat(clusterName) + .as("TransportClient sees the default docker-cluster name") + .isEqualTo(expectedClusterName); + // transportClientV7 {{{ + } } + // } } - @Test - void testElasticsearchCustomMaxHeapSizeInEnvironmentVariable() throws Exception { - long customHeapSize = 1574961152; + // OSS 7.10.2 + @Test + void ossImageHasNoXpackEndpoint() throws IOException { try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE) - .withEnv("ES_JAVA_OPTS", String.format("-Xms%d -Xmx%d", customHeapSize, customHeapSize)) + // ossContainer { + ElasticsearchContainer container = new ElasticsearchContainer( + "docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2" + ) + // } ) { container.start(); - assertElasticsearchContainerHasHeapSize(container, customHeapSize); + Response response = getClient(container).performRequest(new Request("GET", "/")); + assertThat(response.getStatusLine().getStatusCode()).as("OSS image starts").isEqualTo(200); + // The OSS image does not have any feature under Elastic License + assertThat(catchThrowable(() -> getClient(container).performRequest(new Request("GET", "/_xpack/")))) + .as("OSS image does not expose the /_xpack endpoint") + .isInstanceOf(ResponseException.class); } } @Test - void testElasticsearchCustomMaxHeapSizeInJvmOptionsFile() throws Exception { - long customHeapSize = 1574961152; + void ossImageRejectsPassword() { + // The OSS image can not use security feature + assertThat( + catchThrowable(() -> { + new ElasticsearchContainer("docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2") + .withPassword("foo"); + }) + ) + .as("password cannot be set on an OSS image") + .isInstanceOf(IllegalArgumentException.class); + } - try ( - ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE) - .withClasspathResourceMapping( - "test-custom-memory-jvm.options", - "/usr/share/elasticsearch/config/jvm.options.d/a-user-defined-jvm.options", - BindMode.READ_ONLY - ); - ) { + // Non-semantic image tags + + @Test + void nonSemanticLatestTagStillStarts() throws Exception { + // Users sometimes tag custom or older images as :latest. The version part is then not + // a semantic version, but ComparableVersion still treats it as >= 8.0.0, so the + // container applies the 8+ defaults. Starting a 7.x image under that tag must still work + // (the CA cert is missing and is ignored). + tagImage(ELASTICSEARCH_IMAGE_7.asCanonicalNameString(), "elasticsearch-tc-older-release", "latest"); + DockerImageName image = DockerImageName + .parse("elasticsearch-tc-older-release:latest") + .asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch"); + + try (ElasticsearchContainer container = new ElasticsearchContainer(image)) { container.start(); - assertElasticsearchContainerHasHeapSize(container, customHeapSize); + + Response response = getClient(container).performRequest(new Request("GET", "/_cluster/health")); + assertThat(response.getStatusLine().getStatusCode()) + .as("cluster health is available with a non-semantic :latest tag") + .isEqualTo(200); + assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); } } @@ -582,7 +539,17 @@ private RestClient getClient(ElasticsearchContainer container) { private RestClient getAnonymousClient(ElasticsearchContainer container) { if (anonymousClient == null) { - anonymousClient = RestClient.builder(HttpHost.create(container.getHttpHostAddress())).build(); + String protocol = container.caCertAsBytes().isPresent() ? "https://" : "http://"; + anonymousClient = + RestClient + .builder(HttpHost.create(protocol + container.getHttpHostAddress())) + .setHttpClientConfigCallback(httpClientBuilder -> { + if (container.caCertAsBytes().isPresent()) { + httpClientBuilder.setSSLContext(container.createSslContextFromCa()); + } + return httpClientBuilder; + }) + .build(); } return anonymousClient; @@ -592,47 +559,18 @@ private void assertElasticsearchContainerHasHeapSize(ElasticsearchContainer cont throws Exception { Response response = getClient(container).performRequest(new Request("GET", "/_nodes/_all/jvm")); String responseBody = EntityUtils.toString(response.getEntity()); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); - assertThat(responseBody).contains("\"heap_init_in_bytes\":" + heapSizeInBytes); - assertThat(responseBody).contains("\"heap_max_in_bytes\":" + heapSizeInBytes); + assertThat(response.getStatusLine().getStatusCode()).as("nodes JVM stats are available").isEqualTo(200); + assertThat(responseBody) + .as("initial heap size matches the configured value") + .contains("\"heap_init_in_bytes\":" + heapSizeInBytes); + assertThat(responseBody) + .as("maximum heap size matches the configured value") + .contains("\"heap_max_in_bytes\":" + heapSizeInBytes); } private void assertClusterHealthResponse(ElasticsearchContainer container) throws IOException { Response response = getClusterHealth(container); - assertThat(response.getStatusLine().getStatusCode()).isEqualTo(200); + assertThat(response.getStatusLine().getStatusCode()).as("cluster health is available").isEqualTo(200); assertThat(EntityUtils.toString(response.getEntity())).contains("cluster_name"); } - - @Test - void testGetHttpSchemeForElasticsearch7ReturnsHttp() { - try (ElasticsearchContainer container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)) { - container.start(); - assertThat(container.getHttpScheme()).isEqualTo("http"); - } - } - - @Test - void testGetHttpSchemeForElasticsearch8ReturnsHttps() { - try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) - ) { - container.start(); - assertThat(container.getHttpScheme()).isEqualTo("https"); - } - } - - @Test - void testGetHttpSchemeForElasticsearch8WithSslDisabledReturnsHttp() { - try ( - ElasticsearchContainer container = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:8.1.2" - ) - .withEnv("xpack.security.http.ssl.enabled", "false") - ) { - container.start(); - assertThat(container.getHttpScheme()).isEqualTo("http"); - } - } } diff --git a/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java b/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java index 0099b5613ca..2c108d1a7b3 100644 --- a/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java +++ b/modules/elasticsearch/src/test/java/org/testcontainers/elasticsearch/KibanaContainerTest.java @@ -15,8 +15,6 @@ import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.util.EntityUtils; -import org.assertj.core.api.Assertions; -import org.assertj.core.api.Assumptions; import org.junit.jupiter.api.Test; import org.testcontainers.Testcontainers; import org.testcontainers.containers.Container; @@ -29,186 +27,164 @@ import java.io.IOException; import java.nio.charset.StandardCharsets; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.assertj.core.api.Assumptions.assumeThat; +import static org.testcontainers.elasticsearch.ElasticsearchContainerTest.ELASTICSEARCH_IMAGE_LATEST; +import static org.testcontainers.elasticsearch.ElasticsearchContainerTest.ELASTICSEARCH_VERSION_7; +import static org.testcontainers.elasticsearch.ElasticsearchContainerTest.ELASTICSEARCH_VERSION_LATEST; + class KibanaContainerTest { public static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); - private static final String ES_IMAGE = "docker.elastic.co/elasticsearch/elasticsearch:9.2.4"; + private static final String KIBANA_IMAGE = "docker.elastic.co/kibana/kibana:" + ELASTICSEARCH_VERSION_LATEST; - @Test - void cannotCreateKibanaContainerForVersionLessThan8() { - Assertions - .assertThatThrownBy(() -> new KibanaContainer("docker.elastic.co/kibana/kibana:7.17.29")) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("is not supported"); - } + /** + * A Kibana patch intentionally behind the latest Elasticsearch version, to verify mixed-patch compatibility. + */ + private static final String KIBANA_IMAGE_OLDER_PATCH = "docker.elastic.co/kibana/kibana:9.5.0"; + + // Managed mode (latest) @Test - void managedModeCanStartAndReachElasticsearchInSameExplicitNetwork() throws IOException { - // managedModeCanStartAndReachElasticsearchInSameExplicitNetwork { + void managedModeReachesElasticsearchOnSharedNetwork() throws IOException { + // managedModeReachesElasticsearchOnSharedNetwork { try ( Network network = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE).withNetwork(network); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST).withNetwork(network); KibanaContainer kibana = new KibanaContainer(es).withNetwork(network) ) { es.start(); kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertKibanaIsAvailable(kibana); } // } } @Test - void managedModeCannotStartWithOnlyESNetworkExplicit() { - Network network = Network.newNetwork(); + void managedModeReachesElasticsearchWithoutExplicitNetwork() throws IOException { try ( - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE).withNetwork(network); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST); KibanaContainer kibana = new KibanaContainer(es) ) { - Assertions - .assertThatThrownBy(kibana::start) - .isInstanceOf(ContainerLaunchException.class) - .satisfies(ex -> { - Assertions - .assertThat(ex.getCause()) - .isInstanceOf(IllegalStateException.class) - .hasMessageContaining("explicit network"); - }); - } - } + kibana.start(); - @Test - void managedModeCannotStartWithDifferentExplicitNetworks() { - try ( - Network esNetwork = Network.newNetwork(); - Network kibanaNetwork = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE).withNetwork(esNetwork); - KibanaContainer kibana = new KibanaContainer(es).withNetwork(kibanaNetwork) - ) { - Assertions - .assertThatThrownBy(kibana::start) - .isInstanceOf(ContainerLaunchException.class) - .satisfies(ex -> { - Assertions - .assertThat(ex.getCause()) - .isInstanceOf(IllegalStateException.class) - .hasMessageContaining("different networks"); - }); + assertKibanaIsAvailable(kibana); } } @Test - void managedModeUsesCustomNetworkAliasInExplicitNetwork() throws Exception { - final String customEsAlias = "my-custom-es-alias"; - + void managedModeWorksWhenElasticsearchSecurityIsDisabled() throws IOException { try ( - Network network = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE) - .withNetwork(network) - .withNetworkAliases(customEsAlias); - KibanaContainer kibana = new KibanaContainer(es).withNetwork(network) + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withEnv("xpack.security.enabled", "false"); + KibanaContainer kibana = new KibanaContainer(es) ) { kibana.start(); - Assertions.assertThat(kibana.isRunning()).isTrue(); - - // Verify Kibana uses the custom alias (not auto-generated tc-* alias) - Container.ExecResult result = kibana.execInContainer("sh", "-c", "env | grep ELASTICSEARCH_HOSTS"); - - Assertions.assertThat(result.getStdout()).contains(customEsAlias).contains(":9200"); + assertKibanaIsAvailable(kibana); } } @Test - void managedModeCanStartAndReachElasticsearchWithoutExplicitNetwork() throws IOException { + void managedModeWorksWhenElasticsearchTlsIsDisabled() throws IOException { try ( - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withEnv("xpack.security.enabled", "true") + .withEnv("xpack.security.http.ssl.enabled", "false"); KibanaContainer kibana = new KibanaContainer(es) ) { + es.start(); kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertKibanaIsAvailable(kibana); } } @Test - void managedModeCanStartWithoutElasticsearchSecurity() throws IOException { + void managedModeUsesCustomElasticsearchNetworkAlias() throws Exception { + final String customEsAlias = "my-custom-es-alias"; + try ( - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE).withEnv("xpack.security.enabled", "false"); - KibanaContainer kibana = new KibanaContainer(es) + Network network = Network.newNetwork(); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withNetwork(network) + .withNetworkAliases(customEsAlias); + KibanaContainer kibana = new KibanaContainer(es).withNetwork(network) ) { kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertThat(kibana.isRunning()).as("Kibana container is running").isTrue(); + + Container.ExecResult result = kibana.execInContainer("sh", "-c", "env | grep ELASTICSEARCH_HOSTS"); + + assertThat(result.getStdout()) + .as("ELASTICSEARCH_HOSTS uses the custom Elasticsearch network alias") + .contains(customEsAlias) + .contains(":9200"); } } @Test - void managedModeCanStartWithoutElasticsearchHttps() throws IOException { + void managedModeRejectsWhenOnlyElasticsearchHasExplicitNetwork() { + Network network = Network.newNetwork(); try ( - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE) - .withEnv("xpack.security.enabled", "true") - .withEnv("xpack.security.http.ssl.enabled", "false"); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST).withNetwork(network); KibanaContainer kibana = new KibanaContainer(es) ) { - es.start(); - kibana.start(); - - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertThatThrownBy(kibana::start) + .as("managed mode requires Kibana to join the same explicit network as Elasticsearch") + .isInstanceOf(ContainerLaunchException.class) + .satisfies(ex -> { + assertThat(ex.getCause()) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("explicit network"); + }); } } @Test - void externalModeFailsWithConflictingCredentials() { - Assertions - .assertThatThrownBy(() -> { - new KibanaContainer("docker.elastic.co/kibana/kibana:8.0.0") - .withKibanaUsernameAndPassword("user", "pass") - .withElasticsearchServiceAccountToken("token"); - }) - .isInstanceOf(IllegalStateException.class) - .hasMessageContaining("Conflicting Elasticsearch credentials"); + void managedModeRejectsWhenNetworksDiffer() { + try ( + Network esNetwork = Network.newNetwork(); + Network kibanaNetwork = Network.newNetwork(); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST).withNetwork(esNetwork); + KibanaContainer kibana = new KibanaContainer(es).withNetwork(kibanaNetwork) + ) { + assertThatThrownBy(kibana::start) + .as("managed mode rejects Kibana and Elasticsearch on different networks") + .isInstanceOf(ContainerLaunchException.class) + .satisfies(ex -> { + assertThat(ex.getCause()) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("different networks"); + }); + } } @Test - void managedModeFailsWhenSettingElasticsearchUrl() { - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE); - Assertions - .assertThatThrownBy(() -> { - new KibanaContainer(es).withElasticsearchUrl("http://somewhere.over.the.rainbow:9200"); - }) + void managedModeRejectsExplicitElasticsearchUrl() { + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST); + assertThatThrownBy(() -> { + new KibanaContainer(es).withElasticsearchUrl("http://somewhere.over.the.rainbow:9200"); + }) + .as("managed mode cannot also set an explicit Elasticsearch URL") .isInstanceOf(IllegalStateException.class) .hasMessageContaining("Cannot set Elasticsearch URL when using Elasticsearch container"); } - @Test - void failsWhenNoElasticsearchConfigured() { - try (KibanaContainer kibana = new KibanaContainer("docker.elastic.co/kibana/kibana:8.0.0")) { - Assertions - .assertThatThrownBy(kibana::start) - .isInstanceOf(ContainerLaunchException.class) - .satisfies(ex -> { - Assertions - .assertThat(ex.getCause()) - .isInstanceOf(IllegalStateException.class) - .hasMessageContaining("Elasticsearch must be configured"); - }); - } - } + // External mode (latest) @Test - void externalModeCanWorkWithUsernamePassword() throws IOException, InterruptedException { + void externalModeReachesElasticsearchWithUsernamePassword() throws IOException { final String esHostname = "elasticsearch"; - // externalModeCanWorkWithUsernamePassword { + // externalModeReachesElasticsearchWithUsernamePassword { try ( Network network = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE) + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) .withNetwork(network) .withNetworkAliases(esHostname) .withEnv("xpack.security.http.ssl.enabled", "false") @@ -217,43 +193,41 @@ void externalModeCanWorkWithUsernamePassword() throws IOException, InterruptedEx String kibanaSystemPassword = setKibanaSystemPassword(es); try ( - KibanaContainer kibana = new KibanaContainer("docker.elastic.co/kibana/kibana:9.2.2") //this minor version is intentionally below ES version + KibanaContainer kibana = new KibanaContainer(KIBANA_IMAGE) .withNetwork(network) .withElasticsearchUrl("http://" + esHostname + ":9200") .withKibanaSystemPassword(kibanaSystemPassword) ) { kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertKibanaIsAvailable(kibana); } } // } } @Test - void externalModeCanWorkWithoutCredentials() throws IOException { + void externalModeReachesElasticsearchWithoutCredentials() throws IOException { final String esHostname = "elasticsearch"; try ( Network network = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE) + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) .withNetwork(network) .withNetworkAliases(esHostname) .withEnv("xpack.security.enabled", "false") .withEnv("xpack.security.http.ssl.enabled", "false"); - KibanaContainer kibana = new KibanaContainer("docker.elastic.co/kibana/kibana:9.2.4") + KibanaContainer kibana = new KibanaContainer(KIBANA_IMAGE) .withNetwork(network) .withElasticsearchUrl("http://" + esHostname + ":9200") ) { es.start(); kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertKibanaIsAvailable(kibana); } } @Test - void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws Exception { + void externalModeReachesElasticsearchWithTlsAndServiceToken() throws Exception { byte[] caCrt; byte[] nodeCrt; byte[] nodeKey; @@ -269,7 +243,7 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E " ip: [ \"127.0.0.1\" ]\n"; try ( - ElasticsearchContainer setup = new ElasticsearchContainer(ES_IMAGE) + ElasticsearchContainer setup = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) .withEnv("discovery.type", "single-node") .withCopyToContainer( Transferable.of(instancesYml.getBytes(StandardCharsets.UTF_8), 0644), @@ -278,7 +252,6 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E ) { setup.start(); - // Run certutil inside the running container and write outputs inside the container FS Container.ExecResult execResult = setup.execInContainer( "bash", "-lc", @@ -290,9 +263,8 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E "bin/elasticsearch-certutil cert --silent --pem --in /tmp/instances.yml --ca-cert /tmp/out/ca/ca.crt --ca-key /tmp/out/ca/ca.key --out /tmp/out/certs.zip && " + "unzip -o /tmp/out/certs.zip -d /tmp/out" ); - Assertions.assertThat(execResult.getExitCode()).isEqualTo(0); + assertThat(execResult.getExitCode()).as("elasticsearch-certutil generated certificates").isEqualTo(0); - // copy the certificates and key from the container, so we can use them later caCrt = setup.copyFileFromContainer("/tmp/out/ca/ca.crt", IOUtils::toByteArray); nodeCrt = setup.copyFileFromContainer("/tmp/out/es01/es01.crt", IOUtils::toByteArray); nodeKey = setup.copyFileFromContainer("/tmp/out/es01/es01.key", IOUtils::toByteArray); @@ -300,9 +272,7 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E try ( Network network = Network.newNetwork(); - ElasticsearchContainer es = new ElasticsearchContainer( - "docker.elastic.co/elasticsearch/elasticsearch:9.2.4" - ) + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) .withNetwork(network) .withNetworkAliases(esHostname) ) { @@ -311,7 +281,7 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E String kibanaServiceAccountToken = createKibanaServiceAccountToken(es); try ( - KibanaContainer kibana = new KibanaContainer("docker.elastic.co/kibana/kibana:9.2.4") + KibanaContainer kibana = new KibanaContainer(KIBANA_IMAGE) // network is needed only because the ES we try to access via explicit mode is operated by non-public Docker .withNetwork(network) .withElasticsearchUrl("https://" + esHostname + ":9200") @@ -319,12 +289,123 @@ void externalModeCanStartAndReachElasticsearchWithCertAndServiceToken() throws E .withElasticsearchCaCertificate(es.caCertAsBytes().get()) ) { kibana.start(); - String status = getKibanaStatus(kibana); - Assertions.assertThat(status).isEqualTo("available"); + assertKibanaIsAvailable(kibana); + } + } + } + + @Test + void externalModeRejectsConflictingCredentials() { + assertThatThrownBy(() -> { + new KibanaContainer(KIBANA_IMAGE) + .withKibanaUsernameAndPassword("user", "pass") + .withElasticsearchServiceAccountToken("token"); + }) + .as("username/password and a service account token cannot be set together") + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("Conflicting Elasticsearch credentials"); + } + + @Test + void startFailsWhenElasticsearchIsNotConfigured() { + try (KibanaContainer kibana = new KibanaContainer(KIBANA_IMAGE)) { + assertThatThrownBy(kibana::start) + .as("Kibana cannot start without an Elasticsearch URL or container") + .isInstanceOf(ContainerLaunchException.class) + .satisfies(ex -> { + assertThat(ex.getCause()) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("Elasticsearch must be configured"); + }); + } + } + + @Test + void reusesContainerWhenReuseIsEnabled() { + assumeThat(TestcontainersConfiguration.getInstance().environmentSupportsReuse()) + .as("testcontainers.reuse.enable must be true") + .isTrue(); + + // Testcontainers.exposeHostPorts + host.testcontainers.internal lets Kibana reach ES + // from inside the container on any platform (Linux Docker Engine included). + // No withNetwork() on Kibana keeps the hash fully deterministic: + // - no dynamic network ID + // - the random tc-* alias added by GenericContainer's constructor is only serialised + // into the CreateContainerCmd when withNetwork() has been called, so it is absent here + // The host.testcontainers.internal extra-host IP is the same for kibana1 and kibana2 + // because they start in the same JVM (same PortForwardingContainer instance). + // The first Kibana container must stay running while the second one starts, because + // withReuse(true) only skips JVM-shutdown cleanup — an explicit stop() still removes the + // container, so there would be nothing to find. + try ( + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withEnv("xpack.security.enabled", "false") + .withEnv("xpack.security.http.ssl.enabled", "false") + ) { + es.start(); + int esMappedPort = es.getMappedPort(9200); + Testcontainers.exposeHostPorts(esMappedPort); + String esUrl = "http://" + GenericContainer.INTERNAL_HOST_HOSTNAME + ":" + esMappedPort; + + KibanaContainer kibana1 = new KibanaContainer(KIBANA_IMAGE).withElasticsearchUrl(esUrl).withReuse(true); + KibanaContainer kibana2 = new KibanaContainer(KIBANA_IMAGE).withElasticsearchUrl(esUrl).withReuse(true); + + try { + kibana1.start(); + // kibana2 is started while kibana1 is still running; the reuse mechanism should + // find kibana1's container by hash and return the same container ID. + kibana2.start(); + + assertThat(kibana2.getContainerId()) + .as("withReuse(true) reuses the same container on subsequent starts") + .isEqualTo(kibana1.getContainerId()); + } finally { + kibana1.stop(); + kibana2.stop(); + } + } + } + + // Version-specific + + @Test + void rejectsKibanaVersionBelow8() { + assertThatThrownBy(() -> new KibanaContainer("docker.elastic.co/kibana/kibana:" + ELASTICSEARCH_VERSION_7)) + .as("Kibana versions below 8.0.0 are not supported") + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("is not supported"); + } + + @Test + void externalModeAcceptsOlderKibanaPatchThanElasticsearch() throws IOException { + final String esHostname = "elasticsearch"; + + try ( + Network network = Network.newNetwork(); + ElasticsearchContainer es = new ElasticsearchContainer(ELASTICSEARCH_IMAGE_LATEST) + .withNetwork(network) + .withNetworkAliases(esHostname) + .withEnv("xpack.security.http.ssl.enabled", "false") + ) { + es.start(); + String kibanaSystemPassword = setKibanaSystemPassword(es); + + try ( + KibanaContainer kibana = new KibanaContainer(KIBANA_IMAGE_OLDER_PATCH) + .withNetwork(network) + .withElasticsearchUrl("http://" + esHostname + ":9200") + .withKibanaSystemPassword(kibanaSystemPassword) + ) { + kibana.start(); + assertKibanaIsAvailable(kibana); } } } + private static void assertKibanaIsAvailable(KibanaContainer kibana) throws IOException { + assertThat(getKibanaStatus(kibana)).as("Kibana reports overall status available").isEqualTo("available"); + } + private static String setKibanaSystemPassword(ElasticsearchContainer elasticsearch) throws IOException { String kibanaPassword = "kibana-system-" + System.currentTimeMillis(); @@ -444,69 +525,16 @@ private static String getKibanaStatus(KibanaContainer kibana) throws IOException } } - @Test - void withReuseShouldReuseTheSameContainer() { - Assumptions - .assumeThat(TestcontainersConfiguration.getInstance().environmentSupportsReuse()) - .as("testcontainers.reuse.enable must be true") - .isTrue(); - - final String kibanaImage = "docker.elastic.co/kibana/kibana:9.2.4"; - - // Testcontainers.exposeHostPorts + host.testcontainers.internal lets Kibana reach ES - // from inside the container on any platform (Linux Docker Engine included). - // No withNetwork() on Kibana keeps the hash fully deterministic: - // - no dynamic network ID - // - the random tc-* alias added by GenericContainer's constructor is only serialised - // into the CreateContainerCmd when withNetwork() has been called, so it is absent here - // The host.testcontainers.internal extra-host IP is the same for kibana1 and kibana2 - // because they start in the same JVM (same PortForwardingContainer instance). - // The first Kibana container must stay running while the second one starts, because - // withReuse(true) only skips JVM-shutdown cleanup — an explicit stop() still removes the - // container, so there would be nothing to find. - try ( - ElasticsearchContainer es = new ElasticsearchContainer(ES_IMAGE) - .withEnv("xpack.security.enabled", "false") - .withEnv("xpack.security.http.ssl.enabled", "false") - ) { - es.start(); - int esMappedPort = es.getMappedPort(9200); - Testcontainers.exposeHostPorts(esMappedPort); - String esUrl = "http://" + GenericContainer.INTERNAL_HOST_HOSTNAME + ":" + esMappedPort; - - KibanaContainer kibana1 = new KibanaContainer(kibanaImage).withElasticsearchUrl(esUrl).withReuse(true); - KibanaContainer kibana2 = new KibanaContainer(kibanaImage).withElasticsearchUrl(esUrl).withReuse(true); - - try { - kibana1.start(); - // kibana2 is started while kibana1 is still running; the reuse mechanism should - // find kibana1's container by hash and return the same container ID. - kibana2.start(); - - Assertions - .assertThat(kibana2.getContainerId()) - .as("KibanaContainer with withReuse(true) should reuse the same container on subsequent starts") - .isEqualTo(kibana1.getContainerId()); - } finally { - kibana1.stop(); - kibana2.stop(); - } - } - } - private static void applyTls(ElasticsearchContainer c, byte[] caCrt, byte[] nodeCrt, byte[] nodeKey) { final String certDir = "/usr/share/elasticsearch/config/certs"; - // Copy provided materials c.withCopyToContainer(Transferable.of(caCrt, 0644), certDir + "/http_ca.crt"); c.withCopyToContainer(Transferable.of(nodeCrt, 0644), certDir + "/http.crt"); c.withCopyToContainer(Transferable.of(nodeKey, 0644), certDir + "/http.key"); - // Disable ES bootstrap TLS autoconfiguration c.withEnv("xpack.security.autoconfiguration.enabled", "false"); c.withEnv("xpack.security.enabled", "true"); - // Configure ONLY HTTP TLS using exactly the provided files c.withEnv("xpack.security.http.ssl.enabled", "true"); c.withEnv("xpack.security.http.ssl.certificate_authorities", "certs/http_ca.crt"); c.withEnv("xpack.security.http.ssl.certificate", "certs/http.crt"); diff --git a/modules/elasticsearch/src/test/resources/test-custom-memory-jvm.options b/modules/elasticsearch/src/test/resources/test-custom-memory-jvm.options index e3fe586284e..a4a939c4a8a 100644 --- a/modules/elasticsearch/src/test/resources/test-custom-memory-jvm.options +++ b/modules/elasticsearch/src/test/resources/test-custom-memory-jvm.options @@ -1,2 +1,2 @@ --Xms1574961152 --Xmx1574961152 +-Xms1500m +-Xmx1500m