diff --git a/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc b/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc index 072d7b7a..78cc18c0 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc @@ -146,7 +146,6 @@ spec: ... ---- - == Pod overrides The Spark operator also supports Pod overrides, allowing you to override any property that you can set on a Kubernetes Pod. @@ -157,6 +156,24 @@ Read the xref:concepts:overrides.adoc#pod-overrides[Pod overrides documentation] Stackable operators automatically determine the set of needed JVM arguments, such as memory settings or trust- and keystores. Using JVM argument overrides you can configure the JVM arguments xref:concepts:overrides.adoc#jvm-argument-overrides[according to the concepts page]. +=== Overriding `extraClassPath` + +The current implementation of `extraClassPath` overrides is a replace. Thus if you want to override `extraClassPath` you will need to append to the already set defaults you might need e.g. for `spark-connect`: + +[source,yaml] +---- +--- +spec: + sparkImage: + productVersion: 4.1.2 + sparkConf: + "spark.driver.extraClassPath": "/stackable/spark/extra-jars/*:/stackable/spark/connect/spark-connect-4.1.2.jar:/mydependencies/my.jar" #<1> + "spark.executor.extraClassPath": "/stackable/spark/extra-jars/*:/stackable/spark/connect/spark-connect-4.1.2.jar:/mydependencies/my.jar" +---- +<1> First two entries are defaults set by the operator when using `spark-connect`, the last one is user specific. + +Omitting the defaults set by the operator leads to a override and thus to incomplete `extraClassPath`. + === Spark application WARNING: Please note that you can currently not configure the JVM arguments of the spark-submit call, only on the drivers and executors.