Skip to content

Commit 2eb3f97

Browse files
adwk67maltesander
adwk67
and
maltesander
authored
fix: move metastore db credentials into a secret (#452)
* fix: move metastore db credentials into a secret * changelog * changed references in documentation/examples * changelog * Update rust/operator-binary/src/controller.rs Co-authored-by: Malte Sander <contact@maltesander.com> * Update CHANGELOG.md Co-authored-by: Malte Sander <contact@maltesander.com> * consistent ordering --------- Co-authored-by: Malte Sander <contact@maltesander.com>
1 parent 114bc0c commit 2eb3f97

25 files changed

Lines changed: 227 additions & 66 deletions

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,10 @@ All notable changes to this project will be documented in this file.
88

99
- Added documentation/tutorial on using external database drivers ([#449]).
1010

11+
### Fixed
12+
13+
- [BREAKING] Move the metastore `user` and `password` DB credentials out of the CRD into a Secret containing the keys `username` and `password` ([#452]).
14+
1115
### Changed
1216

1317
- BREAKING: Switch to new image that only contains HMS.
@@ -17,6 +21,7 @@ All notable changes to this project will be documented in this file.
1721

1822
[#447]: https://github.com/stackabletech/hive-operator/pull/447
1923
[#449]: https://github.com/stackabletech/hive-operator/pull/449
24+
[#452]: https://github.com/stackabletech/hive-operator/pull/452
2025

2126
## [24.3.0] - 2024-03-20
2227

‎deploy/helm/hive-operator/crds/crds.yaml‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,9 @@ spec:
5050
connString:
5151
description: 'A connection string for the database. For example: `jdbc:postgresql://hivehdfs-postgresql:5432/hivehdfs`'
5252
type: string
53+
credentialsSecret:
54+
description: A reference to a Secret containing the database credentials. The Secret needs to contain the keys `username` and `password`.
55+
type: string
5356
dbType:
5457
description: 'The type of database to connect to. Supported are: `postgres`, `mysql`, `oracle`, `mssql` and `derby`. This value is used to configure the jdbc driver class.'
5558
enum:
@@ -59,17 +62,10 @@ spec:
5962
- oracle
6063
- mssql
6164
type: string
62-
password:
63-
description: The password for the database user.
64-
type: string
65-
user:
66-
description: The database user.
67-
type: string
6865
required:
6966
- connString
67+
- credentialsSecret
7068
- dbType
71-
- password
72-
- user
7369
type: object
7470
hdfs:
7571
description: HDFS connection specification.

‎docs/modules/hive/examples/getting_started/hive-postgres-s3.yaml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,20 @@ spec:
99
clusterConfig:
1010
database:
1111
connString: jdbc:postgresql://postgresql:5432/hive
12-
user: hive
13-
password: hive
12+
credentialsSecret: hive-credentials
1413
dbType: postgres
1514
s3:
1615
reference: minio
1716
metastore:
1817
roleGroups:
1918
default:
2019
replicas: 1
20+
---
21+
apiVersion: v1
22+
kind: Secret
23+
metadata:
24+
name: hive-credentials
25+
type: Opaque
26+
stringData:
27+
username: hive
28+
password: hive

‎docs/modules/hive/examples/getting_started/hive-postgres-s3.yaml.j2‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,20 @@ spec:
99
clusterConfig:
1010
database:
1111
connString: jdbc:postgresql://postgresql:5432/hive
12-
user: hive
13-
password: hive
12+
credentialsSecret: hive-credentials
1413
dbType: postgres
1514
s3:
1615
reference: minio
1716
metastore:
1817
roleGroups:
1918
default:
2019
replicas: 1
20+
---
21+
apiVersion: v1
22+
kind: Secret
23+
metadata:
24+
name: hive-credentials
25+
type: Opaque
26+
stringData:
27+
username: hive
28+
password: hive

‎docs/modules/hive/pages/reference/discovery.adoc‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,21 @@ spec:
2525
clusterConfig:
2626
database:
2727
connString: jdbc:postgresql://postgresql:5432/hive
28-
user: hive
29-
password: hive
28+
credentialsSecret: hive-credentials
3029
dbType: postgres
3130
metastore:
3231
roleGroups:
3332
default: # <3>
3433
replicas: 2
34+
---
35+
apiVersion: v1
36+
kind: Secret
37+
metadata:
38+
name: hive-credentials
39+
type: Opaque
40+
stringData:
41+
username: hive
42+
password: hive
3543
----
3644
<1> The name of the Hive cluster, which is also the name of the created discovery ConfigMap.
3745
<2> The namespace of the discovery ConfigMap.

‎docs/modules/hive/pages/usage-guide/database-driver.adoc‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,7 @@ spec:
145145
clusterConfig:
146146
database:
147147
connString: jdbc:mysql://mysql:3306/hive # <1>
148-
user: hive # <2>
149-
password: hive
148+
credentialsSecret: hive-credentials # <2>
150149
dbType: mysql
151150
s3:
152151
reference: minio # <3>
@@ -167,10 +166,19 @@ spec:
167166
persistentVolumeClaim:
168167
claimName: pvc-hive-drivers
169168
replicas: 1
169+
---
170+
apiVersion: v1
171+
kind: Secret
172+
metadata:
173+
name: hive-credentials # <2>
174+
type: Opaque
175+
stringData:
176+
username: hive
177+
password: hive
170178
----
171179

172180
<1> The database connection details matching those given when deploying the MySQL Helm chart
173-
<2> Plain-text Hive credentials will be replaced in an upcoming release!
181+
<2> Hive credentials are retrieved from a Secret
174182
<3> A reference to the file store using S3 (this has been omitted from this article for the sake of brevity, but is described in e.g. the xref:getting_started/first_steps.adoc[] guide)
175183
<4> Use `envOverrides` to set the driver path
176184
<5> Use `podOverrides` to mount the driver

‎docs/modules/hive/pages/usage-guide/derby-example.adoc‎

Lines changed: 30 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,21 @@ spec:
2020
clusterConfig:
2121
database:
2222
connString: jdbc:derby:;databaseName=/tmp/metastore_db;create=true
23-
user: APP
24-
password: mine
23+
credentialsSecret: hive-credentials
2524
dbType: derby
2625
metastore:
2726
roleGroups:
2827
default:
2928
replicas: 1
29+
---
30+
apiVersion: v1
31+
kind: Secret
32+
metadata:
33+
name: hive-credentials
34+
type: Opaque
35+
stringData:
36+
username: APP
37+
password: mine
3038
----
3139

3240
WARNING: You should not use the `Derby` database in production. Derby stores data locally which does not work in high availability setups (multiple replicas) and all data is lost after Pod restarts.
@@ -62,8 +70,7 @@ spec:
6270
clusterConfig:
6371
database:
6472
connString: jdbc:derby:;databaseName=/stackable/metastore_db;create=true
65-
user: APP
66-
password: mine
73+
credentialsSecret: hive-credentials
6774
dbType: derby
6875
s3:
6976
inline:
@@ -96,6 +103,15 @@ metadata:
96103
stringData:
97104
accessKey: minio-access-key
98105
secretKey: minio-secret-key
106+
---
107+
apiVersion: v1
108+
kind: Secret
109+
metadata:
110+
name: hive-credentials
111+
type: Opaque
112+
stringData:
113+
username: APP
114+
password: mine
99115
----
100116

101117

@@ -131,11 +147,19 @@ spec:
131147
clusterConfig:
132148
database:
133149
connString: jdbc:postgresql://hive-postgresql.default.svc.cluster.local:5432/hive
134-
user: hive
135-
password: hive
150+
credentialsSecret: hive-credentials
136151
dbType: postgres
137152
metastore:
138153
roleGroups:
139154
default:
140155
replicas: 1
156+
---
157+
apiVersion: v1
158+
kind: Secret
159+
metadata:
160+
name: hive-credentials
161+
type: Opaque
162+
stringData:
163+
username: hive
164+
password: hive
141165
----

‎examples/simple-hive-cluster-postgres-s3.yaml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,7 @@ spec:
2222
clusterConfig:
2323
database:
2424
connString: jdbc:derby:;databaseName=/tmp/hive;create=true
25-
user: APP
26-
password: mine
25+
credentialsSecret: hive-credentials
2726
dbType: derby
2827
s3:
2928
inline:
@@ -56,3 +55,12 @@ metadata:
5655
stringData:
5756
accessKey: minio-access-key
5857
secretKey: minio-secret-key
58+
---
59+
apiVersion: v1
60+
kind: Secret
61+
metadata:
62+
name: hive-credentials
63+
type: Opaque
64+
stringData:
65+
username: APP
66+
password: mine

‎examples/simple-hive-cluster.yaml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,7 @@ spec:
1010
clusterConfig:
1111
database:
1212
connString: jdbc:derby:;databaseName=/tmp/hive;create=true
13-
user: APP
14-
password: mine
13+
credentialsSecret: hive-credentials
1514
dbType: derby
1615
metastore:
1716
roleGroups:
@@ -24,3 +23,12 @@ spec:
2423
max: "2"
2524
memory:
2625
limit: 5Gi
26+
---
27+
apiVersion: v1
28+
kind: Secret
29+
metadata:
30+
name: hive-credentials
31+
type: Opaque
32+
stringData:
33+
username: APP
34+
password: mine

‎rust/crd/src/affinity.rs‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,8 @@ mod tests {
4949
clusterConfig:
5050
database:
5151
connString: jdbc:derby:;databaseName=/tmp/hive;create=true
52-
user: APP
53-
password: mine
5452
dbType: derby
53+
credentialsSecret: mySecret
5554
metastore:
5655
roleGroups:
5756
default:

0 commit comments

Comments
 (0)