Skip to content

Commit f1a2ece

Browse files
zhuay-azhuay-a
authored
Merge branch 'Tencent:main' into patch-1
2 parents ac7e014 + 5ac1479 commit f1a2ece

14 files changed

Lines changed: 180 additions & 71 deletions

File tree

‎helm-charts/tca/README.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,11 @@ The command removes all the Kubernetes components associated with the chart and
4949
| `global.imagePullPolicy` | Global Docker image registry | `""` |
5050
| `global.imagePullSecrets` | Global Docker registry secret names as an array | `[]` |
5151

52+
To create a secret to pull an image from a private container image registry or repository:
53+
```Bash
54+
$ kubectl create secret docker-registry ${SECRET_NAME} --docker-server=${DOCKER_SERVER} --docker-username=${DOCKER_USERNAME} --docker-password=${DOCKER_PASSWORD} --docker-email=${DOCKER_EMAIL}
55+
```
56+
5257
### TCA Metric parameters
5358
| Name | Description | Value |
5459
| ------------------------------------------------------- | ------------------------------------------------------- | -------------- |
@@ -302,13 +307,45 @@ Helm will deploy `mariadb` by default. Reference: [bitnami/mariadb](https://gith
302307
| `tca.scmproxy.image.tag` | TCA scmproxy image tag | `latest` |
303308
| `tca.scmproxy.privateScmUrl` | TCA scmproxy private scm url | `""` |
304309
305-
306310
### TCA Client Configuration parameters
307-
| Name | Description | Value |
308-
| -------------------------------------------- | ------------------------------------------------------------------------ | ------------------------- |
309-
| `tca.client.image.repository` | TCA client image repository | `tencenttca/tca-client` |
310-
| `tca.client.image.repository` | TCA client image tag | `latest` |
311-
| `tca.client.enabeld` | Enabled starting TCA client with helm | `true` |
311+
#### Default node pool configuration
312+
| Name | Description | Value |
313+
|------------------------------------------------|----------------------------------------------| ------------------------ |
314+
| `tca.client.image.repository` | TCA client image repository | `tencenttca/tca-client` |
315+
| `tca.client.image.tag` | TCA client image tag | `latest` |
316+
| `tca.client.enabeld` | Enabled starting TCA client with helm | `true` |
317+
| `tca.client.replicas` | The number of nodes in the default node pool | `1` |
318+
| `tca.client.tag` | Tag of the extra node pool | `Codedog_Linux` |
319+
| `tca.client.resources.limits.cpu` | The upper limit of cpu usage | `2000m` |
320+
| `tca.client.resources.limits.memory` | The upper limit of memory usage | `4Gi` |
321+
| `tca.client.resources.requests.cpu` | Required cpu resources | `1000m` |
322+
| `tca.client.resources.requests.memory` | Required memory resources | `2Gi` |
323+
324+
#### Client Node pool expansion operations guide
325+
You can configure node resources on the client.
326+
If you do not manually configure node resources, the default node pool is used.
327+
If the node pool resources are insufficient, expand the capacity of the node pool at any time. For details, refer to 'Procedure for expanding a node pool'.
328+
329+
#### TCA client node pool expansion config
330+
| Name | Description |
331+
|--------------------------------------------------------------|---------------------------------------|
332+
| `extraNodePools.nodepool` | Name of the extended node pool |
333+
| `extraNodePools.nodepool.tag` | Tag of the extra node pool |
334+
| `extraNodePools.nodepool.enabled` | Enabled extending node pool with helm |
335+
| `extraNodePools.nodepool.replicas` | The number of nodes in the extra node pool |
336+
| `extraNodePools.nodepool.resources.limits.cpu` | The upper limit of cpu usage |
337+
| `extraNodePools.nodepool.resources.limits.memory` | The upper limit of memory usage |
338+
| `extraNodePools.nodepool.resources.requests.cpu` | Required cpu resources |
339+
| `extraNodePools.nodepool.resources.requests.memory` | Required memory resources |
340+
341+
#### Procedure for expanding a node pool
342+
1. Initially, deploy the project exclusively utilizing the default node pool.
343+
2. To expand the node pool, navigate to "Background Management > Node Management > Label Management" and add label information.
344+
3. Update the extra node pool configuration in the project file by navigating to "CodeAnalysis/helm-charts/tca/values.yaml" .
345+
4. To add configuration information for extra node pools, refer to the "TCA client node pool expansion config" section under "extraNodePools" in the client section of the TCA.
346+
5. Please note that the "extraNodePools.nodepool" tag can be customized to match the user's requirements for the node pool name. This tag should be the same as the tag name added in Step 2.
347+
6. To enable the current node pool, modify the "enabled" value to "true".
348+
7. Update project deployment.
312349
313350
### TCA Gateway Configuration parameters
314351
| Name | Description | Value |

‎helm-charts/tca/templates/client/configmap-conf.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ data:
3939
[LICENSE_CONFIG]
4040
; [可选]使用自研工具时，需要填写，默认不需要
4141
; license服务器url, base_path, license
42-
URL=
42+
URL={{ .Values.tca.commonConfig.clsServerUrl }}
4343
BASE_PATH=
44-
LICENSE=
44+
LICENSE={{ .Values.tca.commonConfig.clsServerLicense }}
4545

helm-charts/tca/templates/client/deployment.yaml renamed to helm-charts/tca/templates/client/deployment-default.yaml

Lines changed: 4 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
{{- if .Values.tca.client.enabled }}
12
apiVersion: apps/v1
23
kind: StatefulSet
34
metadata:
@@ -10,6 +11,7 @@ metadata:
1011
name: {{ template "tca.fullname" . }}-client
1112
spec:
1213
serviceName: ""
14+
replicas: {{ .Values.tca.client.replicas }}
1315
selector:
1416
matchLabels:
1517
k8s-app: {{ template "tca.fullname" . }}-client
@@ -42,7 +44,7 @@ spec:
4244
image: "{{ .Values.tca.client.image.repository }}:{{ .Values.tca.client.image.tag }}"
4345
imagePullPolicy: {{ .Values.global.imagePullPolicy }}
4446
command: ["python3"]
45-
args: ["codepuppy.py", "start", "-t", "$(CODEDOG_TOKEN)", "--create-from", "docker", "--tag", "CodeDog_Linux"]
47+
args: ["codepuppy.py", "start", "-t", "$(CODEDOG_TOKEN)", "--create-from", "docker", "--tag", "{{ .Values.tca.client.tag }}"]
4648
envFrom:
4749
- configMapRef:
4850
name: client-env
@@ -56,7 +58,7 @@ spec:
5658
exec:
5759
command:
5860
- sleep
59-
- '20'
61+
- '20'
6062
resources:
6163
{{ toYaml .Values.tca.client.resources | indent 12 }}
6264
volumeMounts:
@@ -71,40 +73,4 @@ spec:
7173
path: config.ini
7274
name: {{ template "tca.fullname" . }}-client-config
7375
name: client-config-conf
74-
---
75-
{{- if .Values.tca.client.autoscaling.enabled }}
76-
apiVersion: autoscaling/v2beta2
77-
kind: HorizontalPodAutoscaler
78-
metadata:
79-
name: {{ template "tca.fullname" . }}-client
80-
labels:
81-
k8s-app: {{ template "tca.fullname" . }}-client
82-
app: {{ template "tca.name" . }}
83-
chart: {{ template "tca.chart" . }}
84-
release: {{ .Release.Name }}
85-
heritage: {{ .Release.Service }}
86-
spec:
87-
scaleTargetRef:
88-
apiVersion: apps/v1
89-
kind: StatefulSet
90-
name: {{ template "tca.fullname" . }}-client
91-
minReplicas: {{ .Values.tca.client.autoscaling.minReplicas }}
92-
maxReplicas: {{ .Values.tca.client.autoscaling.maxReplicas }}
93-
metrics:
94-
{{- if .Values.tca.client.autoscaling.targetMemory }}
95-
- type: Resource
96-
resource:
97-
name: memory
98-
target:
99-
type: Utilization
100-
averageUtilization: {{ .Values.tca.client.autoscaling.targetMemory }}
101-
{{- end }}
102-
{{- if .Values.tca.client.autoscaling.targetCPU }}
103-
- type: Resource
104-
resource:
105-
name: cpu
106-
target:
107-
type: Utilization
108-
averageUtilization: {{ .Values.tca.client.autoscaling.targetCPU }}
109-
{{- end }}
11076
{{- end }}

‎helm-charts/tca/templates/client/deployment-extra.yaml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
{{ $myChartName := .Release.Name }}
2+
{{ $myChartService := .Release.Service }}
3+
{{ $Values := .Values}}
4+
{{ $BasePath := $.Template.BasePath}}
5+
{{ $Image := .Values.tca.client.image }}
6+
7+
{{- range $key, $value := .Values.tca.client.extraNodePools }}
8+
{{- if $value.enabled }}
9+
---
10+
apiVersion: apps/v1
11+
kind: StatefulSet
12+
metadata:
13+
labels:
14+
k8s-app: {{ template "tca.fullname" $ }}-client-{{ $key }}
15+
app: {{ template "tca.name" $ }}
16+
chart: {{ template "tca.chart" $ }}
17+
release: {{ $myChartName }}
18+
heritage: {{ $myChartService }}
19+
name: {{ template "tca.fullname" $ }}-client-{{ $key }}
20+
spec:
21+
serviceName: ""
22+
replicas: {{ $value.replicas }}
23+
selector:
24+
matchLabels:
25+
k8s-app: {{ template "tca.fullname" $ }}-client-{{ $key }}
26+
release: {{ $myChartName }}
27+
template:
28+
metadata:
29+
labels:
30+
k8s-app: {{ template "tca.fullname" $ }}-client-{{ $key }}
31+
release: {{ $myChartName }}
32+
spec:
33+
shareProcessNamespace: true
34+
{{- with $Values.global.imagePullSecrets }}
35+
imagePullSecrets:
36+
{{- toYaml . | nindent 8 }}
37+
{{- end }}
38+
{{- if $Values.tca.podAntiAffinity.enable }}
39+
affinity:
40+
podAntiAffinity:
41+
preferredDuringSchedulingIgnoredDuringExecution:
42+
- weight: 1
43+
podAffinityTerm:
44+
topologyKey: "{{ .Values.tca.podAntiAffinity.topologyKey }}"
45+
labelSelector:
46+
matchLabels:
47+
k8s-app: {{ template "tca.fullname" $ }}-client-{{ $key }}
48+
release: {{ $myChartName }}
49+
{{- end }}
50+
containers:
51+
- name: {{ template "tca.fullname" $ }}-client-{{ $key }}
52+
image: "{{ $Image.repository }}:{{ $Image.tag }}"
53+
imagePullPolicy: {{ $Values.global.imagePullPolicy }}
54+
command: ["python3"]
55+
args: ["codepuppy.py", "start", "-t", "$(CODEDOG_TOKEN)", "--create-from", "docker", "--tag", "{{ $value.tag }}"]
56+
envFrom:
57+
- configMapRef:
58+
name: client-env
59+
env:
60+
- name: _HASH_ENV_
61+
value: {{ include (print $BasePath "/client/configmap-env.yaml") $ | sha256sum }}
62+
- name: _HASH_CONFIG_
63+
value: {{ include (print $BasePath "/client/configmap-conf.yaml") $ | sha256sum }}
64+
lifecycle:
65+
preStop:
66+
exec:
67+
command:
68+
- sleep
69+
- '20'
70+
resources:
71+
{{ toYaml $value.resources | indent 12 }}
72+
volumeMounts:
73+
- mountPath: /workspace/client/config.ini
74+
subPath: config.ini
75+
name: client-config-conf
76+
volumes:
77+
- configMap:
78+
items:
79+
- key: config.ini
80+
mode: 420
81+
path: config.ini
82+
name: {{ template "tca.fullname" $ }}-client-config
83+
name: client-config-conf
84+
85+
{{- end }}
86+
{{- end }}

‎helm-charts/tca/templates/ingress.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ metadata:
99
chart: {{ template "tca.chart" . }}
1010
release: {{ .Release.Name }}
1111
spec:
12+
ingressClassName: nginx
1213
rules:
1314
- host: {{ .Values.tca.commonConfig.publicUrl }}
1415
http:

‎helm-charts/tca/values.yaml‎

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ global:
88
imagePullPolicy: IfNotPresent
99
## E.g.
1010
## imagePullSecrets:
11-
## - myRegistryKeySecretName
11+
## -name: myRegistryKeySecretName
1212
##
1313
imagePullSecrets: []
1414

@@ -632,27 +632,44 @@ tca:
632632
## TCA client: TCA client analysing code
633633
##
634634
client:
635+
## @param tca.client.enabled Enable default TCA client starting
636+
## @param tca.client.replicas Indicates the number of nodes in the default node pool
637+
## @param tca.client.tag Tag of the default node pool
638+
## @param tca.client.resources Cpu and memory resource configuration of the default node pool
639+
##
635640
image:
636641
repository: tencenttca/tca-client
637642
tag: latest
638-
## @param tca.client.enabled Enable TCA client starting
639-
##
640643
enabled: true
644+
replicas: 1
645+
tag: Codedog_Linux
641646
resources:
642647
limits:
643-
cpu: "4000m"
644-
memory: "8Gi"
648+
cpu: "2000m"
649+
memory: "4Gi"
645650
requests:
646651
cpu: "1000m"
647652
memory: "2Gi"
648-
## TCA client auto scaling config
649-
##
650-
autoscaling:
651-
enabled: true
652-
minReplicas: 2
653-
maxReplicas: 4
654-
targetCPU: ""
655-
targetMemory: ""
653+
654+
## TCA client node pool expansion config
655+
## @param tca.client.extraNodePools.nodepool Name of the extended node pool
656+
## @param tca.client.extraNodePools.{nodePoolName}.tag Tag of the extra node pool
657+
## @param tca.client.extraNodePools.{nodePoolName}.enabled Enable extra TCA client starting
658+
## @param tca.client.extraNodePools.{nodePoolName}.replicas Indicates the number of nodes in the extra node pool
659+
## @param tca.client.extraNodePools.{nodePoolName}.resources Cpu and memory resource configuration of the extra node pool
660+
##
661+
extraNodePools:
662+
nodepool:
663+
tag: NodePoolTag
664+
enabled: false
665+
replicas: 2
666+
resources:
667+
limits:
668+
cpu: "2000m"
669+
memory: "4Gi"
670+
requests:
671+
cpu: "1000m"
672+
memory: "2Gi"
656673

657674
## TCA gateway: TCA gateway for all servers
658675
##

‎server/configs/requirements.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Django==3.2.25
99
pymysql==1.1.1
1010

1111
# for rest api
12-
djangorestframework==3.15.2
12+
djangorestframework==3.14.0
1313
django-filter==2.4.0
1414
django-import-export==2.5.0
1515
django-cors-headers==3.7.0
@@ -24,7 +24,7 @@ Pygments==2.15.0
2424
django-guardian==2.3.0
2525

2626
# for document
27-
drf-yasg==1.17.0
27+
drf-yasg==1.21.5
2828

2929
# for celery
3030
celery==5.2.3

‎server/projects/analysis/requirements.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,14 @@ Jinja2==3.1.4
2626
markupsafe==2.0.1
2727

2828
# for rest api
29-
djangorestframework==3.11.2
29+
djangorestframework==3.14.0
3030
coreapi==2.3.3
3131
Markdown==3.3.4
3232
django-cors-headers==3.7.0
3333
Pygments==2.15.0
3434

3535
# for document
36-
drf-yasg==1.17.0
36+
drf-yasg==1.21.5
3737

3838
# for file upload and download
3939
urllib3==1.26.19

‎server/projects/file/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Django==3.2.25
99
pymysql==1.1.1
1010

1111
# for rest api
12-
djangorestframework==3.11.2
12+
djangorestframework==3.14.0
1313

1414
# for celery
1515
celery==5.2.3

‎server/projects/login/login/models.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
"""
1111
from django.db import models
1212
from django.utils import timezone
13-
from django.contrib.auth.models import AbstractBaseUser
13+
from django.contrib.auth.models import AbstractBaseUser, UserManager
1414

1515
from login.lib.Cipher import base62_cipher
1616
from login.lib.snowflake import IdWorker, GenerateId
@@ -38,6 +38,8 @@ class UserInfo(AbstractBaseUser):
3838
(0, "unknown")
3939
)
4040

41+
objects = UserManager()
42+
4143
uid = models.CharField(max_length=20, primary_key=True, default=gen_id, unique=True)
4244
nickname = models.CharField(max_length=1000)
4345
avatar_url = models.CharField(max_length=500)

0 commit comments

Comments
 (0)