Ansible playbook for test-operator

[jancervenka]

OpenStack test-operator with an appropriate AnsibleTest CR can execute the lightspeed-tests using the committed Ansible playbook.

How to run the tests:

0. Pre-requisities: OpenStack Lightspeed running in your cluster.

1. Install and run the test-operator with:

# in the test-operator directory
make install
ENABLE_WEBHOOKS=false make run

3. Create a file lightspeed-auth-resources.yaml with the following content:

    ---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: lightspeed-test-role
rules:
- nonResourceURLs: ["/ls-access"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: lightspeed-test-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: lightspeed-test-role
subjects:
- kind: ServiceAccount
  name: default
  namespace: openstack-lightspeed

And run oc apply -f lightspeed-auth-resources. This will grant Lightspeed service access to default service account used by the test-operator.

4. Create a file lightspeed-dummy-resources.yaml with the following content:

---
# These resources must exist before running lightspeed-tests
# Apply once: oc apply -f lightspeed-persistent-resources.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: test-operator-dummy-config
  namespace: openstack-lightspeed
  labels:
    app: lightspeed-tests
    persistent: "true"
    managed-by: test-operator
data:
  clouds.yaml: |
    # Dummy file - not used by lightspeed tests
    # Required by test-operator but lightspeed tests don't need OpenStack
---
apiVersion: v1
kind: Secret
metadata:
  name: test-operator-dummy-secret
  namespace: openstack-lightspeed
  labels:
    app: lightspeed-tests
    persistent: "true"
    managed-by: test-operator
type: Opaque
stringData:
  secure.yaml: |
    # Dummy file - not used by lightspeed tests
    # Required by test-operator but lightspeed tests don't need OpenStack
---
# Dummy SSH key - required by test-operator but not used by lightspeed tests
apiVersion: v1
kind: Secret
metadata:
  name: dataplane-ansible-ssh-private-key-secret
  namespace: openstack-lightspeed
  labels:
    app: lightspeed-tests
    persistent: "true"
    managed-by: test-operator
type: Opaque
stringData:
  ssh-privatekey: |
    # Dummy SSH key - not used by lightspeed tests

And run oc apply -f lightspeed-dummy-resources.yaml. This will create empty configs that are expected by the test-operator but are not actually used for anything. Without the dummy resources, the test operator will not run the tests.

5. Create an AnsibleTest CR file lightspeed-ansible-test-cr.yaml with the following content:

---
apiVersion: test.openstack.org/v1beta1
kind: AnsibleTest
metadata:
  name: lightspeed-tests
  namespace: openstack-lightspeed
  labels:
    app.kubernetes.io/name: test-operator
    app.kubernetes.io/managed-by: kustomize
spec:
  openStackConfigMap: test-operator-dummy-config
  openStackConfigSecret: test-operator-dummy-secret
  ansibleGitRepo: https://github.com/openstack-k8s-operators/lightspeed-tests
  ansibleGitBranch: main
  ansiblePlaybookPath: playbooks/run_lightspeed_tests.yaml

  # Ansible inventory - run tests on localhost
  ansibleInventory: |
    localhost ansible_connection=local ansible_python_interpreter=python3

  debug: true
  containerImage: quay.io/podified-antelope-centos9/openstack-ansible-tests:current-podified

  # Storage class for any PVCs (test results, logs)
  # if running in CRC, use crc-csi-hostpath-provisioner
  # if running in OpenShift cluster, use: local-storage
  storageClass: crc-csi-hostpath-provisioner

  # Enable privileged mode to mount service account token
  privileged: true
  # Resource limits for the test pod
  resources:
    limits:
      cpu: 2000m
      memory: 2Gi
    requests:
      cpu: 1000m
      memory: 1Gi

  # Variables passed to Ansible playbook
  ansibleVarFiles: |
    ---
    # Lightspeed service URL (defaults to in-cluster service)
    lightspeed_url: "https://lightspeed-app-server.openstack-lightspeed.svc.cluster.local:8443"
    lightspeed_timeout: 30
    test_question: "What is OpenStack?"
    # Pytest options
    pytest_args: "-v --tb=short --color=yes"
    # JUnit XML output path (test-operator writable location)
    junit_xml_path: "/var/lib/AnsibleTests/external_files/test-results.xml"

Run oc apply -f lightspeed-ansible-test-cr.yaml to run the test suite.

If all goes well, you should be able to inspect the logs with oc logs lightspeed-tests -n openstack-lightspeed and see at the end:

TASK [Tests completed successfully] ********************************************
task path: /var/lib/ansible/ansible/playbooks/run_lightspeed_tests.yaml:142
ok: [localhost] => {
    "msg": "All RHOS Lightspeed tests passed successfully!"
}

PLAY RECAP *********************************************************************
localhost

The lightspeed-tests pod should report status COMPLETED:

➜  test-operator git:(main) oc get pods
NAME                                                              READY   STATUS      RESTARTS   AGE
lightspeed-console-plugin-bdd77fd5-qq64v                          1/1     Running     0          2d9h
lightspeed-okp-server-66fb4cf76c-68l87                            1/1     Running     0          2d9h
lightspeed-postgres-server-65bdbbcdcb-dsn4p                       1/1     Running     0          2d9h
lightspeed-stack-deployment-5cc6646f98-24czp                      3/3     Running     0          2d9h
lightspeed-tests                                                  0/1     Completed   0          7h57m
openstack-lightspeed-operator-controller-manager-5c9fc4d65cqmcz   1/1     Running     0          2d9h
toolbox-67648944cd-7spr4                                          1/1     Running     0          2d9h

To re-run the tests, delete the CR and create it again:

oc delete ansibletest lightspeed-tests -n openstack-lightspeed
oc apply -f lightspeed-ansible-test-cr.yaml

[lpiwowar]

Nice one!:) 👍 🎉

Three comments:). We have to really figure out the creation of the dummy resources and the Cluster Role related resources. It might not be part of this playbook but we should be relatively certain about it before we merge.

[lpiwowar]

issue (blocking): We have to figure out what to do with the creation of the ClusterRole and the ClusterRoleBinding:

• Can it happen in the ansible test pod? I do not think so but we should check (the mounted SA probably does not have the necessary privileges). If not then where can we put it? Maybe the @malingatembo 's cifmw playbook?

Also, we have to figure out where to create the "dummy" resources:

• Can we create them here for now? If not where will they run?

[jancervenka]

@lpiwowar I would not create the ClusterRole and ClusterRoleBinding in Ansible test pod. I was trying that approach and it is messy (that's why I ended up removing all the auth setup/cleanup from this repository).

The dummy resources need to be created before the AnsibleTest CR is created (and before the playbook is executed). Otherwise, the tests will not start running.

The test-operator will throw errors if the AnsibleTest CR is created before the dummy resources. Also, I feel the dummy resources are not related to the lightspeed-tests but they are part of the test-operator setup. I think it would be better to not create them here.

I think everything (dummy and auth resources) should be created in the same place where the AnsibleTest CR will be created. Which I think is Malinga's cifmw playbook, right?

[lpiwowar]

Which I think is Malinga's cifmw playbook, right?

Sorry! I was too quick with the review yesterday. I'm going to do a proper one soon. You are right, those need to exist prior to the AnsibleTest creation.

I think we have to update the PR from @malingatembo. Maybe you can try to create a stacked PR on top of the @malingatembo's PR and ensure that these resources get created there? 👀 -> https://github.com/openstack-k8s-operators/ci-framework/pulls/malingatembo . It can be a draft for now but once Malinga is done with his job creation we can test the PR you create and potentially merge it with Malinga's PR. Only if you want to. It is a bit of stretch beyond the ticket but it would be nice to do IMO.

For now we are aiming at making this work but in the long term we might want to consider updating test-operator not requiring the dummy resources.

[jancervenka]

I agree! I will prepare the PR 👍

[lpiwowar]

Overall LGTM!:) Thank you! 🎉

I suggest fixing the ansible-lint errors and extending the pre-commit with it.

I'm giving an approve. We have really nothing to lose here and we can polish it later:).

I'm leaving the final merging to @malingatembo though.

[lpiwowar]

suggestion (non-blocking): Let's extend the pre-commit checks with ansible-lint [1]. There are couple of minor things ansible-lint is complaining about:

Installed 29 packages in 147ms
WARNING  Listing 3 violation(s) that are fatal
yaml[truthy]: Truthy value should be one of [false, true]
playbooks/run_lightspeed_tests.yaml:4

yaml[truthy]: Truthy value should be one of [false, true]
playbooks/run_lightspeed_tests.yaml:27

yaml[truthy]: Truthy value should be one of [false, true]
playbooks/run_lightspeed_tests.yaml:106

Read documentation for instructions on how to ignore specific rule violations.

# Rule Violation Summary

  3 yaml profile:basic tags:formatting,yaml

Failed: 3 failure(s), 0 warning(s) in 15 files processed of 20 encountered. Last profile that met the validation criteria was 'min'.

[1]

lightspeed-tests/.pre-commit-config.yaml

Line 1 in 670824b

repos:

[jancervenka]

thank you! @lpiwowar ansible-lint is added to the pre-commit and the lint errors are fixed.