Skip to content

CI: Write the AAS schemas where the tests read them - #639

Merged
zrgt merged 2 commits into
eclipse-basyx:developfrom
leogrox:fix/ci-schema-path
Oct 6, 2026
Merged

zrgt merged 2 commits into
eclipse-basyx:developfrom
leogrox:fix/ci-schema-path

Conversation

@leogrox

@leogrox leogrox commented Sep 20, 2026 •

Copy link
Copy Markdown

Fixes #637.

pr.yml downloads the schema files into ./test/adapter/schema, while the tests read ../schemas/, that is test/adapter/schemas, the path .gitignore already covers (L27). The files never reach the tests, so JsonSerializationSchemaTest and XMLSerializationSchemaTest skip their whole class on every run: that is the skipped=2 in a green build.

Renaming the directory in the workflow is enough to make them run.

Testing

Reproduced the CI layout locally, with the schemas in test/adapter/schema, and both classes skip:

setUpClass (…JsonSerializationSchemaTest) ... skipped 'JSON Schema does not exist at …/test/adapter/json/../schemas/aasJSONSchema.json'
setUpClass (…XMLSerializationSchemaTest) ... skipped 'XSD schema does not exist at …/test/adapter/xml/../schemas/aasXMLSchema.xsd'
OK (skipped=2)

With the files in test/adapter/schemas the 13 tests in those classes run. Two of them fail on develop, which is #636, fixed separately; with both changes the suite is green (282 tests).

Two things I left out on purpose, happy to add either if you want them here rather than in a follow-up: making the absence of the schema fatal in CI (a missing file currently turns into a green build), and pointing the workflow and the tests at one shared constant so the two paths can't drift again.

@hpoeche

hpoeche commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Thank you for spotting and fixing this mistake. As you pointed out in #637, the root cause here is that the test are skipped silently in the CI.
We are currently implementing unittests for the server package, where we focused a similar situation with the integration tests: It is okay to skip them in local dev environments, but the CI must not skip them but fail. We solved this by introducing an environment variable that prevents skipping the tests when set.
You can take a look at our implementation here:
https://github.com/rwth-iat/basyx-python-sdk/blob/aeaa5063ff214012529c4fcc25d0f83930189a6b/server/test/_helper/test_helpers.py#L27

If @s-heppner agrees, it would be very helpful to implement this approach here too.

The workflow downloads them into `test/adapter/schema`, while the tests read
`test/adapter/schemas`, the path `.gitignore` already covers. The files
therefore never reach the tests, `JsonSerializationSchemaTest` and
`XMLSerializationSchemaTest` skip their whole class on every run, and CI has
been reporting `OK (skipped=2)` while validating nothing against the schema.

Signed-off-by: LGUIUX <phdl@outlook.es>
The schema validation tests skip their whole class when the schema files
are missing. That is right for a local run, but it is also what let the
misplaced download in CI go unnoticed.

Follow the approach of the server's Docker integration tests: with
`REQUIRE_SCHEMA_TESTS` set, a missing schema file is an error instead of a
skip. `pr.yml` sets it for the unittest step; local runs are unaffected.

Signed-off-by: LGUIUX <phdl@outlook.es>
@leogrox

leogrox commented Sep 22, 2026

Copy link
Copy Markdown
Author

Thanks, that addresses the actual root cause better than the path fix alone. I've added it here as a separate commit, so it's easy to drop if @s-heppner would rather keep this PR to the path:

  • test/_helper/test_helpers.py gets REQUIRE_SCHEMAS, read from REQUIRE_SCHEMA_TESTS, same shape as your REQUIRE_SERVER_INTEGRATION_TESTS.
  • JsonSerializationSchemaTest and XMLSerializationSchemaTest raise in setUpClass when it's set and the schema file is missing. Otherwise they skip as before.
  • pr.yml sets it on the unittest step.

Checked from sdk/: without the schema files and with the variable set, both classes error out (FAILED (errors=2)); unset or 0, they skip as before. With the files in place this branch still shows the two #636 failures, so #638 needs to go in first; with both, the suite is green (282 tests).

The CouchDB tests skip the same way when the service isn't reachable. I left them alone to keep this about the schemas, but the same switch would work there if you want it.

@zrgt

zrgt commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

@hpoeche please review

@hpoeche hpoeche left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you very much!
@zrgt Please double check the CI changes. Afterwards, I would recommend to approve, so the CI checks can run.

@s-heppner s-heppner modified the milestones: Release 2.2.0, Release 2.3.0 Oct 2, 2026
@s-heppner

Copy link
Copy Markdown
Member

I bumped the milestone for this to version 2.3.0, as we release 2.2.0 today and this is not ready yet.

@zrgt zrgt closed this Oct 6, 2026
@zrgt zrgt reopened this Oct 6, 2026
@zrgt

zrgt commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Closed and reopened to re-trigger CI against the current develop. Re-running the old workflow kept testing the original merge commit, which doesn't include #638, so the schema tests kept failing.

@zrgt
zrgt merged commit c3b28b1 into eclipse-basyx:develop Oct 6, 2026
78 of 86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants