Skip to content

Utilize test data from aas-core3.0-testgen rather than our proprietary set. #109

Description

@zrgt

Issue:
We've been using our proprietary test set for validating the SDK against the metamodel. However, with every change in the metamodel that necessitates an update to our SDK, we find ourselves having to make corresponding adjustments to our test data. This iterative process is resource-intensive and can lead to potential inconsistencies and oversights.

Proposal:
To streamline this process and ensure a more robust testing framework, we recommend transitioning to the test data provided by aas-core3.0-testgen. Here are the key benefits of making this switch:

  1. Comprehensive Coverage: The aas-core3.0-testgen provides an extensive set of test data, ensuring thorough validation of our SDK against a wide range of scenarios.

  2. Automatic Generation: The test data from aas-core3.0-testgen is automatically generated, which means we'll always have the most up-to-date test cases without the manual effort of creating or updating them.

  3. Reduced Maintenance: By relying on this external test set, we can significantly reduce the overhead associated with maintaining our own test data. This will allow our team to focus on enhancing the SDK and addressing other critical priorities.

  4. Consistency with the Community: Adopting a widely-used test set ensures that our SDK is in alignment with community standards and best practices.

Next Steps:
We should initiate a transition plan to integrate aas-core3.0-testgen test data into our testing pipeline. This will involve:

  • Setting up a mechanism to regularly fetch the latest test data from aas-core3.0-testgen.
  • Adapting our testing infrastructure to utilize this new data source.
  • Conducting an initial comprehensive test run to identify any gaps or inconsistencies.
  • Decommissioning our proprietary test set once the transition is complete and verified.

Activity

  1. added
    sdkSomething to do with the `sdk` package
    and removed on Nov 17, 2024
  2. hpoeche commented on Jul 27, 2026

    @hpoeche
    Contributor

    @s-heppner We should definitely reconsider this. The aas-core-works/aas-core-testdatagen repository provides fixed fuzzy tests for v3.1 of the metamodel.
    We already used this locally to discover some minor deviations in our implementation of the SDK.
    We should discuss how we can implement this in our CI in order to increase the conformance with the standard and reduce the risk of missing changes in future versions of the metamodel.

  3. s-heppner commented on Jul 27, 2026

    @s-heppner
    Member

    I agree, though we should evaluate how much resources a whole run of all the test files from testdatagen takes.
    Depending on the intensity, we can decide if this should run on every PR, or only on PRs to main.
    Let's keep it out of scope of the current CI refactor, but take it on very soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhancement of an existing featuresdkSomething to do with the `sdk` package

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions