fix: make EnsembleByKey transformSchema match output

[fallintoplace]

Problem

EnsembleByKey.transformSchema describes the opposite shape from transform:

• with collapseGroup=true, transform returns grouping keys and aggregate columns, while transformSchema keeps every input column
• with collapseGroup=false, transform joins aggregates back to the input, while transformSchema keeps only fields selected from the aggregation column set

This can break pipeline schema validation because the declared schema differs from the DataFrame produced at runtime. The old implementation also fails when transformSchema is called before transform without explicit colNames, and declares FloatType for scalar means that Spark produces as DoubleType.

Changes

• initialize default output names consistently from both transform and transformSchema
• derive grouping fields from keys
• return grouping fields followed by aggregate fields for collapsed groups
• match Spark join ordering for non-collapsed groups: keys, remaining non-overwritten input fields, then aggregates
• declare scalar mean outputs as DoubleType for both float and double inputs
• assert exact schema equality for double and float inputs in both collapse modes, with distinct key and aggregation columns and no explicit output name

Testing

• core/testOnly com.microsoft.azure.synapse.ml.stages.EnsembleByKeySuite (9 tests passed)
• core/Test/scalastyle (0 findings)
• core/Compile/scalastyle (0 findings)

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[github-actions]

Hey @fallintoplace 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[Copilot]

Pull request overview

This PR updates EnsembleByKey’s transformSchema logic so it matches the actual output schema produced by transform, across both collapseGroup=true and collapseGroup=false modes, and adds a regression test to validate schema consistency.

Changes:

• Rebuild transformSchema output fields using keys for grouping fields and consistent output ordering for both collapse modes.
• In non-collapsing mode, model Spark join column ordering and drop/replace conflicting output column names in the predicted schema.
• Add a test asserting transformSchema(schema) equals transform(df).schema for both collapse modes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
core/src/main/scala/com/microsoft/azure/synapse/ml/stages/EnsembleByKey.scala	Adjusts transformSchema field selection and ordering to align with transform behavior.
core/src/test/scala/com/microsoft/azure/synapse/ml/stages/EnsembleByKeySuite.scala	Adds schema-equality coverage for both collapseGroup modes.