Skip to content

[Fix] Fix memory sizes config not taking effect in Yarn mode - #4502

Open
zhang-arvin wants to merge 1 commit into
apache:devfrom
zhang-arvin:fix-memory-sizes-config
Open

[Fix] Fix memory sizes config not taking effect in Yarn mode#4502
zhang-arvin wants to merge 1 commit into
apache:devfrom
zhang-arvin:fix-memory-sizes-config

Conversation

@zhang-arvin

Copy link
Copy Markdown

What problem does this PR solve?

Fixes #3617: The memory sizes configuration in Application Conf (e.g., jobmanager.memory.process.size, taskmanager.memory.process.size) was not taking effect when submitting Flink jobs in Yarn mode.

Root Cause

In FlinkClientTrait.prepareConfig(), the appProperties() from the Application Conf (parsed with flink.property. prefix) were never applied to the Flink configuration. While submitRequest.properties() (dynamic properties from the web UI) were correctly passed as -D flags via buildCliArgs(), the Application Conf properties were silently ignored.

Fix

Added applyAppProperties() method in FlinkClientTrait that applies all appProperties() from the Application Conf to the Flink configuration. This method is called in prepareConfig() after applyEnvProperties(), ensuring that all Flink configuration properties set in the Application Conf are properly propagated to the submit request.

Changes

  • streampark-flink-client-core: Added applyAppProperties() method to FlinkClientTrait and called it in prepareConfig()
  • streampark-flink-client-core test: Added 3 test cases for SubmitRequestTest covering memory config parsing from YAML, JSON, and Properties format Application Conf

Verification

  • All existing tests pass (12/12 in SubmitRequestTest)
  • New tests cover memory configs in YAML, JSON, and Properties formats

@github-actions github-actions Bot added the FLINK label Aug 18, 2026
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] the config of 'memory sizes' in [Application Conf] do not effective

1 participant