[Console] Fix four DTO/entity defects that break Spark and Flink SQL app management - #4497
Open
88fantasy wants to merge 1 commit into
Open
[Console] Fix four DTO/entity defects that break Spark and Flink SQL app management#449788fantasy wants to merge 1 commit into
88fantasy wants to merge 1 commit into
Conversation
…app management Each of these makes a normal UI operation fail outright; all four are reproducible on a clean install. 1. FlinkAppCreateRequest carries no sqlId, though the frontend sends one and FlinkApplicationManageServiceImpl.updateFlinkSqlJob() requires it for an application that has already been started. BeanUtils.copyProperties silently skips the absent field, the entity's sqlId stays null, and the save fails with "Flink sql is null, update flink sql job failed." Only never-started applications were unaffected, since those take the candidate branch. SparkAppCreateRequest and FlinkAppResponse both already declare sqlId — this was a one-sided omission on the Flink request DTOs. 2. SparkAppStateEnum.of(Integer) compares appState.value == state, which unboxes. SparkAppListQueryRequest has a scalar state field that is null whenever the list is not filtered by status, so the copy to SparkApplication leaves state null and shouldTracking() -> getStateEnum() -> of(null) throws NPE: /spark/app/list returns 500 for everyone. Now returns OTHER for null. FlinkAppStateEnum.getState(Integer) has the same unboxing shape but no scalar state field reaching it today, so it is left alone here. 3. SparkEnv.doSetSparkConf() only assigned sparkConf when conf/spark-defaults.conf existed. A stock Spark distribution ships only spark-defaults.conf.template, so for an unmodified Spark home the field stays null and the insert fails on t_spark_env.spark_conf, which is NOT NULL with no default — no Spark home could be registered at all. An absent file is now stored as an empty conf. 4. SparkApplication.k8sImagePullPolicy was a primitive int while SparkAppCreateRequest declares Integer and the Spark frontend never sends the field, so BeanUtils.copyProperties unboxed null and /spark/app/create returned 500 for every UI-created Spark application. The entity field now matches its nullable column.
|
Member
|
I noticed your recent code contributions — they look good. I wanted to ask which AI model you used, including the specific model/version, and how you prompted the model to complete this contribution. We’re currently planning to introduce some guidelines and best practices for AI-assisted coding in our open-source project, so your experience would be very helpful. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What is the purpose of the change
Four small DTO/entity defects, each of which makes a normal UI operation fail outright on a clean install. They are unrelated to each other in cause but identical in shape — a field that does not line up between the request DTO, the entity, and the column — so they are grouped in one PR.
Closes #4491
Closes #4492
Closes #4493
Closes #4494
Brief change log
FlinkAppCreateRequestdeclaressqlId([Bug] Saving an already-started Flink SQL application from the UI always fails (FlinkAppCreateRequest has no sqlId) #4491). The frontend sends it andupdateFlinkSqlJob()requires it, but the DTO had no such field, soBeanUtils.copyPropertiesskipped it and saving any already-started Flink SQL application failed withFlink sql is null, update flink sql job failed.SparkAppCreateRequestandFlinkAppResponsealready declare it — this was a one-sided omission.SparkAppStateEnum.of(Integer)returnsOTHERfor null ([Bug] Spark application list page returns 500 (SparkAppStateEnum.of unboxes a null state) #4492). It compared withappState.value == state, which unboxes;SparkAppListQueryRequest's scalarstateis null for an unfiltered list, so/spark/app/listreturned 500 for everyone.FlinkAppStateEnum.getState(Integer)has the same unboxing shape but nothing reaches it with a null today, so it is deliberately left alone here.SparkEnv.doSetSparkConf()stores an empty conf when the file is absent ([Bug] A stock Spark distribution cannot be registered (spark-defaults.conf is absent, spark_conf is NOT NULL) #4493). A stock Spark distribution ships onlyspark-defaults.conf.template, sosparkConfstayed null and the insert failed ont_spark_env.spark_conf(NOT NULL, no default) — no Spark home could be registered at all.SparkApplication.k8sImagePullPolicyisInteger([Bug] Creating a Spark application from the UI always returns 500 (k8sImagePullPolicy primitive int) #4494), matching its nullable column and theIntegerinSparkAppCreateRequest. As a primitive it unboxed the null the Spark frontend never sends, so/spark/app/createreturned 500 for every UI-created application.Verifying this change
/spark/app/listfrom 500 to 200.conf/spark-defaults.confin an otherwise stock Spark home makes registration succeed with the file's existence as the only changed variable.mvn testonstreampark-console-servicepasses (93 tests, 0 failures).Does this pull request potentially affect one of the following parts
FlinkAppCreateRequestgains an optionalsqlId, andSparkApplication.getK8sImagePullPolicy()now returnsIntegerrather thanint.Documentation