Skip to content

[fix][COM][dist] Fix Spring Boot 2.7 deployment compatibility and transitive dependency issues#5447

Merged
casionone merged 4 commits into
masterfrom
fix-master-compile
Jun 25, 2026
Merged

[fix][COM][dist] Fix Spring Boot 2.7 deployment compatibility and transitive dependency issues#5447
casionone merged 4 commits into
masterfrom
fix-master-compile

Conversation

@aiceflower

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR fixes multiple deployment-blocking issues discovered when building and deploying the current master branch with Spring Boot 2.7. Each fix addresses a distinct failure observed during mvn package -Papache and the subsequent service startup:

  1. snakeyaml 被错误排除导致服务启动失败 (63966cfb0)

    • linkis-commons/linkis-module/pom.xml / linkis-eureka/pom.xml
    • Add snakeyaml back as an explicit compile dependency. The exclusion inherited from spring-boot-starter left Eureka/Manager unable to start (Spring needs snakeyaml to parse application.yml).
  2. 排除错误的传递依赖 jackson-module-scala_2.11 和 hadoop-hdfs-2.7.2 (f605e7142)

    • linkis-dist/pom.xml / linkis-dist/src/main/assembly/distribution.xml
    • Exclude jackson-module-scala_2.11 (Scala 2.11 binary, conflicts with the 2.12 the project uses) and hadoop-hdfs 2.7.2 (legacy Hadoop 2.x artifact that overrides hadoop-hdfs-client 3.3.4 and triggers NoClassDefFoundError on HDFS writes).
  3. 补全 Spring Boot 2.7 部署兼容性配置 (a77d0e328)

    • linkis-dist/pom.xml: add explicit commons-lang:2.6 (Eureka Client depends on its ExceptionUtils but spring-cloud-netflix no longer pulls it transitively).
    • linkis-dist/package/conf/application-engineconn.yml: add the missing Spring Boot 2.7 compatibility keys (spring.main.allow-circular-references, spring.mvc.pathmatch.matching-strategy: ant_path_matcher, spring.mvc.servlet.path, spring.cloud.loadbalancer.cache.enabled: false). The sibling application-eureka.yml and application-linkis.yml already had these on master, but the engineconn yml was missed.
  4. 修复 impala 引擎编译失败 (slf4j-api 缺失) (ee5a186c1)

    • linkis-engineconn-plugins/impala/pom.xml
    • Add slf4j-api as an explicit dependency. Without it, ImpalaEngineConnExecutor.scala fails to compile with Symbol 'type org.slf4j.Logger' is missing from the classpath because linkis-common is <scope>provided</scope> and impala-frontend/impala-minimal-hive-exec exclude all transitive deps.

Why are the changes needed?

Building and deploying the current master from a clean checkout fails end-to-end:

  • mvn package -Papache fails at the impala module.
  • Skipping impala still produces a tarball, but starting services fails with snakeyaml/class loading errors.
  • After working around those, RPC POST /api/rest_j/v1/rpc/receiveAndReply returns 404 because Spring Boot 2.7's default path_pattern_parser is incompatible with springfox 3.0.0 — the engineconn yml was missing the override.
  • After fixing the override, Eureka Client still throws NoClassDefFoundError: org.apache.commons.lang.exception.ExceptionUtils because commons-lang 2.x is no longer transitively available.

Each fix has been validated independently and the full build now completes (BUILD SUCCESS, 18:42 min).

How was this patch tested?

  • mvn clean package -Papache -DskipTests -Dmaven.javadoc.skip=true — full reactor build now passes (was: failure at linkis-engineplugin-impala).
  • Deployed the produced tarball via deploy-upgrade.sh and verified:
    • All 6 services (mg-eureka, mg-gateway, ps-publicservice, cg-linkismanager, cg-entrance, cg-engineconnmanager) start and register with Eureka.
    • Shell engine task (jobId 126376) succeeds with output matching the echo payload.
    • PerformanceInterceptor shows /rpc/receiveAndReply returning in 504 ms (no /error forward = no 404).
    • Eureka heartbeat /rpc/receive steady at 2–3 ms.

Related

  • These were originally prototyped on fix/dependency-versions (an older base). This branch was rebased onto the current master so the changes apply cleanly to the latest application-*.yml layout that master already migrated to.

aiceflower and others added 4 commits June 25, 2026 14:21
问题: linkis-module 和 linkis-eureka 的 pom.xml 中排除了 snakeyaml,
导致 Spring Boot 无法解析 application-eureka.yml 和 application-linkis.yml,
所有服务启动失败。

修复: 在两个模块中添加 snakeyaml 为显式 compile 依赖,覆盖排除策略。

关联部署问题: jackson-module-scala 和 hadoop-hdfs 版本问题由增量构建时
profile 混用导致,-Papache profile 配置正确,干净编译即可解决。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
问题: Maven 传递依赖解析时,scala.binary.version property 在 linkis-common
的依赖声明中未正确传递到 linkis-dist,导致 jackson-module-scala_2.11 (应为 2.12)
和 hadoop-hdfs:2.7.2 (应为 hadoop-hdfs-client:3.3.4) 被错误打包。

修复: 在 linkis-dist/pom.xml 中显式排除这两个错误版本并引入正确版本。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- linkis-dist/pom.xml: 显式引入 commons-lang 2.6 (Eureka Client 依赖其
  ExceptionUtils, 但 spring-cloud-netflix 传递依赖未带入)
- application-engineconn.yml: 补全 spring.main.allow-circular-references +
  spring.mvc.pathmatch.matching-strategy (ant_path_matcher) +
  spring.mvc.servlet.path + spring.cloud.loadbalancer.cache.enabled=false

  eureka.yml 和 linkis.yml 已在 master 上含相关配置,但 engineconn.yml
  遗漏了。缺失会导致 RPC POST /rpc/receiveAndReply 返回 404,引擎启动失败。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
impala 模块编译报错 "Symbol 'type org.slf4j.Logger' is missing from the
classpath",根因是 slf4j-api 不在 impala 编译 classpath 中:

- linkis-common 标记为 <scope>provided</scope>,不传递 slf4j-api
- impala-frontend / impala-minimal-hive-exec 用 <exclusions>*</exclusions>
  排除所有传递依赖,slf4j-api 也被一并排除

Logging.logger 和 Utils.tryAndWarn 隐式参数都依赖 org.slf4j.Logger,
必须显式引入 slf4j-api 才能编译通过。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@casionone casionone left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@casionone casionone merged commit 4b115ab into master Jun 25, 2026
10 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants