Skip to content

TEZ-4754: RelocalizationUtils: input handling improvements - #537

Open
abstractdog wants to merge 1 commit into
apache:masterfrom
abstractdog:TEZ-4754
Open

abstractdog wants to merge 1 commit into
apache:masterfrom
abstractdog:TEZ-4754

Conversation

@abstractdog

Copy link
Copy Markdown
Contributor

Tighten validation of the destination file name used when downloading additional AM resources so unexpected shapes are rejected before the file is written. Adds a small unit test.

@abstractdog
abstractdog requested a lite review from Copilot September 18, 2026 12:32
@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 3m 3s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 3m 17s master passed
+1 💚 compile 2m 44s master passed
+1 💚 checkstyle 0m 20s master passed
+1 💚 javadoc 0m 15s master passed
+0 🆗 spotbugs 0m 34s tez-common in master has 13 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 29s the patch passed
+1 💚 codespell 0m 52s No new issues.
+1 💚 compile 2m 45s the patch passed
+1 💚 javac 2m 45s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 checkstyle 0m 13s the patch passed
+1 💚 javadoc 0m 13s the patch passed
+1 💚 spotbugs 0m 44s the patch passed
_ Other Tests _
+1 💚 unit 61m 35s root in the patch passed.
+1 💚 asflicense 0m 19s The patch does not generate ASF License warnings.
80m 26s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/1/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux 88f9a15edb42 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 0e9f756
Default Java Eclipse Adoptium-21.0.12+8-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/1/testReport/
Max. process+thread count 1361 (vs. ulimit of 5500)
modules C: tez-common U: tez-common
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

URI-scheme traversal and submission-state corruption remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Tightens destination filename validation for relocalized AM resources and adds unit coverage.

Changes:

  • Rejects empty, path-based, absolute, and traversal-like names.
  • Adds tests for valid and invalid destination names.
File summaries
File Description
tez-common/src/test/java/org/apache/tez/dag/utils/TestRelocalizationUtils.java Adds coverage for accepted and rejected destination names.
tez-common/src/main/java/org/apache/tez/dag/utils/RelocalizationUtils.java Validates destination names before downloading resources.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +82 to +91
if (destName.indexOf('/') >= 0 || destName.indexOf('\\') >= 0
|| destName.indexOf('\0') >= 0) {
throw new IllegalArgumentException(
"Resource name must not contain path separators: " + destName);
}
if (destName.equals(".") || destName.equals("..")) {
throw new IllegalArgumentException(
"Resource name must not be a parent-directory reference: " + destName);
}
if (new Path(destName).isAbsolute()) {

@abstractdog abstractdog Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not true, adding a unit test to prove: uriSchemeOpaqueFormIsRejected
new Path(destName) call inside RelocalizationUtils#validateDestName already throwsIllegalArgumentException from Hadoop's URI parser in such cases

// absolute path, a path separator, or a ".." segment — so a submitter
// holding AM modify access cannot land a jar anywhere on the AM host and
// chain it into code execution via the classloader / am-hooks path.
validateDestName(destName);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack, handling with an upfront check

Tighten validation of the destination file name used when downloading
additional AM resources so unexpected shapes are rejected before the
file is written. Adds a small unit test.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 9s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for branch
+1 💚 mvninstall 2m 48s master passed
+1 💚 compile 2m 32s master passed
+1 💚 checkstyle 0m 47s master passed
+1 💚 javadoc 0m 40s master passed
+0 🆗 spotbugs 0m 35s tez-common in master has 13 extant spotbugs warnings.
+0 🆗 spotbugs 1m 0s tez-dag in master has 537 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 7s Maven dependency ordering for patch
+1 💚 mvninstall 2m 10s the patch passed
+1 💚 codespell 0m 49s No new issues.
+1 💚 compile 2m 35s the patch passed
+1 💚 javac 2m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 43s the patch passed
+1 💚 javadoc 0m 39s the patch passed
+1 💚 spotbugs 1m 51s the patch passed
_ Other Tests _
-1 ❌ unit 61m 3s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 0m 43s The patch does not generate ASF License warnings.
80m 59s
Reason Tests
Failed junit tests tez.history.TestHistoryParser
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/3/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux 49a492187ff4 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 0e9f756
Default Java Eclipse Adoptium-21.0.12+8-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/3/testReport/
Max. process+thread count 1377 (vs. ulimit of 5500)
modules C: tez-common tez-dag U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/3/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for branch
+1 💚 mvninstall 2m 52s master passed
+1 💚 compile 2m 32s master passed
+1 💚 checkstyle 0m 45s master passed
+1 💚 javadoc 0m 38s master passed
+0 🆗 spotbugs 0m 35s tez-common in master has 13 extant spotbugs warnings.
+0 🆗 spotbugs 0m 55s tez-dag in master has 537 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 5s Maven dependency ordering for patch
+1 💚 mvninstall 2m 14s the patch passed
+1 💚 codespell 0m 49s No new issues.
+1 💚 compile 2m 34s the patch passed
+1 💚 javac 2m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 40s the patch passed
+1 💚 javadoc 0m 36s the patch passed
+1 💚 spotbugs 1m 51s the patch passed
_ Other Tests _
+1 💚 unit 60m 29s root in the patch passed.
+1 💚 asflicense 0m 38s The patch does not generate ASF License warnings.
80m 13s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/4/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense javac javadoc spotbugs checkstyle codespell detsecrets
uname Linux 1c62eb6b88f0 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 0e9f756
Default Java Eclipse Adoptium-21.0.12+8-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/4/testReport/
Max. process+thread count 1361 (vs. ulimit of 5500)
modules C: tez-common tez-dag U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-537/4/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.3 codespell=2.4.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants