Repository navigation
fix(upload): flatten both path separators in the upload state file name - #1218
Merged
stevemessick merged 3 commits intoOct 23, 2026
Merged
Conversation
ResumableUploadContext.get_upload_info_file_path() turns the path of the file being uploaded into a file name inside the upload state directory. It only replaced os.path.sep, which is "\" on Windows, so a path written with forward slashes (C:/data/train.csv, as produced by Git Bash, MSYS or Path.as_posix()) kept its "/" and pointed into sub-directories of the state dir that do not exist. Saving the state then failed with "[Errno 2] No such file or directory" and the upload aborted. Replace "\", "/" and ":" alike, and add tests for Windows, POSIX and relative paths.
Collaborator
|
Tested locally and LGTM! |
Contributor
|
/gcbrun |
Contributor
|
/gcbrun |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Windows,
kaggle datasets create/kaggle datasets version(and any other command that uses resumable uploads) fails when the folder path is written with forward slashes, for example from Git Bash:Cause.
ResumableUploadContext.get_upload_info_file_path()flattens the uploaded file's path into a state-file name withpath.replace(os.path.sep, "_"). On Windowsos.path.sepis\, so/survives, and the name points into sub-directories of the state dir that do not exist. A relative-ppath happens to work because the paths below the folder are built withos.path.join.Fix. Replace
\,/and:alike (re.sub(r"[\\/:]", "_", path)), so the state file is always a single file directly inside the state dir. On POSIX,\is a legal file-name character, so the only effect there is a different name for the temporary state file.The state-file name changes for such paths, so an upload interrupted before upgrading restarts instead of resuming. On Windows those uploads could not save their state anyway.
Tests. I added
TestUploadInfoFilePathtotests/unit/test_upload_helpers.py. It covers Windows paths with/,\and mixed separators, POSIX paths and relative paths, and it writes a state file for a forward-slash Windows path. On the current main, 4 sub-cases and the write test fail; with the fix, all pass.I ran
tests/uniton Windows (Python 3.13). The only failure is the existingtest_symlinked_state_dir_is_rejected, which fails on unpatched main too, because creating a symlink there needs a privilege (WinError 1314).I also added a line to the
### Nextsection of the CHANGELOG.