Skip to content

SOLR-18320 follow-up: streaming zipDirectory, drop dead passthrough wrapper - #5260

Merged
epugh merged 2 commits into
apache:mainfrom
epugh:SOLR-18320-simplify-followup
Oct 28, 2026
Merged

epugh merged 2 commits into
apache:mainfrom
epugh:SOLR-18320-simplify-followup

Conversation

@epugh

@epugh epugh commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

I was too quick on the merge and realized I had some self review comments I had not dealt with.

Summary

Small cleanup follow-up to #4672 (already merged), from a /simplify pass that landed one commit after that PR was squash-merged, so it never made it in.

  • Utils.zipDirectory(Path) was shaped around "return bytes for an HTTP body" - fine for its two other callers (CreateTool, DownloadConfigSet), but it forced RemoteSolrBackend.uploadConfigSet to buffer the entire configset zip in memory just to write it straight back out to a temp file it already had open (Files.write(tempZip, Utils.zipDirectory(configDir))). Added a streaming zipDirectory(Path, OutputStream) overload with the shared walk/skip-hidden/normalize-separator logic in one place; the byte[]-returning version now delegates to it, and RemoteSolrBackend calls the streaming form directly against Files.newOutputStream(tempZip), restoring the original single-pass I/O.
  • Removed CreateTool.zipConfigSet(Path), a pure one-line passthrough to Utils.zipDirectory with no behavior of its own. Both call sites (the production upload call and CreateToolTest) now call Utils.zipDirectory directly.
  • Added a test confirming the two zipDirectory overloads produce equivalent archives - compared by decoded entry name/content rather than raw bytes, since ZipEntry defaults to the current time when none is set explicitly, so two back-to-back calls aren't guaranteed byte-identical.

Test plan

🤖 Generated with Claude Code

- Utils.zipDirectory(Path) was the only shape used by two of its three
  callers (CreateTool, DownloadConfigSet), which both feed the bytes into
  an HTTP request body - fine for them, but it forced RemoteSolrBackend to
  buffer the whole configset in memory just to write it straight back out
  to a temp file it already had open. Added a streaming
  zipDirectory(Path, OutputStream) overload with the walk/skip-hidden/
  normalize-separator logic in one place; zipDirectory(Path) now delegates
  to it via a ByteArrayOutputStream, and RemoteSolrBackend calls the
  streaming form directly against Files.newOutputStream(tempZip),
  restoring the original single-pass I/O.
- Removed CreateTool.zipConfigSet(Path), a pure one-line passthrough to
  Utils.zipDirectory with no behavior of its own; both call sites (the
  production upload call and CreateToolTest) now call Utils.zipDirectory
  directly.
- Added a test confirming the two zipDirectory overloads produce
  equivalent archives (compared by decoded entry name/content, not raw
  bytes - ZipEntry defaults to the current time when unset, so two
  back-to-back calls aren't guaranteed byte-identical).

Verified: CreateToolTest, CreateToolPicocliTest, and
TestSchemaDesignerConfigSetHelper (14 tests) all pass; full compileJava
compileTestJava across the repo is clean; ./gradlew tidy clean.
@epugh

epugh commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

/no-changelog

@epugh
epugh merged commit 0524480 into apache:main Oct 28, 2026
7 of 8 checks passed
@dsmiley

dsmiley commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

I wish I had a chance to review. The title says "streaming zipDirectory" but I don't see it. The following method replacement actually streams the zip (meaning, doesn't use an intermediate temp file). I will get this in at some point in the future when I return to SolrBackend stuff:

  public void uploadConfigSet(Path configDir, String name) throws SolrServerException, IOException {
    new ConfigSetAdminRequest.Upload()
        .setConfigSetName(name)
        .setUploadContent(new RequestWriter.ContentWriter() {
          @Override
          public void write(OutputStream os) throws IOException {
            // stream it directly out; no intermediate temp file :-)
            Utils.zipDirectory(configDir, os);
          }

          @Override
          public String getContentType() {
            return "application/zip";
          }
        })
        .process(adminClient);
  }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants