ByteBufferCleaner to check Unsafe access in Java 23+

[maxxedev]

In Java 23+, ByteBufferCleaner should check for access to deprecated memory-access methods in sun.misc.Unsafe

• Read the contribution guidelines for this project.
• Read the ASF Generative Tooling Guidance if you use Artificial Intelligence (AI).
• [] I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute? Not used
• Run a successful build using the default Maven goal with mvn; that's mvn on the command line by itself.
• Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible, but it is a best practice.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Each commit in the pull request should have a meaningful subject line and body. Note that a maintainer may squash commits during the merge process.

[garydgregory]

Converting to draft until I can have a closer look, needs clean ups anyway.

[garydgregory]

There are several problems here:

• The build fails
• Using java.lang.management adds a new module dependency which some people will complain about. We could disallow this through Checkstyle but we currently do not.
• It's not clear to me this is the correct implementation since the JEP is just a deprecation.
• There is a bug where clearWritable() is not called.

[maxxedev]

build fails

fixed

* Using `java.lang.management` adds a new module dependency which some people will complain about. We could disallow this through Checkstyle but we currently do not.

we just disable Unsafe access on Java 23+ now

* It's not clear to me this is the correct implementation since the JEP is just a deprecation.

deprecation is for removal. Plus, most users would want the library to not trigger scary warnings.

* There is a bug where `clearWritable()` is not called.

fixed

[garydgregory]

@maxxedev The builds for this PR have been broken since last week.

[maxxedev]

@garydgregory please review again :)

[garydgregory]

Hello @maxxedev,

Thank you for updating this PR.

I don't think this is acceptable as-is because it now violates the write once, run anywhere Java principle: the behavior changes depending on which Java version the application is running on, with no way for the user to control or opt out of that change. I think that is particularly problematic for a deprecated API; the fact that an API is deprecated should not mean its behavior changes depending on the runtime version.

If you'd like to pursue this change further, I suggest making the new behavior explicitly opt-in. That would preserve the existing behavior by default while still allowing users who want the new behavior to enable it.

[maxxedev]

I don't think this is acceptable as-is because it now violates the write once, run anywhere Java principle: the behavior changes depending on which Java version the application is running on [...]. I think that is particularly problematic for a deprecated API; the fact that an API is deprecated should not mean its behavior changes depending on the runtime version.

Right now on JDK23+, this warning is shown:

WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::invokeCleaner has been called by org.apache.commons.io.input.ByteBufferCleaner$Java9Cleaner
WARNING: Please consider reporting this to the maintainers of class org.apache.commons.io.input.ByteBufferCleaner$Java9Cleaner
WARNING: sun.misc.Unsafe::invokeCleaner will be removed in a future release

That's JDK telling the library to change the behavior by not using Unsafe on JDK23+. commons-io library should be a good citizen and automatically follow the suggestion set by JDK.

 
In any case, I added an clean(boolean) flag for users to opt-in or control this behavior.

[garydgregory]

Hello @maxxedev

Thank you for your update.

Please see my comment in the PR. Also, the new setClean(false) path is exercised for MemoryMappedFileInputStream with a static mock of ByteBufferCleaner.clean. For BufferedFileChannelInputStream the test only checks the internal cleaned flag, not that ByteBufferCleaner.clean is not invoked. A Mockito verification analogous to the MemoryMappedFileInputStream test would close the coverage gap.

[garydgregory]

The catch needs an inline comment. unsafeMemoryAccessDeprecated() parses the spec version as an integer. For the Java 8 value of "1.8" the parse would fail and the method correctly returns false. Adding a comment would make it obvious the behavior is still correct on Java 8 and not just luck.

[maxxedev]

To hopefully make the code clearer, I added specific check for "1.8".

[garydgregory]

Hello @maxxedev

I'm sorry but the PR has gotten worse IMO and confusing:

• BufferedFileChannelInputStream.clean() and MemoryMappedFileInputStream.cleanBuffer() now test only the user flag clean, not ByteBufferCleaner.isSupported(). If cleaning is requested on a platform where the cleaner cannot be instantiated, ByteBufferCleaner.clean() will throw IllegalStateException` on close. Previously the code silently skipped cleaning when unsupported.
• getCleanerMaybe() returns null for Java 23+, making isSupported() false, yet ByteBufferCleaner.clean() falls back to getCleaner() which will still attempt reflective Unsafe access. The intent to disable cleaning on Java 23 is therefore only partially realized and may cause repeated reflective initialization on every clean() call.
• When INSTANCE == null the clean method calls getCleaner() each time, creating a new Java8Cleaner/Java9Cleaner via reflection on every close.
• Why does one input stream track clean and cleaned while the other doesn't?

Perhaps a simpler alternative would be to create a NoOpCleaner() instead of using null-checks.

[maxxedev]

• Previously the code silently skipped cleaning when unsupported.
• getCleaner() code does not work well

Fixed to silently skip, and improved getCleaner() handling.

 

Why does one input stream track clean and cleaned while the other doesn't?

BufferedFileChannelInputStream extends InputStream, and uses its own cleaned flag.
MemoryMappedFileInputStream extends AbstractInputStream, and uses their isClosed flag

I've changed BufferedFileChannelInputStream to also extend AbstractInputStream, and be more like MemoryMappedFileInputStream

 
thanks

[garydgregory]

@maxxedev

Thanks for the contribution and helping to improve this component.

I have quite a few comments on this one, so I've opened PR #872 to address them. I'll double-check #872 tomorrow against this PR.

In the meantime, here are notes for future work:

• Regression: FileChannel leaks if cleaning fails for any reason. The main code uses a try-finally which should be used here as well.
• The behavior is not fully opt-in since a new caller to ByteBufferCleaner will not fully clean the buffer on Java >= 23.
• When clean is set to false, the safe clean code in ByteBufferCleaner is not called.
• Don't loose the private keyword on getCleaner().
• @since is missing from new public methods.
• Fix Javadoc grammar.
• ByteBufferCleaner.isSupported() Javadoc is now wrong.
• The ByteBufferCleaner class Javadoc makes no mention of the new behavior, it should.
• Same for ByteBufferCleaner.clean(ByteBuffer).
• Same for builder method's setClean(boolean)
• In one of the tests, the result of final boolean expectedCleanOnClose = ByteBufferCleaner.isSupported(); is never used, the call has no side effects, so can be completely removed.
• In other tests, the result of final byte[] expectedData = Files.readAllBytes(file); is not used, that can be removed as well.
• No need to qualify name ByteBufferCleaner.Java*Cleaner" inside ByteBufferCleaner`. There is no reason to have changed this.
• Follow IO's Javadoc conventions: A setter methods "Sets...".
• Follow IO's Javadoc conventions: Sentences end in a period.

Thank you.

[garydgregory]

Don't loose the private keyword.

[maxxedev]

@garydgregory

I had hoped that opening this PR would make it less work for you but I think it's turning out to be quite the opposite. So it sounds good that you just do it yourself with a new PR. I really appreciate your patience and insightful comments. You are a great programmer.