Skip to content

feat: option to drop compiled function bodies - #251

Closed
andreas-karlsson wants to merge 1 commit into
bytecodealliance:mainfrom
andreas-karlsson:drop-compiled-function-bodies
Closed

andreas-karlsson wants to merge 1 commit into
bytecodealliance:mainfrom
andreas-karlsson:drop-compiled-function-bodies

Conversation

@andreas-karlsson

Copy link
Copy Markdown
Collaborator

Fixes #249.

Adds withDropCompiledFunctionBodies(boolean) to MachineFactoryCompiler.Builder. When it's enabled, compile() drops the parsed bodies of compiled functions from the module and keeps only the ones the interpreter may still run: the interpreted functions, plus any function they reach through tail calls. Nothing is dropped if an indirect tail call makes those targets unknown, or when the compilation is loaded from the cache.

var factory = MachineFactoryCompiler.builder(module)
        .withDropCompiledFunctionBodies(true)
        .compile();
var instance = Instance.builder(module).withMachineFactory(factory).build();

The module is changed in place (WasmModule.dropFunction), so it can only be run with the returned machine factory afterwards. Accessing a dropped body throws an IllegalStateException.

Heap after compile, on two of our modules:

Data section main This PR
3.5 MB module 3.2 MB 15 MB 5 MB
14 MB module, 5k functions 12.4 MB 72 MB 18 MB

Parse and compile times are unchanged.

Other things I tried:

  • Not modifying the WasmModule by instead recreating a filtered copy. This became awkward as both the Instance and the compiled machine factory need the copy, which meant a new compile method returning the module and factory together.
  • Parsing function bodies lazily, keeping them through weak references. It saves memory during load as well, but it made parse and compile 10–20% slower and had more moving parts. Happy to share it if that's interesting.

A note about the compiler modifying the WasmModule.

I thought this was unacceptable at first and jumped through hoops to avoid it. In the end I've accepted that the compiler "cannibalizes" the module. One API change that would avoid it is a new type CompiledWasmModule that is both a WasmModule and a machine factory (Function<Instance, Machine>). compile() could return a copy of the module with the compiled bodies dropped, leaving the input untouched, and an Instance could use it as its default machine factory:

var compiledModule = MachineFactoryCompiler.builder(module)
        // with the input untouched, this could even become the default
        .withDropCompiledFunctionBodies(true)
        .compile();
var instance = Instance.builder(compiledModule).build();

It would require WasmModule to be non-final, and the new type is confusingly similar to the existing CompiledModule, which might be merged into it. It also breaks binary compatibility: code compiled against an older version, a library for example, fails with NoSuchMethodError when run with the newer one, because compile()'s signature includes its return type. If you think this is still preferable, I'm happy to evolve the PR in that direction.

@andreas-karlsson

Copy link
Copy Markdown
Collaborator Author

@andreaTP sorry for the PR churn, this was a journey. I've now gone full circle and think a slight modification of what I tried first is the clear winner. Closing in favour of #252.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runtime compiler retains excessive memory

1 participant