Repository navigation
Add support for wasm32-unknown-unknown #23
Description
Activity
There seem to be 3 main projects to get lua working on wasm
https://github.com/vvanders/wasm_lua
https://github.com/serprex/luwa
https://github.com/ceifa/wasmoonwith https://github.com/ceifa/wasmoon seeming to be the only one that recently had changes
Reacted by Alex Orlenko, Gabriel Francisco and Yesterday17I would love to see this implemented! Could somebody summarise the necessary tasks needing to be done for this to run on wassm ?
I would love to see this implemented! Could somebody summarise the necessary tasks needing to be done for this to run on wassm ?
Step 1: get Lua to compile to WASM or use one of the existing projects for that.
Step 2: find a way to correctly compile said lua VM to wasm together with mlua. Presumably you want them to get compiled in such a way that they share the same WASM memory (So... probably need to end up with 1 wasm file?) to reduce the overhead of FFI.
Step 3: Find some way to get mlua to communicate with the bundled lua. (I am assuming that the normal API's won't work, or that at least some extra plumbing is required.)
Also, there are JS implementations of Lua. Those might be easier to get working but are probably slower than a version compiled in wasm, especially if the wasm version can use the same memory as the rest of the program. https://github.com/fengari-lua/fengari-web is used by the teal playground.
Also, assuming you indeed want to reduce FFI overhead as much as possible and thus make the luavm use the same memory as the rest of the program, then I would imagine that at most only 1 instance of the VM is running. As a result, the functions to create a new Lua instance should probably be disabled and instead replaced with one to get a global instance.
Reacted by Maksymilian Mozolewski, Erlend Sogge Heggen and bernsteiningI would expect that if you want to spawn new VM's that they all have their own memory. Which means that the only way to "share" memory is by copying it over. I'm also not sure if 2 WASM modules can communicate directly with each other. If they can't, you are looking at going through JS. Which means that every value needs to be copied twice.
Lua itself has ways to do sandboxing. Though doing that properly is of course not the easiest.
I will try to add some info.
as @lenscas mentioned, fengari is just a lua vm implemented in JS. wasmoon is a wrapper around lua54 C VM compiled into wasm, and drives it using JS.anyway, both of them are useless for our usecase. if you separate out the lua vm from the main rust app as two individual wasm modules, there's nothing for mlua to do. both modules need to share memory and communicate with each other smh, but its more like a lua app and rust app talking to each other and not "embedding" lua into rust app.
I cannot know for sure, but you should be able to embed a lua vm inside rust wasm (no JIt though). there's two ways:
First, use a rust lua VM like Hematita. this should allow us to usewasm32-unknown-unknowntarget +wasm_bindgen(which allows rust <-> web JS interactions. used by most tools liketrunkor eframe).
BUT apparently, the ABI used is not C ABI compatible. and fixing it would be breaking change forwasm_bindgenand its not happening anytime soon. more info wasm-bindgen/wasm-bindgen#2209 . i am not completely sure if hematita will help, gotta ask the developer himself, but the project is being put on hold for the moment.Second, use a C lua VM (bindings) and compile it to wasm. like i said,
wasm32-unknown-unknowntoolchain is not available for C code. so, we must usewasm32-unknown-emscriptenwhich brings us to the issue rustwasm/team#291 . this seems like the most doable thing at the moment.now, my C/C++ (or cmake) skills are absolute garbage, so, dealing with lua54's building process (by copying from wasmoon) is too hard for me. BUT, fortunately,
luau_srccrate used cmake and i was able to successfully do a hello world example using mlua + luau using emscripten target with basically zero effort.- just clone mlua repo.
- add a basic
main.rsin the src folder
use mlua::Lua; fn main() { println!("inside main"); for i in 0..5 { Lua::new() .load(&format!("print('hello lua, times: {i}')")) .exec() .expect("failed to execute lua"); } }
- compile the above with
cargo build --features=luau,vendored --target=wasm32-unknown-emscriptento produce amlua.jsandmlua.wasmfiles. - create a basic html page in crate root folder with
python -m http.server. don't forget to use the right path for themlua.jsscript!!!
<script src="./target/wasm32-unknown-emscripten/debug/mlua.js"></script> <h1>Hello World</h1>
- check console logs to see if there's the
hello luaoutput
now, technically, lua54 is also doable because wasmoon can do it. but i will leave that task to someone who can understand what's happening in https://github.com/ceifa/wasmoon/blob/main/build.sh .
just to point out again, this issue is about compilation to
wasm32-unknown-unknownwhich is not possible due to ABI issues (which will be fixed at some point according to the linked issue). i don't really understand these things completely, but if you want to know more, there's zulip for wasmtime which has nice helpful people, and there's rust-wasm team channel on officlal rust discord server.PS: #40 seems to be a duplicate of this.
Reacted by Erlend Sogge Heggen and Theo Parisa lot of crates usually only support
wasm32-unknown-unknownand seems like winit does the same 😿 which makes this useless for embedding in apps / games.
but emscripten already supports GLFW / SDL to a decent extent https://emscripten.org/docs/porting/multimedia_and_graphics/OpenGL-support.html#opengl-es-extensions , so if rust SDL2 library gets emscripten support Rust-SDL2/rust-sdl2#884 this will be tremendously useful formeeveryone who wants lua scripting on rust web apps / games.@coderedart fantastic and thorough exploration, thank you!
winit emscripten support seems to be a no-go, and so for engines like bevy I imagine it's an unlikely solution, so it looks like the only way for this to be useful in engines like
bevyis via this issuereply in that thread
Nope, there hasn't been any progress :( Everyone working in this space has been busy with other things.
i guess this issue will go back to hibernation. i will just keep my eye on hematita.
I plan soon to start porting Lua 5.3 from C to Rust which should bring
wasm32-unknown-unknownas well asno_std(hopefully) support to mlua.
Seems it's the only way (apart from rewriting from scratch).
Why Lua 5.3? Because it's fairly modern, fast and less complex than Lua 5.4 (20k vs 24k LoC).
Also the goal is to keep C API layer backward compatible.i will just keep my eye on hematita.
hematita is mostly a POC than a real implementation. It even does not has GC support.
Reacted by Red Artist, Maksymilian Mozolewski, Andy Weidenbaum and Erlend Sogge HeggenReacted by Red Artist, Maksymilian Mozolewski, Andy Weidenbaum and Erlend Sogge HeggenI am happy to get anything, but just wanted to know why lua 5.3 instead of 5.1 (which works on luau / luajit)? the whole internet including apps like mpv seem to stick to 5.1 / 5.2 because of backwards compatibility.
I rather see 5.3 on the Web than 5.1.
There js a compatible library that brings 5.1 and luaJIT to 5.3 compatibility wise. I don't think something similar exists for 5.3 to 5.1.
Reacted by Red ArtistThat would be interesting, but also sounds like a big project,
Are there any attempts at bringing lua to pure rust which are somewhat complete? It's probably worth reaching out to people who have tried this before.
https://github.com/cjneidhart/lua-in-rust
Seems to be a wip lua impl
Reacted by Alex Orlenko and Maksymilian Mozolewski11 remaining items
I am very interested in this working with
wasm32-unknown-unknownfor my project. If there is anything I can do to help or any pointers to work on this, please let me know!I am very interested in this working with
wasm32-unknown-unknownfor my project. If there is anything I can do to help or any pointers to work on this, please let me know!Can i plug kyren's https://github.com/kyren/piccolo ? She was the original creator of rlua iirc and it's a completely pure rust solution, compiles to wasm32-unknown-unknown, and relies on a garbage collection arena with it's own lifetime and seems to cover everything PUC Lua offers.
( I also made a shoddy Lua interpreter that I hope to add luau-style type casting to eventually but it's order of magnitudes worse then piccolo https://github.com/auxnon/silt-lua I wanted it to be drop-in replacement for mlua for wasm builds but honestly just use piccolo lol)
Reacted by Anthony DitoI made a custom version of lua53-sys that compiles a patched lua53 with a custom libc and libcppthrow. It works and compiles for wasm32-unknown-unknown. https://github.com/speedy-lex/lua53-sys
Reacted by elliotsayes, Gaspard Legendre, Polanas and Hugues MorissetAn update on wasm exceptions, it's up to Phase 5, "The Feature is Standardized".
As of writing, of the 14 runtimes monitored on webassemby.org, 9 have full support, 2 have experimental, and 3 have no support.
So I don't know what else is needed for it, but C++ exceptions via WASM EH should only need the right flags set now.
https://emscripten.org/docs/tools_reference/settings_reference.html#support-longjmp
https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fwasm-exceptionsI believe we just need shims to supplement
mluas needs from the std lib. On either the Rust or C++ side. Which, unless I'm wrong, should mainly be some File IO related functions.Reacted by Shane Celis and Hugues MorissetReacted by Maksymilian Mozolewski, Eddy S. and Hugues MorissetPerhaps https://github.com/CppCXY/lua-rs is another option, as it can be compiled to wasm-unknown-unknown.
Reacted by Duncan FreemanReacted by Joscha RohmannPerhaps https://github.com/CppCXY/lua-rs is another option, as it can be compiled to wasm-unknown-unknown.
But why go for a vibe coded version of Lua that, unless I am mistaken adds nothing new to the table? As stated above, the missing pieces on the Wasm side should be fixed now and the ABI problems that rust has are in the process of being fixed or already fixed.
There are fewer and fewer blockers basically everyday and it might already be possible to get it all working if someone is willing to spend the time.
If a rust based implementation is needed then why not something like https://github.com/lumen-oss/ottavino ?
It is already used by projects (both the project it forked and the fork I linked) and brings new functionality to the table aside from the usual "it is written in rust"
But why go for a vibe coded version of Lua that, unless I am mistaken adds nothing new to the table? As stated above, the missing pieces on the Wasm side should be fixed now and the ABI problems that rust has are in the process of being fixed or already fixed.
There are fewer and fewer blockers basically everyday and it might already be possible to get it all working if someone is willing to spend the time.
If a rust based implementation is needed then why not something like https://github.com/lumen-oss/ottavino ?
It is already used by projects (both the project it forked and the fork I linked) and brings new functionality to the table aside from the usual "it is written in rust"
From a performance perspective, this is currently the fastest, and it passes the vast majority of the official Lua test suite. From an implementation perspective, it is the most complete. Even the project you mentioned, https://github.com/lumen-oss/ottavino, has its string algorithms ported from lua-rs.
Reacted by ianmclaughlin-poReacted by Duncan Freeman and Ryan ChouWhat's the current status on this? Is there any way I can try to build mlua for wasm32-unknown-unknown right now?
What's the current status on this? Is there any way I can try to build mlua for wasm32-unknown-unknown right now?
From my understanding, everything on the rust and Wasm specification side is now pretty much resolved. So the next step is that someone takes (nightly) rust and tries it out. Then seeing what they run into, if it can be solved by passing specific arguments to the compiler or if something more involved is needed. And of course document it.
I tried building with
cargo +nightly build --target wasm32-unknown-unknown, but it failed at mlua-sys, which does not know how to build for wasm32-unknown-unknown. I only added mlua to the Cargo.toml, and I tried with the latest version and directly from the github repo. I'm not too sure where to go from here, especially since im unfamiliar with testing crates. Do I also need to include a specific version of mlua-sys or lua itself?@speedy-lex version compiles for wasm32-unknown-unknown you can easily integrate it with mlua by changing some lines in Cargo.toml
I'm working on
wasm32-unknown-unknownsupport. I have demo with transpiled/ported Lua 5.5 to Rust. The only problem it requiresc_variadicfeature that should be stable in Rust 1.99 (rust-lang/rust#44930)Reacted by Esper Thomson, Ryan Chou, Hugues Morisset and Noah WeberWhat's the current status on this? Is there any way I can try to build mlua for wasm32-unknown-unknown right now?
From my understanding, everything on the rust and Wasm specification side is now pretty much resolved. So the next step is that someone takes (nightly) rust and tries it out. Then seeing what they run into, if it can be solved by passing specific arguments to the compiler or if something more involved is needed. And of course document
I already have tested and used it in wasm32-unknown-unknown but it was almost a year ago so a lot of stuff can probably be removed.
See: https://github.com/speedy-lex/lua53-sys@khvzak Since 1.99 is stable soon i was wondering how far your work has come? :>
Would this also work with luau?
No pressure just interest since i would need this for my project and would maybe look into working towards it if its not over the finish line soon 😇Ty and have beautiful day ahead!
Reacted by Maciej Główka
At the moment, attempting to build for
wasm32-unknown-unknowncauses the following panic:don't know how to build Lua for wasm32-unknown-unknown.I know running Lua in wasm isn't the most efficient thing in the world, but I feel like the functionality should be added.
I'd be happy to look into this, I just want to get the OK before I go and implement it.