Skip to content

Conversation

@kossnocorp
Copy link

@kossnocorp kossnocorp commented Dec 23, 2025

This PR makes tree-sitter-ruby crate compile to Wasm.

See reproduction repo for detailed steps and more details about the described issues. The repo branch fix-env contains fully resolved problem using both tree-sitter and tree-sitter-ruby forks.


This is 2/2 PRs required to address the problem:


Build Issues

Missing #include <stdlib.h>

The problem starts when trying to build simple Rust code using tree-sitter-ruby to Wasm:

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn parse_ruby(source: &str) -> String {
    let mut parser = tree_sitter::Parser::new();
    parser
        .set_language(&tree_sitter_ruby::LANGUAGE.into())
        .unwrap();
    let tree = parser.parse(source, None).unwrap();
    tree.root_node().to_sexp()
}

To demonstrate the problem, I used wasm-pack build --target nodejs:

use wasm_bindgen_test::*;

#[wasm_bindgen_test]
fn test_parse() {
    let result = tree_sitter_ruby_wasm_repro::parse_ruby("def foo; end");
    assert!(result.contains("method"));
}

It fails with missing #include <stdlib.h> error:

[INFO]: 🎯  Checking for the Wasm target...
   Compiling serde_core v1.0.228
   Compiling equivalent v1.0.2
   Compiling serde_json v1.0.145
   Compiling hashbrown v0.16.1
   Compiling ryu v1.0.21
   Compiling itoa v1.0.16
   Compiling memchr v2.7.6
   Compiling tree-sitter-ruby v0.23.1
   Compiling wasm-bindgen-macro v0.2.106
   Compiling serde v1.0.228
   Compiling indexmap v2.12.1
   Compiling wasm-bindgen v0.2.106
   Compiling js-sys v0.3.83
warning: tree-sitter-ruby@0.23.1: In file included from src/parser.c:1:
warning: tree-sitter-ruby@0.23.1: src/tree_sitter/parser.h:10:10: fatal error: 'stdlib.h' file not found
warning: tree-sitter-ruby@0.23.1: #include <stdlib.h>
warning: tree-sitter-ruby@0.23.1:          ^~~~~~~~~~
warning: tree-sitter-ruby@0.23.1: 1 error generated.
error: failed to run custom build command for `tree-sitter-ruby v0.23.1`

Caused by:
  process didn't exit successfully: `/wrkspc/volumen/subs/tree-sitter-ruby-wasm-repro/target/debug/build/tree-sitter-ruby-c1bd2abb184a2868/build-script-build` (exit status: 1)
...

Duplicate symbol __assert_fail

After fixing the problem in tree-sitter-ruby tree-sitter-ruby/tree-sitter-ruby@c9b9144a046be3232512e2155c86dcdbd4d97597) by adding required std::env::var("TARGET").unwrap() == "wasm32-unknown-unknown" block, the build starts failing with:

"rust-lld: error: duplicate symbol: __assert_fail"

This is addressed by making __assert_fail static inline in tree-sitter tree-sitter/tree-sitter@d046c97.

Missing Shims Issue

After the build is fixed, an attempt to run wasm-pack build --target nodejs fails with the missing env error indicating missing included symbols:

[INFO]: 🎯  Checking for the Wasm target...
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.05s
[INFO]: ⬇️  Installing wasm-bindgen...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running unittests src/lib.rs (target/wasm32-unknown-unknown/debug/deps/tree_sitter_ruby_wasm_repro-ec779e9e1f46f1e9.wasm)
no tests to run!
     Running tests/wasm.rs (target/wasm32-unknown-unknown/debug/deps/wasm-a085ddde8065e3b0.wasm)
node:internal/modules/cjs/loader:1424
  throw err;
  ^

Error: Cannot find module 'env'
Require stack:
- /tmp/.tmpHRzcNd/wasm-bindgen-test.js
- /tmp/.tmpHRzcNd/run.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1421:15)
    at defaultResolveImpl (node:internal/modules/cjs/loader:1059:19)
    at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1064:22)
    at Module._load (node:internal/modules/cjs/loader:1227:37)
    at TracingChannel.traceSync (node:diagnostics_channel:328:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:245:24)
    at Module.require (node:internal/modules/cjs/loader:1504:12)
    at require (node:internal/modules/helpers:152:16)
    at Object.<anonymous> (/tmp/.tmpHRzcNd/wasm-bindgen-test.js:5:18)
    at Module._compile (node:internal/modules/cjs/loader:1761:14) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/tmp/.tmpHRzcNd/wasm-bindgen-test.js', '/tmp/.tmpHRzcNd/run.cjs' ]
}

Node.js v24.12.0
Error: Node failed with exit_code 1
...

To find the missing symbols, I used thefind_wasm_import script:

wasm-pack pack
wasm2wat pkg/tree_sitter_ruby_wasm_repro_bg.wasm > tmp/pkg.wat
rg -e 'import "env"' tmp/pkg.wat
#=> 18:  (import "env" "iswlower" (func (;0;) (type 3)))
#=> 19:  (import "env" "memchr" (func (;1;) (type 4)))
#=> 20:  (import "env" "iswupper" (func (;2;) (type 3)))

The missing symbols are iswlower, memchr, and iswupper.

After fixing the missing symbols issue, the tests still fail with the same error, although import "env" does not show in the wat file anymore.

After inspecting the test wasm file, we see that now strchr is missing:

rm -f target/wasm32-unknown-unknown/debug/deps/tree_sitter_ruby_wasm_repro-*.wasm
wasm-pack test --node || echo
wasm2wat target/wasm32-unknown-unknown/debug/deps/tree_sitter_ruby_wasm_repro-*.wasm > tmp/test.wat
rg -e 'import "env"' tmp/test.wat
#=> 43:  (import "env" "strchr" (func $strchr (type 6)))

The code is being optimized away, so to make sure we don't miss any new symbols, we should use the dev profile:

wasm-pack build --dev --target nodejs
wasm2wat pkg/tree_sitter_ruby_wasm_repro_bg.wasm > tmp/dev.wat
rg -e 'import "env"' tmp/dev.wat
#=> 23:  (import "env" "strchr" (func $strchr (type 5)))

Since nothing new shows up, we can be sure that tree-sitter-ruby and tree-sitter fixes are sufficient to build successfully to Wasm.

The fix adds all required functions and includes. Making the tests (wasm-pack test --node) finally pass:

[INFO]: 🎯  Checking for the Wasm target...
   [...]
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.28s
[INFO]: ⬇️  Installing wasm-bindgen...
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running unittests src/lib.rs (target/wasm32-unknown-unknown/debug/deps/tree_sitter_ruby_wasm_repro-00bf6fedec583183.wasm)
no tests to run!
     Running tests/wasm.rs (target/wasm32-unknown-unknown/debug/deps/wasm-1ced0a350c9a9b71.wasm)
running 1 test
test test_parse ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 filtered out; finished in 0.13s

   Doc-tests tree_sitter_ruby_wasm_repro

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant