Compiler error with malformed symbol generated

[earwig]

I'm tracking down a strange Emscripten error on a large project. Here's a minimal reproducible example:

#include <setjmp.h>

namespace foo {
    class Bar {
      public:
        void func() {}
    };
}

int main() {
    foo::Bar bar = foo::Bar();
    jmp_buf env;
    setjmp(env);
    bar.func();
}

The error (full text collapsed):

$ em++ -Wall main.cpp -o main.js
emscripten:ERROR: emscript: failure to parse metadata output from wasm-emscripten-finalize. raw output is:
{
...
  "declares": [
    "getTempRet0",
    "__invoke_void_%"class.foo::Bar"*",
    "emscripten_longjmp",
    "setTempRet0",
    "emscripten_resize_heap",
    "emscripten_memcpy_big"
  ],
...
}

Traceback (most recent call last):
...
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 869, in load_metadata_wasm
    metadata_json = json.loads(metadata_raw)
...
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 9 column 22 (char 146)

Full error

$ em++ -Wall main.cpp -o main.js
emscripten:ERROR: emscript: failure to parse metadata output from wasm-emscripten-finalize. raw output is:
{
  "staticBump": 536,
  "tableSize": 2,
  "initializers": [
    "__wasm_call_ctors"
  ],
  "declares": [
    "getTempRet0",
    "__invoke_void_%"class.foo::Bar"*",
    "emscripten_longjmp",
    "setTempRet0",
    "emscripten_resize_heap",
    "emscripten_memcpy_big"
  ],
  "externs": [
  ],
  "exports": [
    "__wasm_call_ctors",
    "malloc",
    "saveSetjmp",
    "testSetjmp",
    "free",
    "main",
    "__errno_location",
    "fflush",
    "stackSave",
    "stackRestore",
    "stackAlloc",
    "realloc",
    "setThrew"
  ],
  "namedGlobals": {
    "__data_end" : "1560"
  },
  "invokeFuncs": [
  ],
  "mainReadsParams": 1,
  "features": [
  ]
}

Traceback (most recent call last):
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/em++.py", line 14, in <module>
    sys.exit(emcc.run(sys.argv))
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emcc.py", line 2182, in run
    emscripten.run(tmp_wasm, final_js, memfile)
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 964, in run
    return temp_files.run_and_clean(lambda: emscript(
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/tools/tempfiles.py", line 105, in run_and_clean
    return func()
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 964, in <lambda>
    return temp_files.run_and_clean(lambda: emscript(
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 387, in emscript
    metadata = finalize_wasm(infile, memfile, DEBUG)
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 566, in finalize_wasm
    return load_metadata_wasm(stdout, DEBUG)
  File "/usr/local/Cellar/emscripten/2.0.7/libexec/emscripten.py", line 869, in load_metadata_wasm
    metadata_json = json.loads(metadata_raw)
  File "/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@3.8/3.8.6/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 9 column 22 (char 146)

A symbol named __invoke_void_%"class.foo::Bar"* is created, which is then dumped into JSON without any escaping.

Indeed, after setting EMCC_DEBUG=1 and inspecting the generated object file, the oddly named symbol is present:

$ llvm-objdump -t /.../emscripten_temp/main_0.o

main_0.o: file format wasm

SYMBOL TABLE:
00000001 g     F CODE .hidden __original_main
00000000         *UND* __stack_pointer
00000000       F *UND* malloc
00000000 l     O DATA .L__const.main.bar
00000000       F *UND* saveSetjmp
00000000       F *UND* getTempRet0
000001ad  w    F CODE .hidden _ZN3foo3Bar4funcEv
00000000       O *UND* __THREW__
00000000       F *UND* __invoke_void_%"class.foo::Bar"*
00000000       O *UND* __threwValue
00000000       F *UND* testSetjmp
00000000       F *UND* emscripten_longjmp
00000000       F *UND* setTempRet0
00000000       F *UND* free
000001cd g     F CODE .hidden main

I'm not sure if the bug is just (1) wasm-emscripten-finalize needs to escape symbols properly, or also (2) the symbol name is wrong. Certainly a symbol name with a double quote in it feels extremely wrong, but I'm not sure if it actually is.

Environment:

• macOS 10.14.6 (I think this might be macOS-specific)
• emcc 2.0.7
• Apple clang 11.0.0
• emscripten_temp.zip

[sbc100]

This problem is due to a mismatch between the binaryen and llvm versions used in your project.

The re-writing of these symbols used to be part of binaryen and it is now done by llvm.

This means that you have a binaryen version that is recent enough to not do this work, but an llvm version that is not new enough to do it.

I believe this is perhaps an issue with the homebrew package. Switching to emsdk for the installation would fix it, or you can push for the homebrew package to be updated. See emscripten-ports/SDL2_mixer#2.

[earwig]

Thanks! Using emsdk solved my problem. Sorry, I should have tried that.

[thomaseizinger]

I am unfortunately running into the same issue on recent versions of emsdk. I tried 2.0.9, 2.0.7 and master. This is the error I am getting:

emscripten:ERROR: emscript: failure to parse metadata output from wasm-emscripten-finalize. raw output is:

{
  "staticBump": 14280,
  "tableSize": 216,
  "declares": [
    "__cxa_find_matching_catch_2",
    "getTempRet0",
    "__resumeException",
    "__invoke_void_[0xi8]*_i32_{}*_[3xi32]*_%"core::panic::Location"*",
    "__invoke_void_i32*_i32",
    "__invoke_void",
    "__invoke_{i32.i32}_i32*",
    "__invoke_{i32.i32}_i32_i32",
    "__invoke_void_[0xi8]*_i32_%"core::panic::Location"*",
    "__invoke_void_i32_i32_%"core::panic::Location"*",
    "__invoke_void_{i8*.i32}*_{i8*.i32}*_i32",
    "__invoke_void_{i8*.i32}*_i8*_i32",
    "__invoke_%"core::panic::Location"*_%"core::panic::Location"*",
    "__invoke_void_%"std::panicking::begin_panic::{{closure}}<&str>"*",
    "__invoke_void_{}*_[3xi32]*_i32*_%"core::panic::Location"*",
    "__invoke_i8*_i32_i32",
    "__invoke_{{}*.[3xi32]*}_{}*_[3xi32]*",
    "__invoke_void_{i8*.i32}*_{i8*.i32}*",
    "__invoke_void_{}*",
    "__invoke_i32*_i32*",
    "__invoke_void_%"alloc::vec::Vec<usize>"*_%"alloc::vec::Vec<usize>"*_i32",
    "__invoke_void_%"alloc::vec::Vec<usize>"*_%"alloc::vec::Vec<usize>"*",
    "__invoke_void_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_i32",
    "__invoke_void_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_%"core::option::Option<core::cell::Cell<externref::Slab>>"*",
    "__invoke_void_%"externref::Slab"*_%"externref::Slab"*_i32",
    "__invoke_void_%"externref::Slab"*_%"externref::Slab"*",
    "__invoke_void_%"alloc::vec::Vec<usize>"*",
    "__invoke_i32",
    "__invoke_{i32.i32}",
    "__invoke_{i32.i32}_i32*_i32*",
    "__invoke_i8_i32*_i32*",
    "__invoke_%"externref::Slab"*_%"core::cell::UnsafeCell<externref::Slab>"*",
    "__invoke_void_%"externref::Slab"*_%"externref::Slab"*_%"externref::Slab"*",
    "__invoke_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_%"core::cell::UnsafeCell<core::option::Option<core::cell::Cell<externref::Slab>>>"*",
    "__invoke_void_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_%"core::option::Option<core::cell::Cell<externref::Slab>>"*_%"core::option::Option<core::cell::Cell<externref::Slab>>"*",
    "__invoke_void_%"core::option::Option<core::cell::Cell<externref::Slab>>"*",
    "__invoke_i32*",
    "__invoke_void_i32*_%"core::cell::Cell<externref::Slab>"*",
    "__invoke_i1",
    "__invoke_i32_%"core::cell::Cell<externref::Slab>"*",
    "__invoke_void_%"alloc::vec::Vec<usize>"*_i32",
    "__invoke_i32_%"externref::Slab"*",
    "__invoke_void_%"externref::Slab"*_%"core::cell::Cell<externref::Slab>"*_%"externref::Slab"*",
    "__invoke_void_%"externref::Slab"*",
    "__invoke_void_%"externref::Slab"*_i32",
    "__invoke_i32*_%"alloc::vec::Vec<usize>"*",
    "__invoke_i32*_i32*_i32",
    "__invoke_i1_%"core::fmt::Formatter"*_%"core::fmt::Arguments"*",
    "__invoke_{i8*.i8*}_i8*_i8*",
    "__invoke_void_%"core::fmt::Arguments"*_%"core::panic::Location"*",
    "__invoke_{i32.i32}_i32",
    "__invoke_{i32*.i32}",
    "__invoke_i32_{i32.i32}*",
    "__invoke_void_%"core::result::Result<().alloc::collections::TryReserveError>"*_i32",
    "__invoke_{i8*.i32}_%"alloc::alloc::Global"*_i32_i32",
    "__invoke_i32*_i8*_i32",
    "__invoke_i32_i8*_i32",
    "__invoke_i32_i32",
    "__invoke_{i32*.i32}_i32*",
    "__invoke_{i32.i32}_{i32.i32}*",
    "__invoke_void_i32*_i32*",
    "__invoke_void_i32*_i32*_i32",
    "__invoke_void_{i32*.i32}*_i32",
    "__invoke_void_{i32*.i32}*",
    "__invoke_i32*_{i32*.i32}*",
    "__invoke_{i32*.i32}_{i32*.i32}*",
    "__invoke_i32_i32*",
    "__invoke_i32_%"alloc::vec::Vec<usize>"*",
    "pthread_mutexattr_init",
    "pthread_mutexattr_settype",
    "pthread_mutexattr_destroy",
    "__invoke_{[0xi32]*.i32}_i32*_i32",
    "__invoke_{[0xi32]*.i32}_[0xi32]*_i32",
    "__invoke_i1_i32*",
    "__invoke_{i32*.i32}_i32",
    "__invoke_void_%"alloc::vec::Vec<usize>"*_i32_i32",
    "__invoke_{[0xi32]*.i32}_i32*_i32_i32",
    "__invoke_void_%"sys::unix::condvar::Condvar"*",
    "__invoke_void_%"io::error::Error"*_i8_[0xi8]*_i32",
    "__invoke_i1_{}*_[3xi32]*_%"core::fmt::Arguments"*",
    "__invoke_void_%"core::result::Result<().io::error::Error>"*_{}*",
    "__invoke_void_%"core::result::Result<().io::error::Error>"*_{}*_%"core::fmt::Arguments"*",
    "__invoke_i64*_%"core::option::Option<alloc::string::String>"*",
    "pthread_rwlock_unlock",
    "__invoke_void_%"panicking::default_hook::{{closure}}"*_{}*_[3xi32]*",
    "__invoke_i32*_%"core::panic::PanicInfo"*",
    "pthread_rwlock_rdlock",
    "pthread_condattr_init",
    "pthread_condattr_setclock",
    "pthread_condattr_destroy",
    "__gxx_personality_v0",
    "fd_write",
    "__sys_getcwd",
    "environ_sizes_get",
    "environ_get",
    "proc_exit",
    "args_sizes_get",
    "args_get",
    "main",
    "__cxa_uncaught_exceptions"
  ],
  "externs": [
  ],
  "exports": [
    "__externref_table_alloc",
    "__externref_table_dealloc",
    "__externref_drop_slice",
    "__externref_heap_live_count",
    "__wbindgen_exn_store",
    "__wbindgen_malloc",
    "__wbindgen_realloc",
    "__wbindgen_free",
    "memset",
    "malloc",
    "free",
    "rustsecp256k1_v0_2_0_context_create",
    "rustsecp256k1_v0_2_0_context_destroy",
    "rustsecp256k1_v0_2_0_default_illegal_callback_fn",
    "rustsecp256k1_v0_2_0_default_error_callback_fn",
    "__errno_location",
    "rust_eh_personality",
    "ntohs",
    "htons",
    "htonl",
    "fflush",
    "_get_tzname",
    "_get_daylight",
    "_get_timezone",
    "_start",
    "stackSave",
    "stackRestore",
    "stackAlloc",
    "setThrew",
    "_ZSt18uncaught_exceptionv",
    "__cxa_can_catch",
    "__cxa_is_pointer_type",
    "memalign"
  ],
  "namedGlobals": {
    "__data_end" : "15304"
  },
  "invokeFuncs": [
  ],
  "features": [
    "--enable-mutable-globals"
  ]
}

I am using emscripten through the Rust target wasm32-unknown-emscripten. The invalid symbol names are symbols of the Rust standard library.

The re-writing of these symbols used to be part of binaryen and it is now done by llvm.

This sounds like something may need to happen on the Rust side as well to generate correct symbols that can be processed by newer versions of binaryen?

I can successfully compile using emsdk 1.40.1. Was version 2.0 the first one where functionality was removed from binaryen?

[kripken]

@thomaseizinger I believe that change was only in 2.0.7, based on the changelog.

If you see that error, perhaps you are not using a new-enough LLVM, or are using a too-new emscripten. I think @tlively checked and the last known good combination for rust (given rust's current LLVM at the time) was 1.39.20.

[thomaseizinger]

@thomaseizinger I believe that change was only in 2.0.7, based on the changelog.

Thanks, I will try with 2.0.6 then to see if it works!

If you see that error, perhaps you are not using a new-enough LLVM, or are using a too-new emscripten. I think @tlively checked and the last known good combination for rust (given rust's current LLVM at the time) was 1.39.20.

I am using the latest Rust stable version as of today (1.48). According to this PR, 1.48 should be using LLVM 11.

Could it possibly be that Rust is invoking LLVM in a way that doesn't generate compatible symbol names?

[tlively]

@thomaseizinger If you're using the latest Rust, you should really be using Emscripten 1.39.20, otherwise you're risking random things breaking on you (like this, but also possibly other things).

[thomaseizinger]

@thomaseizinger If you're using the latest Rust, you should really be using Emscripten 1.39.20, otherwise you're risking random things breaking on you (like this, but also possibly other things).

That is good advice, thank you!

I am curious, what needs to happen for latest emscripten to be compatible with latest rust?

[tlively]

The incompatibility comes from using different versions of LLVM. Emscripten uses tip-of-tree LLVM but Rust uses LLVM releases (with a few custom modifications). Emscripten 1.39.20 was the first Emscripten release after the LLVM 11.0 branch cut, so its version of LLVM is most similar to Rust's version of LLVM and least likely to be incompatible.

For the latest Emscripten to be compatible with the latest Rust, we would need to support using released versions of LLVM with Emscripten, not just tip-of-tree LLVM. This would be quite a bit of extra work for us, and so far it's been better to just tell folks which old version of Emscripten to use.