[pull] master from JuliaLang:master #2142
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some testing by @BenChung has revealed how hard it is to load a JuliaC-built library into a program that has already loaded a very old version of libstdc++. Even with probing disabled, `libjulia-internal.so` fails because of missing GLIBCXX symbols. We use so little of the C++ standard library in `libjulia-internal.so` that it's worth the tradeoff to link it statically: it barely changes the size of the resulting library, removes a medium-size library we have to ship in trimmed bundles, and solves some of our hermeticity issues when being loaded by other software. `libjulia-codegen.so` uses it more extensively, and we expect to be able to load it as a plugin for `opt`, meaning it may have to remain dynamically linked. This PR contains a series of changes to enable statically linked libstdc++ by default and mitigate the size impact: - We enable `--gc-sections` when building with gcc/ld.bfd. This saves us some code already, since we can trim out a lot of libLLVMSupport/libLLVMTargetParser. On macOS, we can use `-dead_strip` to save some space even though the rest of these changes are not applicable. - Two flags for `-static-libstdc++` and `-static-libgcc`, called `USE_RT_STATIC_LIBSTDCXX` and `USE_RT_STATIC_LIBGCC`, are added and enabled by default. - Most of the additional code that gets linked into `libjulia-internal.so` is related to locales for iostreams. Replace these with standard C IO and LLVM helpers that don't have weird locale-related behaviour. - (NOT IMPLEMENTED) `--gc-sections` removes unused executable code, but leaves us with a lot of irrelevant debug info. I tested the effect of `llvm-dwarfutil --garbage-collection` and saw pretty good savings, but that is not included in this PR because BinaryBuilder doesn't have a new enough version of the LLVM tools. | Change | Size of `libjulia-internal.so` (KiB) | |--------------------------------------|--------------------------------------| | No change | 14524 | | Linker GC enabled | 13220 | | -static-libstdc++ and -static-libgcc | 22136 | | Excise iostreams | 15036 | | DWARF GC (not in this PR) | 11488 | This is a comparison of symbols that were added and removed. Even though 15 times more code is removed than added, the resulting `libjulia-internal.so` has a similar size to the original because of the additional debug info.
Under normal circumstances we never deallocate JITted code, and JITLink will never abandon in-flight allocations. Unfortunately, on platforms where we use a code model that require relocations to not be too large, the linker can bail out if an allocation gets placed somewhere unlucky. [1] This change forces the use of MapperJITLinkMemoryManager on the platforms where we use a non-large code model so that this can't happen, and removes the jl_unreachable() calls so that the JITLink error gets reported nicely instead of failing in our code. Since every JITLink platform currently uses a non-large code model, we should no longer use JLJITLinkMemoryManager by default anywhere (except when it has been enabled on a RTDyLD platform, like when sanitizers are enabled). It is still necessary to have this change for #60031 to go forward, else the previously RTDyLD platforms run out of memory. Thanks to @IanButterworth and @giordano again. [1] https://buildkite.com/julialang/julia-master/builds/52316/steps/canvas?sid=019aba85-6699-45e3-b173-55f1420063ca#019aba85-66c0-40f1-80ba-8c5b4da56dba/934-1268
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )