Backports for 1.13.0-beta1 #60312
Open
Backports for 1.13.0-beta1 #60312
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some testing by @BenChung has revealed how hard it is to load a JuliaC-built library into a program that has already loaded a very old version of libstdc++. Even with probing disabled, `libjulia-internal.so` fails because of missing GLIBCXX symbols. We use so little of the C++ standard library in `libjulia-internal.so` that it's worth the tradeoff to link it statically: it barely changes the size of the resulting library, removes a medium-size library we have to ship in trimmed bundles, and solves some of our hermeticity issues when being loaded by other software. `libjulia-codegen.so` uses it more extensively, and we expect to be able to load it as a plugin for `opt`, meaning it may have to remain dynamically linked. This PR contains a series of changes to enable statically linked libstdc++ by default and mitigate the size impact: - We enable `--gc-sections` when building with gcc/ld.bfd. This saves us some code already, since we can trim out a lot of libLLVMSupport/libLLVMTargetParser. On macOS, we can use `-dead_strip` to save some space even though the rest of these changes are not applicable. - Two flags for `-static-libstdc++` and `-static-libgcc`, called `USE_RT_STATIC_LIBSTDCXX` and `USE_RT_STATIC_LIBGCC`, are added and enabled by default. - Most of the additional code that gets linked into `libjulia-internal.so` is related to locales for iostreams. Replace these with standard C IO and LLVM helpers that don't have weird locale-related behaviour. - (NOT IMPLEMENTED) `--gc-sections` removes unused executable code, but leaves us with a lot of irrelevant debug info. I tested the effect of `llvm-dwarfutil --garbage-collection` and saw pretty good savings, but that is not included in this PR because BinaryBuilder doesn't have a new enough version of the LLVM tools. | Change | Size of `libjulia-internal.so` (KiB) | |--------------------------------------|--------------------------------------| | No change | 14524 | | Linker GC enabled | 13220 | | -static-libstdc++ and -static-libgcc | 22136 | | Excise iostreams | 15036 | | DWARF GC (not in this PR) | 11488 | This is a comparison of symbols that were added and removed. Even though 15 times more code is removed than added, the resulting `libjulia-internal.so` has a similar size to the original because of the additional debug info. (cherry picked from commit f36882f)
(cherry picked from commit af36919)
Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com> (cherry picked from commit 841148d)
(cherry picked from commit cb2ceb9)
Somewhat of a companion to #60248. For a small application that has just started up `fork()` is not a huge concern, but it's quite heavy-handed for Julia- as-a-library scenarios where resident memory may already be large. Many soft-embedded targets also do not support fork() well, so it is good for our compatibility to adjust this. Rather than relying on the linker to do all of the heavy lifting, this changes our `libstdcxx` probe sequence to directly parse the `ld.so.cache` and `libstdc++.so.6` files. As long as we can expect `/etc/ld.so.cache` to be the same path on all Linux systems, this seems to be a reliable way to locate system libraries. (cherry picked from commit ac4ee59)
(cherry picked from commit 44ecbcf)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.