Finalize the build process for our runtime library

[icmccorm]

Our build process for the BSAN runtime (./x.py build bsan-rt) involves building both a Rust component and a C++ component. We always build the Rust component as a staticlib in order to import all of the object files from compiler-builtins, which we need to avoid infinite loops from our interceptors.

We link the LLVM library in two different ways depending on the architecture. On MacOS, sanitizer runtimes are built as dynamic libraries, which is necessary for interceptors to work. So, we build the C++ component as a .dylib and we add the Rust component to our include path, passing the linking flag -lbsan_rt. We set the include path in our Rust build script, but all other steps are handled by LLVM in CMakeLists.txt.

The Rust toolchain builds sanitizers as static libraries on all other platforms. There is no simple way to link together two independently built static libraries within CMake¹. Instead, we build the Rust and C++ components separately and manually link them together using both llvm-ar and llvm-objcopy, like so:

llvm-objcopy -w --keep-global-symbol=bsan_* ../build/libbsan_rust.a ./libbsan_rust.a
llvm-ar -x libbsan_rust.a
OBJS=$(find . -name *.o) llvm-ar -r $OBJS libbsan_cpp.a

We need to use llvm-objcopy to hide the symbols from compiler-builtins; otherwise, they will clash with the symbols exported by the standard library that user code links against.

This solution feels hacky, but it does not require any configuration settings outside of those that are already enabled by default when building the Rust compiler (rust.llvm-tools). All forum posts about this issue that I could find suggest executing a similar script through CMake as a custom built step².

One possible alternative would be to build both components as separate static libraries and then configure rustc to link against both of them when BorrowSanitizer is enabled (-Zsanitizer=borrow). However, I tried implementing this and it failed due to missing symbols that should have been available through -lc—I'm not sure why this happened. It would be worth trying this again, but what we have works for now.

Footnotes

1. "I am not sure if CMake provides a (simple) way to do this." - (link) ↩

2. https://stackoverflow.com/questions/2157629/linking-static-libraries-to-other-static-libraries ↩