merge main into amd-staging (#753)

Author: z1-cciauto

bolt/docs/PacRetDesign.md bolt/docs/PointerAuthDesign.mdbolt/docs/PacRetDesign.md renamed to bolt/docs/PointerAuthDesign.md

@@ -10,6 +10,10 @@ intended audience is BOLT developers. The document is an updated version of the
 in assembly, or `OpNegateRAState` in BOLT sources. In this document, I will use
 **negate-ra-state** as a shorthand.
 
+Note: there are two resolutions for CFI:
+- Call Frame Instruction: individual DWARF instruction, e.g. negate-ra-state
+- Control Flow Integrity: a security mechanism, e.g. pointer authentication.
+
 ## Introduction
 
 ### Pointer Authentication
@@ -104,9 +108,9 @@ negate-ra-state CFIs will become invalid during BasicBlock reordering.
 ## Solution design
 
 The implementation introduces two new passes:
-1. `MarkRAStatesPass`: assigns the RA state to each instruction based on the CFIs
-    in the input binary
-2. `InsertNegateRAStatePass`: reads those assigned instruction RA states after
+1. `PointerAuthCFIAnalyzer`: assigns the RA state to each instruction based on
+    the CFIs in the input binary
+2. `PointerAuthCFIFixup`: reads those assigned instruction RA states after
     optimizations, and emits `DW_CFA_AARCH64_negate_ra_state` CFIs at the correct
     places: wherever there is a state change between two consecutive instructions
     in the layout order.
@@ -129,7 +133,7 @@ instruction.
 This special case is handled by adding an `initialRAState` bool to each BinaryFunction.
 If the `Offset` the CFI refers to is zero, we don't store an annotation, but set
 the `initialRAState` in `FillCFIInfoFor`. This information is then used in
-`MarkRAStates`.
+`PointerAuthCFIAnalyzer`.
 
 ### Binaries without DWARF info
 
@@ -146,7 +150,7 @@ In summary:
 - pointer auth is used, and we have DWARF CFIs: passes run, and rewrite the
   negate-ra-state CFI.
 
-### MarkRAStates pass
+### PointerAuthCFIAnalyzer pass
 
 This pass runs before optimizations reorder anything.
 
@@ -173,9 +177,9 @@ what we have before the pass, and after it.
 | autiasp                       | negate-ra-state | signed   |
 | ret                           |                 | unsigned |
 
-##### Error handling in MarkRAState Pass:
+##### Error handling in PointerAuthCFIAnalyzer pass:
 
-Whenever the MarkRAStates pass finds inconsistencies in the current
+Whenever the PointerAuthCFIAnalyzer pass finds inconsistencies in the current
 BinaryFunction, it marks the function as ignored using `BF.setIgnored()`. BOLT
 will not optimize this function but will emit it unchanged in the original section
 (`.bolt.org.text`).
@@ -188,16 +192,17 @@ The inconsistencies are as follows:
 Users will be informed about the number of ignored functions in the pass, the
 exact functions ignored, and the found inconsistency.
 
-### InsertNegateRAStatePass
+### PointerAuthCFIFixup
 
-This pass runs after optimizations. It performns the _inverse_ of MarkRAState pa s:
+This pass runs after optimizations. It performs the _inverse_ of PointerAuthCFIAnalyzer
+pass:
 1. it reads the RA state annotations attached to the instructions, and
 2. whenever the state changes, it adds a PseudoInstruction that holds an
    OpNegateRAState CFI.
 
 ##### Covering newly generated instructions:
 
-Some BOLT passes can add new Instructions. In InsertNegateRAStatePass, we have
+Some BOLT passes can add new Instructions. In PointerAuthCFIFixup, we have
 to know what RA state these have.
 
 > [!important]
@@ -224,7 +229,7 @@ freely. The only special case is function splitting. When a function is split,
 the split part becomes a new function in the emitted binary. For unwinding to
 work, it needs to "replay" all CFIs that lead up to the split point. BOLT does
 this for other CFIs. As negate-ra-state is not read (only stored as an Annotation),
-we have to do this manually in InsertNegateRAStatePass. Here, if the split part
+we have to do this manually in PointerAuthCFIFixup. Here, if the split part
 starts with an instruction that has Signed RA state, we add a negate-ra-state CFI
 to indicate this.
 

bolt/include/bolt/Passes/MarkRAStates.h …ude/bolt/Passes/PointerAuthCFIAnalyzer.hbolt/include/bolt/Passes/MarkRAStates.h renamed to bolt/include/bolt/Passes/PointerAuthCFIAnalyzer.h bolt/include/bolt/Passes/MarkRAStates.h renamed to bolt/include/bolt/Passes/PointerAuthCFIAnalyzer.h

@@ -1,32 +1,33 @@
-//===- bolt/Passes/MarkRAStates.cpp ---------------------------------===//
+//===- bolt/Passes/PointerAuthCFIAnalyzer.h -------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
-// This file implements the MarkRAStates class.
+// This file implements the PointerAuthCFIAnalyzer class.
 //
 //===----------------------------------------------------------------------===//
-#ifndef BOLT_PASSES_MARK_RA_STATES
-#define BOLT_PASSES_MARK_RA_STATES
+#ifndef BOLT_PASSES_POINTER_AUTH_CFI_ANALYZER
+#define BOLT_PASSES_POINTER_AUTH_CFI_ANALYZER
 
 #include "bolt/Passes/BinaryPasses.h"
 #include <mutex>
 
 namespace llvm {
 namespace bolt {
 
-class MarkRAStates : public BinaryFunctionPass {
+class PointerAuthCFIAnalyzer : public BinaryFunctionPass {
   // setIgnored() is not thread-safe, but the pass is running on functions in
   // parallel.
   std::mutex IgnoreMutex;
 
 public:
-  explicit MarkRAStates() : BinaryFunctionPass(false) {}
+  explicit PointerAuthCFIAnalyzer(const cl::opt<bool> &PrintPass)
+      : BinaryFunctionPass(PrintPass) {}
 
-  const char *getName() const override { return "mark-ra-states"; }
+  const char *getName() const override { return "pointer-auth-cfi-analyzer"; }
 
   /// Pass entry point
   Error runOnFunctions(BinaryContext &BC) override;

…de/bolt/Passes/InsertNegateRAStatePass.h …nclude/bolt/Passes/PointerAuthCFIFixup.hbolt/include/bolt/Passes/InsertNegateRAStatePass.h renamed to bolt/include/bolt/Passes/PointerAuthCFIFixup.h bolt/include/bolt/Passes/InsertNegateRAStatePass.h renamed to bolt/include/bolt/Passes/PointerAuthCFIFixup.h

@@ -1,27 +1,28 @@
-//===- bolt/Passes/InsertNegateRAStatePass.h ------------------------------===//
+//===- bolt/Passes/PointerAuthCFIFixup.h ----------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
-// This file implements the InsertNegateRAStatePass class.
+// This file implements the PointerAuthCFIFixup class.
 //
 //===----------------------------------------------------------------------===//
-#ifndef BOLT_PASSES_INSERT_NEGATE_RA_STATE_PASS
-#define BOLT_PASSES_INSERT_NEGATE_RA_STATE_PASS
+#ifndef BOLT_PASSES_POINTER_AUTH_CFI_FIXUP
+#define BOLT_PASSES_POINTER_AUTH_CFI_FIXUP
 
 #include "bolt/Passes/BinaryPasses.h"
 
 namespace llvm {
 namespace bolt {
 
-class InsertNegateRAState : public BinaryFunctionPass {
+class PointerAuthCFIFixup : public BinaryFunctionPass {
 public:
-  explicit InsertNegateRAState() : BinaryFunctionPass(false) {}
+  explicit PointerAuthCFIFixup(const cl::opt<bool> &PrintPass)
+      : BinaryFunctionPass(PrintPass) {}
 
-  const char *getName() const override { return "insert-negate-ra-state-pass"; }
+  const char *getName() const override { return "pointer-auth-cfi-fixup"; }
 
   /// Pass entry point
   Error runOnFunctions(BinaryContext &BC) override;

bolt/lib/Core/Exceptions.cpp

@@ -572,7 +572,7 @@ bool CFIReaderWriter::fillCFIInfoFor(BinaryFunction &Function) const {
       if (Function.getBinaryContext().isAArch64()) {
         // Support for pointer authentication:
         // We need to annotate instructions that modify the RA State, to work
-        // out the state of each instruction in MarkRAStates Pass.
+        // out the state of each instruction in PointerAuthCFIAnalyzer Pass.
         if (Offset != 0)
           Function.setInstModifiesRAState(DW_CFA_remember_state, Offset);
       }
@@ -583,7 +583,7 @@ bool CFIReaderWriter::fillCFIInfoFor(BinaryFunction &Function) const {
       if (Function.getBinaryContext().isAArch64()) {
         // Support for pointer authentication:
         // We need to annotate instructions that modify the RA State, to work
-        // out the state of each instruction in MarkRAStates Pass.
+        // out the state of each instruction in PointerAuthCFIAnalyzer Pass.
         if (Offset != 0)
           Function.setInstModifiesRAState(DW_CFA_restore_state, Offset);
       }
@@ -652,15 +652,15 @@ bool CFIReaderWriter::fillCFIInfoFor(BinaryFunction &Function) const {
         // BasicBlocks, which changes during optimizations. Instead of adding
         // OpNegateRAState CFIs, an annotation is added to the instruction, to
         // mark that the instruction modifies the RA State. The actual state for
-        // instructions are worked out in MarkRAStates based on these
+        // instructions are worked out in PointerAuthCFIAnalyzer based on these
         // annotations.
         if (Offset != 0)
           Function.setInstModifiesRAState(DW_CFA_AARCH64_negate_ra_state,
                                           Offset);
         else
           // We cannot Annotate an instruction at Offset == 0.
           // Instead, we save the initial (Signed) state, and push it to
-          // MarkRAStates' RAStateStack.
+          // PointerAuthCFIAnalyzer's RAStateStack.
           Function.setInitialRAState(true);
         break;
       }

bolt/lib/Passes/CMakeLists.txt

@@ -17,18 +17,18 @@ add_llvm_library(LLVMBOLTPasses
   IdenticalCodeFolding.cpp
   IndirectCallPromotion.cpp
   Inliner.cpp
-  InsertNegateRAStatePass.cpp
   Instrumentation.cpp
   JTFootprintReduction.cpp
   LongJmp.cpp
   LoopInversionPass.cpp
   LivenessAnalysis.cpp
   MCF.cpp
-  MarkRAStates.cpp
   PatchEntries.cpp
   PAuthGadgetScanner.cpp
   PettisAndHansen.cpp
   PLTCall.cpp
+  PointerAuthCFIAnalyzer.cpp
+  PointerAuthCFIFixup.cpp
   ProfileQualityStats.cpp
   RegAnalysis.cpp
   RegReAssign.cpp

bolt/lib/Passes/IdenticalCodeFolding.cpp

@@ -377,7 +377,8 @@ namespace bolt {
 void IdenticalCodeFolding::initVTableReferences(const BinaryContext &BC) {
   for (const auto &[Address, Data] : BC.getBinaryData()) {
     // Filter out all symbols that are not vtables.
-    if (!Data->getName().starts_with("_ZTV"))
+    if (!Data->getName().starts_with("_ZTV") && // vtable
+        !Data->getName().starts_with("_ZTCN"))  // construction vtable
       continue;
     for (uint64_t I = Address, End = I + Data->getSize(); I < End; I += 8)
       setAddressUsedInVTable(I);
@@ -437,8 +438,9 @@ void IdenticalCodeFolding::markFunctionsUnsafeToFold(BinaryContext &BC) {
   NamedRegionTimer MarkFunctionsUnsafeToFoldTimer(
       "markFunctionsUnsafeToFold", "markFunctionsUnsafeToFold", "ICF breakdown",
       "ICF breakdown", opts::TimeICF);
-  if (!BC.isX86())
-    BC.outs() << "BOLT-WARNING: safe ICF is only supported for x86\n";
+  if (!BC.isX86() && !BC.isAArch64())
+    BC.outs()
+        << "BOLT-WARNING: safe ICF is only supported for x86 and AArch64\n";
   analyzeDataRelocations(BC);
   analyzeFunctions(BC);
 }

bolt/lib/Passes/MarkRAStates.cpp …lt/lib/Passes/PointerAuthCFIAnalyzer.cppbolt/lib/Passes/MarkRAStates.cpp renamed to bolt/lib/Passes/PointerAuthCFIAnalyzer.cpp bolt/lib/Passes/MarkRAStates.cpp renamed to bolt/lib/Passes/PointerAuthCFIAnalyzer.cpp

@@ -1,12 +1,12 @@
-//===- bolt/Passes/MarkRAStates.cpp ---------------------------------===//
+//===- bolt/Passes/PointerAuthCFIAnalyzer.cpp -----------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
-// This file implements the MarkRAStates class.
+// This file implements the PointerAuthCFIAnalyzer class.
 // Three CFIs have an influence on the RA State of an instruction:
 // - NegateRAState flips the RA State,
 // - RememberState pushes the RA State to a stack,
@@ -16,10 +16,10 @@
 // the RA State of each instruction, and save it as new MCAnnotations. The new
 // annotations are Signing, Signed, Authenticating and Unsigned. After
 // optimizations, .cfi_negate_ra_state CFIs are added to the places where the
-// state changes in InsertNegateRAStatePass.
+// state changes in PointerAuthCFIFixup.
 //
 //===----------------------------------------------------------------------===//
-#include "bolt/Passes/MarkRAStates.h"
+#include "bolt/Passes/PointerAuthCFIAnalyzer.h"
 #include "bolt/Core/BinaryFunction.h"
 #include "bolt/Core/ParallelUtilities.h"
 #include <cstdlib>
@@ -31,7 +31,7 @@ using namespace llvm;
 namespace llvm {
 namespace bolt {
 
-bool MarkRAStates::runOnFunction(BinaryFunction &BF) {
+bool PointerAuthCFIAnalyzer::runOnFunction(BinaryFunction &BF) {
 
   BinaryContext &BC = BF.getBinaryContext();
 
@@ -110,7 +110,7 @@ bool MarkRAStates::runOnFunction(BinaryFunction &BF) {
   return true;
 }
 
-Error MarkRAStates::runOnFunctions(BinaryContext &BC) {
+Error PointerAuthCFIAnalyzer::runOnFunctions(BinaryContext &BC) {
   std::atomic<uint64_t> FunctionsIgnored{0};
   ParallelUtilities::WorkFuncTy WorkFun = [&](BinaryFunction &BF) {
     if (!runOnFunction(BF)) {
@@ -132,8 +132,8 @@ Error MarkRAStates::runOnFunctions(BinaryContext &BC) {
 
   ParallelUtilities::runOnEachFunction(
       BC, ParallelUtilities::SchedulingPolicy::SP_INST_LINEAR, WorkFun,
-      SkipPredicate, "MarkRAStates");
-  BC.outs() << "BOLT-INFO: MarkRAStates ran on " << Total
+      SkipPredicate, "PointerAuthCFIAnalyzer");
+  BC.outs() << "BOLT-INFO: PointerAuthCFIAnalyzer ran on " << Total
             << " functions. Ignored " << FunctionsIgnored << " functions "
             << format("(%.2lf%%)", (100.0 * FunctionsIgnored) / Total)
             << " because of CFI inconsistencies\n";