aya: Support loading programs with ksyms #1372

altugbozkurt07 · 2025-10-22T21:25:48Z

Hi,

this commit covers the first iteration of supporting programs with ksyms variables in aya. I tried to replicate the functionality based on this commit. So to wrap it up;

BTF Parsing and Collection (aya-obj/src/obj.rs):

Added collect_ksyms_from_btf() to parse .ksyms DATASEC from BTF
Implemented ExternCollection to store extern symbol metadata
Created dummy variable handling for function externs in DATASEC
Added BTF fixup logic via fixup_ksyms_datasec() to make externs kernel-acceptable

Extern Type System (aya-obj/src/extern_types.rs):

Implemented ExternDesc to track extern functions and variables
Added resolve_extern_ksyms() for resolving externs against kernel BTF
Created separate resolution paths for functions (resolve_extern_function_internal) and variables (resolve_extern_variable_internal)
Added type compatibility checking using types_are_compatible()
For variables, if they cant be resolved against target kernel btf, resolution through kallsyms is implemented as a fallback

Relocation Support (aya-obj/src/relocation.rs):

Added relocate_externs() to patch LD_IMM64 and CALL instructions with resolved kernel BTF IDs
Implemented instruction patching for both function calls and variable references.

Integration (aya/src/bpf.rs):

Integrated extern resolution into the EbpfLoader flow
Added kernel BTF loading and extern resolution before program loading

What we are missing:

1- We need more integration tests to cover different use-cases to make sure everything is properly implemented.
Some of the tests that needs to be covered are:

Weak symbols tests
Extern variable tests
Error handlings tests(non-existent extern symbol, function with wrong signature, etc)
BTF fixup tests

2- We are missing externs in kernel modules, this is something that is supported by libbpf, at this point it should not be that hard to implement it but before that i would like to first make sure the changes i introduce to aya aligns with the aya infra structure in terms of API desing.

3- Kconfigs are also missing, but again once this iteration goes through, it would be easier to extend this to cover Kconfigs as well.

I would really appreciate your feedback on this and let me know if i am missing anything else, or my implementation is not properly covering the described use-cases.

This change is

netlify · 2025-10-22T21:25:54Z

✅ Deploy Preview for aya-rs-docs ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`1527344`
🔍 Latest deploy log	https://app.netlify.com/projects/aya-rs-docs/deploys/6948609bf45e8000082a22ce
😎 Deploy Preview	https://deploy-preview-1372--aya-rs-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

vadorovsky

Thanks for working on this and kudos for making it work!

The major issue causing the CI failures is usage of std which needs to be guarded.

I still need to play with the code, but I left some nitpicks about dosctrings and comments.

Reviewable status: 0 of 11 files reviewed, 25 unresolved discussions

aya-obj/src/lib.rs line 79 at r1 (raw file):

pub mod btf;
/// ksym implementation

Let's remove this. If you want to have a top-level documentation of the module, add it in the top of the module file with !//.

aya-obj/src/extern_types.rs line 1 at r1 (raw file):

use std::{

This is the reason why CI is failing. The aya-obj crate supports no_std, so everything requiring std needs to be guarded with #[cfg(feature = "std")]. See the other modules

I still need to take a deeper look, but given that this whole ksym detection/sanitization mechanism clearly requires allocations and string operations, we could probably just guard the whole module with #[cfg(feature = "std")].

Another option would be trying to make it no_std, but I don't think it's possible.

aya-obj/src/extern_types.rs line 58 at r1 (raw file):

            }

            // Dispatch to appropriate resolver

I would remove all the comments in this function except that one.

aya-obj/src/extern_types.rs line 99 at r1 (raw file):

    fn resolve_kallsyms(&mut self) -> std::result::Result<(), KsymResolveError> {
        use std::{

Move the imports up.

aya-obj/src/extern_types.rs line 122 at r1 (raw file):

        let file =
            File::open("/proc/kallsyms").map_err(|e| KsymResolveError::KallsymsReadError {
                error: e.to_string(),

We shouldn't include other errors with string.thiserror can wrap other error types inside your own one.

aya-obj/src/extern_types.rs line 227 at r1 (raw file):

    /// Internal: Resolve a single extern variable
    /// Returns Some(btf_id) on success, None for weak externs not found

Code snippet:

/// Resolves a single exterm variable. Returns BTF ID if found, otherwise
/// returns `None`.

aya/src/bpf.rs line 97 at r1 (raw file):

}

pub(crate) static KERNEL_BTF: LazyLock<Option<Btf>> = LazyLock::new(|| match Btf::from_sys_fs() {

btw, I still need to play with the code myself, but I don't like having it as a global variable. If it's really difficult to pass it across functions, we could make it a part of some struct and write methods.

aya/src/bpf.rs line 1195 at r1 (raw file):

    /// Error resolving extern kernel symbols (functions and variables)
    #[error("kernel symbol resolution error: {0}")]
    KsymResolve(#[from] aya_obj::KsymResolveError),

Usually we are trying to keep the error variants in EbpfError very general, and treat each variant as a category of errors. Given that these two are about ksyms, I would try to pack them in one error type first (let's stay, KsymError), then include it here.

test/integration-test/bpf/ksysm.bpf.c line 18 at r1 (raw file):

extern void bpf_rcu_read_lock(void) __attribute__((section(".ksyms")));
extern void bpf_rcu_read_unlock(void) __attribute__((section(".ksyms")));

Would be nice to add a similar Rust eBPF program.

aya-obj/src/btf/btf.rs line 819 at r1 (raw file):

    /// This modifies extern functions and variables to make them acceptable to the kernel:
    /// - Functions: Change linkage to GLOBAL, fix param names, replace with dummy var in datasec
    /// - Variables: Change linkage to GLOBAL_ALLOCATED, replace type with int

Code snippet:

/// Fixes up BTF for `.ksyms` datasec entries containing extern kernel symbol and
/// makes it acceptable by the kernel:
///
/// * Changes linkage of extern functions to `GLOBAL`, fixes parameter names, injects
///   a dummy variable representing them in datasec.
/// * Changes linkage of extern variables to `GLOBAL_ALLOCATED`, replaces their type
///   with `int`.

aya-obj/src/btf/btf.rs line 825 at r1 (raw file):

        dummy_var_id: Option<u32>,
    ) -> Result<(), BtfError> {
        // Get dummy var info and int type ID upfront

I would try to make this comment more descriptive (for the whole let ... else block`) and remove all the others.

aya-obj/src/relocation.rs line 184 at r1 (raw file):

    pub fn relocate_externs(&mut self) -> Result<(), EbpfRelocationError> {
        for (name, extern_desc) in &self.externs.externs {
            eprintln!(

Use debug!, instead of eprintln!

aya-obj/src/relocation.rs line 246 at r1 (raw file):

fn relocate_externs<'a, I: Iterator<Item = &'a Relocation>>(
    fun: &mut Function, // ← Different first parameter (not &mut self)

Remove this comment, it brings no value. It's not even a method, so why would it have self?

aya-obj/src/obj.rs line 872 at r1 (raw file):

        };

        info!("Found/created int type_id: {}", int_type_id);

I would make it debug! and print it only if we create the int type.

aya-obj/src/obj.rs line 890 at r1 (raw file):

        // After creating dummy_var
        info!("Created dummy_var type_id: {}", dummy_var_id);

debug!

aya-obj/src/obj.rs line 898 at r1 (raw file):

    }

    /// Collect extern kernel symbols from BTF DATASEC entries

Function/method documentation should be written using the 3rd person, so starting from "Collects". And if you use full sentences, finish them with ..

Let's also be consistent with capitalizing (or not capitalizing) names. We either write "DATASEC" or "datasec" everywhere. I'm leaning towards the latter, since it's just a convenient shortened form of "data section", not really a proper name.

Code snippet:

/// Collects extern kernel symbols from BTF datasec entries.

aya-obj/src/obj.rs line 900 at r1 (raw file):

    /// Collect extern kernel symbols from BTF DATASEC entries
    fn collect_ksyms_from_btf(&mut self) -> Result<(), ParseError> {
        // Find .ksyms DATASEC

Remove all these comments, the code is self-explanatory enough.

aya-obj/src/obj.rs line 927 at r1 (raw file):

    }

    /// Find .ksyms DATASEC in BTF, returns (id, datasec) if found

Code snippet:

/// Searches for the `.ksyms` datasec in BTF, returns it if found.

aya-obj/src/obj.rs line 945 at r1 (raw file):

    }

    /// Check if datasec contains any functions

Code snippet:

/// Checks if datasec dontains any functions.

aya-obj/src/obj.rs line 954 at r1 (raw file):

    }

    /// Collect extern descriptors from datasec entries

Code snippet:

/// Collects extern descriptors from datasec entries.

aya-obj/src/obj.rs line 961 at r1 (raw file):

        for entry in &datasec.entries {
            let Some(extern_desc) = self.process_datasec_entry(btf, entry)? else {
                continue; // Skip non-extern entries

Remove this comment.

aya-obj/src/obj.rs line 970 at r1 (raw file):

    }

    /// Process a single datasec entry, returning ExternDesc if it's an extern

Aside from what I mentioned earlier, try to use ``` for type names. If the types are inside the same crate, you can even link them like:

Code snippet:

/// Processes a single datasec entry, returns [`ExternDesc`] if it's an extern.

aya-obj/src/obj.rs line 978 at r1 (raw file):

        let btf_type = btf.type_by_id(entry.btf_type)?;

        // Extract extern info based on type

Remove these overly explicit comments.

aya-obj/src/obj.rs line 1017 at r1 (raw file):

    }

    /// Apply BTF fixups for .ksyms DATASEC

Code snippet:

/// Applies BTF fixups for `.ksyms` datasec.

aya-obj/src/obj.rs line 1026 at r1 (raw file):

    }

    /// Helper to find symbol by name

Don't start function/method documentation with "Helper", use the 3rd person and explain what it does.

Code snippet:

/// Return a symbol with the given `name` from the symbol table.

vadorovsky · 2025-11-01T09:46:10Z

aya-obj/src/btf/btf.rs

+        let size = mem::size_of::<i32>() as u32;
+
+        for (i, &type_id) in entry_type_ids.iter().enumerate() {
+            let type_kind = match &self.types.types[type_id as usize] {


This conversion to string is unnecessary, given that you're not using it anywhere else. You could just make the match statement below work directly on BtfType:

diff --git a/aya-obj/src/btf/btf.rs b/aya-obj/src/btf/btf.rs index 5a94a606..f0a52e20 100644 --- a/aya-obj/src/btf/btf.rs +++ b/aya-obj/src/btf/btf.rs @@ -875,14 +875,9 @@ impl Btf { let size = mem::size_of::<i32>() as u32; for (i, &type_id) in entry_type_ids.iter().enumerate() { - let type_kind = match &self.types.types[type_id as usize] { - BtfType::Func(_) => "func", - BtfType::Var(_) => "var", - _ => return Err(BtfError::InvalidDatasec), - }; - + let type_kind = &self.types.types[type_id as usize]; match type_kind { - "func" => { + BtfType::Func(_) => { let (func_name, proto_id) = { let BtfType::Func(f) = &self.types.types[type_id as usize] else { return Err(BtfError::InvalidDatasec); @@ -914,7 +909,7 @@ impl Btf { debug!("DATASEC {datasec_name}: FUNC {func_name}: fixup offset {offset}"); } - "var" => { + BtfType::Var(_) => { let var_name = { let BtfType::Var(v) = &self.types.types[type_id as usize] else { return Err(BtfError::InvalidDatasec); @@ -929,7 +924,7 @@ impl Btf { debug!("DATASEC {datasec_name}: VAR {var_name}: fixup offset {offset}"); } - _ => unreachable!(), + _ => return Err(BtfError::InvalidDatasec), } if let BtfType::DataSec(d) = &mut self.types.types[datasec_id as usize] {

vadorovsky · 2025-11-01T10:04:43Z

aya/src/bpf.rs

            obj.relocate_btf(btf)?;
        }
+
+        if let Some(kernel_btf) = kernel_btf() {


The BTF parsed from the kernel is already a part of EbpfLoader:

aya/aya/src/bpf.rs

Line 164 in 948b855

btf: Btf::from_sys_fs().ok().map(Cow::Owned),

And we provide a method for people to load custom BTF if they want, instead of the one from sysfs:

aya/aya/src/bpf.rs

Lines 175 to 195 in 948b855

/// Sets the target [BTF](Btf) info.

///

/// The loader defaults to loading `BTF` info using [Btf::from_sys_fs].

/// Use this method if you want to load `BTF` from a custom location or

/// pass `None` to disable `BTF` relocations entirely.

/// # Example

///

/// ```no_run

/// use aya::{EbpfLoader, Btf, Endianness};

///

/// let bpf = EbpfLoader::new()

/// // load the BTF data from a custom location

/// .btf(Btf::parse_file("/custom_btf_file", Endianness::default()).ok().as_ref())

/// .load_file("file.o")?;

///

/// # Ok::<(), aya::EbpfError>(())

/// ```

pub fn btf(&mut self, btf: Option<&'a Btf>) -> &mut Self {

self.btf = btf.map(Cow::Borrowed);

self

}

So you can just use self.btf and remove the function and static variable. self.btf is already used in the if statement above (&btf is unpacked from Self at the top of the method), you can just combine them together

diff --git a/aya/src/bpf.rs b/aya/src/bpf.rs index 4740a6af..2b7e4ad2 100644 --- a/aya/src/bpf.rs +++ b/aya/src/bpf.rs @@ -94,22 +94,6 @@ pub fn features() -> &'static Features { &FEATURES } -pub(crate) static KERNEL_BTF: LazyLock<Option<Btf>> = LazyLock::new(|| match Btf::from_sys_fs() { - Ok(btf) => { - debug!("Loaded kernel BTF"); - Some(btf) - } - Err(e) => { - debug!("Failed to load kernel BTF: {}", e); - None - } -}); - -/// Returns a reference to the kernel BTF if available. -pub fn kernel_btf() -> Option<&'static Btf> { - KERNEL_BTF.as_ref() -} - /// Builder style API for advanced loading of eBPF programs. /// /// Loading eBPF code involves a few steps, including loading maps and applying @@ -526,10 +510,7 @@ impl<'a> EbpfLoader<'a> { if let Some(btf) = &btf { obj.relocate_btf(btf)?; - } - - if let Some(kernel_btf) = kernel_btf() { - obj.resolve_typed_externs(kernel_btf)?; + obj.resolve_typed_externs(btf)?; obj.resolve_typeless_externs()?; }

vadorovsky · 2025-11-01T10:32:32Z

test/integration-test/bpf/ksyms.bpf.c

+#include <bpf/bpf_core_read.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <vmlinux.h>


The build error will go away if you move #include <vmlinux.h> to the top - all the bpf/* headers need the integer type definitions from vmlinux.h (they're different for different architectures).

Suggested change

#include <bpf/bpf_core_read.h>

#include <bpf/bpf_helpers.h>

#include <bpf/bpf_tracing.h>

#include <vmlinux.h>

#include <vmlinux.h>

#include <bpf/bpf_core_read.h>

#include <bpf/bpf_helpers.h>

#include <bpf/bpf_tracing.h>

vadorovsky

Some more comments about code organization, the kind of comments you were asking for. 😄

vadorovsky · 2025-11-01T17:22:04Z

aya-obj/src/obj.rs

    }

+    fn create_dummy_ksym_var(&mut self) -> Result<u32, ParseError> {
+        let btf = self.btf.as_mut().ok_or(ParseError::NoBTF)?;


This method operates only on (program's) BTF, so I think we could make it a method of Btf, not Object - let's maybe create aya-obj/src/btf/extern_types.rs file and move it there under impl Btf block.

When we do that, we can get rid of this ok_or and throwing an error.

vadorovsky · 2025-11-01T17:50:16Z

aya-obj/src/obj.rs

+    }
+
+    /// Collects extern kernel symbols from BTF datasec entries.
+    fn collect_ksyms_from_btf(&mut self) -> Result<(), ParseError> {


I think same here - this method is collecting ksyms from program's BTF. It could belong to Btf instead of Obj...

vadorovsky · 2025-11-01T17:50:32Z

aya-obj/src/obj.rs

+            return Ok(());
+        };
+
+        if Self::datasec_has_functions(self.btf.as_ref().unwrap(), &datasec) {


...and then we could get rid of this unwrap.

vadorovsky · 2025-11-01T17:50:48Z

aya-obj/src/obj.rs

+    }
+
+    /// Searches for the `.ksyms` datasec in BTF, returns it if found.
+    fn find_ksyms_datasec(&self) -> Result<Option<(u32, DataSec)>, ParseError> {


Same with this method

vadorovsky · 2025-11-01T17:50:57Z

aya-obj/src/obj.rs

+    }
+
+    /// Checks if datasec contains any functions.
+    fn datasec_has_functions(btf: &Btf, datasec: &DataSec) -> bool {


vadorovsky · 2025-11-01T17:51:24Z

aya-obj/src/obj.rs

+
+    /// Collects extern descriptors from datasec entries.
+    fn collect_extern_entries(&self, datasec: &DataSec) -> Result<Vec<ExternDesc>, ParseError> {
+        let btf = self.btf.as_ref().ok_or(ParseError::NoBTF)?;


vadorovsky · 2025-11-01T17:52:13Z

aya-obj/src/obj.rs

+    }
+
+    /// Processes a single datasec entry, returns [`ExternDesc`] if it's an extern.
+    fn process_datasec_entry(


vadorovsky · 2025-11-01T18:24:08Z

aya-obj/src/obj.rs

+        let btf = self.btf.as_mut().ok_or(ParseError::NoBTF)?;
+        let dummy_var_id = self.externs.dummy_ksym_var_id;
+
+        btf.fixup_ksyms_datasec(datasec_id, dummy_var_id)


Instead of having Object::fixup_ksyms_btf method, we could call the Btf::fixup_ksyms_datasec in Btf::fixup_and_sanitize, somewhere here

aya/aya-obj/src/btf/btf.rs

Line 498 in 948b855

pub(crate) fn fixup_and_sanitize(

altugbozkurt07

Reviewable status: 0 of 15 files reviewed, 36 unresolved discussions (waiting on @vadorovsky)

aya/src/bpf.rs line 97 at r1 (raw file):