Skip to content

Conversation

@epheo
Copy link

@epheo epheo commented Aug 25, 2025

Implements SK_REUSEPORT support to enable programmable socket selection within SO_REUSEPORT groups.

  • #[sk_reuseport] macro for eBPF programs
  • SkReuseport program type with load/attach methods
  • SkReuseportContext with sk_reuseport_md field access
  • ReusePortSockArray map for socket management
  • SK_PASS/SK_DROP constants for return values
  • Documentation and usage examples
  • Some integration tests

This allows load balancing decisions to be made programmatically with SO_REUSEPORT set.

Fixes: #215

Submitting as draft for now but would appreciate an early feedback regarding the overall direction.


This change is Reviewable

@netlify
Copy link

netlify bot commented Aug 25, 2025

Deploy Preview for aya-rs-docs ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 6f8fd23
🔍 Latest deploy log https://app.netlify.com/projects/aya-rs-docs/deploys/68ac500f41379f0008709f26
😎 Deploy Preview https://deploy-preview-1328--aya-rs-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mergify mergify bot added aya This is about aya (userspace) aya-bpf This is about aya-bpf (kernel) aya-obj Relating to the aya-obj crate test A PR that improves test cases or CI labels Aug 25, 2025
Implements SK_REUSEPORT support to enable programmable socket
selection within SO_REUSEPORT groups.

- #[sk_reuseport] macro for eBPF programs
- SkReuseport program type with load/attach methods
- SkReuseportContext with sk_reuseport_md field access
- ReusePortSockArray map for socket management
- SK_PASS/SK_DROP constants for return values
- Documentation and usage examples
- Some integration tests

This allows load balancing decisions to be made programmatically
with SO_REUSEPORT set.

Fixes: aya-rs#215
@epheo epheo force-pushed the BPF_PROG_TYPE_SK_REUSEPORT branch from a9f1fbd to 6f8fd23 Compare August 25, 2025 11:59
@epheo epheo marked this pull request as ready for review September 2, 2025 21:20
@vadorovsky vadorovsky requested a review from tamird September 5, 2025 11:13
Copy link
Member

@tamird tamird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tamird reviewed 16 of 19 files at r1, all commit messages.
Reviewable status: 16 of 19 files reviewed, 6 unresolved discussions


test/integration-test/src/tests/sk_reuseport.rs line 56 at r1 (raw file):

        aya::programs::ProgramType::SkReuseport
    )));
}

not entirely clear to me what the point of these tests is

Code quote:

#[tokio::test]
async fn sk_reuseport_load() {
    let mut ebpf = Ebpf::load(crate::SK_REUSEPORT).unwrap();
    let prog: &mut SkReuseport = ebpf
        .program_mut("select_socket")
        .unwrap()
        .try_into()
        .unwrap();

    // Test that the program loads successfully
    prog.load().unwrap();

    // Test that it's properly loaded
    let info = prog.info().unwrap();
    assert_eq!(
        info.program_type().unwrap(),
        aya::programs::ProgramType::SkReuseport
    );
}

#[tokio::test]
async fn sk_reuseport_loaded_programs_iteration() {
    let mut ebpf = Ebpf::load(crate::SK_REUSEPORT).unwrap();
    let prog: &mut SkReuseport = ebpf
        .program_mut("select_socket")
        .unwrap()
        .try_into()
        .unwrap();

    let programs = loaded_programs().collect::<Result<Vec<_>, _>>().unwrap();
    assert!(!programs.iter().any(|p| matches!(
        p.program_type().unwrap(),
        aya::programs::ProgramType::SkReuseport
    )));

    prog.load().unwrap();
    sleep(Duration::from_millis(500)).await;

    let programs = loaded_programs().collect::<Result<Vec<_>, _>>().unwrap();
    assert!(programs.iter().any(|p| matches!(
        p.program_type().unwrap(),
        aya::programs::ProgramType::SkReuseport
    )));
}

test/integration-test/src/tests/sk_reuseport.rs line 72 at r1 (raw file):

    // Test that the map has the correct properties
    let fd = map.fd();
    assert!(!fd.as_fd().as_raw_fd() < 0, "Map fd should be valid");

really? these tests (and comments) look like they have been generated by an LLM

Code quote:

    // Test that the map has the correct properties
    let fd = map.fd();
    assert!(!fd.as_fd().as_raw_fd() < 0, "Map fd should be valid");

test/integration-test/src/tests/sk_reuseport.rs line 114 at r1 (raw file):

    }
    
    // Manually bind and listen

these comments are ... annoying. they just repeat the code and don't help me understand the overall intent at all. Also, why do we need to use these low-level APIs for creating a network socket?


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 82 at r1 (raw file):

//!     if ctx.ip_protocol() == 6 {  // IPPROTO_TCP
//!         let ret = unsafe {
//!             bpf_sk_select_reuseport(

are we really telling people to use this bindings function? do we want to implement a method on ctx?


test/integration-ebpf/src/sk_reuseport.rs line 33 at r1 (raw file):

    let _bind_inany = ctx.bind_inany();
    let _data = ctx.data();
    let _data_end = ctx.data_end();

seems like maybe you need to actually do something with these?

Code quote:

    // Test accessing context fields
    let _len = ctx.len();
    let _hash = ctx.hash();
    let _ip_protocol = ctx.ip_protocol();
    let _eth_protocol = ctx.eth_protocol();
    let _bind_inany = ctx.bind_inany();
    let _data = ctx.data();
    let _data_end = ctx.data_end();

ebpf/aya-ebpf/src/maps/reuseport_sock_array.rs line 60 at r1 (raw file):

        self.def.get() as *mut _
    }
}

missing newline

@epheo epheo marked this pull request as draft September 5, 2025 13:14
@vadorovsky vadorovsky self-requested a review September 5, 2025 13:54
Copy link
Member

@vadorovsky vadorovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vadorovsky reviewed 11 of 19 files at r1, all commit messages.
Reviewable status: 16 of 19 files reviewed, 17 unresolved discussions (waiting on @epheo and @tamird)


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 82 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

are we really telling people to use this bindings function? do we want to implement a method on ctx?

We definitely want to implement a safe method.


-- commits line 4 at r1:
s/Implements/Implement/

Let's stick to imperative sentences in commit messages.


aya/src/maps/sock/reuseport_sock_array.rs line 30 at r1 (raw file):

/// use aya::maps::ReusePortSockArray;
/// use aya::programs::SkReuseport;
/// use std::os::fd::{AsRawFd, FromRawFd};

Let's organize these imports a bit:

  • std first
  • group imports from each crate together, e.g.
    • std::{net::TcpListener, os::fd::{AsRawFd, FromRawFd}};
    • use aya::{maps::ReusePortSockArray, programs::SkReuseport};

aya/src/maps/sock/reuseport_sock_array.rs line 48 at r1 (raw file):

/// 
/// // Bind and listen (setup details omitted for brevity)
/// // ... bind(socket_fd, &addr, addr_len) and listen(socket_fd, backlog) ...

I would actually include the full setup - it will add just a few lines, but people are going to have a full example to play with.


aya/src/maps/sock/reuseport_sock_array.rs line 71 at r1 (raw file):

/// use aya::programs::SkReuseport;
/// use std::net::TcpListener;
/// use std::os::fd::{AsRawFd, FromRawFd};

Same as above about organizing impors.


aya/src/programs/sk_reuseport.rs line 16 at r1 (raw file):

/// SO_ATTACH_REUSEPORT_EBPF socket option constant.
const SO_ATTACH_REUSEPORT_EBPF: i32 = 52;

We have these variables available in auto-generated bindings here:

pub const SO_ATTACH_REUSEPORT_EBPF: u32 = 52;

pub const SO_DETACH_REUSEPORT_BPF: u32 = 68;

But the problem is that they are in aya-ebpf-bindings, which is never imported in any of the user-space crates. Perhaps we need to re-think that - this is probably not the first and not the last time when a constant from that crate would be useful also in the user-space.


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 20 at r1 (raw file):

//! # Advanced Socket Selection
//!
//! For explicit socket selection, use `bpf_sk_select_reuseport()` with socket arrays:

Same here


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 41 at r1 (raw file):

//!     // Select specific socket using helper
//!     let ret = unsafe {
//!         bpf_sk_select_reuseport(

Again, use the wrapper method here


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 102 at r1 (raw file):

//! ```

use core::ffi::c_void;
use crate::cty::c_void;

Our cty crate/module handles cross-complilation of BPF modules, we shouldn't use the core types.


ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 133 at r1 (raw file):

    /// Returns the total packet length.
    #[expect(clippy::len_without_is_empty)]
    pub fn len(&self) -> u32 {

I think usize is more appropriate here - we could just cast using as usize, since casting smaller integer to a bigger one is fine.


ebpf/aya-ebpf/src/maps/reuseport_sock_array.rs line 12 at r1 (raw file):

/// `ReusePortSockArray` is used to store sockets that participate in SO_REUSEPORT
/// groups. eBPF programs of type `BPF_PROG_TYPE_SK_REUSEPORT` can use this map
/// with the `bpf_sk_select_reuseport()` helper to select specific sockets for

Once you write the wrapper method, please link it here


aya-ebpf-macros/src/lib.rs line 658 at r1 (raw file):

///     
///     let ret = unsafe {
///         bpf_sk_select_reuseport(

We shouldn't recommend direct usage of raw bindings to users, Instead, we should wrap bpf_sk_select_reuseport as a new, safe method of SkReuseportContext:

impl SkReuseportContext {
    pub fn select_reuseport(
        &self,
        map: &ReusePortSockArray,
        key: u32,
        flags: u64,
    ) -> Result<(), c_long> {
        let ret = unsafe {
            bpf_sk_select_reuseport(
                self.as_ptr().cast(),
                map.as_ptr(),
                &key as *const u32 as *mut _,
                flags,
            )
        };
        if ret == 0 { Ok(()) } else { Err(ret) }
    }
}

@tamird tamird requested a review from vadorovsky September 5, 2025 16:15
Copy link
Member

@tamird tamird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, sorry about the tone of my review! I thought @vadorovsky was the author. Thanks for your contribution!

Reviewable status: 16 of 19 files reviewed, 17 unresolved discussions (waiting on @epheo and @vadorovsky)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aya This is about aya (userspace) aya-bpf This is about aya-bpf (kernel) aya-obj Relating to the aya-obj crate test A PR that improves test cases or CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for BPF_PROG_TYPE_SK_REUSEPORT

3 participants