-
Notifications
You must be signed in to change notification settings - Fork 369
aya: add support for BPF_PROG_TYPE_SK_REUSEPORT #1328
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for aya-rs-docs ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
Implements SK_REUSEPORT support to enable programmable socket selection within SO_REUSEPORT groups. - #[sk_reuseport] macro for eBPF programs - SkReuseport program type with load/attach methods - SkReuseportContext with sk_reuseport_md field access - ReusePortSockArray map for socket management - SK_PASS/SK_DROP constants for return values - Documentation and usage examples - Some integration tests This allows load balancing decisions to be made programmatically with SO_REUSEPORT set. Fixes: aya-rs#215
a9f1fbd to
6f8fd23
Compare
tamird
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tamird reviewed 16 of 19 files at r1, all commit messages.
Reviewable status: 16 of 19 files reviewed, 6 unresolved discussions
test/integration-test/src/tests/sk_reuseport.rs line 56 at r1 (raw file):
aya::programs::ProgramType::SkReuseport ))); }
not entirely clear to me what the point of these tests is
Code quote:
#[tokio::test]
async fn sk_reuseport_load() {
let mut ebpf = Ebpf::load(crate::SK_REUSEPORT).unwrap();
let prog: &mut SkReuseport = ebpf
.program_mut("select_socket")
.unwrap()
.try_into()
.unwrap();
// Test that the program loads successfully
prog.load().unwrap();
// Test that it's properly loaded
let info = prog.info().unwrap();
assert_eq!(
info.program_type().unwrap(),
aya::programs::ProgramType::SkReuseport
);
}
#[tokio::test]
async fn sk_reuseport_loaded_programs_iteration() {
let mut ebpf = Ebpf::load(crate::SK_REUSEPORT).unwrap();
let prog: &mut SkReuseport = ebpf
.program_mut("select_socket")
.unwrap()
.try_into()
.unwrap();
let programs = loaded_programs().collect::<Result<Vec<_>, _>>().unwrap();
assert!(!programs.iter().any(|p| matches!(
p.program_type().unwrap(),
aya::programs::ProgramType::SkReuseport
)));
prog.load().unwrap();
sleep(Duration::from_millis(500)).await;
let programs = loaded_programs().collect::<Result<Vec<_>, _>>().unwrap();
assert!(programs.iter().any(|p| matches!(
p.program_type().unwrap(),
aya::programs::ProgramType::SkReuseport
)));
}test/integration-test/src/tests/sk_reuseport.rs line 72 at r1 (raw file):
// Test that the map has the correct properties let fd = map.fd(); assert!(!fd.as_fd().as_raw_fd() < 0, "Map fd should be valid");
really? these tests (and comments) look like they have been generated by an LLM
Code quote:
// Test that the map has the correct properties
let fd = map.fd();
assert!(!fd.as_fd().as_raw_fd() < 0, "Map fd should be valid");test/integration-test/src/tests/sk_reuseport.rs line 114 at r1 (raw file):
} // Manually bind and listen
these comments are ... annoying. they just repeat the code and don't help me understand the overall intent at all. Also, why do we need to use these low-level APIs for creating a network socket?
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 82 at r1 (raw file):
//! if ctx.ip_protocol() == 6 { // IPPROTO_TCP //! let ret = unsafe { //! bpf_sk_select_reuseport(
are we really telling people to use this bindings function? do we want to implement a method on ctx?
test/integration-ebpf/src/sk_reuseport.rs line 33 at r1 (raw file):
let _bind_inany = ctx.bind_inany(); let _data = ctx.data(); let _data_end = ctx.data_end();
seems like maybe you need to actually do something with these?
Code quote:
// Test accessing context fields
let _len = ctx.len();
let _hash = ctx.hash();
let _ip_protocol = ctx.ip_protocol();
let _eth_protocol = ctx.eth_protocol();
let _bind_inany = ctx.bind_inany();
let _data = ctx.data();
let _data_end = ctx.data_end();ebpf/aya-ebpf/src/maps/reuseport_sock_array.rs line 60 at r1 (raw file):
self.def.get() as *mut _ } }
missing newline
vadorovsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vadorovsky reviewed 11 of 19 files at r1, all commit messages.
Reviewable status: 16 of 19 files reviewed, 17 unresolved discussions (waiting on @epheo and @tamird)
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 82 at r1 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
are we really telling people to use this bindings function? do we want to implement a method on ctx?
We definitely want to implement a safe method.
-- commits line 4 at r1:
s/Implements/Implement/
Let's stick to imperative sentences in commit messages.
aya/src/maps/sock/reuseport_sock_array.rs line 30 at r1 (raw file):
/// use aya::maps::ReusePortSockArray; /// use aya::programs::SkReuseport; /// use std::os::fd::{AsRawFd, FromRawFd};
Let's organize these imports a bit:
stdfirst- group imports from each crate together, e.g.
std::{net::TcpListener, os::fd::{AsRawFd, FromRawFd}};use aya::{maps::ReusePortSockArray, programs::SkReuseport};
aya/src/maps/sock/reuseport_sock_array.rs line 48 at r1 (raw file):
/// /// // Bind and listen (setup details omitted for brevity) /// // ... bind(socket_fd, &addr, addr_len) and listen(socket_fd, backlog) ...
I would actually include the full setup - it will add just a few lines, but people are going to have a full example to play with.
aya/src/maps/sock/reuseport_sock_array.rs line 71 at r1 (raw file):
/// use aya::programs::SkReuseport; /// use std::net::TcpListener; /// use std::os::fd::{AsRawFd, FromRawFd};
Same as above about organizing impors.
aya/src/programs/sk_reuseport.rs line 16 at r1 (raw file):
/// SO_ATTACH_REUSEPORT_EBPF socket option constant. const SO_ATTACH_REUSEPORT_EBPF: i32 = 52;
We have these variables available in auto-generated bindings here:
| pub const SO_ATTACH_REUSEPORT_EBPF: u32 = 52; |
| pub const SO_DETACH_REUSEPORT_BPF: u32 = 68; |
But the problem is that they are in aya-ebpf-bindings, which is never imported in any of the user-space crates. Perhaps we need to re-think that - this is probably not the first and not the last time when a constant from that crate would be useful also in the user-space.
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 20 at r1 (raw file):
//! # Advanced Socket Selection //! //! For explicit socket selection, use `bpf_sk_select_reuseport()` with socket arrays:
Same here
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 41 at r1 (raw file):
//! // Select specific socket using helper //! let ret = unsafe { //! bpf_sk_select_reuseport(
Again, use the wrapper method here
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 102 at r1 (raw file):
//! ``` use core::ffi::c_void;
use crate::cty::c_void;
Our cty crate/module handles cross-complilation of BPF modules, we shouldn't use the core types.
ebpf/aya-ebpf/src/programs/sk_reuseport.rs line 133 at r1 (raw file):
/// Returns the total packet length. #[expect(clippy::len_without_is_empty)] pub fn len(&self) -> u32 {
I think usize is more appropriate here - we could just cast using as usize, since casting smaller integer to a bigger one is fine.
ebpf/aya-ebpf/src/maps/reuseport_sock_array.rs line 12 at r1 (raw file):
/// `ReusePortSockArray` is used to store sockets that participate in SO_REUSEPORT /// groups. eBPF programs of type `BPF_PROG_TYPE_SK_REUSEPORT` can use this map /// with the `bpf_sk_select_reuseport()` helper to select specific sockets for
Once you write the wrapper method, please link it here
aya-ebpf-macros/src/lib.rs line 658 at r1 (raw file):
/// /// let ret = unsafe { /// bpf_sk_select_reuseport(
We shouldn't recommend direct usage of raw bindings to users, Instead, we should wrap bpf_sk_select_reuseport as a new, safe method of SkReuseportContext:
impl SkReuseportContext {
pub fn select_reuseport(
&self,
map: &ReusePortSockArray,
key: u32,
flags: u64,
) -> Result<(), c_long> {
let ret = unsafe {
bpf_sk_select_reuseport(
self.as_ptr().cast(),
map.as_ptr(),
&key as *const u32 as *mut _,
flags,
)
};
if ret == 0 { Ok(()) } else { Err(ret) }
}
}
tamird
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, sorry about the tone of my review! I thought @vadorovsky was the author. Thanks for your contribution!
Reviewable status: 16 of 19 files reviewed, 17 unresolved discussions (waiting on @epheo and @vadorovsky)
Implements SK_REUSEPORT support to enable programmable socket selection within SO_REUSEPORT groups.
This allows load balancing decisions to be made programmatically with SO_REUSEPORT set.
Fixes: #215
Submitting as draft for now but would appreciate an early feedback regarding the overall direction.
This change is