Skip to content

Conversation

@matheus23
Copy link
Member

Description

When I look at qlog files of iroh-bench runs, I can see that PATH_ACK processing is severely delayed when using a single-threaded tokio runtime for each endpoint.
This disappears when I give iroh-bench two worker threads. This allows the EndpointDriver and the ConnectionDriver to each run independently of each other, thus the endpoint driver won't be delayed when processing PATH_ACKs.
When the processing was delayed, this had an effect on the apparent RTT and in turn on congestion control.

Screenshots

Before: Note all the blue lines going in from waaaaay back in history all into one place on the left (those are the PATH_ACKs that get handled way too late):
image

After: The PATH_ACKs get handled in time, they're way more interspersed with stream frames:
image

Change checklist

  • Self-review.

@matheus23 matheus23 self-assigned this Dec 16, 2025
@github-actions
Copy link

github-actions bot commented Dec 16, 2025

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/3780/docs/iroh/

Last updated: 2025-12-16T16:58:58Z

@matheus23 matheus23 closed this Dec 16, 2025
@matheus23 matheus23 force-pushed the matheus23/more-stable-iroh-bench branch from d357e3a to 94caac0 Compare December 16, 2025 15:55
Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered making this a command-line option?

I can imagine testing using a single thread is still useful to make sure it still behaves well in that usecase. One day we'd like to support constrained environments...

@matheus23 matheus23 reopened this Dec 16, 2025
Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving in case you want to merge without additional work

@matheus23
Copy link
Member Author

Have you considered making this a command-line option?

I have not...

I can imagine testing using a single thread is still useful to make sure it still behaves well in that usecase. One day we'd like to support constrained environments...

Yeah - I suspect that'll require some changes to quinn's pacer or sth like that?
Perhaps there's another easy fix that makes tokio choose the EndpointDriver more often more fairly? Some kind of yielding? Maybe this is related and might help? https://tokio.rs/blog/2020-04-preemption

@n0bot n0bot bot added this to iroh Dec 16, 2025
@github-project-automation github-project-automation bot moved this to 🏗 In progress in iroh Dec 16, 2025
@matheus23 matheus23 changed the title chore(iroh-bench): Use two worker threads for each endpoint in iroh-bench chore(iroh-bench): Allow configuring the number of worker threads for each endpoint in iroh-bench Dec 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🏗 In progress

Development

Successfully merging this pull request may close these issues.

3 participants