Askrene parallel solving support #8723
Open
+1,226
−814
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g an entore response. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We reimplemented this redundantly: hash_scid was called short_channel_id_hash, so I obviously missed it. Rename, and implement hash_scidd helper too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is fairly simple. We do all the prep work, fire off the child, and it continues all the way to producing JSON output (or an error). The parent then forwards it. Limitations (fixed in successive patches): 1. Child logging currently gets lost. 2. We wait for the child, so this code is not a speedup. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We just shim rq_log for now, but we'll be weaning the child process off that soon. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Lagrang3
approved these changes
Dec 4, 2025
Collaborator
Lagrang3
left a comment
Lagrang3
left a comment
There was a problem hiding this comment.
Very clever parallelization!
We want to make it clear when future generations edit the code, which routines are called in the child (i.e. all the routing), and which in the parent. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Weird that it was in askrene.c Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now there's only one file clearly shared by both parent and child. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Notably no access to the struct command and struct plugin. Note: we actually *do* mess with askrene->reserves, but the previous code used cmd to get to it. Now we need to include a non-const pointer in struct route_query. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `askrene` now runs routing in parallel. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Queue them before we query local channels, so they don't use stale information. Changelog-Added: Config: `askrene-max-threads` to control how many CPUs we use for routing (default 4). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fork logic itself is pretty simple, so do that directly in askrene.c, and then call into "run_child()" almost as soon as we do the fork. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes sure it cannot interfere with the parent askrene's connection to lightningd, for example. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This speeds them up, and exercises the askrene parallel code. Before: test_real_data: 260s test_real_biases: 173s After: test_real_data: 133s test_real_biases: 106s And this is because much of the time is spent uncompressing the gossmap and startup. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell
force-pushed
the
guilt/askrene-parallel
branch
from
c3fb159 to
8ffb1fc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is actually fairly simple: we fork() at the point we're going to call the solver, and the child runs until it either produces an error message or a JSON result.
The main changes are in refactoring to make it clear which parts of the code run in the child, and which run in the parent.