Skip to content

Conversation

@saganatt
Copy link
Collaborator

Rewrite groupTable() to use the O2 table iterators instead of manually iterating over Apache Arrow chunked columns.
Credits to @mytkom for the idea and most of the implementation.

It gives a 2x speed-up to the creation of a blocked combination policy -- see the results (mean of 5 benchmark runs) in the table below.
This does not affect the running time of creation of a combination generator and generation of combinations.

Table size Real time after the simplification (ns) Real time before the simplification (ns)
4 282.39 713.76
8 402.68 1,121.59
16 646.18 1,911.54
32 1,102.81 3,364.43
64 2,090.18 6,332.45
128 3,945.50 12,165.50
256 7,946.86 23,717.70
512 16,024.30 46,894.60
1024 32,856.40 95,093.30
2048 67,987.50 192,691.00
4096 150,429.00 392,947.00
8192 336,427.00 825,430.00
16384 716,285.00 1,674,110.00
32768 1,467,430.00 3,434,070.00
65536 3,011,900.00 6,886,320.00
131072 6,172,810.00 13,968,900.00
262144 12,517,400.00 28,290,700.00
524288 25,900,100.00 57,190,100.00
1048576 58,248,200.00 120,218,000.00
2097152 136,993,000.00 262,650,000.00

FYI @mytkom @chefxxx @MykhailoShamrai

@saganatt saganatt requested a review from a team as a code owner July 11, 2025 11:40
@github-actions
Copy link
Contributor

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

Please consider the following formatting changes to AliceO2Group#14501
@alibuild
Copy link
Collaborator

Error while checking build/O2/fullCI_slc9 for e74662b at 2025-07-15 09:36:

No log files found

Full log here.

@ktf
Copy link
Member

ktf commented Aug 11, 2025

Looks good, thanks. Which tests did you do to validate the results?

@saganatt
Copy link
Collaborator Author

saganatt commented Aug 11, 2025

Looks good, thanks. Which tests did you do to validate the results?

I ran event mixing tutorial tasks (file eventMixing.cxx in O2Physics/Tutorials/src) and checked that the console output of these tasks doesn't change with this PR.

I performed the benchmark with the following code that is not merged yet, but I can make a PR for that after my vacations: https://github.com/saganatt/AliceO2/blob/mixing-benchmark-simplify-group-table/Framework/Core/test/benchmark_EventMixingParts.cxx

@ktf ktf merged commit 5df80a7 into AliceO2Group:dev Aug 11, 2025
11 checks passed
@saganatt saganatt deleted the simplify-group-table branch September 10, 2025 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants