DPL Analysis: improve grouping performance further

[aalkin]

@ddobrigk @ktf

By skipping most of the heavy processing for the zero-size slices and avoiding allocations for the values_counts arrow function - just calculating sizes and offsets in one loop over the index column - I was able to reduce the processing time for an extra 1 second per 100 000 events.

Profiling demonstrates that the remaining ~3 seconds per 100 000 events are pretty much all in creation of non empty slices, which involves string lookup of columns and binding of the iterators. In addition, there is a considerable fraction of time taken by deconstruction of shared pointers to arrow tables when the slices go out of scope.

The "naive" code by @ddobrigk, since it does not construct any tables at all, avoids the overhead completely. But we cannot, of course, use this as a generic approach.

The next improvement I can do is to avoid constructing empty arrow table slices at all, while still providing the soa::Table slices for the user with the zero size and no bindings. That should additionally reduce the processing time by about 15%. Switching from string-based column lookup to the order-based would give a huge performance boost as well, but for that we need to determine if we still have any data left where the branch order in the stored trees differs from the column order in the data model.

[github-actions]

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

[alibuild]

Error while checking build/O2/fullCI_slc9 for 67d8b69 at 2025-09-03 12:00:

## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile digi.log


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
grep: error-log.txt: binary file matches
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/b522e39ff42b738b2d174361b2b190cedc6cb6d1/slc9_x86-64/o2checkcode/1.0-local417/etc/modulefiles
++ cat
--

Full log here.

[ktf]

@ddobrigk could you check with your benchmark?

[ddobrigk]

I'm running the benchmarks now - @aalkin can you confirm these changes are for the standard sorted grouping and not unsorted? Thanks!

[ddobrigk]

Hi @ktf @aalkin, sorry for not writing before: here is the outcome of the benchmarks. It seems the new code has better performance in the asymptotic regime of large groups and is essentially unaltered for small group cases: see attached figure. I would say this can be merged so that we improve the asymptotic behaviour. Thanks a lot for the work!

Maybe a question to @aalkin : that unsorted performance trend is still pretty bad, at 1000x worse than the custom solution in the large-group limit. It would be good if we could improve that at some point as well...

[ktf]

Merging following @ddobrigk comments. We should have some sort of benchmark / validation suite for O2Physics and run it in the CI.