Skip to content

Conversation

@sawenzel
Copy link
Collaborator

@sawenzel sawenzel commented Nov 18, 2025

This commit expands on #14427 and fixes the issue brought up in https://its.cern.ch/jira/browse/O2-4671.

After debugging/testing it turns out that the approach taken via a customer streamer for std::vector<o2::tpc::PadFlags> does not take effect in the ROOT/IO because apparently ROOT still prefers to use the CollectionProxy for std::vector and does not employ the custom streamer.

Instead, after discussion with @pcanal, this commit proposes to implement a custom stream just for the mData data member of CalArray<o2::tpc::PadFlags>. This is the only place where we use o2::tpc::PadFlags in IO and it fixes the problem when reading CCDB objects containing such data.

I have verified that the following code

o2-ccdb-downloadccdbfile -p TPC/Calib/IDC_PadStatusMap_A -t 1731274461770 -d ./ -o tpc_idc.root --no-preserve-path
root tpc_idc.root
gFile->Get<o2::tpc::CalDet<o2::tpc::PadFlags>>("ccdb_object")

correctly executes the custom streamer function.

Note that there is also no need to make the code ROOT version dependent. We need to fix the reading in any case and the writing will just stay the same.

Concerning situations, where future classes will write data containing std::vector<o2::tpc::PadFlags> we should be protected by the fact that this bug has been fixed >= ROOT 6.36 in any case.

This commit relates also to

root-project/root#17009

This commit expands on AliceO2Group#14427
and fixes the issue brought up in https://its.cern.ch/jira/browse/O2-4671.

After debugging/testing it turns out that the approach taken via a customer streamer
for std::vector<o2::tpc::PadFlags> does not take effect in the ROOT/IO because
apparently ROOT still prefers to use the CollectionProxy for std::vector and does not
employ the custom streamer.

Instead, after discussion with @pcanal, this commit proposes to implement a
custom stream just for the mData data member of CalArray<o2::tpc::PadFlags>.
This is the only place where we use o2::tpc::PadFlags in IO and it fixes the problem
when reading CCDB objects containing such data.

I have verified that the following code
```
o2-ccdb-downloadccdbfile -p TPC/Calib/IDC_PadStatusMap_A -t 1731274461770 -d ./ -o tpc_idc.root --no-preserve-path
root tpc_idc.root
gFile->Get<o2::tpc::CalDet<o2::tpc::PadFlags>>("ccdb_object")
```
correctly executes the custom streamer function.

Note that there is also no need to make the code ROOT version dependent. We need to fix the reading in any
case and the writing will just stay the same.

Concerning situations, where future classes will write data containing std::vector<o2::tpc::PadFlags>
we should be protected by the fact that this bug has been fixed >= ROOT 6.36 in any case.

This commit relates also to

root-project/root#17009

The commit also re-enables dictionary creation of related classes
and adds a dictionary for CalArray<o2::tpc::PadFlags> previously missing.
@github-actions
Copy link
Contributor

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

@sawenzel sawenzel requested a review from ktf November 18, 2025 14:54
@ktf
Copy link
Member

ktf commented Nov 18, 2025

As discussed privately, I think indeed the current code right now works by accident in certain builds due to the "static initialisation order fiasco" accidentally being the one we need. Given this improves over it, I am all for it.

@sawenzel
Copy link
Collaborator Author

The problem is not related to static initialization but rather that ROOT ignores custom streamer functions on std C++ collections. The following code demonstrated this:

root [0] auto f=[](TBuffer& R__b, void* objp) { std::cout << "Hello\n"; return; };
root [1] std::vector<int> vec(10);
root [2] auto cl = TClass::GetClass<std::vector<int>>();
root [3] cl->SetStreamerFunc(f);
root [4] TFile out("outfile.root", "CREATE");
root [5] TTree tree("data","data");
root [6] tree.Branch("branch", &vec);
root [7] tree.Fill();

because the writing of vec does not print anything. However, using custom streamers on the level of members of C++ classes works fine, which is done with this PR.

@sawenzel sawenzel merged commit 2439bbe into AliceO2Group:dev Nov 19, 2025
13 checks passed
@sawenzel sawenzel deleted the swenzel/calarray_streamer branch November 19, 2025 13:39
@ktf
Copy link
Member

ktf commented Nov 19, 2025

I am puzzled. I developed the code with a similar small case reproducer and there is a different behavior in O2Physics with my previous patch removed. Do you see the same without the pragma link C++ class std::vector < o2::tpc::PadFlags> + ?

@sawenzel
Copy link
Collaborator Author

Yes I tried with Philippe the cases with and without the #pragma and it didn't make a difference.

sawenzel added a commit to sawenzel/AliceO2 that referenced this pull request Nov 25, 2025
A continuation of the CalDet<TPCFlags> saga, possibly related
to https://its.cern.ch/jira/browse/O2-4671

Tests on ARM, even after deployment of the custom streamer
in AliceO2Group#14830, still showed
segfaults in TPC digitization.

With the relevant code isolated into a unit test in
AliceO2Group#14850, it was possible to
do a valgrind study. This showed Invalid reads to the mData of CalArray.

Thereafter, putting assert statements showed that we often access
CalArray<PadFlags> data slightly out of bounds - irrespective of custom
streamer or not. This then either indicates a problem in the code logic
or a problem with the calibration CCDB objects. This should clearly be
fixed.

In the meantime, this commit adds a protection against invalid accesses
and returns a trivial answer as well as an error message. This is in any
case better than undefined behaviour.

In addition, this commit introduces possibility to switch off the custom
streamer for further studies.
sawenzel added a commit that referenced this pull request Nov 25, 2025
A continuation of the CalDet<TPCFlags> saga, possibly related
to https://its.cern.ch/jira/browse/O2-4671

Tests on ARM, even after deployment of the custom streamer
in #14830, still showed
segfaults in TPC digitization.

With the relevant code isolated into a unit test in
#14850, it was possible to
do a valgrind study. This showed Invalid reads to the mData of CalArray.

Thereafter, putting assert statements showed that we often access
CalArray<PadFlags> data slightly out of bounds - irrespective of custom
streamer or not. This then either indicates a problem in the code logic
or a problem with the calibration CCDB objects. This should clearly be
fixed.

In the meantime, this commit adds a protection against invalid accesses
and returns a trivial answer as well as an error message. This is in any
case better than undefined behaviour.

In addition, this commit introduces possibility to switch off the custom
streamer for further studies.
sawenzel added a commit to sawenzel/AliceO2 that referenced this pull request Nov 26, 2025
A continuation of the CalDet<TPCFlags> saga, possibly related
to https://its.cern.ch/jira/browse/O2-4671

Tests on ARM, even after deployment of the custom streamer
in AliceO2Group#14830, still showed
segfaults in TPC digitization.

With the relevant code isolated into a unit test in
AliceO2Group#14850, it was possible to
do a valgrind study. This showed Invalid reads to the mData of CalArray.

Thereafter, putting assert statements showed that we often access
CalArray<PadFlags> data slightly out of bounds - irrespective of custom
streamer or not. This then either indicates a problem in the code logic
or a problem with the calibration CCDB objects. This should clearly be
fixed.

In the meantime, this commit adds a protection against invalid accesses
and returns a trivial answer as well as an error message. This is in any
case better than undefined behaviour.

In addition, this commit introduces possibility to switch off the custom
streamer for further studies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants