Skip to content

Conversation

@ktf
Copy link
Member

@ktf ktf commented Aug 10, 2025

Thanks to the newly added binary view columns we can finally support
proper CCDB integration in analysis.

In order to do so, the user needs to create a TIMESTAMPED table, i.e. a table
which is an extension of another one where the timestamps for each rows
are provided. The extra columns of such timestamped table will be CCDB columns
where the iterator of each provides access for one specified CCDB object.

Notice the PR duplicates the CCDB code run in reconstruction and the additional
device will never be added to the topology if running online, so there are no expected
side effects.

@ktf ktf requested a review from a team as a code owner August 10, 2025 13:39
@ktf ktf mentioned this pull request Aug 10, 2025
@github-actions
Copy link
Contributor

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

@ktf ktf changed the title DPL Analysis: centralised CCDB support in analysis WIP DPL Analysis: centralised CCDB support in analysis Aug 10, 2025
@alibuild
Copy link
Collaborator

Error while checking build/O2/fullCI_slc9 for a7b2795 at 2025-08-10 18:36:

## sw/BUILD/O2-latest/log
[ERROR] invalid workflow in /sw/BUILD/05b14db3cfde306e932e5b730c7debb85c783acd/O2/stage/bin/o2-testworkflows-analysis-ccdb: In spec simple-ccdb-fetcher input specification 1 requires binding, description and origin to be fully specified (found :AOD/TOFCALIB/1)
ninja: build stopped: subcommand failed.

Full log here.

@alibuild
Copy link
Collaborator

Error while checking build/O2/fullCI_slc9 for be4cb11 at 2025-08-15 11:38:

## sw/BUILD/O2-latest/log
CMake Error at cmake/O2AddLibrary.cmake:76 (add_library):
CMake Error at cmake/O2AddLibrary.cmake:76 (add_library):

Full log here.

@ktf
Copy link
Member Author

ktf commented Aug 16, 2025

Still to do:

  • actual fetching
  • Making the fTimestamp column consigurable

@alibuild
Copy link
Collaborator

Error while checking build/O2/fullCI_slc9 for 35a1907 at 2025-08-16 23:54:

## sw/BUILD/O2-latest/log
CMake Error at cmake/O2AddLibrary.cmake:76 (add_library):
CMake Error at cmake/O2AddLibrary.cmake:76 (add_library):

Full log here.

@ktf ktf changed the title WIP DPL Analysis: centralised CCDB support in analysis Actually working Aug 20, 2025
@ktf
Copy link
Member Author

ktf commented Aug 20, 2025

With this the example (o2TestAnalysisCCDB.cxx) of the centralised CCDB for analysis tasks actually works.

I need to do some cleanup before I can merge this, and I need to check I did not break the non analysis CCDB access, but we should not be far.

@ktf ktf changed the title Actually working WIP DPL Analysis: centralised CCDB support in analysis Aug 20, 2025
@alibuild
Copy link
Collaborator

Error while checking build/O2/fullCI_slc9 for 71abb21 at 2025-08-20 21:53:

++ cp /sw/slc9_x86-64/O2/14567-slc9_x86-64-local1/compile_commands.json .
+++ python3 -c 'import json, os; print(os.path.commonpath([x["file"] for x in json.loads(open("compile_commands.json").read()) if "sw/BUILD" not in x["file"] and "G__" not in x["file"] and x["file"].endswith(".cxx")]))'
++ O2_SRC=/sw/SOURCES/O2/14567-slc9_x86-64/0
++ [[ -e /sw/SOURCES/O2/14567-slc9_x86-64/0/CMakeLists.txt ]]

Full log here.

Thanks to the newly added binary view columns we can finally support
proper CCDB integration in analysis.

In order to do so, the user needs to create a TIMESTAMPED table, i.e. a table
which is an extension of another one where the timestamps for each rows
are provided. The extra columns of such timestamped table will be CCDB columns
where the iterator of each provides access for one specified CCDB object.

Notice the PR duplicates the CCDB code run in reconstruction and the additional
device will never be added to the topology if running online, so there are no expected
side effects.
@ktf ktf changed the title WIP DPL Analysis: centralised CCDB support in analysis DPL Analysis: centralised CCDB support in analysis Aug 22, 2025
@ktf
Copy link
Member Author

ktf commented Aug 25, 2025

Despite the large size, this PR only introduces new features so it should be relatively safe to merge. In particular I took extra care in making sure the code is not enabled at all in online more and I duplicated anything which might have been used also by the reconstruction CCDB fetcher, to minimise risks. This has also been tested on HyperLoop with no regressions in Correlations and Histograms, as expected. Merging.

@ktf ktf merged commit d375e63 into AliceO2Group:dev Aug 25, 2025
15 of 17 checks passed
mhemmer-cern pushed a commit to mhemmer-cern/AliceO2 that referenced this pull request Sep 9, 2025
Thanks to the newly added binary view columns we can finally support
proper CCDB integration in analysis.

In order to do so, the user needs to create a TIMESTAMPED table, i.e. a table
which is an extension of another one where the timestamps for each rows
are provided. The extra columns of such timestamped table will be CCDB columns
where the iterator of each provides access for one specified CCDB object.

Notice the PR duplicates the CCDB code run in reconstruction and the additional
device will never be added to the topology if running online, so there are no expected
side effects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants