Storing numpy arrays as attributes of a detector

[rgeneaux]

Hello,
I have a question regarding backgrounds for a detector. The detector plugin emits several types of data so using the usual background functionality does not work well. Instead, another take_background method is triggered by cliking on a bool_push in the parametertree, and the background is saved as a class attribute as an np.ndarray.

However I would also like to save it together with any data that is being saved by the detector (either by daq logger, daq scan, or just save button). I thought of including it to the list of DataFromPlugins (with do_plot=False) but this means we'll just save many copies of the background, for instance at each scan position. Another way would be to store it in the parametertree of the detector (hidden, probably), but I don't know which Parameter type would support an np.ndarray! Has anyone already tried something similar?
Thanks!

[seb5g]

Various questions arises:

• why is the usual background not working well, could you develop?
• you could also save it as a h5file from the detector then load it from a filename (in a settings) into your plugin to do the background substration
• parametertree is a bad idea because the content is saved as a binary string within the node attributes. Each attribute is limited to a certain size that would quickly be overiden by the numpy array saved as a binary

[rgeneaux]

Yes, the viewer outputs several data: two images (or spectra if full vertical binning) corresponding to "pump on" or "off", then another one with the resulting dA/A, +possibly other things depending on options, like the timestamps of each image. It also does things differently depending on whether you want to see each acquisition, or if you treat incoming data by chunk. Another thing is that there is actually two backgrounds, for on and off spectra (because of pump scattering).
All that makes me think that the normal background will not work, we don't want to take the background of the timestamps, or of dA/A, for instance!

The h5file seems like a good idea, especially since the filename will be saved in the parametertree so we will be able to match each dataset with its proper background file. Thanks!

[rgeneaux]

We were indeed able to solve the problem using an h5file, thanks!

For reference, some details:
When we take the backgrounds, they are also saved to an h5file. The name of this file is saved as a read-only parameter of the plugin. This way, any data that is taken will have a "link" to the correct background that is in use.

this is what the save function looks like:

 def save_background(self, dte):
        # Saving as viewer attribute
        ponoff = dte.get_data_from_name("Pump On/Off").data
        self.bkg_poff, self.bkg_pon = ponoff

        # Saving to h5 file for future post-processing
        h5saver = H5Saver(save_type='detector')
        h5saver.settings.child("base_name").setValue("Background")
        h5saver.init_file(update_h5=True, custom_naming=False)

        self.settings["camera_settings", "current_bkg_file"] = h5saver.settings["current_h5_file"]

        self.module_and_data_saver = module_saving.DetectorSaver(self)
        self.module_and_data_saver.h5saver = h5saver

        detector_node = self.module_and_data_saver.get_set_node()
        self.module_and_data_saver.add_data(detector_node, dte)

        h5saver.close_file()

To be able to use the add_data method right away, I had to add two attributes to the plugin

def ini_attributes(self):
        self.title = "Camera"
        self.ui = None

[seb5g]

looks like a good use of advanced saving mechanisms ! Nice to see people starting to use them

[seb5g]

Actually you should probably have use a DataToExportSaver that would make your life even easier:

you could then use just a path (of your h5) as an argument and a with statement, like

with DataToExportSaver('the/path/to/h5.h5', save_type='detector') as saver:
    do stuff

that would automatically and properly close the file without the need for all the stuff about H5Saver and extra attributes