2. Inspecting and processing data during the experiment¶
2.1. Data display during experiment¶
Karabo provides a customised GUI scene that displays relevant data during an experiment. E.g. the last captured corrected-Xray image from the detector, and some crucial other parameters such as train and pulse ID, beam energy, detector position, and for pump-probe experiments the pump-probe laser state, pump-probe laser delay, etc.
Specific scenes can be developed for each experiment’s needs. An example is shown in Figure Fig. 2.1.
2.2. Integrating data analysis pipelines¶
Given enough prior notice, an on-the-fly pipeline can be created in collaboration with the users and made available to European XFEL experiments. Hence, calculation (like integration, correlations, dynamic ROI selection, peak detection, etc.) can be made available on-the-fly.
2.3. Data stream to user tools¶
Karabo bridge is a proxy interface to stream data to tools that are not integrated into the Karabo framework. It can be configured to provide any detector or control data. This interface is primarily for online data analysis (near real-time), but the extra_data Python package can also stream data from files using the same protocol.
We provide Karabo Bridge Clients to receive this data in Python and C++, but you can also write your own code to receive the data if necessary.
In order to test tools with Karabo bridge, we provide
a simulated server in the
karabo_bridge Python package.
This sends nonsense data with the same structure as real data.
To start a server, run the command:
The number (4545) must be an unused local TCP port above 1024.
2.4. How to use it at the EuXFEL¶
Karabo bridge devices (from which karabo bridge clients can connect to and receive data) are running at all instruments.
Data from MHz detectors at the EuXFEL (AGIPD, LPD, DSSC) might have a different data shape in file and in data streams. For each train data the shape in file follows: (pulse index, slow scan axis, fast scan axis). For performance constraints, the default shape offered for this data via data stream is: (module number, fast scan, slow scan, pulse index). This configuration can be changed on demand during beamtime and be switched between 2 shapes:
- online: (module number, fast scan, slow scan, pulse index)
- file-like: (pulse index, module number, slow scan, fast scan)
If you wish to use one or the other, communicate it to your contact person at the instrument.
Having file-like data shape online will add delay to the data stream, you might want to avoid this option if faster feedback is important for your experiment.
2.4.1. SPB instrument¶
2 bridge devices are permanently running and data can be requested from them. they are respectively sending raw and calibrated data of the AGIPD detector. The following table gives connection details:
|AGIPD RAW DATA||10.253.0.51||45000|
|AGIPD CORRECTED DATA||10.253.0.51||
2.4.2. FXE instrument¶
2 bridge devices are permanently running and data can be requested from them. they are respectively sending raw and calibrated data of the LPD detector. The following table gives connection details:
|LPD RAW DATA||10.253.0.53||4500|
|LPD CORRECTED DATA||10.253.0.53||4501|
2.4.3. MID instrument¶
1 bridge device is permanently running and data can be requested from it. It sends calibrated data of the AGIPD detector. The following table gives connection details:
|AGIPD CORRECTED DATA||10.253.0.171||
Any data pipeline can be temporarily set-up on request