Continuous pointing drift in horizontal direction.

It may trigger a vacuum spike at SPB OH (MKB_PSLIT section) at 21:07 pm.

DOOCS panels are again incorrectly reporting the FXE shutter as opened when it is in fact closed. This had been fixed after it was last reported in February (elog:24), but reappeared now.

We had some trouble with the DAQ. It was not able to start runs properly.

We got the following error messages:

initially:

[00:01:27]: Problem of communication with data manager

[00:01:30]: Start new run ...

[00:01:38]: Failed starting data recording

DM: error at registerRun (Request to MDC timeouted)

after trying to start the run again:

[00:05:35]: Start new run ...

[00:05:35]: Failed starting data recording

DM: error at registerRun (Problem of communication with MDC

#Info: MDC-1100: MDC web server error: Error creating run

#Details: {'run_number': ['must be unique per proposal', 'has already been taken'], 'experiment_id': ['has already been taken']})

Cycling the DAQ resolved the issue.

X-ray delivery
- 12.72 keV, 10Hz, ~1.4 mJ/pulse, stable delivery 

Optical laser delivery
- 487 nm ns laser for pump-probe
- 800 nm PP laser for jet illumination, stable delivery

Achievements/Observations
- AGIPD BG optimized (DiffPump realigned)
- IRDA focus optimized (EHC_CRL arm10 (1mm pinhole) inserted)
- Completed dark state measurement of the main sample (more than 4 hours of measurement time)
- Collected PP (pump-probe) dataset with 5 us time delay (about 3 hours of measurement time; Jet was quite stable during the 3 hrs measurement)

Issues
- AGIPD module Q3M2 was not healthy. After another power cycle, it recovered.
- X-ray pulse energy was relatively low in the beginning of the shift. It was partially recovered after running the launch optimization by BKR.

Pulse energy fluctuation observed after full train operation in SA2 starts.

X-ray delivery

• 12.72 keV, 10 Hz, NOT stable delivery, pointing drastically changed in the afternoon, presumably due to the train-on-demand operation at SASE2

Optical laser delivery

• 487 nm ns laser (not used)
• 800 nm PP laser for sample illumination

Achievements/Observations

• Beam is aligned for IRDa
• lysosyme date was collected for JF4M geometry

Issues

• X-ray pointing changed in the afternoon, we have to figure out why and whether we can avoid losing trajectory
• Beam is currently clipped and EHC_SCR, @Romain Letrun , @Adam Round please take a look tomorrow
• Detector background was not optimal, might require more work tomorrow
• Roadrunner RX motor did not move, in Karabo, was giving error messages @Adam Round please take a look, maybe resetting the roadrunner outside of Karabo can help
• AGIPD MFPGA2 issue (keep acquiring) which was solved by power cycle

User experiment transient grating, Cris Svetina. (LTP)

• Used 7.11 keV from ACC with very good performance, above 4 mJ.
• Collected pump-probe data on user sample most of the time today
• FXE will operate in 24h mode for the rest of the delivery week

User experiment transient grating, Cris Svetina. (LTP)

• Used 7.11 keV from ACC with very good performance, above 4 mJ.
• Collected pump-probe data on user sample most of the time today, only minor interruptions due to accelerator outages
• After the outage at 21h, SASE came back with 17 eV lower photon energy, so we had to re-calibrate the undulator spectrum
• FXE will operate in 24h mode for the rest of the delivery week

X-ray delivery
- 9.3 keV, >3 mJ/pulse, beam down around 30min

Optical laser delivery
- 800nm PP laser for jet illumination, stable delivery

Achievements/Observations
- Collected a lot of data (from midnight to 6am) with different sample concentrations and mixing delays with the users

User experiment transient grating, Cris Svetina. (LTP)

• Used 7.12 keV from ACC with very good performance, above 4 mJ.
• Collected pump-probe data on user sample
• Sudden timing jump of around 20 ps. Could not find out why this happened some time between 9h today and 18h.
• Needed to add camera to DAQ. Solved this on instrument side using the DAQ assistant. Great tool!

X-ray delivery
- 9.3 keV, >3 mJ/pulse, stable delivery 

Optical laser delivery
- 800nm PP laser for jet illumination, stable delivery

Achievements/Observations
- Optimised hexapod position for SFX
- Collected granulovirus data for geometry calibration
- Started user sample delivery but clogged the nozzle a few times

Preparation for user experiment transient grating, Cris Svetina

• Used 7.12 keV from ACC with very good performance, above 4 mJ
• Got spatiotemporal overlap with transient gratings using monochromatic beam
• One issue solved by DOC: Picomotors could not be scanned in karabacon, even though this worked well two/three weeks ago

BKR called PRC around 06:10AM to inform on vacuum issues at SA1 beamline leading to a lost beam permission for the North Branch.

VAC-OCD found an ion pump in the PBLM / XTD9 XGM presented a spike in pressure that triggered valves in the section to close.

After reopening valves the beam permission was restored; further investigations and monitoring in coming days will continue. Currently all systems work normally.

https://in.xfel.eu/elog/Vacuum-OCD/252

11:15 SASE1/SPB called that they observed and issue with the SASE1 XGM/XTD2: about every 35 seconds the power reading jumps from 4 Watts to zero and back. It seems to be related to some automated re-scaling of the Keithlyes, which coincides. The fast signal and downstream XGMs do not show this behavior. The concern is that this may affect the the data collection later today. 

12:15 The XGM expert was informed via email, however, there is no regular on-call service today. If this cannot be fixed in the next few hours, the PRC advises to use in addition the reading of the downstream XGM for data normalisation. 

13:15 Theo fixed the problem by disabling auto-range. Needs follow-up on Monday, but for this weekend should be ok. 

**X-ray delivery**

- 6 keV, 3.2 mJ/pulse, stable delivery

**Optical laser delivery**

- 800nm PP laser for jet illumination, stable delivery

**Achievements/Observations**

- NKB focus checked and IUT_Slits optimized
- AGIPD quadrants position finalized
- Measured a lot of water BG runs and few runs with Au nanoparticles (sphere with ~50 nm in diameter)

**Issues**

- SideMic data not saved properly (empty array).
- Found relatively high BG when nozzle is IN since the distance in between nozzle tip & the interaction region is ~100um (even 80 um in some cases).
- DA issue (DAQ was stuck in changing) although all the DA looked healthy. DOC fixed the issue (temporarly) together with Egor, Florian, and Steffen.
- Runs callibration were not working

it is not clear why, but the beam position on BIU2 has changed over time, even more since we checked last. it is off in both directions

we see the GATT aperture projection drifting, so it is a change in the beam transport, not in the source.

see the beam, cut by the DIAG1 slit, compared to the cross it should be centred to.

the motor encoders in the beamline have not changed since the beginning, when the beam was pointing correctly.

There are now two watchdogs for the XTD2 attenuator running at the same time

• SA1_XTD2_WATCHDOG/MDL/ATT1_MONITOR020
• SA1_XTD2_OPTICS_PROTECTION/MDL/ATT1

that are configured differently, which results in different pulse limits (see attachment).

X-ray delivery

6 keV, 2.4 mJ/pulse, stable delivery

Optical laser delivery

800nm PP laser, stable delivery

Achievements/Observations

NKB focus cslits optimized
Some progress with AGIPD background although we still need to improve
Catcher and injector motors (X, Z) are aligned well; with defroster, had under 5mm offset in X
Managed to get a jet and overlap with X-rays.

Issues

Beckhoff flowmeters were disconnected and we were not sure which ports to connect, maybe spent 45min to connect to the right ports and restart device.
Helium flow meter was on but not updating, had to restart it too.
Marco noted actual measured flow at beckhoff VS HPLC set flow was perhaps too low, not sure
Vacuum with injector improved, issue was related to nozzle preparation, however even after this fix the chamber pressure was just over the AGIPD interlock. After tuning gas/liquid flow managed to go under it, but possibly this can also be improved.
Often number of pulses was being changed without us requesting it.
One AGIPD module stopped working after moving quadrant.
DAQ data aggregator crashed a few times, Steffan Hauf will look into this issue after we finished the shift.
Could not move one of the AGIPD quadrant (Q4) due to interlock
Background in Q1 and Q3 needs to be optimized, and/or quadrants needs to be moved.

SASE1 bunch number was reduced to 10 pulses for no obvious reason. 

Traced it back to a malfunctioning of the watchdog for the solid attenuator (see attached error message)

Deacttivated the watchdog for SASE1 solid attenuator. Re-activated it again.

Issue: SASE1 (SPB?) was requesting 200 pulses at 6 keV with full power with all solid attenuators in the beam. This was a potentailly dangerous situation for the solid attnenuator (!)

After that the team left to go home. Nobody is available at FXE.  Set pulse number to 1.

Checked status of solid attenuator. Seems that plate 1&2 have some vertical cracks (see attached picture)