SASE1 XTD2 attenuator arm issue

On monday 7.4.2025, one of the SASE1 attenuator arms (SA1_XTD2_ATT/PNACT/A9) with a 2mm Silicon attenuator got stuck when inserted
EEE reports in the XO meeting today that DOC / EEE helped to unblock and remove it from the beam
(from the karabo-history, I assume this was on monday between 10h00 and 10h30).
This arm A9 is now blocked/disabled with the attenuator out of beam to prevent this to happen again until this issue is fixed.
To fix the issue for good and to re-enable operation of arm A9, an access to the tunnel XTD2 would be needed.
The group in charge of this beamline component is XRO. So far there is no access request.
CTRL (Wajid) reminds that also the karabo device needs to be blocked / unblocked.

This week, SPB/SFX receive SASE1 beam.
SPB/SFX (Chan Kim and Tomas Popelar) confirm that they can work this week without this attenuator arm,
in particular because they have their own attenuators and will not use the XTD2 attenuators.
Thus, for operations it seems that this access could wait at least until next week tuesday.

SASE3 Apple-X undulator cell 26 issue - SOLVED

During the night, an issue appeared with the SASE3 undulator cell 26 (2nd last Apple-X undulator).
This cell could not be operated and was in status open.
BKR therefore couldn't tune for linear vertical polarization as required by the SASE3 users at SXP.
A ZZ access was discussed and prepared with the experts from the undulator group.

However, they now found a way to repair the issue from remote without access (info from Mikhail Yakopov / undulator group at 10h20).
The issue is solved.

PRC will contact PES spectrometer experts at XPD to tune polarization monitoring with BKR.

Beam Downtime

During last night, apparently the photon beam was down in all SASE beamlines during two longer periods:

• between 2:48am and 3h23am (35 min) on 14.4.2025
• between 5:09am and 6h59am (1 hour 50 min) on 14.4.2025

MID called and reported issue with Dachs terminals at the user labs (probably no power). TS was informed. Because the lab are currently accessible (doors are locked open), TS will take care about it on Monday.

7:20 FXE reports a problem with laser synchronisation (yellow warning light and could not change delay). BKR informs the DESY laser sync group. 

7:51 DESY sync group reported that there was a CPU failing about 6 am this morning which led to the yellow warning light. However, synchronisation was very likely all the time ok withing 5-10fs. The CPU was now reset and everything should be wokring now. 

5:49 HED called and they have problems with the Laser Sync signal. A change of laser delay is not possibe. We supsect that it is similar to earlier this week at FXE, where a crash of a sync CPU caused a similar error. BRK is informed and tries to reach somebody from sync Group via email (at 6:35, no OCD phone number available) . If this does not work, we will work with the shifts starting at 7:00 on this (DOC and new BKR crew). 

8:45 (see also DESY elog entries): After several attempts LBsync timing works, at least for now. One problem was that the crate was constantly rebooting by itself. Now it is stable, but error could happen again. In that case, BKR should activate expert via email. 

Attachements: Location of LBsync server status in DOOCS and status of XFELcpuSYNCEXP1-3 for last light

7:15 SPB reports large background from non-lasing SASE3 bunches. BKR is informed.  

7:50 BKR analysed situation. Everything concerning the orbits is OK, so the background is due to the standard spontaneous background (about < 1% level of FEL beam). With SASE3 running currently 255 lasing bunches and SASE1 single bunch the intensities will be on a similar level. Recommendation to SPB to close SRAs from 3 mm to maybe 1.5 mm to reduce background. 

On 29.01 evening the SA3 pre-absorber was interlocked due to M5 mirror.

With the help of Maurizio V., it was found the M5 RY motor has to be driven to 400um position to release the interlock condition and permit beam to SXP and SQS. This relates to a limit switch which was in a undesired location and about 1mm out of position.

A ZZ is organized to happen now, 31.01 at ~12:40, in order to reset a cabinet belonging to one of the Apple-X undulators.

This is agreed with North Branch instruments and RC/BKR.

**Update 13:25: ZZ was successful; SA1 and SA3 beams restored at 13:25

Saturday 01.Feb around 2AM machine went down due to problems with cold magnet at A20:

https://ttfinfo.desy.de/elog/XMLlist?file=/XFELelog/data/2025/05/31.01_n/2025-02-01T07:06:16-04.xml&xsl=/elogbook/xsl/elog.xsl&picture=true

During the early morning BKR used a diffrent trajectory at LINAC to continue lasing, altough with poor performance.

A Desy expert was found (voluntary basis) to made a second ZZ in the morning between 10:30 and 12:30 which fixed the faulty relay for the quadrupole/cold magnet quench system.

Beam was restored to FXE, HED and SQS instruments at ~14:00, including the change of photon energies in SA1 and SA2 as well as pointing corrections and necessary fine tuning.

X-ray delivery

• 11.0 keV, up to 178 pulses at 564 kHz, 650 uJ.

Optical laser delivery

• illumination and pp lasers

Achievements/Observations

• Performed detector and ice detection program.

Issues

• Run Controller was crashing several times during the day. DOC was notified. They discovered that the run controller server was down and fixed the problem by restarting it.
• PP laser was not available during the day until ~ 18:30
• Some troubles with and homing of SmarAct stages after update from to the current control version. DOC was requested to roll back to the latest stable release.

X-ray delivery

• 11.0 keV, up to 352 pulses at 1.1 MHz, 900 uJ.

Optical laser delivery

• ns illumination and pp lasers

Achievements/Observations

• Finished with Ice detection program
• Optimized the beamline alignment for the next experiment
• Checked the NKB focusing with CRL collimation: The collimated scheme resulted in smaller focus size and significantly higher intensity at focus

Issues

• PAM mirror mount is not reacheable with the SmarAct motor

X-ray delivery
-12.4 keV, 202 pulses, 0.5 MHz, 1mJ/pulse

Optical laser delivery
- ns laser (660 nm wavelength) for jet illumination

Achievements/Observations
- NKB focus optimized
- One PCS user program successfully finished
- Second PCS user program was just started

Issues
- Right before we could start with beam defocusing, the beam was gone around 8:45 pm. BKR called to let us know that there will be no beam today (cryo issues)

X-ray delivery

• 12.4 keV, 1.1mJ/pulse, 202 pulses/train at 0.56MHz
• Focused NKB beam was used

Optical laser delivery

• ns laser for jet visualization

Achievements/Observations

• Successfully finished all PCS beamtimes
• Stable jet with PEO sheath

Issues

• No beam in the morning
• Unstable jet with ethanol sheath

X-ray delivery

• 18.8 keV, 260 uJ pulse energy

Optical laser delivery

• No optical laser

Achievements/Observations

• Experimental setup optimized, both instrumental side and PIXIE side
• X-ray beam focused to the PIXIE and transmission measurement was done with several different PIXIE operation conditions.

Issues

• No beam for ~4.5 hours due to ZZ.
• X-ray beam not found with IRDa InlineMic after focusing X-ray beam to IRDa sample plane.
• InlineMic Y motor is in error states. It has to be fixed during the day.
• upstream kapton window for the downstream vacuum pipe broken. The glue has become loose. No damage by X-ray was visible.

The image intensifier in the SQS Exit Slit imager had been exchanged on request of SQS, Thomas Baumann.
Model: Hamamatsu MCP C9546-03-Y001 against C9546-01.
The position of the YAG screen on the camera is slightly shifted which may slightly affect the spectral axis, a few pixels vertically, and approximately 15 pixels horizontally.
The image scale by zoom had not been changed, as before 13 µm/pixel.

The SA3 FEL imager is now protected via the Big Brother server, which creates a beam limit on the bunch pattern server if necessary.
contact Andreas Koch, Lars Fröhlich for more details or consult the imager manual, section 2.9:
https://docs.xfel.eu/share/page/site/xfelwp74/document-details?nodeRef=workspace://SpacesStore/1766b430-7c48-4c46-bb2d-1774d69e1246
demonstration 27.1.2025 to Jan Grünert, Harald Sinn, Andreas Galler.

Function: The Big Brother limits the number of pulses in SA3 via a beam limit in case of danger, individually for each screen of the FEL imager, AND only if an imager screen is inserted. The EPS is valid as before.
This means that if the EPS interlock for the FEL imager is enabled (default setting, 2 pulses maximum), the intervention by the Big Brother could reduce pulses to 1 or 0.
The potential limit of the number of pulses for each screen is shown in the FEL imager scene, as before.

Best practice: Use the GATT to limit the pulse energy. Avoid to operate with a strongly saturated image. Use low absorbing ND filters.

eLogs posts to: eLog Operation-2025,  Zulip SASE3_INSTRUMENT_COMMISSIONING, ttfinfo