On Shift: TS, FT, CK, AR, PS

X-ray delivery
8.951 keV, 1.6 mJ up to 200 pulses at 0.5 MHz (intesity drops off at end of train)

Achievements/Observations
Beam trajectory and focusing re-established at 8.951 keV
Temporary beamstop verified position for sweeping - confirmed by eric
Beam sweeping in synchronisation with pulse trains observed!
slit window assembly remounted and beam is going through to back of hutch in perparation for optimisation next shft

End of in-house 8496

X-ray delivery

15.13 keV 30 pulses @0.5 MHz and 64 pulses (customized pattern), 0.85-1.0 mJ stable delivery

Achievements/Observations

• We used the customized pattern and it worked well, thanks to everyone involved
• One fuse was blew, TS OCD had to come on site
• Interlock was brocken accidently with the shutter open, which tripped the beam in all SASEs

this morning, ZZ was done together with Sylvia Huynh (EEE) to solve softmono pre-mirror motor issue https://redmine.xfel.eu/issues/177392

The issue was solved and also mechanical functionality of the stage was checked.

After ZZ, the beamline remained interlocked. Hutches will be searched and closed tonight to allow final checks on Tuesday

Beam is now aligned back to HED for hand-over to Naresh (see picture)

points achieved:

-vibometer measurements on M2 at high energy, 2,1 and 3 mrad setting

-tried again to find beam through pblm grating, but no success

Dear all,

we calibrated the mirror vertical motors to their geometrically neutral.

Please be aware that saved values for beamline settings may no longer be valid. This mainly applies to the vertical motor positions, but will have an impact on Tx and Ry values as well.

As starting point, use XRO Optics Setpoints 'C' - this setting was last used to bring beam to SPB

20.4.2024 / 22h51:
DOC informed PRC that the SASE2 topic broker had to be restarted.
The EPS power limit was reset by PRC to the previous value of 40 W.

DOC and DRC are working on the restart. HED is in hardware setup configuration change (not currently taking beam, thus no harm to experiment),
will start taking beam with new users sunday 7am, but would need before this some test runs for the detectors. 

Currently the SA2 topic watchdogs (for ATT, CRL, imagers) cannot be restarted - they crash quickly after restarting.
DOC/DRC are investigating this issue. Until the watchdogs are successfully restarted, extra care must be taken when using ATT, CRL, imagers in SA2.

Update 21.4.2024 / 1h43 by DRC:
In a similar fashion to the incident of last 5.04, the Broker process for the SA2 and LA2 topic has restarted.
This was noticed by the DOC crew and by the HED crew in parallel.
With the help of the CTRL OCD (Micheal Smith) we restarted both topics. As a lesson learned from the last interruption, the POWER_LIMIT
middlelayer device has been set up by the PRC, so that when the DOOCS bridges will be restarted by the DESY-MCS personnel,
the maximum power will not be set to 0.
Please check in due time if all systems are working as intended.

Dear MID colleagues, Svitozar (PRC), Bolko and Shan (RCs),
This morning we faced the situation where MID was limited to 400 pulses (mode M) via the EPS signals. The current RF window set at the machine allows to the requested 450 pulses.
The limiting conditions were the inserted graphite filter (#2) and the diamon screen at the MID Pop-in (#17). After a discussion with experts (and thanking Naresh, Wolfgang and Andreas), both conditions in EPS mode F (see attached) are now disabled, as the current SA2 beam offer safe condition for both devices.
This special situation with disabled EPS signals in mode F is in place until Tuesday 05.Nov as MID will stay at ~12.28keV and with HXRSS (500-600uJ). Please note that in case of either a change in photon energy, substantial increase of photon flux, longer SA RF window or the wish to insert YAG screen at the SA2 Pop-in, these two mentioned conditions have to be enabled again at the SA2 EPS Mode F prior these actions take place.
After the beamtime ends on Tuesday 7AM, either me, Naresh or the PRC will make sure these two EPS conditions are re-enabled.
In case of questions please do not hesitate in contacting me.
Best wishes,
Frederik

**Update Mon 15:00: The beamtime delivery to MID is extended up to Tuesday 9AM. At this moment the EPS signals will be enabled again by Frederik W.-F.

A ZZ is taking place today, 31.01 at 11:30, in XTD9 to verify and fix the M3 Pop-in screen which is currently blocking the beam. It is not yet clear if this is an electrical or mechnical problem.  

** Update 13:05: ZZ intervention is finished and successful. A mechanical issue due to a loose coupling between motor and spindle was fixed. Machine is running again and SPB/SFX can use the beam.

5.61 keV setpoint in SA1 was confirmed to correspond to 5.660 keV (+/- 0.003 keV), see attached.

The user expt #5644 can proceed as planned.

The xray beam drifted significantly on wednesday and thursday (18. and 19.) but then was not drifting on friday (20.), see attached trendline of the M2 actuator that kept the beam at fixed position on M3.

Today also seems that beam pointing is stable. But unclear what happened earlier...

This is to report some findings during operation that should be looked into (mostly to avoid in the future, because both negatively affect data quality of the user expt at FXE):

• At the moment, the 99:1 mode affects 4 to 5 pulses out of 100 in SA1, see first screenshot that shows in the top right the IPM reading with each blue point denoting the average pulse energy in a train. This was not always the case. After initial setup on thursday it did affect only a single train out of 100.
• On 27.10., the switching of SA2 between 1 and multiple bunches affected SA1 beam heavily, see the jumps in SA1 pulse energy in the second screenshot shown here:
  • The pulse energy dropped by 15% and
  • the pointing changed vertically by 150 um on the FXE PI (data not shown, but available from karabo trendlines)

Edit 8.11.: Added screenshot (grafana/ctrend) to show that the vertical displacement was permanent (not corrected by the feedback)

Concerning damage of the SA1 FEL imager, Operation-2024 eLog #206

The screen position of the YAG screen - when pressing the screen selection button - has been moved by 1 mm so that the damaged area is out of the field of view, see attached screenshots (damage visible under visible light exposure).
The screenshots show the cross-hair, i.e. its nominal beam position and the circular damaged area at its original position of the screen and the new position.

More information on the damage can be found in the damage report, entry #35:
https://docs.xfel.eu/share/page/site/xfelwp74/document-details?nodeRef=workspace://SpacesStore/fd862f30-c6a3-4a5e-9a65-817cbcebb2d9
Also, there is a comment in ttfinfo: https://ttfinfo.desy.de/XFELelog/show.jsp?dir=/2024/47&pos=2024-11-19T09:39:31

- M3 TX moved to 0 in steps of 1 mm and beam on FXE Pop-In corrected with M3 RY (+) and M2 RY

- Started calibration with M3 Rz in steps of 0.5 (+) and corrected with M2 RY and RZ to find a position with a minimum of parasitic motion

- Iterations with M3 TY

- CRL 3 and 4 were inserted (using find target)

- The backlash correction was disabled

- Curve for M3 Rz position with Ry correction done. M2 RY/RZ corrections to move beam up and down to have it on the screen

- Curve for M3 Ty position with Ry correction done.

- Curve for M3 Rx position with Ry correction done

M3 vertical motors calibrated to 0

What has happened:
During euler motion calibration of M1, we were driving the vertical motors by a few mm back and forth. At some point, the Rz motor went into error, as we attempted to go further in negative direction. Resetting the axis several times did not solve the problem. The vertical motors were then moved to 0 position. DOC was called and helped to overwrite some parameters that triggered the error.
When resuming the calibration routine, the beam suddenly got lost and the vacuum interlock triggered V0 to close. Vacuum in M1 recovered quickly, but on M2 popIn, only the direct beam could be seen. At this point, the conclusion was that one of the motor connections must have come loose, which caused the vacuum event and removed M1 from the beam.

It was then decided, that we break the interlock, vent the chamber and re-connect the motor if possible.

The situation we found was the following:
-Rz connection lead-screw to bellow was detached
-mirror sitting on the mirror holder
-significant InGa spillage on chamber bottom, window flanges opposite to the mirror AND a lot of drops on the mirror surface

Pictures are attached.

Intervention:
The lead-screw and nut were exchanged. There was significant wear on the bearing plate, connecting the lead-screw to the bellow.
When trying to install the new lead-screw with a spare bearing plate, it was found that apparently the thread on the bellow side was deformed as well, and the spare part did not fit to the thread. Since the old bearing plate was the only possible way to connect the lead-screw to the bellow, it was cleaned, lubricated and re-used. The position was manually adjusted around 0 position (66,7mm protrusion) and the motor connected.

Recommendations:
- since the damaged bearing plate was re-installed, it should be avoided to move Rz. The motor should be permanently interlocked and only operated with absolute care for initial alignment.
- the mirror surface is severely contaminated with InGa, it must be checked if a position with acceptable beam can be found (only to be done by XRO as this needs Rz to be moved)
- the affected bellow has to be exchanged to facilitate replacement of the damaged bearing plate. As this requires the mirror chamber to be opened, this only makes sense in connection with replacement of the mirror + cleaning of the chamber.

We fine tuned geometry parameters for linear effects on X motion (parallel offset) for all three mirrors.

dTz was corrected for M2 and M3 to -6.5mm (same geometry)

dTz was corrected for M1 to +6mm

for M2 we were able to do a parabola scan Y motion to check for lrTz

lrTz was then corrected to -433mm

correction in the same direction for M3 did not yield better results

Final mirror positions at HED - see picture

RC / BKR informs : Beam is down (since 12h10), all beamlines affected

The reason is failure of a quadrupole (power supply) downstream of the dogleg.
The MPS crew is working on recovery, access to XTL ongoing.
RC will inform when new estimates come in on when we can restart beam operation.

Currently affected on Photon side:
- SA1: Naresh (XPD) and Marco (DET) working on Gotthard-II commissioning. Naresh is present at BKR.
- SA2/SA3: Antje + Mikako (XRO) working on SA2 mirror flipping tests and SA3 exit slit check. Both are located at SXP control hutch.
Other XRO members will come in for the afternoon/evening shift work. Antje will inform them via chat.

At 21:29:20 today, the number of bunches in SASE1 was increased from 1 to 352 without intervention of the SPB/SFX shift crew. 30 seconds later, the number of bunches was reduced to 1, again without intervention from the SPB/SFX shift crew (see attachments 1 and 2).

We were in the middle of a measurement at that time and the sudden increase from 10 mW to 3.5 W average power resulted in the destruction of the sample we were using.

AGIPD and JUNGFRAU are safe and were not at risk at the time of the incident, but this is merely by luck.

The origin of the change in number of bunches is unclear to us and needs to be investigated. It goes without saying that this is extremely dangerous for equipment and the damages could have been even worse.

Like after the previous winter maintenance period, the shutter of FXE appears as opened in DOOCS panels even though it is closed (https://in.xfel.eu/elog/OPERATION-2023/58).

The cross on the FEL imager, which we use as a reference for BKR crew, is not centred on the GATT aperture. when was this changed, and why?

att. 1 and 2 show the current position (the saturated image aids the positioning of the GATT aperture).

att. 3 is from two weeks ago. then, the cross was centred on the GATT aperture.

The status of the XTD2 attenuators has been checked during the experiment last week and many were found to cause disturbances in the wavefront ranging from mild to severe.

Attachment 1: No attenuator
Attachment 2: 75 um CVD - Fringes visible in beam
Attachment 3: 150 um CVD - Strong fringes in beam
Attachment 4: 300 um CVD - Looks more or less ok
Attachment 5: 600 um CVD - Weak distortion in the beam profile
Attachment 6: 1200 um CVD - Ok
Attachment 7: 2400 um CVD - Ok
Attachment 8: 500 um Si - Not ok