Instability observed today at 14:05, reported to PRC and BKR, including RC: We saw RMS jitter amplitude of up to 4 mm at FXE imager. The jitter was also seen by the upstream XGM with 100 um RMS amplitude. So it is something from the source.

This instability is suddenly appearind/disappearing and was absent over night and started this morning at 7h to occasionally turn on. It appear approx. everz ~30 mins for a few minutes.

Connerning the phase shifter adjustment after undulator gap scan via karabo:

At 19:30 FXE tested the scan change via Karabo with BKR. When the energy was changed with step size of 100 eV, the intensity of X-ray stays unchanged. This means the phase shifter adjusment is corrrected and it is working now.

Many thanks for your help.

FXE interlock in fault state. See photo attached with original error message from 6:53h. It cannot be reset by acknowledging. BKR is informed and they are contacting MPS OCD.

Summary for sept., 23rd

6:55 SPB just handed the beam over to FXE when the power glitch killed the interlocks and brought down ACC

11:30 After a lot of communication with PRC, Wolfgang Clement and DOC, the beam is back in SA1. We request change to 5.5 keV

• It took most time to get the AIBS recovered
• Interlock was reset by Wolfgang Clement on-site
• Pandora X-05 in SXP hutch had no power, but recovered after TS restored power in SA3

12:30 We finished alignment and collected some user data (XRD)

16:00 Failure of the optical laser shutter in ILH. Seems not operational.

• Adhoc procedure is being established to circumvent the faulty laser shutter (coordinated with SLSO, LSO, GL and Management)

18:00 FXE is able to take user data again (XAS)

23:30 Beam handed over to SPB (thanks for the extra time to finish a promising measurement)

7:00 Power glitch during SPB operation - they were supposed to hand the beam back at 7:30h

• Power glitch brought down FXE interlock system with beam stop monitor alarm that could not be reset
• SA1 PPL running fine, seems unaffected

9:10 Beam back in SA1, but still at 7 keV and cannot be changed by BKR to the req'd 5.8 keV, 94 kHz.

10:00 Beam back with FXE at req'd photon energy

Collected pump-probe diffraction and XAS data for the user expt. No further issues.

• Received the beam from day shift, aligned to beam stop (low signal due to 4m air scatter, only 700uJ upstream)
  • ZYLA5 camera was having issues, restarted without rotating to limit dropped trains.
  • 1.2mm CVD solid attenuator in SA2 seems to be damaged
• We have trouble with our SEPS interlock after changes last week
  • We are supposed to be able to close the shutter (upstream of CRL3) without removing CRL3 to break hutch interlock, but it still sets bunches to zero. This was raised with DOC, and directly with Mikako who implemented this last week
• We aligned our beam through HED's CRL3
• Establish overlap with Relax laser, and timing on YAG (RF-lock only)
  • Issues with new camera ROI's being too large limiting readout to 3.3Hz
• Start cryojet at 22:00, see if we can squeeze in some laser shots to get signal levels tonight
• Alignment/transmission scans with XFEL planned overnight.

Shift summary from Monday 18th 7am until Saturday 7am

MONDAY

• Aligned beamline until the beamstop. Macro for recovering mirror positions worked well. Minimal fine tuning required.
• Changed energy to 8.4 keV to align spectrometes on Ni kalpha emission.
• Characterized focal spot of the x-rays at interaction point via edge scans. Focus of ~4 um as requested by users.
• Aligned all laser imaging systems to the interaction point

TUESDAY

• Verified the xfel photon energy with the HED spectrometers. The energy was off by 30 eV. Tuned the energy for real 7.9 keV, in the BKR screen appears as 7.87 keV
• Beamline transmission scans. Focusing CRLs in the optics hutch give 55% transmission. This means from the 700 uJ from XGM1 we get around 250uJ at the interaction point (accounting also for ~75% transmission through the mirrors)
• First ReLaX + X-ray test shots. Fine tuned some timings, filtering on the Streaked Optical Pyrometry, etc...
• Cancelled night shift, due to no X-rays on Wednesday. Usually at this point we would align targets over night and shoot them during the day.

THURSDAY

• Beam was not exactly at the same position at the XFEL imager as Monday. We needed to recheck the beamline alignment and overlap of all our beams at the interaction point. (We need to overlap in a 5x5 um2 beamspot.
• Rechecked the X-ray energy. This time it was 110 eV off. The BKR number is 7.79 keV for real 7.9 keV.
• Further checkes on spatial and timing overlap.
• Essentially, we could recover the conditions from Tuesday evening at Thursday late afternoon.
• Late evening: uTCA1 stopped working We switched the Mastertimer to uTCA2 waiting for an opportunity on Friday to solve it. We also observed an issue with an outdated trainId from the PPU being stored in the DAQ. We contacted DOC for both issues.
• Late evening, series of laser shots to estimate the optimal laser conditions for the experiment.
• First full data sequence collection.

FRIDAY

• We are in data collection mode. Aligning targets and shooting them.

SATURDAY

• Power glitch at 6:50 am on Saturday morning. HED is minimally affected, beam goes down
• Beam comes back after ~1hr, but down again 30 mins later due to MPS issue (only for SASE2) which requires a server "DIXHEXPS1.3" to be reset, which could not be located.
• Simultaneously SASE2 karabo broker has pile up issues resulting in extreme latency. DOC begins to work on the problem and the whole SASE 2 topic goes down, services do not reappear. We are informed that this may require a restart of the whole broker.

achievements:
- Viking spectrometer set up in 8th order around 1280eV and calibrated

problems:
- problems with Viking motors. Took couple of hours to solve, Martin came on site
- problems with moving exit slit (Blade_Bottom motor does not move). Not solved by EEE on call, may be mechanical problems. Called run coordinator, XRO on call (Maurizio) is going to inspect the slit in the tunnel today. Operation of SCS relies on exit slit available (now it opened to ca. 10mm)

Around 6:55 there was a power glitch. Power in the experimental hutch and rack room went down in SASE3, error message in Xray interlock system and no power for PP laser.

During morning TS turned the power on and interlock issues were solved by DESY. SCS could restart large part of SCS systems, however got help of DOC only after 2:30p.m. Similarily, PP laser was needed help of DOC and got this help only around 2p.m. The afternoon shift was mainly bringing the instrument in operation with helps of DOC. Around 9p.m. we realized that PP laser is not in operational state because of large fluctuations, this was not communicated to SCS at all. After calling the Laser group, SCS was told that the laser group will work on recover of the laser on Sunday morning. Without laser we could not continue user program, therefore  at least 24h of user operation were lost.

Update for elog 99:

Now Gotthard timing is changed to new setting for 0.5 MHz.

attached screenshot with gothard timing setting

Machine changed from 0.5 MHz to 1.1MHz new screenshot attached2

Reply to elog103:

Spoke with Machine RC Thomas Wamsat and SQS.

Machine RC has some ideas to try. SQS have agreed that they can give 30 min for the machine to optimize the intensity.

SA1 has no impact while the machine trying to improve the power for SA3.

Update at 21.17hr

Machine RC has optimized for higher pulse intensity for SQS from 900uJ to 1600uJ. But still, the problem reported in elog103 exists, However, SQS can work now and overnight.

Thanks to Thomas Wamsat for his effort to support.

Machine RC Raimond informed PRC that there is an issue with the A4 module that needs to fix to have a continuous operation. To fix this they need 20 min.

Agreed with all instruments (SPB, SQS, and HED) that at 13:30hr machine will take over the beam for 20 min.

Update: 13:55hr

Machine finished fixing the issue for the A4 module and beam handover to all beamlines.

Issue with SA1_XTD9_PBLM and it is stuck.

Hi All,

After discussion with SPB, they will slightly move the PBLM screen where they don't see any distortion in the beam and are willing to continue to work till Monday. 

The conclusion from EEE (Leandro) and DOC is that from PLC everything looks fine and it might be with mechanics for this we need to physically look at the device. 

As SPB can work like this till Monday. I proposed on Tuesday with someone who has knowledge of PBLM device can organize the ZZ

Next week Jan is PRC

Dear FXE colleagues,

There is an issue with SA1_XTD9_PBLM and it is stuck. Please when you use the beam in the night, coordinate with SPB first. or call PRC no. 9-4800

- Received a call from machine RC (Sergey Tomin)

- There is an issue with DOOCS to Karabo bridge, Tim and DESY experts are looking into this and came to the conclusion that it can't be resolved now and need more time something like 1 week. (probably next week). Due to this PySpectrometer for SA3 is not available, as SQS would like to set up a spectrometer to measure pulse duration. However, it can work in Karabo and take data.

FXE (Fred) confirmed that they don't use beam for night shifts from Friday to Sunday.

HED reported that they are unable to go 352 pulses and while changing/dailing the no. of pulses in the bunch Pattern it is showing the EPS is limited to 300 pulses. It seems on the SA2_MAIN scene the link for bunch Pattern apparently using the macros scene, which is preventing it from going to 352 pulses. Now I have updated the link with a new scene of MDL for the bunch pattern. In the attachment, you will find both scenes. One toward the left is MDL bunchPattern configurator and one towards the right is Macros bunchPattern. 

Now the issue is solved. 

thanks

Naresh

PRC wk 22

- SA2 photon beamline

Today morning found the CRL1: 5 and 6 arm and Pop-in 45-I are still IN. I have retracted them.

SA2 photon beamline

- SRA position were 7 mm x 7 mm, MID reported they see the hallow of beam shadow at MID Popin

- I have closed the SRA to 3.5 mm x 3.5 mm

SA1 photon beam line status

- CRL1 lenses are in the beam. 

- The shutter between XTD2 and XTD9 is closed (The reason is not known)

I will retract the CRL1 lenses. Asked the machine to open the XTD2 and XTD9 shutter.

SA1 photon beam line status

- CRL1 lenses are in the beam. 

- The shutter between XTD2 and XTD9 is closed (The reason is not known)

I will retract the CRL1 lenses. Asked the machine to open the XTD2 and XTD9 shutter.