Group Selection Page HELP Controls Group Data Analysis Group ITDM WP76 DET Vacuum EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Operation PSPO XO TS Migrated2Zulip
OPERATION-2025 OPERATION-2024 OPERATION-2023 OPERATION-2022 OPERATION-2021 OPERATION-2020 OPERATION-2019 SASE1-2018 SASE1-2017 Experiment data
  OPERATION-2023 logbook, Contact: F.Wolff-Fabris, A.Galler, L.Samoylova, H.Sinn, Page 20 of 24  Not logged in ELOG logo
ID Date Author Group Subjectdown
  117   31 Mar 2023, 05:10 Benjamin, Yi-Ping, Ru-Pan, AndreasSCSLate Shift Summary

Achievements

  • Optimized BOZ for Co and Fe
  • Estabilished Stable liquid jet flow with 30 mM Fe(bpy)3

Problems

  • Jet froze several times
  • Chamber warmed up earlier in the day. This led to a "dusting" of CNCbl on many surfaces and in the large turbo. We cleaned the turbo with ethanol.

Plan for Night Shift

  • Continue measuring stitic XAS with BOZ scheme.
  120   02 Apr 2023, 04:46 Benjamin van KuikenSCSLate Shift Summary

Achievement

  • Measured tXAS at the Fe L-edge of Fe(bpy)3 using both delay and energy scans. 

Issues

  • The cold traps ran empty on the chamber resulting in a pressure spike that interlocked the DSSC
  • The rough vacuum of the chem DPS degraded causing TP5 to go into error. This resulted in valve closing and stopping the measurements
  • Jet froze twice
  441   07 Nov 2023, 15:57 Janni Eidam KW44 - Leak in Solid Attenuator

There was a leak in the Solid Attenuator that increased on Saturday morning when SPB moved the arm number 4 (see last entry ID 236).

We were lucky that a balance developed between the leak and the pumping of the ion pumps, so by setting the setpoints of the three most affected ionpumps to 5x10-5 mbar we could survive the weekend and save the beamtime. 

On Monday Maik and I then entered the tunnel to fix the leak. The need of breaking the interlock to bring a dewar and a pumpcart was already discussed on Saturday/Sunday with the PRC (Jan Grünert) and BKR. So Maik and I could enter on Monday afternoon at around 13:20. We found out that the bellow of the arm number 4 is the leaky one which we removed then and put a blind flange. So the XTD 2 solid attenuator has two removed arms now (see photo).

After about half an hour of pumping both ion pumps on the Solid Attenuator were already in the good 10-6 mbar range. We also set back the setpoint of the three pumps P10160I, P10170I and P10180I to the default value 5x10-6 mbar and put the bridge for the end switch on the connector (see picture). The whole work including cleanup in the tunnel could be finished around 19:15 and the tunnel search crew could then set the interlock back.

So the acute problem with the leak could be solved for first now, but there was no time for the activation of the NEC pump. This should be kept in mind for the next shutdown, when we hopefully have spare parts and can anyway replace the arms number 4 and 5. 

We packed the removed actuator and diamond plate in foil and they are next to the radiation protection release box now and waiting for release (see photo). Email was sent to radiation protection officer.

Thanks to Maik for help and a nice working time yesterday and to Michaela for helping us to bring all the things to the XHE3 shaft building! Thanks also to Raul for talking to BKR as PRC and arranging the final tunnel access appointment!

Attachment 1: 20231106_180901.jpg
20231106_180901.jpg
Attachment 2: 20231106_175457.jpg
20231106_175457.jpg
Attachment 3: 20231106_175612.jpg
20231106_175612.jpg
Attachment 4: 20231106_185102_(1).jpg
20231106_185102_(1).jpg
  19   29 Oct 2022, 11:03 Jaydeep Patel Jet alignment test- obeserver

Nozzle type

Nozzle Aperture(µm)

Sample

Jet Speed(m/s)

He flow rate

Liquid flow rate

Repetition rate

 

      Beam Attenuation

(%)

 

Run No.

GDVN

B

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

75

Water

 

25

10

80

0.5

122

30

13

60

0.5

9

123

40

20

30

0.5

23

124

45

25

30

0.5

23

125

50

32

30

0.5

23

126

Buffer 1

 

25

10

80

0.5

31

141

30

14

60

0.5

31

140

40

20

30

0.5

31

139

45

26

30

0.5

31

138

50

32

30

0.5

9

137

Buffer 2

 

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Crystal size 1µm

25

10

80

0.5

16

131

30

13

60

0.5

16

130

40

20

30

0.5

16

129

45

25

30

0.5

16

128

50

32

30

0.5

16

127

Crystal size 2-4µm

25

10

80

0.5

12

132

30

13

60

0.5

9

133

40

20

30

0.5

9

134

45

25

30

0.5

9

135

50

32

30

0.5

9

136

Crystal size 5 µm

25

10

80

0.5

9

142

30

13

60

0.5

9

143

40

20

30

0.5

31

144

45

25

30

0.5

29

145

50

32

30

0.5

29

146

DFFN

JKMH8.11

 

8.11=75-70-70

 

 

 

 

 

 

 

 

 

 

75

water

25

13

80

0.5

 

 

30

13

40

0.5

 

 

40

20

20

0.5

 

 

45

25

15

0.5

 

 

50

32

15

0.5

 

 

Buffer1

 

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Buffer2

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Crystal size 1 µm

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Crystal size 2-4 µm

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Crystal size 5 µm

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

GDVN

C/C4

 

C=100-90-100

C4=100-75-100

 

 

 

 

 

 

 

100

Water

26

15

80

0.5

58%

172

30

 20

 60

0.5

58%

171

40

35

25

0.5

58%

170

45

37

 25

0.5

58%

174

50

 40

18

0.5

58%

175

Buffer 1

26

 

 

 

 

180

30

 

 

 

 

179

40

 

 

 

 

178

45

 

 

 

 

177

50

 

 

 

 

176

Buffer 2

25

 

 

 

 

 

30

 

 

 

 

 

40

 

 

 

 

 

45

 

 

 

 

 

50

 

 

 

 

 

Crystal size 1 µm

25

 

 

 

31%

181

30

 

 

 

31%

182

40

 

 

 

31%

183

45

 

 

 

31%

184

50

 

 

 

31%

 

185

Crystal size 2-4 µm

25

 

 

 

31%

187

30

 

 

 

31%

188

40

 

 

 

31%

189

45

X

X

X

X

X

50

X

X

X

X

X

Crystal size 5 µm

25

20

80

0.5

16/ 31%

147/190c4

30

30

60

0.5

31%

192

40

40

25

0.5

31%

193

45

40

15

0.5

 

 

50

40

10

0.5

 

 

  3   16 Jan 2023, 19:50 Jan GruenertPRCIssue

Problem: Beam operation in SASE2 is prohibited by SEPS / MPS
Origin of problem: HED-CRL3
(this was already an issue during the past weekend)

HED (Karen) today realigned CRL3 and brought the lenses into the safe OUT state, see entry https://in.xfel.eu/elog/OPERATION-2023/2
In karabo then everything seemed find, SEPS switches all green / OUT, see ATT#1.
Nevertheless, in DOOCS the signal safety_HED_CRL_OPT was still red, see ATT#2.

The MPS safety_HED_CRL_OPT signal from the Beckhoff HED PLC was missing.
Onsite investigation by EEE (Leandro and Bernard) revealed that the Beckhoff crate BKF000476 was offline (switched off).
Switching the crate ON fixed the missing signal problem.

Presumably crate BKF00476 was not powered back on since after the planned power-cut in December 2022,
and because it is not part of PLC EtherCAT loop, the Nagios system didn't inform EEE about the crate status.

Result: Problem solved, beam operation in SASE2 can resume normally.

Attachment 1: JG_2023-01-16_um_16.30.12.png
JG_2023-01-16_um_16.30.12.png
Attachment 2: JG_2023-01-16_um_16.30.50.png
JG_2023-01-16_um_16.30.50.png
  4   17 Jan 2023, 10:25 Jan GruenertPRCIssue
SASE3 peration issue - SOLVED
 
Controls issue, the machine cannot make beam in SASE3.
From BKR / DOOCS the problem description is: Big Brother is blocking SASE3 beam due to an EPS power limit of 0 Watt.

Looking into the SASE3 karabo topic, many devices are not instantiated, ATT #1.
The state of the PREABS  preabsorber is not known and it is not found in the device topology either.
 
SA3_XTD10_PREABS/DCTRL/OPEN.off
SA3_XTD10_PREABS/SWITCH/OUT_BEAM_1.state
SA3_XTD10_PREABS/DCTRL/LED_POWER
SA3_XTD10_PREABS/DCTRL/OPEN.state
 
Info to DRC and PRCs.
Two actions resolved the issue for now:
- XO changed in karabo the EPS power limits to the values used before the WMP (40 W).
- PRC and VAC instantiated the EPS-loop PLC devices
 
If this fully resolves the problem will be known when the machine will try to make beam again (now switching to hybrid 10 Hz / 5 Hz mode).
 
Another small problem at BKR is that the 3 big wall monitors are now working (ticket to ITDM issued).
Attachment 1: JG_SASE3_no_beam_permission_2023-01-17_09-13-31.png
JG_SASE3_no_beam_permission_2023-01-17_09-13-31.png
  6   22 Jan 2023, 18:40 Jan GruenertPRCIssue

SASE1 blocked to 1 bunch/train by EPS interlock (sic, not 2 but 1) --- PARTIALLY RESOLVED

Several issues:

1) MODEM device is broken, in state UNKNOWN, see ATT#1 and ATT#2. Fixed with deinstatiating and then using BECKHOFF_ASSISTANT to re-instantiate. --> ATT#3

2) Several devices in XTD9 are causing that MODES is interlocked: I restart broken devices of the FXE-PBLM and SPB-PBLM.
Then there is a broken MDL of SPB_XTD9_SCR called SPB_XTD9_SCR/MDL/BIU. This MDL is not in the SA1-topic system topology but shown with (X) on the SA1_MAIN scene, and this screen is blocking the interlock (see ATT#4)

SPB_XTD9_SCR/VSWITCH/HOLSTER_X_LIMITL is triggering the interlock SA1_SYS_EPS2MPS/DCTRL/MODES and causing that the machine cannot go to more than 2 bunches per train.
Even when going to SPB-topic this issue cannot be solved - all devices in SPB-topic related to this SCR seem fine.

3) To allow SASE1 tuning by the machine, BKR is closing shutter XS3. This avoids any beam into XTD9, resolved the interlock problem and allows the machine to continue SASE tuning.

4) We don't know what had blocked the beam not only to 2 but actually to 1 bunch / train (no traces of problems in Big Brother).

 

Attachment 1: JG_2023-01-22_um_17.43.36.png
JG_2023-01-22_um_17.43.36.png
Attachment 2: JG_2023-01-22_um_18.17.31.png
JG_2023-01-22_um_18.17.31.png
Attachment 3: JG_2023-01-22_um_18.23.20.png
JG_2023-01-22_um_18.23.20.png
Attachment 4: JG_2023-01-22_um_18_1.37.21.png
JG_2023-01-22_um_18_1.37.21.png
  13   02 Feb 2023, 06:37 Chan KimSPBIssue

SPB_IRU_DPA/MOTOR issue

X stage looks okay, but Y, AX, and AY values have been changed by 2-3mm. (see attachment 1 and 2)

 

SPB_IRU_SLIT-1/MOTOR issue

All 4 blades (top, bottom, left, and right) have been changed by 0.1-0.5mm. (see attachment 3)

 

SPB_IRU_SIDEMIC/MOTOR and SPB_IRU_INLINEMIC/MOTOR issue

Not fully checked, but Y & Z motors of SideMic and AX & AY motors of InlineMic have been changed by ~1mm and ~0.4 karabo unit. (see attachment 4 and 5)

Attachment 1: Screenshot_from_2023-02-02_06-09-52.png
Screenshot_from_2023-02-02_06-09-52.png
Attachment 2: Screenshot_from_2023-02-02_06-11-30.png
Screenshot_from_2023-02-02_06-11-30.png
Attachment 3: Screenshot_from_2023-02-02_06-17-24.png
Screenshot_from_2023-02-02_06-17-24.png
Attachment 4: Screenshot_from_2023-02-02_06-30-00.png
Screenshot_from_2023-02-02_06-30-00.png
Attachment 5: Screenshot_from_2023-02-02_06-31-44.png
Screenshot_from_2023-02-02_06-31-44.png
  17   05 Feb 2023, 15:18 Harald SinnPRCIssue

15:15  SCS called and reported an issue with one of the motors of the SCS exit slit. DOC already looked at the problem and could not make the motor move. XRO (Maurizio) is now informed and looks at the problem

17:10 ZZ access to XTD by Maurzio and Mikako

18:00 ZZ finished. Unfortunatly the motor is completely blocked, even with cables disconnected one cannot move it. 

  30   13 Feb 2023, 19:02 Frederik Wolff-FabrisPRCIssue

SA3 SCS exit slit: the upper blade was detached from the motor and fall into closed position. Maurizio V. and F. Wolff-Fabris tried to move it out of the beam by placing spacers to hold the blade up.

Bottom blade was moved down with the motors. Currently both motors are disconnected.

*** Update 14.02 10AM:

This morning the exit slit was clipping the beam. Another ZZ by M. Dommach, M. Petrich and F. Wolff-Fabris was done to move up the top blade by a further 8mm. 

After that intervention SCS confirmed to see the beam at the Pop-in and at the experimental hutch.

  58   28 Feb 2023, 07:17 Romain LetrunSPBIssue

In the public status display, it looks like FXE is taking the beam even though their shutter is closed.

Attachment 1: Screenshot_at_2023-02-28_06-38-30.png
Screenshot_at_2023-02-28_06-38-30.png
  78   14 Mar 2023, 10:56 Harald SinnPRCIssue

The longitudinal feedback on SA1/3 was swicthed off by mistake over night. When it will be switched on, the beam may wiggle a bit an SA1+SA3 for short. 

We agreed with SCS and FXE that this will be done today at 11:30 

 

  79   14 Mar 2023, 14:40 Harald SinnPRCIssue

In order to optimise the performance of SASE2 self seeding, a stronger compression setting is required. 

The RC proposed to increase the time window between SASE2 and SASE1/3 to avoid performance degradation and cross talk effects on SASE1/3. 

I discussed with SCS and FXE instruments and there seems to be nothing against moving the time trigger by 30 micro seconds later, as proposed by BKR. 

The agreed bunch train lengths can be still achieved with the new setting. The change will be implemented now (Tue 14:45)

  80   14 Mar 2023, 20:26 Harald SinnPRCIssue

SCS could only use two pulses, because the SXP imager was still in the beam and SXP shutter open. 

I removed the SXP imager, so SCS can now get the full beam. The SXP shutter is still open and SXP instrument contacted via email to find out, if the shutter is intentionally open. 

21:13: SXP confirmed that they will close the shutter tomorrow. The imager was just forgotten in the beam. 

  99   23 Mar 2023, 11:38 Romain LetrunSPBIssue

When using 1.13 MHz X-ray repetition rate, SASE1 HIREX is capturing two pulses in a single frame giving the impression that there are more peaks within the spectrum than there are in reality.

This was verified by recording data with one and  two pulses per train, which are shown in attachment.

The spectral integral/intensity in GOTTHARD frame 6, the first with data, is twice larger with two pulses compared to one pulse per train (attachments 1 and 2).

The number of peaks is consistently higher with two pulses compared to one pulse (attachment 3).

Attachment 1: 20230323_HIREX_integral.png
20230323_HIREX_integral.png
Attachment 2: 20230323_HIREX_mean_spectrum.png
20230323_HIREX_mean_spectrum.png
Attachment 3: 20230323_HIREX_peaks.png
20230323_HIREX_peaks.png
  111   28 Mar 2023, 15:26 Angel Rodriguez FernandezMIDIssue

The door from MID side entrance was smashed and we lost the radiation permit.

The acciedent happened between 11 (when the door was lest use by MID members) and 14:19 (when the picture was taken).

It is possible that we will have the permit in the next hours to countinue.

The Door was repared by TS (thank you for the fast answer) and at 15:40 Eric gave us the right to proceed after the crosscheck of all the systems.

Attachment 1: IMG_8399.JPG
IMG_8399.JPG
  123   04 Apr 2023, 08:37 Frederik Wolff-FabrisPRCIssue

TUE 08:30AM:

The SA3 Cell 17 & 18 Phase shifters do not move and encoders are damaged. A ZZ is needed to exchange the encoders. Instruments are informed and access will take place Tuesday morning. Machine has separated the north branch.

*** Update 10:20AM:

ZZ is finished; beam in North Branch to restart soon.

 

  127   07 Apr 2023, 23:57 Yifeng JiangFXEIssue

After receiving beam from SPB, we noticed that there is a werid shape of beam in the FEL imager which was not observed yesterday.

Maybe someone left X-ray component in the beam.

Attachment 1: 2023-04-07-235513_774809068_exflqr51474.png
2023-04-07-235513_774809068_exflqr51474.png
  128   08 Apr 2023, 10:29 Andreas GallerPRCIssue

Changed EPS Interlock condition for SPB_XTD9_IMGPI/SWITCH/SCREEN_OUT

SPB/SFX called DOC this morning for being limitied to two pulses by the EPS (Instrument should have called PRC instead)
After tracing the issue to the SPB_XTD9_IMGPI/SWITCH/SCREEN_OUT switch SPB/SFX decided to take this imager out of the interlock, confirming on camera that no screen was inserted.
EEE and DOC investigated the issue without conclusion https://redmine.xfel.eu/issues/151238
 

SPB_XTD9_IMGPI/SWITCH/SCREEN_OUT disabled in EPS Mode30 and Mode400
for todays SPB/SFX shift we'll keep it like this, PRC will check with FXE when they take over at 23:00

  130   08 Apr 2023, 23:44 Yifeng JiangFXEIssue

We notice that 0.5 mm Silicon ATT in SASE1 causes holo beam profile in FEL imager. Maybe this silicon att has been damaged.

Attachment 1: 2023-04-08-234240_066894160_exflqr51474.png
2023-04-08-234240_066894160_exflqr51474.png
Attachment 2: 2023-04-08-234217_329678202_exflqr51474.png
2023-04-08-234217_329678202_exflqr51474.png
ELOG V3.1.4-7c3fd00