Group Selection Page HELP DOC Controls Group Data Analysis Group ITDM WP76 DET Vacuum XRO EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Photon Commissioning Team Operation PSPO XO TS
OPERATION-2024 OPERATION-2023 OPERATION-2022 OPERATION-2021 OPERATION-2020 OPERATION-2019 SASE1-2018 SASE1-2017 Experiment data
  OPERATION-2024 logbook, Contact: F.Wolff-Fabris, A.Galler, H.Sinn, Page 2 of 11  Not logged in ELOG logo
ID Date Author Groupdown Subject
  99   10 May 2024, 00:04 Rebecca BollSQSIssue

our pp laser experienced timing jumps by 18 ns today. this happened for the first time at ~7:30 in the morning without us noticing and a couple of more times during the morning. we realized it at 15:30 due to the fact that the signal on our pulse arrival monitor (X-ray/laser cross correlator downstream of the experiment) disappeared. we spent some time to investigate it, until we also see this jump in two different photodiodes different triggers (oscilloscope and digitizer) so it was clear that it is indeed the laser timing jumping. we also see the laser spectrum and intensity change for the trains that are offset in timing.

we called laser OCD at ~16:30. they have been trying to fix the problem by readjusting the instability zone multiple times and by adjusting the pockels cell, but this did not change the timing jumps. at 21:30 they concluded that there was nothing more they could do tonight and a meeting should happen in the morning to discuss how one can proceed.

effectively, this means that we lost the entire late and night shifts today for the pump-probe beamtime and have some data of questionable quality from the early shift as well.

  107   17 May 2024, 10:42 Dirk RaiserSQSStatus

Laser status:

After the laser issues listed in https://in.xfel.eu/elog/OPERATION-2024/99 the late shift was finally able to find a timing setting for the amplifier trigger that was beyond stable settings under usual conditions, but which allowed to continue taking data runs for the rest of the beamtime reliably. The users were already able to reveal indications of a pump-probe signal in their data.

The issue itself remains but didn't show up in form of unstabilities anymore. After the beamtime LAS took some effort to investigate and narrow down to either a faulty trigger going into the front-end amplifier or on the front-end amplifier itself. To the best of our knowledge this circumstance will need to involve EEE support or even a service from the manufacturer and cannot be solved solely by LAS. LAS ensured that the highest priority will be to deliver stable laser pulses for next week's beamtime as well.

FEL status:

we found the FEL poynting to be very stable troughout the last weeks beamtime (despite some short down-times), which allowed to reliably verify the spatial overlap.

  110   25 May 2024, 02:38 Rebecca BollSQSIssue

Today in the afternoon, SASE1 was tuned (a description of the activities in att 4). We now realize that this has in some form changed the SQS pulse parameters.

att 1 shows ion spectra recorded under nominally the same photon parameters before (run 177) and after (run 247)

it's also visible in the history of the SASE viewer that the behavior of the SASE3 pulse energy is different before and after 15:45, which is right when SASE1 was tuned. it clearly started fluctuating more. however, we don't think that the pulse energy fluctuations themselves are the reason for the large change in our data, we rather suspect a change in the pulse duration and/or the source point to cause this. however, we have no means to characterize this now after the fact.

it is particularly unfortunate that the tuning happened exactly BEFORE we started recording a set of reference spectra on the SASE3 spectrometer, which were supposed to serve as a calibration for the spectral corelation analysis to determine the group duration, as well as for a training data set to use the PES with machine learning as a virtual spectrometer for the large data set taken in the shifts before.

Attachment 1: Run_177_vs_247.png
Run_177_vs_247.png
Attachment 2: Screenshot_from_2024-05-25_00-44-46.png
Screenshot_from_2024-05-25_00-44-46.png
Attachment 3: Screenshot_from_2024-05-25_01-03-29.png
Screenshot_from_2024-05-25_01-03-29.png
Attachment 4: Screenshot_from_2024-05-25_01-36-54.png
Screenshot_from_2024-05-25_01-36-54.png
  112   26 May 2024, 21:56 Thomas BaumannSQSStability Issue

We asked the machine for two color mode in the morning. Tuning gave good results at first (1.6mJ second color, 100uJ first color), but as soon as any of the colors was switched off and on again the pulse energy dropped (~600uJ) and BKR had to retune to even get a fraction of the pulse energy back back. The same happened when changing the delay

 

  5   31 Jan 2024, 23:26 Chan KimSPBShift summary

31.01.2024 (Day shift)

X-ray delivery

  • 10 keV
  • 1200 - 400 uJ (significant pulse energy slope within train)

Optical laser delivery

  • None

Achievements/Observations

  • Flat fields and timing scans were measured for AGIPD (1.1MHz, normal CDS mode)

Issues

To do for next shift

  • Continue with the detector program
  7   01 Feb 2024, 21:56 Romain LetrunSPBIssue

At 21:29:20 today, the number of bunches in SASE1 was increased from 1 to 352 without intervention of the SPB/SFX shift crew. 30 seconds later, the number of bunches was reduced to 1, again without intervention from the SPB/SFX shift crew (see attachments 1 and 2).

We were in the middle of a measurement at that time and the sudden increase from 10 mW to 3.5 W average power resulted in the destruction of the sample we were using.

AGIPD and JUNGFRAU are safe and were not at risk at the time of the incident, but this is merely by luck.

The origin of the change in number of bunches is unclear to us and needs to be investigated. It goes without saying that this is extremely dangerous for equipment and the damages could have been even worse.

Attachment 1: Screenshot_from_2024-02-01_21-52-15.png
Screenshot_from_2024-02-01_21-52-15.png
Attachment 2: Screenshot_from_2024-02-01_21-31-53.png
Screenshot_from_2024-02-01_21-31-53.png
  8   02 Feb 2024, 00:04 Romain LetrunSPBShift summary

X-ray delivery

  • 1-352 pulses at 10 keV, 1200 - 400 uJ.
  • Significant pulse energy slope within the train. Action on the kickers was eventually performed leading to some reduction of the slope, though remaining still far from ideal. Decision to take the action took a very long time (2.5 hours).
  • Several beam delivery interruptions through the afternoon due to trips of A16 before is was taken off beam.
  • At 21:29:20, the number of bunches in SASE1 was changed from 1 to 352 without intervention of the SPB/SFX shift crew resulting in the destruction of a sample (see elog:7).

Optical laser delivery

  • 800 nm, 50 fs, 1-113 pulses at 564 kHz.
  • Brief intensity and spectral fluctuations as already observed in the past with non-optimal AMPHOS regulation. Detail will be communicated to LAS.

Achievements/Observations

  • Measured flat fields for AGIPD for all remaining configurations.
  • Confirmed timing of JUNGFRAU.
  • Established pump-probe timing at IRDa.

Issues

  • Several issues with AGIPD controls: timeout when trying to stop sending data, two hemispheres in different states, etc. Experts informed.
  • Destruction of the sample following number of pulses incident in the evening caused a considerable loss of time to replace the sample and resume operation.
  • XTD2 500 um Si is damaged (leftover from last run).
  10   02 Feb 2024, 23:31 Romain LetrunSPBShift summary

X-ray delivery

  • 1-64 pulses at 10 keV, ~1.1 mJ.
  • Overall stable delivery.

Optical laser delivery

  • NA

Achievements/Observations

  • Measured flat fields for JUNGFRAU for all configurations. Detector program completed.

Issues

  • Mishap of lit frame finder with custom bunch patter.
  • Apparent issue to process JUNGFRAU darks caused by lack of disk space. Solved by DOC/CAL.
  • Ghost run in myMdC upsetting the run controller. Solved by DOC and cause under further investigation.
  16   08 Feb 2024, 23:14 Tokushi SatoSPBShift summary

X-ray delivery

  • 1-350 at 12.45 keV, 1700 uJ. Stable delivery

Optical laser delivery

  • None

Achievements/Observations

  • fixing many issues with the cube and helium leaks
  • collected many quality diffraction data for both jet with and without samples
  • data on Nendu mutant (~2.5 hours of data collection) with two different conditions

Issues

  • AGIPD data aggregator stopped responding and it took about 1 h to be fixed by DOC, which interrupted the data collection

  • some strange streak diffraction appears on JF4M and this has to be removed by scanning X- and Y- nozzle motors carefully
  18   10 Feb 2024, 00:12 Tokushi SatoSPBShift summary

X-ray delivery

  • 1-350 at 12.45 keV, 1700 uJ. Stable delivery

Optical laser delivery

  • None

Achievements/Observations

  • collected about over 15 hours of quality data from different samples (thanks to FXE) with extremely stable jet.

Issues

  • Issue with data processing in the morning and from the day before due to DAQ; only half of the detector data were calibrated; this was solved by switching to the old DAQ
  19   10 Feb 2024, 23:32 Tokushi SatoSPBShift summary

X-ray delivery

  • 1-350 at 12.45 keV, 1750 uJ. Stable delivery

Optical laser delivery

  • None
    

Achievements/Observations

  • completed Uwe and Petra data.
  • the jet is almost established for <mixing experiments

Issues

  • We spent almost 7 hours working to establish the mixing injector jet
    
  21   11 Feb 2024, 23:04 Tokushi SatoSPBShift summary

X-ray delivery
1-350 at 12.45 keV, 1700 uJ. Stable delivery

Optical laser delivery

None

Achievements/Observations

- collected data at different delay times 14s and 18s with over 7 hours of data collection
- the jet was stabilized again by removing the catcher and decreasing transmission to ~30%
- removing the catcher is critical to get stable jet, that allowed us to run for over 20 runs without stop.

Issues

- The jet was extremely unstable for 1.5 shifts; we needed to stop the runs intermittently with intervals less than 10 mins. 
- It seems difficult in principle to go for longer delay times with the current setting; may be elongating the distance in Y-axis with 400 pixel if the jet stable may help to reach 20s.

  22   13 Feb 2024, 01:02 Tokushi SatoSPBShift summary

X-ray delivery

  • 1-350 at 12.45 keV, 1600 uJ. Stable delivery

Optical laser delivery

  • None

Achievements/Observations

  • collected data at different delay times 20s and 40s
  • we installed new mixing injector to allow for longer delay times; and collected complete dataset with it
  • completed four different delay times for the users 14s, 18s, 20s and 40s

Issues

  • spent longer times to make stable jet which cause many interruptions and hutch searches almost every 5 mins for several hours.
  • it was extremely difficult to mix with delay at 20s with original setup.
  • nozzles broken during the day, costing several hours to prepare new ones
  24   13 Feb 2024, 08:52 Romain LetrunSPBIssue

Like after the previous winter maintenance period, the shutter of FXE appears as opened in DOOCS panels even though it is closed (https://in.xfel.eu/elog/OPERATION-2023/58).

Attachment 1: Screenshot_from_2024-02-13_08-51-02.png
Screenshot_from_2024-02-13_08-51-02.png
  26   15 Feb 2024, 09:32 Romain LetrunSPBShift summary

X-ray delivery

  • 1 to ~332 pulses at 6 keV, 4.5 mJ
  • Stable delivery
  • Number of pulses limited by the Photon Beam Power local limit that is set to 16 W. Is this intentional? All other limits are set to 40 W

Achievements/Observations

  • Aligned NKB trajectory
  • Achieved NKB focus
  • Reduced background scattering on AGIPD
  • Started data collection

Issues

  • Identified a couple of problems with the new DAQ that we were testing. Controls group informed and investigating
  28   21 Feb 2024, 23:00 Tokushi SatoSPBShift summary

X-ray delivery: stable and flat over 250 pulses per train. 1.1 MHz with 400 uJ

Optical laser delivery: NA

Achievements/Observations:

  • Beam optimization and characterization
    - Short pulses tuned
    - HIREX setup and used to characterize the number of SASE SPIKES per pulse (~10)

Issues

  • 10 SASE spikes per pulse is more than desired
    - discussion with BKR and AccRC regarding possibilities for optimization (feedback in the morning)
    -HIREX data proc is not correct as dark processing fails

    - DOC informed and working on it
    - one issue was gothard II appeard to have timed out - does the number of triggers need increasing?
    -sample jetting setup

    - electrospray working but saw exactly 1 hit in run 32
    - tried to alighn with silver nano-cubes but had to switch to sucrose to increase hit rate to aid alignment
    - no hits with sucrose seen with scanning X +- 1mm

  30   22 Feb 2024, 22:52 Tokushi SatoSPBShift summary

X-ray delivery: stable and flat over 190 pulses per train. 1.1 MHz with 200 uJ short intervention needed to improve intensity after it dropped

Optical laser delivery: NA

Achievements/Observations:

  • Beam optimization and characterization
    - Short pulses tuned and number of spikes reduced
  • X-ray focus FWHM calculation found to have a bug in pixel size used
    - rest to 0.2 FWHM of focused beam now better than 1.5 microns
  • Laser illumination installed to enable particle visualization to aid alignment and feedback
  • Sample injection and data collection for several hours
    - Hits seen from Silver cubes
    - very low hit rate from copper
    - cubes, and spheres

Issues:

  • DAQ - multiple crashes and restarts needed throughout the shift
  • Proc pipeline - some runs fail to processwhile adjacent runs collected with same conditions (run repeater) process without errors
  • Sample injection - copper samples continue to be difficult to inject
    - very low hit rates (few hits seen per run)
    - clogging issues
  • Focus drift - focus was tuned at around 3 pm, by 9 when we could check again focus was >3 microns FWHM
  • Beam intensity reduced during the shift and BKR needed to tune it back
    - returned intensity to 200 uJ - they asked if we wanted to continue but did not as this could have resulted in more SASE spikes
  31   23 Feb 2024, 23:03 Tokushi SatoSPBShift summary

X-ray delivery:

stable and flat over 190 pulses per train. 1.1 MHz with ~200 uJ

Optical laser delivery:

NA

Achievements/Observations:

Beam optimization and characterization
Sample injection and data collection for ~10 hours

reasonable hit rates observed but highly fluctuating, making alignment optimisation slow

Issues:
DAQ - restarts needed throughout the shift
Sample injection - copper samples dont give high hit rate and increaseing sample concentration led to clogging
Focus drift - focus was tuned at around 3:30 pm ( from 4 um back to 1.5 um)
-again at 19:30 when we checked again focus was >2microns FWHM returned to 1.5

  33   25 Feb 2024, 00:11 Johan BieleckiSPBShift summary

X-ray delivery
9.10 keV; 190 pulses per train @1.1 MHz with roughly 140-200 uJ; observed some fluctuation in the photon flux

Optical laser delivery
ns laser for particles visualization

Achievements/Observations

  • Beam optimization and characterization
  • X-ray focus FWHM calculation found to have a bug in pixel size used
  • Sample injection and data collection for >10 hours
  • collected data on copper cubes with average hit rates of ~5%

Issues

  • DAQ - multiple crashes and restarts needed throughout the shift; at certain point we encountered three crashes within less than an hour; and called DAQ to intervene
  • Gotthard II at Hirex stopped acquiring and we had to change the number of the acquisition time to ~10 days to allow continuous running until the end of the beamtime.
  • hit rates were low and sonication of samples needed to be done several time to prevent clogging issues
  • Focus drift - we observed significant drift in the beam; a refocus then had to be check several times to focus back the beam to nm
  • beam intensity fluctuated a lot during the shift
  35   25 Feb 2024, 22:54 Tokushi SatoSPBShift summary

X-ray delivery

  • 9.10 keV; 190 pulses per train @1.1 MHz with roughly 170-200 uJ; stable delivery

Optical laser delivery

  • ns laser for particles visualization

Achievements/Observations

  • Sample injection and data collection for ~11 hours
  • collected data on copper cubes with better hit rates than previous shifts; ~15%

Issues

  • We had laser interlock issues, the magnet in door went to error state and this was fixed by Klaus Witt call around 22:15.
  • data migration issues; it was very slow and DOC did not figure out the cause for this.
ELOG V3.1.4-7c3fd00