Group Selection Page HELP Controls Group Data Analysis Group ITDM WP76 DET Vacuum EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Operation PSPO XO TS Migrated2Zulip
Technical integration and coordination SASE1 Status and SASE3 Readiness
  Technical integration and coordination logbook, Contact: A. Violante, G. Wellenreuther, Page 18 of 31  Not logged in ELOG logo
ID Datedown Author Location Severity Subject
  265   04 Oct 2021, 09:30 Andreas GallerGeneralInformationNotes XO Integration Meeting 04.10.2021

General Announcements
none

 

CTRL
- late train IDs at XGM, being sorted

 

DA
- nothing to report

 

EEE

DET

ITDM
- nothing to report

XRO
-

LAS
- Failure of Power SUpply on AMPHOS in SA3, had been replaced over the weekend,
- observed jump on ODL in SA2 around midnight --> being investigated

UNSYS
- nothing to report
- PKG status to be discussed

 

SPB/SFX
- nothing major to report

 

FXE

 

MID
- Feedback for M2 stopped working, started calculating wrong values

HED
- nothing to report

SCS
- problems with DAQ: DA1 stuck --> DOC and ITDM are aware

SQS
- nothing to report

 

AOB

 

 

 

  264   29 Sep 2021, 09:49 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 29.09.2021
General Announcement:
- Tomorrow 30.09 is the deadline to submit the winter maintenance task list. In case of questions please do not hesitate in contacting XO.

CTRL:
- Nothing to report; systems are fine.

DA:
- Nothing to report.

EEE:
- Nothing to report.

VAC:
•	Nothing to report.

XPD:
- High pulse energy in SA3: about 15-16mJ was seen at 1keV and XPD confirmed the accuracy of this value. By the other side, XPD is checking for potential damages in the YAG screens (not yet identified but 
this is close to the limit).
- A. Koch sent information on the screen limits and corresponding watchdogs. Info will be shared at the readiness mailing list.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Lasers are running. Question on spectral stability in SA3 from Monday: LAS investigated but they lack some diagnostics; LAS will need to work with SQS colleagues (Gotthard) to further complement 
diagnostic.

UNSYS:
- Alarm that PKG of SA3 is off. Nevertheless, SA3 is well performing with good temperatures.

SPB:
- Issue resolved by DRC in seeing the beam.

FXE:
- Nothing to report.

MID:
- All fine at MID; all in progress.

HED:
- Setting up experiment.

SCS:
- Progress with new setup.

SQS:
- Nothing to add; in touch with LAS.
  263   27 Sep 2021, 10:06 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 27.09.2021
General Announcement:
- The subscription for the User and Operation mailing list can be found at the regular announcements. Please forward the e-mail to new colleagues who are interested in getting the meeting link. 
- For any required operation that is not in the normal envelope, please keep in mind to discuss with XO in advance, to better planning and organization together with Desy-MXL / BKR colleagues. 
If possible, please let XO know with weeks in advance.

CTRL:
- No major updates from CTRL to be announced.

DA:
- Quiet blue week in terms of support.

EEE:
- Nothing to report.

VAC:
- Nothing to report for VAC.
- From PRC / Raul V.: photon intensity >15mJ in SA3 seems to be real. Spectrum analysis is under way and confirms >14mJ and it may be related to the chirp studies.

XPD:
- Self-seeding studies during the weekend (13keV @ HED): seeded signal of about 400mJ - data to be analyzed.

DET:
- Nothing to report.

ITDM: 
- Last week Gitlab update: some bug fixes were done during the weekend. In case of issues please let ITDM know. Gitlab is now available at the distributed link: gitlab.xfel.eu.

XRO:
- SA2 mirrors changed to platinum stripe.

LAS:
- Nothing to report.

UNSYS:
- Nothing to report.

SPB:
- Nothing to report.

FXE:
- Nothing to report.

MID:
- All fine at MID - thanks to XRO to have changed the optics.

HED:
- Nothing to report.

SCS:
- Nothing to report.

SQS:
- Issues with spectral stability of PP-lasers. To be addressed by LAS.
  262   22 Sep 2021, 09:47 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 22.09.2021
General Announcement:
- A reminder that in case staff members suffer from cold symptoms, please remain at home and do home-office.
- TS informs the restrooms at XHEXP1 are out of service. The fix is under way.

CTRL:
- Update of Beckhoff mc2 devices at all tunnels and experiments was done yesterday. No substantial problems occurred and first checks show also no issues. Please check if past configurations are used as it 
may cause a crash. 
- PP-Laser device (PPL multi-pattern device) will be update at instruments and down during this time. In case instruments do not want this upgrade, please contact CTRL / Valerii Bondar.
- No reports from DOC.

DA:
- Nothing to report.

EEE:
- Fast electronics: adding uTCA for SCS.
- Cabling installation at HED ongoing.

VAC:
- Nothing to report.

XPD:
- Yesterday ZZ with Gotthard was done and went as planned. It is now working nicely.
- XPD is prepared for the activities related to the blue/development week.
- Wavefront measurements were successfully done last Monday.

DET:
- First test of DSSC at SQS taking place today and tomorrow.

ITDM: 
- Reminder for the Gitlab update taking place tomorrow at 16:00.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Issue with SA3 quad movers that were blocked on Monday. A re-start of Movers and phase shifter was done which solved the issue.

SPB:
- Nothing to report.

FXE:
- Nothing to report.

MID:
- All fine at MID.

HED:
- Nothing to report.

SCS:
- Not present.

SQS:
- All good at SQS.
  261   20 Sep 2021, 10:19 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 20.09.2021
General Announcement:
- On last week's cryogenic failure at the machine: the root cause is tracked to a failed temperature sensor that bring down several pumps.

CTRL:
- Time slot for Beckhoff update of Mc2 motors is tomorrow between 09:30-11:00. This update is for all topics in tunnels and experiments. 
Change of parameters must not occur (for the ones which are stored in Beckhoff). Nevertheless, it is good for groups to check parameters that are not stored. 
An e-mail will be sent to communicate it and a second one when the update is finished.

DA:
- Nothing to report.

EEE:
- EEE supports the mentioned loop updates (care of PLC system).
- Cabling installation in FXE and HED have started and are ongoing during this week.

VAC:
- Nothing to report.

XPD:
- Nothing to report.

DET:
- Nothing to report.

ITDM: 
- Gitlab update takes this week, on Thursday at 16:00. An e-mail will be sent to readiness.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Nothing to report.

SPB:
- Nothing to report.

FXE:
- Stable operation during the weekend with a problem solved by DOC. A beam jitter was also observed on the weekend related to a feedback.

MID:
- All fine at MID.

HED:
- Finished user experiment; nothing to report.

SCS:
- Nothing to report.

SQS:
- Not present.
  260   15 Sep 2021, 10:37 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 15.09.2021
General Announcement:
- Machine stop running yesterday around 21:00 due to problems with the cold compression. A repair was done during the night but there are not yet stable conditions. It is unclear at what time today the 
machine will be back. 
By the other side, gun conditioning was done during the night which may help to have less gun trips.
- Steffen H. on behalf of DRCs reminds the instruments to please attend the jour-fix meeting on Mondays. 

CTRL:
- Beckhoff upgrade is set to take place next Tuesday 21.09 at 09:30 (during ZZ access). It relates to only Karabo devices with Mc2 motor loops; the PLC continues to run. 

DA:
- Nothing to report.

EEE:
- An update of SA2 motion loop to integrate the shutter / Gotthard is to take place next Tuesday 21.09 morning. Nerea J. will coordinate with CTRL (Valerii B.) and Naresh K. on the best time.

VAC:
- Nothing to report.

XPD:
- Nothing to report.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Reported last Monday that SA1 PP-laser was down due to air conditioning. In fact, it was a broker chiller that is repaired already Monday was a broker chiller. The PP-laser in SA1 is available since 
yesterday.

UNSYS:
- Nothing to report.

SPB:
- Change of setup happened yesterday. It will be ready for beam at 23:00 if possible.

FXE:
- Nothing to report; not yet clear if program will be affected due to the beam down.

MID:
- All fine at MID.

HED:
- Got beam yesterday morning and made use of it.
- Had issues with Interlocks (see operation-2021 e-log). A DACHS terminal stopped working and could not do hutch search. The Dachs terminal was replaced by Desy colleagues. 

SCS:
- Issue with the GATT mentioned last meeting: Jan-Torben D. has contacted XO (F. Wolff-Fabris). Discussions with further groups will take place.

SQS:
- Not present.
  259   13 Sep 2021, 10:04 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 13.09.2021
General Announcement:
-	 

CTRL:
- Next week CTRL will upgrade Beckhoff devices containing MC2 loops. This is done to avoid the PLC to crash. This is only Beckhoff devices and is for all tunnels and loops. This is a Karabo devices upgrade 
and not the PLCs. 
Naresh K. says systems are still needed on Monday morning. A. Galler tells that a ZZ will take place on Wednesday and this may be a better time for the update. XO and Valerii B. will discuss offline.

DA:
- Nothing particular; DOC and OCDs were quiet.

EEE:
- Nothing to report.

VAC:
- Yesterday effect of the power glitch at 15:03. VAC-OCD checked with SCS - the interlock reacted and GATT stopped for a while.
- SCS comments that the problem with GATT gave 300x more photons than expected to the sample and detector. A better safety / interlock is to be prepared. 
Raul V. says that topic was first discussed in 2019 with SQS (using the B4C block in the interlock loop for the GATT). It may require interaction from SA3 instruments, VAC, XRO, CTRL.

XPD:
- Small confusion on the SA3 XGMs during the weekend: BKR mixed the SCS and SQS XGM's not working and this triggered a few reactions on the weekend. A better way to do not mix up XGMs will be made. 
Please look to the right device before informing experts on possible errors.

DET:
- PnCCD detector is brought back to operation.

ITDM: 
- Next week (not yet date) a follow up of the Gitlab update will take place. Mostly likely on Wednesday; details will come by e-mail.
- Update of the GPFS metadata: to be replaced on a time without activity. Shall be transparent.

XRO:
- Nothing to report.

LAS:
- Lost SA1 PP-laser due to change of temperature and humidity at midnight this morning. No information when it will be made available for use.

UNSYS:
- Nothing to report.

SPB:
- Looks fine apart the fact it did get only 188 and not 202 bunches.

FXE:
- Nothing to report.

MID:
- All fine at MID - experiment running smooth; seeded beam running well. Only issue that Gotthard in tunnel has triggered Big Brother a couple of times. It was not a real signal. Naresh K. will talk to 
Ulrike B.

HED:
- All fine at HED.

SCS:
- Nothing else to add.

SQS:
- Successfully brought up pnCCD. It replaced physically one sensor. Reasons for the crash is understood at SQS (bad communication among the involved people). 
  258   08 Sep 2021, 10:06 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 08.09.2021
General Announcement:
- Quick report of yesterday incident of water running down to XHEXP1: a washing machine in the clean room at ground level leaked; the water went down to the SA4/5 area of XHEXP1. 
Both ground floor and XHEXP1 floors were wet. The washing machine is taken out of service.

CTRL:
- The ticketing system "control-integration" is working again and can be used.

DA:
- Nothing to report.

EEE:
- Fast electronics announcement: tomorrow late afternoon EEE will apply the fix to DSSC camera to correct train-IDs.

VAC:
- A test on the signal optimization in SA3 vacuum loop was done. VAC found a fix that will be applied during the shutdown (together with CTRL, EEE).

XPD:
- Nothing to report.

DET:
- pnCCD risk analysis: DET says that there is a path to avoid accident again and has been discussed with instrument. The misbehavior of the bottom module is to be investigated and replaced.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Improvement of sync system in SA3 laser hutch took place this week. The laser is available to instruments.

UNSYS:
- Nothing to report.

SPB:
- Nothing to report.

FXE:
- Nothing to add.

MID:
- All fine at MID.

HED:
- Nothing to report.

SCS:
- Nothing to report.

SQS:
- Nothing to report.
  257   06 Sep 2021, 11:00 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 06.09.2021
General Announcement:
- XO thanks the instruments for posting in the Operations e-log.
- Jan G. says calorimeter measurements last Friday on HED hutch took place in order to measure transmission of pulse energy in the hutch and compare with tunnel HED-XGM. 
It is found a factor 2 loss - half beam intensity arriving at the end of the hutch. 
- Lorenz K. says that the Barista machine triggered a fire alarm. TN did a check and may be caused by the self-cleaning program of the machine by producing enough steam.
- Call for tasks to be carried on next Winter maintenance period were sent out to the are sent out via e-mail last Friday. Groups shall submit one group list by 30.09.2021.

CTRL:
- From tomorrow morning all the mails sent to the "controls integration" ticketing service will be offline due to an update. Please contact DOC instead. An e-mail will be sent as soon as the service is 
back.

DA:
- Nothing to report.

EEE:
- OCD calls from SQS last week were solved. 
- Nerea J. and F. Wolff-Fabris say the issue with SQS EPS is temporarily solved by now (masked signal at the Big Brother). SQS has to plan when it wants the EPS running.

VAC:
- Quiet week in terms of OCD.
- Today in the morning: please refrain to use this morning any SA3 device like GATT.

XPD:
- Nothing to report.

DET:
- Follow up of the pnCCD detector: upper part of the detector has scratches after Thursday accident. SQS will replace parts this week. The lower part of the detector has also some issues (noisy). 
It is unclear if the detector can be used.

ITDM: 
- Today and tomorrow ITDM carries the integration of the offline storage into the new system (new storage blocks; it shall be unnoticed by the users). The old one goes out by replacement.

XRO:
- Apart of the beam point instability @ HED (stochastic instabilities that appear from time to time) there is nothing else to report.

LAS:
- Change repetition rate at SA2 PP-laser this week.
- SA1 has a higher jitter than normal that has to be investigated on-site.

UNSYS:
- All fine at Undulators.

SPB:
- Running night shifts: on SAT night some X-ray trains not arriving; It did not disturb operation but a bit disturbing.

FXE:
- Lost of pulses and many issues on Saturday such as instability and tuning without prior communication. To FXE team.
- F. Wolff-Fabris says the beam became more stable on Sunday afternoon after the restart of a server and the feedback for the compression.

MID:
- All fine at MID.

HED:
- Suffered from beam jitter: measurements were done with different mirrors settings, CRL configs; MID branch. Analysis ongoing this week (Sebastian Goede).

SCS:
- Nothing to report.

SQS:
- Nothing to report.
  256   01 Sep 2021, 10:36 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 01.09.2021
General Announcement:
- A change of meeting room occupancy will happen soon (increase by some %) and will be posted at each meeting room door. 

CTRL:
- Together with EEE, CTRL updated the pnCCD system and is working again. Hardware train-IDs acquisition was used by SQS yesterday afternoon and was successful.

DA:
- Nothing to report.

EEE:
- Nothing special to report.

VAC:
- Nothing to report.

XPD:
- Nothing to report.

DET:
- Follow up for the pnCCD today (offline calibration fixed). Detector is being cooled down and then data will be collected. DET supports CTRL that one had to make better plan earlier.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Nothing to report; Cell#16 in SA3 got repaired on Monday.

SPB:
- Nothing to report.

FXE:
- Nothing to report.

MID:
- All fine at MID.

HED:
- Yesterday the beam was available for delivery later than 15:00 due to issues with machine tuning.

SCS:
- Not present.

SQS:
- Not present.
  255   30 Aug 2021, 10:08 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 30.08.2021
General Announcement:
- XO will send next Wednesday the call for the Winter maintenance tasks. This time there will be a ticketing system and the call will be also send to Readiness list but please check with your group leader 
to submit a single task list per group.
- Please we ask instruments to place shift summaries at the Operations E-log.
- Suren K. asks TS for the trainings of TS staff as some people do not have access to tunnels areas. Lorenz K. will verify with TS members.

CTRL:
- Broker pileup issue at FXE on Wednesday: resolved with tracing on specific devices. Work is ongoing to prevent future incidents.

DA:
- Nothing to report.

EEE:
- Test on turbo pump on SQS to take place (by Bernard B. and VAC).

VAC:
- Nothing to report besides the test. This morning also a PLC test takes place.

XPD:
- Nothing to report.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- There is a problem with cell#16/SA3. A ZZ is currently ongoing to fix it.

SPB:
- Nothing to report apart of the intensity slope last week. It was improved during the weekend but still an issue for data collection.

FXE:
- Nothing to report.
- BYU trigger delay change was observed: it was fixed but root cause still to be found.

MID:
- Problem with AGIPD processing data (automatic calibration did not work): under analysis/fix.

HED:
- Not present.

SCS:
- Strong contribution of SA1 dedicated bunches last week; one need to have zero bunches at SA1 to be able to see the SA3 beam. Not interleaved mode may solve it but was not yet tested/tried.

SQS:
- Nothing to report.
  254   25 Aug 2021, 10:08 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 25.08.2021
General Announcement:
- A power glitch happened this morning at around 5:46AM. The drop of voltage was seen in two stations (about 0.6kV) and it's smaller than the last Monday's one. In case groups find affected systems, please 
let XO know.

CTRL:
- Last night DOC informed on a Broker pileup at FXE. The FXE loop was restarted and mitigated the situation. Further investigations are ongoing and will be coordinated with FXE.

DA:
- Nothing to report.

EEE:
- EEE helps on the Loop 6 pileup reported by CTRL.
- Call from DOC reporting connection issue to Karabo (broker PLC was replaced and had to be configured the connection).

VAC:
- Effects of power glitch seem to be none. Cooling water for the vacuum systems was not affected.

XPD:
- XGM in SCS being offline due to surge of pressure. This was informed by e-mail from SCS last night; interlock tripped due to high-voltage. XPD checked and put back during the night. 
Laurent M. says it was not clear why the pressure was increased. It is under investigation and SCS will inform XPD.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Amphos were not switched OFF due to power glitch. PP-laser in SA3 are under maintenance.

UNSYS:
- Nothing to report.

SPB:
- Issue with EPS at 14:10 yesterday: beam permission was blocked by EPS despite both shutters closed and all gate valves open. The signals had to be manually restarted (this info was not transmitted and 
EPS blocked beam permission). 
Issue appeared when they use beam for the first time; EEE / Nerea J. will further investigate.

FXE:
- Nothing to report.

MID:
- After the power glitch the SA2 beam intensity decreased by 10%. BKR will not do any interaction right now. It is still fine for MID to work in such conditions.

HED:
- Nothing to report.

SCS:
- Apart of the XGM issue, nothing else to add.

SQS:
- Nothing to report.
  253   23 Aug 2021, 10:03 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 23.08.2021
General Announcement:
- There were two minor power glitches on Saturday morning but with negligible effects to machine.

CTRL:
- Nothing to report.

DA:
- Nothing to report.

EEE:
- Update of EPS-SA1 loop was done this morning in order to enable the FXE imagers modes.

VAC:
- Nothing to report.

XPD:
- Nothing to report.

DET:
- DSSC is installed back to SCS and put into operation last Friday.

ITDM: 
- Nothing to report.

XRO:
- Not present.

LAS:
- All AMPHOS are ON and LAS is preparing PP-lasers for instruments today.

UNSYS:
- Issue with cell5/SA1 phase shifter: M. Yakopov confirms a power cycle was made to recover it.

SPB:
- Problem with Karabo (misread 2 of vacuum gauge values): the problem was a terminal of the crate was broken. After the terminal has changed, it works fine. 

FXE:
- Nothing to report.

MID:
- All fine at MID.

HED:
- Nothing to report.

SCS:
- Nothing to report.

SQS:
- Nothing to report.
  252   18 Aug 2021, 10:48 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 18.08.2021
General Announcement:
- In the aftermath of the severe power glitch, XO will be grateful if groups drop a message of what was affected in your groups. This will help us to understand effects of the latest power glitch.
- Status of machine: last night we had 2.3mJ in SA3 and 0.5mJ in SA1. Right now machine is under tuning - good chance of beam available for FDs today.

CTRL:
- Follow-up to the power glitch. CTRL had to restart the Karabo systems due to the main network to the brokers being down for minutes. Few follow-ups later that day but in general things have recovered. 
If groups experience troubles after running systems, please write ticket to CTRL.

DA:
- DA has deployed at SPB the latest Karabo 2.11.4. This concludes the cycle to all instruments.

EEE:
- Nerea J. informs Andreas K. detected that FXE imagers needs to be debugged by restarting the EPS loop. This will take out beam permission and may be organized for Monday morning. 
- Clock and control system was downgraded (confirmation from Bruno F.).

VAC:
- Nothing to report.

XPD:
- XPD had to restart manually many devices, macros and MDLs. One had to fix the broken Karabo bridges.
- No hardware and XGM gas supply were affected; they are running fine.
- SASE recovery only happened Tuesday afternoon and FD in SA2 was cancelled.
- A. Koch asks if Monday it is a good day for the IMG intervention. XO will check with MXL.

DET:
- AGIPD in SPB was the only detector ON during the power outage. It quickly recovered as it was in power voltage.
- DSSC being moved this week among SA3 instruments.

ITDM: 
- Network issue due to the power glitch (distribution layers switchers); connectivity of different parts of network affected. This in turn affected Karabo. 
- Gitlab upgrade today at 13:00.

XRO:
- M. Vannoni informs that to evaluate better tunnel devices (initially seems OK), one needs the beam back to check them in full.

LAS:
- PLCs and Karabo issues were found; in touch with EEE to make redundant power supply to PLCs. 
- Lasers went OFF and plan is to have PP-Ls available by Friday evening.

UNSYS:
- Problem with SA1/cell27 on Monday as motor drivers went in error state after the power glitch. It was recovered remotely. 
- Tables for 3keV @ 14GeV were updated on Monday.

SPB:
- Problem with Karabo and vacuum gauges (EEE is working on them).
- Problem with PP-laser shutter last week (unable to set interlock). It is now solved.

FXE:
- Detectors were OFF and therefore no issues due to the power glitch.
- Similar Karabo problems as described earlier.

MID:
- All fine at MID. Summary will of the effects of power glitch will be sent soon to XO.

HED:
- Lost the tests for the wavefront sensor as machine was down (FD). Plan to use beam tonight if it comes back. On Friday Karen A. plans to use beam for tests.
- Ulf Z. has sent a report to XO; Detectors were OFF and unaffected.

SCS:
- Not present.

SQS:
- Not present.
  251   16 Aug 2021, 10:07 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 16.08.2021
General Announcement:
- On frequent power glitches: it happened twice over the weekend. Machine was not affected and maybe this time the PP-lasers also. The origin is on the electrical power grid of Vatenfall and it is unclear 
how long it takes to fix it.
- TS requires location knowledge of all office chairs. In case you have taken yours to home, please let TS know.

CTRL:
- Reports from DOC with delayed train of DOOCS XGM for HED. It will be investigated.
- Reminder that it is a blue week and the DOC does not run. The DRC is available via phone.

DA:
- Issues with the offline calibration system: changes of data migration (how it works). Hot fixes are ready to be deployed (probably today); please check the migration and the auto-run; DA will update 
after the deployment.

EEE:
- Nothing to report.

VAC:
- No problems with power glitches.

XPD:
- XPD side: all fine.
- As PRC, Jan G. says that pump-probe laser at FXE was affected on Friday morning (Laser unavailable until the night). On Saturday and Sunday the lasers were not affected due to the power glitches.

DET:
- Nothing to report.

ITDM: 
- Gitlab update taking place this week.

XRO:
- Not present.

LAS:
- None of the PP-lasers went down this weekend in relation to the power glitches.

UNSYS:
- Nothing to report.

SPB:
- Adrian M. asks that, in case supply problems continue to happen, would that mean EuXFEL makes proactive action for operations? 
A. Galler comments that DOB will discuss the issue with lasers (exchange of pumps that are vulnerable to the power glitches is planned for the winter shutdown). From machine side, as it is largely 
unaffected, the operations calendar won't change.

FXE:
- Nothing to add, no news on clock and control upgrade.

MID:
- All fine at MID.

HED:
- Not present.

SCS:
- Nothing to report.

SQS:
- Dismount the DSSC and give back to SCS this week. 
- SQS expect beam for the safety tests on Wednesday and Thursday.
  250   11 Aug 2021, 10:45 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 11.08.2021
General Announcement:
- There was another power glitch last night. It seems that accelerator was not affected. If XFEL groups find affected devices, please contact XO.
- From healthy task force the Covid-19 measures stay in place as they are.

CTRL:
- Nothing major to report.

DA:
- Nothing to report.

EEE:
- Nothing to report.

VAC:
- Nothing to report.

XPD:
- Running at SA3 at the edge of the energy range (3keV is at limit of the undulator tables (K near 4)). Sara C. says that with higher electron beam energy this won't be a problem. 
In addition, the problem was with only few cells and Suren A. is in contact with BKR. Jan G. says that will get in touch with Suren A.
- HED reported problems with beam pointing being instable. XPD and XRO will take a look on it and include XO for the analysis.

DET:
- Small hiccups but nothing major. Follow-up on the pipeline for DSSC and firmware for other detectors.
- Jolanta S.-D. is not aware of LPD/FXE update and will check with EEE/CTRL/FXE.

ITDM: 
- Announcement: ITDM would like to upgrade GitLab on next Wednesday 18.08 from early afternoon 13:00 and may take until the evening. Users of GitLab are contacted and no access to GitLab during that 
action.

XRO:
- Nothing to report. Maurizio V. will follow the HED beam instability.

LAS:
- Affected by power glitch: all PP-lasers / AMPHOS went down and warmed. LAS would need a solution for power glitches, considering power backups, UPS or batteries, as the controllers for this type of the 
pump goes down. 
- Jan G. says that XPD has a bunch of unused UPS and could offer to LAS.

UNSYS:
- Problem with cell6#SA3 quad mover is solved. Coupler was exchanged and it works. UNSYS has more spare parts and is in touch with Beckhoff.
- Sara C. says P. Gessler informed Karen had problem when changing undulator energies. Suren A. could not find any issue with Doocs / Unsys side. 
Patrick G. talked to Olaf H. and firstly BKR must try to change energies and if it works, then instruments shall try. Karen A. says BKR could moved energy and had no issues - the issue was in HED side. 
P. Gessler proposes that Unsys check with Olaf H. Further discussion will follow offline.

SPB:
- Couple of issues: 1. SBP Pop-in reached limit switch and had to be changed; 2. Affected by the PP-laser downtime overnight.
- A. Koch says there was about 0.5mm steps lost. He is looking into it if it is mechanical, motor, or control issue.

FXE:
- Wajid E. says work on the clock and control is ongoing for the LPD.

MID:
- Al fine and use of PP-laser is not needed currently.

HED:
- Nothing else to report besides of what was discussed during Unsys report.

SCS:
- Not present.

SQS:
- Not present.
  249   09 Aug 2021, 10:26 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 09.08.2021
General Announcement:
- XO reminds that the timing system maintenance is taking place now.
- Power glitch happened at around 5:15AM today. Accelerator had a downtime; please check components that use cooling water / pumps to check if they were affected or not.
- Liubov S. reported that the PRC phone could not be started and maybe calls were not answered.

CTRL:
- Follow-up on the timing downtime maintenance: CTRL advices instruments to please check DAQ afterwards. 
- Bug reports for cameras - 1. For Lima Cameras updated for new Karabo 2.1: new library has new features on rotation (a fix can be deployed); 2. Rotation in non-instantiated cameras won't work: one has to 
instantiate them initially.

DA:
- Nothing to report.

EEE:
- Downgrading clocking and control firmware today after this meeting. For questions please talk to Bruno F.

VAC:
- Power glitch propagated to the tunnel (shortage of cooling water) but it did not affect VAC systems 
(update 10:30AM: GATT-SA3 injection of gas stopped due to the water shortage being above 5-10seconds but less than 30s, as pumps continue running - SCS could set it up again quickly). 
- GATT loop: Steffen H. and Raul V. are setting up update and please refrain to use it today.

XPD:
- Issue with XGM in SCS hutch: it got shock vented with N2 last week. Fortunately, high voltage was not ON. Turbo pumps have issues with controller; on Thursday and Sunday the pressure gauge dismounted. 
XPD is on contact with SCS on it. 
Remark: Theo M. is on holidays and for XGM issues please contact Jan G.
- A. Koch asks Steffen H. if the Lima/Bassler camera fixes uses only DAQ and not GUI. Steffen H. says this is correct and for Karabo 2.11 only. A. Koch will send a list of cameras to Steffen H. 

DET:
- Nothing to report. DET aware of clock control downgrade.

ITDM: 
- Today afternoon ITDM starts activities in Influx: downtime of few minutes.
- Reminder on closing proposals: Please do it ASAP. If not done by next week a policy on non-accessed data will be done.

XRO:
- Nothing to report.

LAS:
- The cooling water for SA1 and SA3 AMPHOS went down on Saturday morning, bringing PP-lasers - due to another power glitch on SAT early morning. 
- Cooling water in the LAS R&Dlab remains down as of now.
- PP-Lasers will come up this afternoon but requires time to thermalize. It was put OFF in a controlled way this morning due to the timing system maintenance. The AMPHOS was not OFF during today's power 
glitch.

UNSYS:
- Issue with cell6/SA3: power cycle of whole cell is ongoing. Suren K. asks if a possibility to go to tunnel can be done. ZZ is not yet requested but will contact PRC to fix the coupler.

SPB:
- Complications with the delivered bunch pattern last week: coincidence of needing to drop bunches for detectors commissioning at same time SCS use particular bunch pattern. 
Nevertheless, it worked well enough for getting data, apart of few lost time to set the pattern.

FXE:
- Nothing to report.

MID:
- All fine at MID.

HED:
- Nothing to report.

SCS:
- Not present.

SQS:
- All good at SQS.
  248   04 Aug 2021, 10:19 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 04.08.2021
General Announcement:
- There was a power glitch last night around 10:30PM at the Desy power distribution. XHQ and XHEXP1 seemed to be not affected - probably devices that gets cooling water from Desy could be affected. 
Please inform XO if it is found any affected equipment in the XHEXP1 side.
- Raul V. asks on the fix CTRLs has applied to the vacuum issues last week. Steffen H. will give details offline.

CTRL:
- Bi-directions of clones may not have been working properly and CTRL is investigating and preparing a solution for it. 

DA:
- Nothing to report.

EEE:
- Small issue with FXE timing (strange behavior of RTMs). It happens when communication to main timing system is interrupted and it affects mainly the FXE PLCs. Result is that devices may not correct 
trigger.
- Cable installation currently ongoing in HED and scheduled at FXE (cw33). 

VAC:
- Last night status of critical pumps was checked and seems that there was not water interruptions (very first time for a power glitch). These pumps stay working normally during the night.

XPD:
- No issues to report due to the power glitch.

DET:
- Jungfrau activities at SPB were delayed because of beam delivery issues with the number of bunches.

ITDM: 
- Nothing to report, also regarding to the power glitch.

XRO:
- Not present.

LAS:
- Unaffected from the power glitch. PP-Lasers are working.

UNSYS:
- Everything is fine with Undualtors.
- SA3/cell6 quad mover has a problem unrelated to the power glitch. As agreed with BKR, it stays as it is Monday when corrective actions to recover it will be made.

SPB:
- Beam delivery problems yesterday regarding the bunch number: A. Galler says that accordingly to W. Decking, when running in interleaved mode it may cause that problem if the suppressions does not work 
correctly. 
Further investigations are under way by MXL. If this is an issue for equipment BKR can install an interlock on the Big Brother and probably a detailed discussion with interleaved mode instruments can help 
in this direction.

FXE:
- Nothing to add.

MID:
- Looks like the beam today is very unstable. Data were collected and the stability task force will be contacted.

HED:
- Nothing to report.

SCS:
- Not present.

SQS:
- Yesterday a situation happens when 1 PLC went OFF unintentionally, causing a pump went OFF afterwards. SQS asks if this can be followed in a way the pump does not goes OFF in such events. 
Raul V. comments that 5Hz turbopumps at the tunnel have a configuration to prevent it by disabling the automatic switch in the component configuration. 
Nerea J. says that they can take both tunnels and SQS used configurations to bring them together. 
  247   02 Aug 2021, 12:06 Frederik Wolff-FabrisGeneralInformationNotes XO Integration Meeting 02.08.2021
General Announcement:
-	 

CTRL:
- Deployment announcement: there are two more motors to be updated on ARAVIS cameras (photonic and Bassler). CTRL proposes to do this afternoon. 
Cameras devices will be down after update, aimed to enable train-ID corrections. Please instruments check on Tuesday if cameras are back.

DA:
- Nothing to report.

EEE:
- uTCA CPU in SA1 balcony room is not reachable. Bruno F. is investigating.
- CPU of LOOP16-SQS is not reachable. Nerea J. will check it today.
- Issues with LPD and AGIPD are being tested in the lab and the check at FXE will be done later today.

VAC:
- Some OCD calls related to instability of SA3 vacuum loop. DOC is informed and shall be working on that.

ME:
- Alberto De Fanis asked if ME is present, willing to discuss topics. A. Galler says group was around a couple of times and if groups have questions please use this meeting series to address.

XPD:
- No issue to report.
- Steffen H. will check if some XPD tunnel devices are affected by the ARAVIS update.

DET:
- Nothing to report.

ITDM: 
- Influx problem on Thursday (disabling live-view on Karabo): with help of Influx data support this was fixed. The cluster is up and running well. 
The part of the old data during the time it was down was written by Grafana and will be synchronized back to the cluster during this week (tests are needed).
- Following on Friday for the Influx third node that went down (not related to the previous incident): DOC was blind for a couple of hours. Investigation ongoing. It's not related to Grafana. 
Instruments were not affected.
- Reminder of reviewing and closing proposals for the second half of this year. Please look at the sent e-mail and make a cleanup. 

XRO:
- Not present.

LAS:
- All PP-lasers are running.

UNSYS:
- Issue with phase shifter in SA3 for the 3keV use: problem is that at 14 GeV the tables of phase shifters go to 25.3mm gap and this is in the upper limit for making 3 keV. 
Phase shifter tables will be extended to a larger gap. Better to use 3keV at higher electron energies.
- For the UOM's mention on the update of phase shifters on the chicanes, this has to be prioritized. XO will discuss with Sara C.

SPB:
- Minor issues with SPB imager being saturated. A. Koch adds that it will be followed up and he is aware and will analyze better.
- Organization limitation: RF discussion for last weekend. The BKR / Operators were not aware they would use 404 bunches and costed 2 hours.

FXE:
- Nothing to report.

MID:
- All fine at MID.
- Laser went down for 12 hours due to the issues with HED interlocks on Wednesday (action taken due to the restart of laser interlock performed by SRP).

HED:
- Not present.

SCS:
- Issue on the scanning speed of monochromator. Parameters will be adjusted with Marijan S. today.

SQS:
- Loop for moving DSSC got a fault but has been addressed by EEE.
  246   28 Jul 2021, 09:29 Arvid GebhardtGeneralInformationNotes XO Integration Meeting 28.07.2021

General Announcement:

- No general announcement.

CTRL:

- Problem with cameras / DAQ: fixed yesterday and tested at SA2/MID.

- Issues with SA3 vacuum loop: work ongoing and further investigations are needed.

DA:

- Nothing to report.

EEE:

- Cables available for HED to be installed next week – date to be discussed offline.

- Fast electronics side: working on FXE and HED issues;

- Nerea J. reports on the VAC/SA3 issues: instability with devices going into an unknown state. Alessandro S. works at the night shift in the DOC and will try to fix a permanent fix during the night.

VAC:

- SA3 issues are affecting also the gas attenuator (not only general SA3 vacuum loop but also GATT is affected, which has its own loop). There are no physical events - seems to be electronic. Stability issues to connect devices; Seems to be more related to infrastructure (PLCs). Further investigations are needed.

ME:

- Not present.

XPD:

- SPB imager: change setting of motors (it shall work now) and next week when A. Koch is back further investigations will be done.

DET:

- During the night after 02:00AM: SPB AGIPD K-bridge went OFF. Under investigation. Detector recovered this morning.

ITDM:

- Nothing to report.

XRO:

- Not present.

LAS:

- Nothing special to report.

UNSYS:

- Nothing to report.

SPB:

- Issues with wings of Agipd, causing also interlock trigger. Under investigation.

- 11PM last night: laser interlock triggered. It happened also in the past and SPB will get in touch with Safety today.

- XTD9 IMG: currently working but a permanent fix is welcome in the future.

FXE:

- Not present.

MID:

- All fine.

HED:

- Major issues with Agipd detector yesterday (thanks to DOC for help): 1. Could not kill the service and CTRL investigated; 2. Clock and control did not work (it has worked at CFEL last week). The issue was solved by changing the clock controller back to old configuration (EEE helped). Bruno F. comments that the clock control had to be downgraded in this detector (it worked in other detectors, only this one did not work). Further investigations will follow.

SCS:

- Change of speed from the monochromator was not possible. EEE and CTRL are working on it.

- Next week will run with a special train picker / bunch sequence pattern as the one used last week.

SQS:

- Vacuum problem as mentioned.

- Commissioning DSSC for future beamtimes.

ELOG V3.1.4-7c3fd00