General Announcement:

- /

CTRL:

- Karabo 2.10 FXE was sucessful.

- go on with SPB and go on with other instruments. On Thurthday 3rd all SASE3 topics.

- Connenting Gui clients, if gateway there is no need to write a tiket this week. If local installation, please make an update.

- network upgrade takes place at SASE1 tomorrow. Karabo will be updated later.

DA:

- /

EEE:

- MircoTCA updates today and on going. Timing won´t be avaliable for short period.

VAC:

- /

XPD:

- /

DET:

- /

ITDM:

- Tomorrow network update SASE1, but there no Karbo update.

- at the beginning of January DESY-IT will do some work on data center / maxwell clust. interuptions are expected. Schedule is still be to expacted.

XRO:

- Not present.

LAS:

- Tests on the mircoTCA failt. More tests needs to be done.

- LAS request comunication from instruments go thru XO.

UNSYS:

- Not present.

SPB:

- Aigipd intervention is running well. To be tested by the end of this week.

FXE:

- cable installation on going.

MID:

- all fine

HED:

- /

SCS:

- Not present.

SQS:

- /

General Announcement:
- There is no UOM or XO Int. Meeting next Friday 27.11. We will continue with XO Int. Meeting next Monday 30.11.

CTRL:
- Karabo Deployment in FXE takes place today. Schedule for full Karabo deployment was sent to the Readiness mailing list.
- Two SA2/SA3 Beckhoff devices topics will be updated today (5-10 minutes downtime).

DA:
- No issues or announcement to report.

EEE:
- PLC side: tunnel updates to be started with CTRL.
- Change requests also to start today in a couple of instruments.

VAC:
- Not present.

XPD:
- PES/MEA: there are individual tasks ongoing.

DET:
- Nothing major to report. AGIPD intervention is goes faster than planned.

ITDM: 
- Offline storage update was done and is available since Tuesday.
- Online cluster update has started: SA1 already available since Tuesday. Today ITDM will finish SA2/SA3. Crosscheck and start DAQ will follow.

XRO:
- Not present.

LAS:
- Nothing special to report.

UNSYS:
- Not present.

SPB:
- AGIPD work is ongoing and fine.

FXE:
- Wille company is installing cables in the hutch.
- Issue with craning on SA3 / SXP activities above FXE area: FXE asks for coordination to avoid interrupting activities. 
Gerd W. says it shall not be a problem as crane will be used for few period. Martin K. will inform his phone number for arranging craning times.

MID:
- All fine at MID.

HED:
- All fine at HED.

SCS:
- Shutdown activities ongoing.
- Situation of JJ X-ray will be clarified by SCS staff. 

SQS:
- Nothing to report.

General Announcement:
- An e-mail to the readiness and stgl mailing lists was sent regarding the calendar share and updated task list. Please contact xo-planning@xfel.eu in case of any issue.
- Desy has made mandatory for the tunnel works the use a community mask all the time and a FFP2 mask if distance between two persons is less than 1.5m.

CTRL:
- CTRL sent requests to the DA colleagues. S. Hauf asks the colleagues to take a look and have a discussion afterwards. 
- Karabo deployment on FXE will take place on Wednesday. 
M. Turcato says one DET person must be back to work on that day to switch Detector off and on by again: Wed afternoon or Thursday morning it shall be OK.

DA:
- Nothing to report.

EEE:
- PLCs will start shutdown with planned activities on today and tomorrow in SA3 instruments.

VAC:
- Not present.

XPD:
- Plan for XGM/SCS is to keep it running (under vacuum). XPD reminds SCS people to keep XGM on vacuum. 

DET:
- AGIPD intervention: is progressing quite well and advancing. 
A. Violante says it is progressing faster than expected and many small issues were solved during the weekend; first quadrant was already re-mounted and tested. 
By the end of this week an update on time planning will be made. 

ITDM: 
- Today ITDM starts the update of offline storage system (unavailable all day).
- The update on the online cluster system will start today in SA1 and then SA2 and SA3. It will not be available until Wednesday.

XRO:
- Working on the winter shutdown.

LAS:
- Carrying on planned maintenance work.

UNSYS:
- Not present.

SPB:
- AGIPD intervention is going well and is under control.

FXE:
- Cabling works starting today in the hutch.
- A. Galler asks if calibration of LPD was done. D. Khakhulin mentions that the raw data was used and calibration constants were not used. 
M. Turcato says one has to discuss together with calibration team.

MID:
- All fine.

HED:
- Nothing to report.

SCS:
- Nothing to report.

SQS:
- Nothing to report.

General Announcement:
- Updates on the maintenance schedule will be made available in Alfresco. Guidelines in how to integrate calendars will be sent to readiness mailing list, for example.
- Please feel free to contact XO by to share any thoughts related to the Maintenance.

CTRL:
- Successful tests of HED pulser were done on Monday;
- CTRL is preparing Karabo 2.10 deployment: a schedule will be sent around per e-mail. FXE starts on 25/11. Remaining areas take place on Tue/Wed/Thu of week 49. 
SPB will be discussed later due to the AGIPD work.

DA:
- DA has received two messages from CTRL related to Karabo and data analysis deployment: first about deployment and second about bridge.

EEE:
- Nothing to report.

VAC:
- Nothing to report.

XPD:
- XPD is involved on MID tests this week;
- Shimadzu camera tests/development is cancelled due to AGIPD intervention at SPB.

DET:
- Nothing to report.

ITDM: 
- Planning for next week maintenance and working with AGIPD/SPB intervention.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- A problem with SA2/cell37 coupler is found and will be fixed during shutdown.

SPB:
- Work on AGIPD is going OK. It has been found the expected minor issues. SPB expects AGIPD experts on site.

FXE:
- Nothing to report.

MID:
- Fast-ADC issue is under investigations together with EEE/Bruno F.

HED:
- Nothing to report.

SCS:
- Not present.

SQS:
- Nothing to report.

General Announcement:
-	 

CTRL:
- Further improvements on the SA3 vacuum loop will continue during the shutdown.

DA:
- No issues to report.

EEE:
- 1 OCD call from SCS call related to PLC: hardware issue.
- Fast electronics side: MID timing system problem is still present. Frequency that this issue happens is reduced.
- For the Winter maintenance, EEE will ship some crates to Poland. Due to Covid-19 situation, only crates with non-critical changes for next run will be sent. EEE is in 
closed contacts with instruments.

VAC:
- 1 OCD during the weekend was related to GATT and promptly solved.

XPD:
- Nothing to report.

DET:
- Last Friday problem with offline jobs was solved (it had to be followed by IT).
- SPB activity for AGIPD baseline intervention/correction to start today.

ITDM: 
- Maintenance on the offline storage will happen next Monday 23.11. An e-mail was sent this morning. The system will be OFF and all data will not be accessible that day.
- ITDM will also start the maintenance of online cluster on 23.11. They will not be available until the end of maintenance (about 3 days).

XRO:
- Nothing to report.

LAS:
- OCD calls from SCS: 1. observed crystal degradation in SCS side; 2. laser pattern server could not be changed (LAS forward the message to CTRL).
- All other PP-lasers are fine.

UNSYS:
- BKR tested air coils in the last SA2 undulators cells for performing work on Wednesday for photon beam alignment (there was a problem with Cell37/SA2 not responding).

SPB:
- Instrument and beam delivery were running successfully last week;
- This week SPB starts working with AGIPD: one aim to work as fast as possible to finish before Christmas break.

FXE:
- D3 is currently preparing the tests.

MID:
- Continues to use beam this week for detector tests. 

HED:
- Tested AGIPD during last weekend. Currently HED is changing setup to use High Intensity laser this week.

SCS:
- Not present.

SQS:
- Nothing to report.

General Announcement:
- XO is in contact with some groups regarding the winter shutdown, in order to get additional information or solve potential conflicts. 
If there are any question, please contact us in this meeting or via e-mail.
- Steffen adds on the idea of using FXE for initial deployment of updated Karabo version. It will take place after 23/Nov.
- Jan asks how XO needs to be informed for tasks that groups deal with each other. A. Galler says XPD is on the list for setting a date to discuss this. 
Groups are encouraged to communicate with other support groups beforehand.

CTRL:
- Misbehaving timing server at MID yesterday was initially solved by EEE. EEE replaced the hardware of the timing system itself but problem continued. 
System seems now to be healthy but from time to time there are small timing interruptions. Investigation is ongoing and will be discussed in the digitizer meeting of Data 
department.
- One CTRL ticket on Doocs device at SCS: it was a problem with custom installation. CTRL asks to be informed in case groups have custom installations. This may pose a 
risk when deploying new updates.

DA:
- AGIPD train-ID issue: a small workaround until shutdown is in place. One has to set the individual channel and seems to be now working. Next intervention to take place 
during Winter maintenance time.

EEE:
- Nothing else to report aside of the MID timing system.

VAC:
- Nothing to report.

XPD:
- SA3 XGM was switched to Ne as requested by instruments.

DET:
- Preparation for activities on AGIPD at SPB/SFX during the winter time is ongoing.

ITDM: 
- Pre-announcement: at the start of the maintenance, ITDM has activity on the computer cluster. A separate announcement will be sent.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Nothing to report.

SPB:
- No issues to report.
- Intervention date of AGIPD: SPB requires due to highest priority work that between 16/11-18/12 no interruptions of electrical power or the need of EQP may happen. 

FXE:
- Crosstalk SA1/SA3 was not seen yesterday as SQS used only 1 bunch.

MID:
- Nothing to add besides what CTRL and EEE have mentioned.

HED:
- Next week HED does not use X-ray beam: commissioning of the high power laser and test of pulse on demand are planned. A. Galler says it pulse on demand was fixed by 
CTRLs and can be used. 
- Question on use of ADCs: Bruno says some trains will not be recorder or be repeated. This may be related to the timing issue like in MID. 
But data can be acquired. It is under investigation accordingly to Bruno, with CTRL, DA, ITDM.

SCS:
- Not present.

SQS:
- Nothing to report.

General Announcement:
- Maintenance summary task list has been made available at the S:\ folder as well as in the XO Alfresco page.

CTRL:
- On Thursday laser1 went down: error on deployment related to fix zero train ID.
- Lasers safety deployment is fixed.
- SA3 vac loop: progress was made and new version of Karabo needs re-start this morning.
- SA3 fixed MC2 motor issues. Coordinated with instruments to do tests today.

DA:
- 1 OCD was received and solved during the weekend.

EEE:
- Announces that on tomorrow's PBS there will be a EEE presentation on the Digitizer instruments.

VAC:
- Nothing to report.

XPD:
- No issues. The switch of XGM settings at different energies went well.

DET:
- Nothing to report.

ITDM: 
- Meeting with MID on train-ID is set. There is a mitigation error plan which will be implemented.

XRO:
- Nothing to report. Minor issues on CTRLs but are not stoppers.

LAS:
- PP-laser for SPB run was interrupted for 45 minutes.
- Issue with synchronization / motor becomes unstable, relating to motor slippage. Hardware will be replaced during the shutdown.

UNSYS:
- Nothing to report.

SPB:
- Finished user experiment: went OK but machine not very stable. Intensity and pointing very unstable. After beam losses timing came quite different. 
Data for pointing quantification to be analyzed.

FXE:
- Unstable beam the whole week. 
- Crosstalk between SA3 and SA1: sometimes BKR could fix/work on it, sometimes not. This happens when FXE runs at 150 bunches and SA3 varying 1-100. 
SA1 intensity goes down by factor 2.

MID:
- DET and ITDM contacted during the weekend.

HED:
- Not present.

SCS:
- Not present.

SQS:
- Crosstalk SA1/SA3: night shift crew said it is not a problem for next week if run without Interleaved.

General Announcement:
- XO reminds on the 4.5MHz operation: MXL tested a 12hs switching mode and this will be used along the week. 
The night shift operates at 4.5 MHz and day shift at 2.25MHz to avoid sending unnecessary bunches to the dumps.

CTRL:
- Issues with MC2 motors (bug in moving, mainly in SA3). CTRL has a fix but it is hard to find a proper time slot this week to apply it. The idea is to deploy next Monday 
only in SA3 first. 
If issues are found beforehand, please contact CTRL.
- SQS mentions a bug in the mirror observed yesterday which relates to the above described issue. SQS has to observe it to verify it moves all the way and then one can 
wait until Monday for the fix.

DA:
- Nothing to report.

EEE:
- Crate issue in SA2: it was not needed to exchange the crate. A reboot on Monday afternoon helped to bring system back. 
Crate works as normal. During winter shutdown EEE will replace crate and power modules.
- MID and LAS reported issues with train-ID / timing servers. Triggers have not been delivered in 10Hz. This is in discussion with DESY. 
One solution is an update of the DOOCS timing serves that do not show errors which does not interrupt timing servers. In MID this may be related to data recording. 
For the PP Laser further discuss with Desy colleagues will be made. This affects only MID because the use of old version of Doocs server. 
- MCP problems reported by XPD: mismatch on the firmware version and EEE plans to upgrade this week. 
- Preparation for the winter maintenance is ongoing and installations can start on 23/11.

VAC:
- VAC would like to thank EEE and CTRL for the SA3 loop vacuum fix.

XPD:
- XPD found another issue with MCP: wrong assigned triggers. A ticket will be created. 

DET:
- Update of MID AGIPD is still an issue but this is not critical at the moment.
- CTRL has accessed the LPD detector issue.

ITDM: 
- K. Wrona informs ITDM has the reservation of Maxwell for data analysis during next two weeks. 
The user groups have to be informed that one has to use Maxwell reservation to quick analyze data.

XRO:
- Nothing to report.

LAS:
- SA1 SPB PP-laser had to increase energy and is on work.
- SA2 PP-laser missing trigger from around 7PM yesterday. Laser went OFF due to this and luckily LAS was watching it and was able to switch ON quickly. LAS is in touch 
with EEE and Desy. 
A fallback solution could be to generate triggers by hardware but there is no progress from Desy side. LAS will prepare specifications for this module.

UNSYS:
- Small hiccup in one SA2 undulator that was remotely corrected.

SPB:
- Trying to align beam: SPB sees elliptical rings. Andreas G. suggests if this continues please communicate to the PRC.

FXE:
- No further information on issues with the LPD.

MID:
- Tomorrow it takes place the AGIPD train-ID meeting.

HED:
- Vacuum valves in differential pumping system were strangely closed on Friday 2AM. HED discovered this only yesterday. Pressure slowly went up. HED will further 
investigate.

SCS:
- Not present.

SQS:
- 4.5MHz operation had gun trips due to the overheating of the dump. 
F. Wolff-Fabris comments that was due to the radiation alarms on the dumps that removed beam permission by about one hour around 4AM.

General Announcement:
- A. Galler reminds the attendance that this meeting is also an opportunity for making comments/questions on the new Covid-19 rules that start to take place.

CTRL:
- Main issue last days was related to the SA3 VAC PLC. The applied fix was left running during the weekend. So far it works faster and reacts faster for diagnostics. 
CTRL is getting some statistics in order to get the final solution. If something goes wrong, please call OCD.

- No OCD calls during the weekend.

DA:
- No issues or news.

EEE:
- uTCA crate of SA2 balcony room is not reachable. A quick fix on Friday was not successful and EEE needs to power cycle or replace the whole crate. 
This will interrupt timing for PP-laser and the signals sent to MPS. Meeting between EEE, lasers, instruments and XO will take place. 
- (Note after meeting: the power cycle of the actual crate will be done today after lunch. If this does not helps then immediately exchange of crate will be done 
afterwards. 
PP-laser will be brought to a safe state and takes about 4-5 hours to restart it).

VAC:
- Nothing to report.

XPD:
- Worked at BKR with MCP detectors. XPD reported to CTRL an issue with ADC - some channels see no signal or not standard ones. 
EEE knows and runs some scripts - this happens from time to time and may affect recording data.
- In general, from XGM side, Theo M. has prepared a way to switch settings to extreme changes of photon energies.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Motor problem with SA1/cell31 last week was repaired and undulator should be working fine.

SPB:
- STAB campaign last week went fine and collected data that has to be analyzed.

FXE:
- Nothing to add.

MID:
- Waiting for invitation to a meeting on the AGIPD train-ID.

HED:
- Requests the beam delivery schedule in order to know if has to schedule people for Tuesday night.

SCS:
- Not present.

SQS:
- Nothing to report.

General Announcement:
-	 

CTRL:
- SA3 VAC loop - 2 approaches to find out the root cause (discussed also with EEE):
1. Thursday 8-11:00AM to test the whole loop by partitioning batches of devices. This may pinpoint the single device that cause the problem.
2. Reproduce the problem at lab level. Based on the reproducibility this approach will be continued in parallel. 

DA:
- Nothing to report.

EEE:
- PLC colleagues are troubleshooting SA3 VAC loop.
- DET reported that measurements are carrying on wrong train-IDs. Tests are being made in the lab (not reproducible) but EEE is working on it.

VAC:
- SA3 VAC loop is main activity - interaction with Michaela P.

XPD:
- Group mainly busy with Photon Diag Conference this week.
- Supporting STAB and not heard any major issues in this effort.

DET:
- Report on the train-ID. This affects all the AGIPDs including the mini-half.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Preparing for next week operation.

UNSYS:
- Nothing to report.

SPB:
- Collected lot of data last few days: SPB/SFX thanks LAS and accelerator teams for support.

FXE:
- Nothing to report.

MID:
- All fine.

HED:
- All fine.

SCS:
- Issue with door interlock: Eric B. has investigated this morning. Problem seems to be with the maglocks and has been solved for now but a long term has to be worked on.

SQS:
- Nothing to report.

General Announcement:
-	 

CTRL:
- SA3 vacuum loops problems: seems to compromise some experiments. Beckhoff loops are changing states very quickly and compromises the whole system. 
CTRL ask for the instruments to give info on the type of command given that makes motors change states. CTRL investigates together with EEE as more data has been produced 
as compared to the past. 
This problem does not happen in SA1/SA2 and may be related to the PES which will be checked this afternoon.
- Please call the new OCD numbers: 6541 and 6542. Specially on weekends it is the fast way to contact. This number is valid since last spring.

DA:
- Nothing to report.

EEE:
- Nothing to report.

VAC:
- Nothing to report except of SA3 vacuum loops.

XPD:
- Not present.

DET:
- XO asks on the train-ID progress of Agipd. Monica T. says that the plan is to correct in the DAQ level during the next shutdown. 
K. Wrona adds that the fixing takes long and the current investigation focus on doing in the DAQ level during January.

ITDM: 
- Maxwell nodes used to login in the Maxwell will be rebooted today, causing access limitations.

XRO:
- Nothing to report.

LAS:
- Bunch pattern in demand: LAS assume XO will take care of this topic. Andreas G. says XO will take care but need to be informed on what is needed. 
Tomasz J. mentions it needs some coordination between instruments and machine.

UNSYS:
- Couple of issues in SA3 undulator cell: it got stuck, mostly during machine development times by running out of range. This was corrected during the weekend.

SPB:
- Moving on with WP8. Continuing today.

FXE:
- Nothing to add.

MID:
- MID makes a general reminder on the train-ID development with AGIPD: MID expects to have a meeting to discuss a long term solution plus the test procedure for 
implementation after the quick fix. 
This has to be planned in advance.

HED:
- Nothing to report.

SCS:
- Nothing to report.

SQS:
- SQS has suffered during the weekend with the VAC issue.
- CTRL side can look at the PES this afternoon.

General Announcement:
- XO thank the effort made by preparing the activity lists for the maintenance period. In few cases XO is contacting groups back with questions and/or comments.
- The Winter maintenance period was adjusted yesterday. The ending period is now on January 13th, 2021, instead of January 4th. 

CTRL:
- CTRL is reaching conclusions on recent Beckhoff problems: 
1. CTRL would like to upgrade most of installations in instruments next week (upgrade of motors software). This upgrade is in general a step forward for the PLC 
connections when it is dropped (re-starting). 
2. For SASE installations, CTRL will make next week during machine days. 
3. The GATT will not be upgraded now – this will start during Winter maintenance period. Upgrade of GATT is a priority for CTRL (has been added to the maintenance list 
submitted to XO). 

DA:
- Nothing to report.

EEE:
- Intervention in SA3 PP-laser was done.

VAC:
- No new issues to report. 
- Maintenance on SA3 vacuum loop: in case of problems, one has to restart in PLC level with EEE staff and may result a bit longer recovery. 
Steffen H. comments it has been worked on and may be solved in the next day or so. The restart procedure is easy but lengthy. CTRL/OCD knows what to do.

XPD:
- One of vacuum pumps lost Beckhoff connection.

DET:
- Nothing to report.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- Nothing to report.

UNSYS:
- Nothing to report.

SPB:
- Not present.

FXE:
- Nothing to report.

MID:
- All is super at MID.

HED:
- All sounds great at the moment.

SCS:
- Nothing to report.

SQS:
- Nothing to report.

General Announcement:
- A Vacuum issue in one of the dumps brought machine down at around 1:30AM. Only around 7AM machine could re-start running and tuning again.
- XO thanks the groups who have already sent the maintenance lists. For the remaining groups, please contact at the xo-planning@xfel.eu.

CTRL:
- Few minor OCD calls during the weekend: One relates to a camera triggering.
- Issue with SA3 vacuum loop seems to be currently fixed but needs further investigations.
- Update of Beckhoff motors is planned for this week. CTRL will get in touch with Instruments.

DA:
- Nothing special to report.

EEE:
- Nothing to report.
- Question to XO on Maintenance: Could we have a draft of tasks list before Nov. 6th? 
Andreas G. answers that XO is working on it and as soon as lists are finished this will be published. In case info on partial parts is already needed, may be provided 
depending on the progress.

VAC:
- SA3 vacuum loop issue was mentioned by CTRL (see above).

XPD:
- Nothing to report.

DET:
- Nothing special to report.

ITDM: 
- Update will be done on DA acquisition of HED. This has been communicated with instrument personnel.

XRO:
- No bad news to report.

LAS:
- SA3 PP-laser presents small power drop in the laser beam: LAS will take care this week.
- SA1 PP-laser has a problem with ADC and stop acquiring data.
- Need to update firmware of ADC cards to get train ID's in SA3 PP-laser. This was requested to EEE. This may bring laser down again and may not be ready by Thursday. XO, 
LAS and EEE will discuss offline. 

UNSYS:
- Nothing to report.

SPB:
- Last week was smooth.

FXE:
- Problems with tunnels last night at around 1:00AM. More than half the shift was lost with no beam delivery: main user data was therefore not acquired. 

MID:
- Nothing to report.

HED:
- Beam was lost between 1AM to 7AM. Last data has been taken between 8:00 and 9:00AM. 
- During the weekend, beam drift was observed at Pop-in at the end of the tunnel. There are data and has to be analyzed.
- Discussion on implementation of Dynamic trigger will take place.

SCS:
- Not present.

SQS:
- Minor issue with Data: one data aggregator was not working properly. OCD and SQS personnel have solved it. 
K. Wrona comments that too many properties overloaded the aggregator. A long term solution will be worked on.

General Announcement:

• .

CTRL:

• problems in SA3
  • VAC loop unstable, devices cannot be brought into a stable state --> what used to be a restart every  two weeks, cannot be recovered these days
  • issues with motors developing a short time out
    • somewhere there is a beavy load on Beckhoff loops
    • TM comments: at times the movements happen very late after issuing the command
  • influxDB sees high load
    • any modification of or running a new macro?
    • Please let CTRL know
  • the issue started with VAC --> devugging shoud start there

DA:

• extrafoam OCD call

EEE:

• working together with CTRL on SA3 issue
• OCD call from SCS but call was not correctly frowarded

VAC:

• not present

XPD:

• no issues

DET:

• OCD call from SCS --> was fixed

ITDM:

• nothing to report

XRO:

• nothing to report

LAS:

• no issues

UNSYS: 

• nthing to report

SPB/SFX:

• all fine

FXE:

• people on shift could not initiate a hutch search
• GABS access right missing (Gebietsabsuche)
  • introduced without informing FXE

MID:

• waiting for invitation AGIPD trainID solution

HED:

• JUNGFRAU issue presists
• HexaPOD is working again
  • work around

SCS:

SQS:

• DAQ issue
  • no data was recorded without
• vac is an issue but not endangering
• motor issue is vital to the instrument

General Announcement:

• .

CTRL:

• issues SA3 vacuum loop
  • gets more and more diffcult to recover
    • problem not clear, PLC comminucation is lost without obvious reasons
    • no software changes took place from either CTRL nor EEE ... component issue?
• fastadc issues --> EEE
• mirror in SA3 not always following a move ommand XTD10 M1

DA:

EEE:

• OCD at HED --> hexapod
• fineDelay

VAC:

• preventive action in XTD10 for GATT scroll pump

XPD:

• SA2 imager XTD6 pop in 45 after M3 small mark on the imager --> imager still usable

DET:

• nothing to report

ITDM:

• OCD call data aggregator at MID
  • need to understand better how to avoid the situation
• users complained to maxwell service (DESY directly) about limited resources becuase of high load by other users
  • there are reserved nodes --> use them!
• issues with scratch
  • more than 300 TB had been accumulated within a relatively short time
    • it seems that data was calibrated to scratch not to proc
    • MID comments --> a special calib mode was used that could not save to proc which is why it was saved to scratch
• problem with one of the proxy servers --> degraded performance of data migration

XRO:

LAS:

• issue at SA1 with Galvo
  • after shift change to FXE, laser was not working --> Galvo was out of phase
  • FXE resetted something (not clear) and the laser came back
  • LAS investigating the issue

UNSYS: 

• nothing special
  • a couple of times the undulators opened suddenly without any command issued.
  • O. Henssler from DESY proveds the logs

SPB/SFX:

• coupling with SA1 and SA2 (SA2 using train on demand) is an issue
  • beam walks off the XTD9 XGM aperture
• offset mirrors M2
  • pitch movement got stiff
• smaract cotnrollers
  • most fixed by increasing timeout windows

FXE:

• shutter can be opened despite closed vavales

MID:

• used pulse on demand
  • users were happy

HED:

• damage on M3 mirror
  • due to focusing scheme --> 9keV and intermediate focus --> during alignment unfortunately the focusing point came too close to M3
• meeting with XPD and XRO at 11:30
• hexapod issue reported to EEE is not quite fixed
  • ´status is not reporting the correct status of the hexapod
  • hexapod started working at some point and fell back to weird operation later again
• CRL arm7 seems to be at the limit of the x-axis
  • if FEL stays where it is, XTD1 chamber might need to be moved
• JUNGFRAU issue
  • up to a point the data is fine in the preview
    • after some runs data seen in the preview is not written to file

SCS:

• nothing to report

SQS:

• l

General Announcement:
- Discussion on access rights based on FXE issue:
- The new Dachs system does not send notification that access rights will expire. Every staff member has to identify at the system when his/her rights will 
expire at each location. 
- Thomas T. says EuXFEL has made a request to Desy in order to modify this and currently it is at the pipeline at Desy.  
- Jan G. mentions that for the labs in his responsibilities he can see the access rights are being expired soon and he manually notifies the people.

CTRL:
- Two OCDs on Tuesday night: 1. SCS motor not moving (was fixed); 2. One SPB smarACT device server crashed (fixed). 
- CTRL is following up both incidents as well as a motor in HED/SA2.

DA:
- Agreement with CTRL (see last Monday's minutes) was done.

EEE:
- Nothing to reports on EDS, PLC.
- CPU for SA1 PP-laser was replaced last Monday. No issues observed since then.

VAC:
- Nothing to report.

XPD:
- No news to report.
- SA3/XGM operation this week (change of settings due to different photon energies range) is less problematic than initially thought. 
Procedure has been integrated to the BKR operators who will make the adjustments at each shift change.

DET:
- Nothing special.

ITDM: 
- Nothing new.

XRO:
- All ongoing, nothing to report.

LAS:
- SA1 PP-laser is running again after exchange of CPU.
- All PP-lasers running.

UNSYS:
- Tests with quad movers: helping with STAB task force.

SPB:
- SmarACT issue: communication error between local control and Karabo.
- SPB reports beam instability issue when SA3 changes, for example, the photon energy or bunch number. BKR was contacted and is aware of it. 

FXE:
- Issue with Interlock yesterday: new access rights for all staff in FXE. GABS has blocked the search prevention. 
Zunaira A. mentions that Frederico L., Dmitry K., Martin K. and Peter Z. can grant access to the staff members. Zunaira A. is not allowed to provide this 
access grants.

MID:
- All fine.

HED:
- User experiment this week. 
- The CRL1 in XTD1 had issue while moving arm in y-direction. A ticket was sent to CTRL. Contact for this ticket is Thomas Preston.

SCS:
- User beamtime this week.

SQS:
- Two color operation with magnetic chicane ongoing this week during day time, making use of the PES and XGM.
- One issue with saving digitizer settings. We open a ticket to ITDM.

General Announcement:
-  XO has sent to the support groups and instruments the call for the winter maintenance tasks. Deadline for submission is Oct. 16th. 

CTRL:
- Friday night incident with signals of SA3 vacuum loop. It took hours to recover the system.
- A deployment/restart of online cluster nodes will be done today meaning 10 minutes downtime. Devices / clone devices shortly unavailable during the 
deployment.
- CTRL reminds that tickets that may require Doocs bridge are better worked if sent during the week time. 

DA:
- No major issues to report.
- DA will carry an offline discussion with CTRL on the cluster nodes deployment. CTRL says this is for the CPU nodes.

EEE:
- Nothing major to report. No incidents during the weekend.
- Looking into PL/fast ADC mTCA issue with SA1 PP-laser. To be discussed separately with LAS, XO after the meeting: next solution is to exchange CPU - 
needing coordination with ITDM and CTRL.

VAC:
- No major issues. Karabo SA3 VAC loops look fine now.

XPD:
- PES: ZZ on Friday to fix issue with a broken gauge, mostly likely related to the power failure. Cables had to be switched. PES works again. 
- Tests next week using PES: Joakim L. will set the system.
- Change of SA3 XGM settings during this week: Theo M. will change setup during weekdays.

DET:
- Nothing major to report.

ITDM: 
- No OCD calls during the weekend.

XRO:
- Nothing to report.

LAS:
- Shutdown AMPHOS in SA3 PP-laser (SCS/SQS): LAS asks for one single point of communication with the instruments; This laser needs work today morning to do 
not compromise the week. 
- E-mail for communication can be scs@xfel.eu and sqs@xfel.eu where all group members receive the message. 
- SA2 PP-laser is ready for next week.
- SA1 PP-laser still show problems with ADC cards. CPU will be exchanged - to be discussed after this meeting.

UNSYS:
- ZZ intervention last week.
- Couple of hiccups remotely fixed.

SPB:
- Nothing to report, besides issues with PP-laser.

FXE:
- ADQ with different sampling rates issue was solved by EEE.
- Plan is to work on the Jungfrau commissioning.

MID:
- MID requests if there is planned a meeting to discuss the train-ID issue. Steve A. comments that yes, a meeting will be organized next week. 

HED:
- Asks on the request for a pulse on demand test during today. F. Wolff-Fabris says this has been sent to the machine RCs and will be discussed during the 
RC meeting at 10:00. 
As soon as a decision is done this will be communicated.

SCS:
- Nothing to report.

SQS:
- Nothing to add apart of PP-laser issue.

General Announcement:
- XO informs the Call for the Winter Maintenance tasks will be sent to the support groups and Instruments today. 
Please note it is not named "shutdown" anymore are critical services such as basic infrastructure must be maintained. At least in this round it will be called as "Winter 
Maintenance" to avoid confusion among the staff. 
The tasks list shall be sent to xo-planning@xfel.eu. 
- In case of need to go to tunnel via ZZ, please contact the PRC at 9-4800.
- Gerd W. informs on the SXP construction work that starts on Nov. 23rd. The plan aims for a minimal impact in SA3 area at XHEXP1. 
There will be new connections to the cooling water system in SA3and civil constructions. 
Cooling water will be cut from Nov. 23rd until Christmas. This means rack rooms and laser will be affected. 
- Gerd W. answers Raul V.: the cut cooling water supply does not involve tunnels. 
- Tommaso M. says instruments need to have permanent cooling water supply. 
Gerd W. states the electronics must be shut down and will not be able to re-start before Christmas as it takes time to empty the system, weld, do pressure tests and fill 
it up again. 
- XO suggests that SA3 instruments must coordinate closely with Gerd and also involve MB (Serguei M.).
- Nerea J. asks if this applies to SA3 balcony room also. Gerd W. says that the rack rooms will be affected. 
- Gerd informs there is a series of meetings taking place on Thursdays at 14:00 and this construction work can be addressed. The SCS and SQS group leaders have been 
invited to this series. 
If someone else wants to participate, please contact Gerd W. for getting the Zoom link, 
- Monica T. says that this lack of cooling water may impact the DSSC move to SQS by delaying its test.

CTRL:
- CTRL experiences some login hiccups related to information sent to InfluxDB. This will be fixed Sunday night (Monday at 02:00AM). There is no reason this issue will 
affect devices until the fix is done.

DA:
- Train-ID investigation for AGIPD offset is ongoing.  DA understands from now on DET is responsible for the leading efforts. 
- Monica T. comments it has been identified the issue related to the AGIPD firmware (2019 version, accordingly to Jolanta). This will require a significant change and to 
do it now is not an option. 
The good news is that this is a systematic issue and DET know how to take care of it.
- A. Galler asks the small task force on the train-ID to make a small presentation during next XO Integration Meeting. 

EEE:
- ADC in PP-laser (SA1) is confirmed to be solved.

VAC:
- Nothing to report.

XPD:
- Nothing to report.

DET:
- Issue with DSSC yesterday: one Interlock signal was permanently enabled. EEE solved after restarting the device. It will be further investigated. 
ITDM: 
- Desy/IT requested ITDM a time slot to fix redundant parts of the routers between "ours" and "accelerators". To do so Desy/IT and MXL suggests to apply the fix (do not 
expect side effects) today before lunch time. 
No relation to the timing system of lasers is expected (low risk). None in this meeting's audience raises an issue against the proposed time.  

XRO:
- Nothing to report.

LAS:
- All 3 PP-lasers are up. Currently LAS still make some studies and instruments must send a message in case the need of use. Lasers are ready to be provided to 
instruments.

UNSYS:
- A ZZ is currently ongoing to fix issues with failed couplers in SA3 control cabinets.
- Beckhoff replies the cause of failure is either an overload of current or relations to temperature. Temperature seems not to be the issue (accordingly to UNSYS).

SPB:
- Nothing to report.

FXE:
- Plan to test digitizer (HQ412): issue is that sampling rate is not reproducible as in the lab. Probably it will happen tomorrow together with PP-laser use.

MID:
- The vacuum issue with the monochromator is under control.

HED:
- Preparing the user next week (that uses the pp-laser).

SCS:
- Nothing to report.

SQS:
- Nothing to report.

General Announcement:
- Andreas G. informs the winter maintenance period will happen in two months approximately. The call for tasks will go out this week to the support groups and 
instruments. 
- XO informs Desy and TS are investigating the Thursday's event of power cut but do not yet have a clear reason for it.
- Jan G. informs that SCS hutch will be closed and searched this afternoon for the FD on Gain Studies.

CTRL:
- Notification service informed on a broken event. After a Karabo shutdown the FXE topic is fine again. 
- Power cut: CTRL sees no issues as leftovers.
- Investigation of missing data in DOOCS (10 seconds outdated): CTRL has two colleagues looking into it.
- Supporting EEE and LAS on the ongoing working with PP-lasers.
- Update of dataloggers has to be made again. It means a 2 minutes downtime during the course of today/afternoon if nobody has something against.

DA:
- Question to CTRL: a notification service was forward to us. DA needs further description on that. Steffen H. says it was related to the python server and this is not 
deployed by CTRL. 
This is the reason why was forward to DA.

EEE:
- Follow up from Friday recover on the power glitch of Thursday: some cards in SA1 balcony rooms were not reachable anymore on Friday. These lead to a complete restart of 
the whole crate (means interruption of timing). After additional local intervention communication to the MPS was restored. 
- Changed the fast ADC of PP laser in SA1.

VAC:
- Nothing to report.

XPD:
- No issues to report.

DET:
- Agipd was power up after the power cut and had no issues.

ITDM: 
- Nothing to report.

XRO:
- Nothing to report.

LAS:
- SA3 PP-laser: problems with SQS site. It misses the power energy (dropped down) and is under investigation. Maybe it is related to some optics assembling.
- SA2 PP-laser: power supply for AMPHOS was lost on Friday. The spare part arrives today at 11:00 and will be changed this afternoon.
- SA1 PP-laser: problem with uTCA (no ADC running and not possibility to login). Crate hangs after every few hours. Working together with EEE (100% in EEE side).

UNSYS:
- The problems with cell1/SA2 not moving were fixed by Suren K.
- Cell2/SA3 hardware problems (couplers) found this morning relates to an Ethernet card which needs to be replaced by ZZ. This cell is not operational.
- Phase shifters in SA3, cell4 and cell5 are not moving. The hardware works and it may be some higher level controls.
- Couplers were sent to Beckhoff for further investigation in order to find solution for decreasing the number of failures in future.

SPB:
- Nothing to report.

FXE:
- Due to issues with PP-laser the FXE did not test the timing tool during the weekend.

MID:
- All fine: still minor vacuum issue in the optical room but MID hopes to be solved soon.

HED:
- Pulse on demand tool was tested last Monday.

SCS:
- Nothing to report.

SQS:
- Nothing to add.

General Announcement:
- XO is anticipating in one week the delivery table preparations to gain more time to spot possible problems related to beam delivery. The cw42 table was sent to the 
instruments last Monday.

CTRL:
- CTRL confirms but updates on the dataloggers and time servers were done successfully.
- LAS brought some issues to CTRL that are under investigations.

DA:
- Nothing to report.

EEE:
- EEE conducted yesterday the SA1/SA2 ATT interlock logic modification. Summary can be found in these links:
https://in.xfel.eu/elog/SASE1+technical+commissioning/1982
https://in.xfel.eu/elog/SASE2+technical+commissioning/830

VAC:
- VAC reminds in general that EuXFEL staff shall check their access rights in the DACHS database. To have trainings up to date is not enough. 
In case your rights are expired, please contact Martin Dommach via e-mail. 

XPD:
- FD week had a good start with GATT bleaching studies. Today there is the FD K-mono 4-Bounce ongoing.
- Machine switched back to 14 GeV on Monday; XGM/SA1 gas was exchanged back; graphite screen in SA1 is inserted.

DET:
- Nothing to report.

ITDM: 
- The SASE2 online data storage was diagnosed and reconfigured.

XRO:
- A. Trapp adds on the solid attenuators interlock modification for SA1/SA2: from now it is not possible to remove the diamonds if the silicon is in. 
For more info please check the e-log or contact M. Vannoni: 
https://in.xfel.eu/elog/SASE1+technical+commissioning/1982
https://in.xfel.eu/elog/SASE2+technical+commissioning/830
- Yesterday there was a ZZ in XTD9 for SPB to inspect BPLM: seems to be OK but minor issues to be resolved for the winter shutdown.

LAS:
- SA1 in maintenance: no risk for FXE during the weekend. It must be operational tomorrow morning. 
- SA2: issue with ADC cards - the NOPA is lost. Reported to CTRL.
- SA3: no issues.

UNSYS:
- ZZ in SA2/Cell31 is scheduled for tomorrow (24.09) morning at 09:00AM.

SPB:
- Involved in the update of attenuator. 
- Helping the stability task force activities this week.

FXE:
- FXE informs the second Gotthard is back: Karabo can't start or reset. CTRL suggests that when there is an urgent case one shall call the OCD. Otherwise sending a CTRL 
ticket is most suitable.
- Last week issue with digitizer: A new card was installed but has different sampling frequency. EEE is aware and is looking into it.
- FXE comments it is hard for the instrument staff to attend the two Graphana trainings due to night shifts/compensation, etc. FXE asks on possibility of additional 
trainings. 
Steffen H. says one can organize an extra training. A blue week is more suitable for the instrument staff. The extra training will be organized for the coming days/weeks. 

MID:
- Train-ID issue is progressing.
- MID asks for help from technical and engineering staff to find parts with M20 holes adaptors in order to lift granite stone.
** Post meeting update: VAC group helps MID.

HED:
- Not present.

SCS:
- Nothing to report.

SQS:
- Nothing to report.