ID |
Date |
Author |
Location |
Severity |
Subject |
434
|
12 Jul 2023, 10:18 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 12.07.2023 |
General Announcement:
- There were two power glitches last night observed at Desy which switched OFF some cooling water pumps. Please check your systems and let XO know if you were also affected.
- SEPS Interlock tests driven by RPOs with help from Maurizio V. and CTRL colleagues are now scheduled to take place on Monday 17.07 instead of tomorrow.
This is because the required bunch patterns may not be provided by machine in due time this week. A tentative schedule for each instrument/SASE will be made in the coming day(s).
- Martin Doering mentions an e-mail was sent out to groups asking for the feedback on tasks that were postponed or cancelled. Please inform xo-planning if you have tasks in these situations.
- Steffen H. says starting today responsibility for the Data Acquisition services (Karabo devices and data group configuration) has been transferred from ITDM to CTRL.
So CTRL will be the support contact together with DOC. A small task force develops the DAQ at CTRL. Please contact CTRL and Steffen H. in case of questions.
CTRL:
- Deploy Karabo beckoff loops due to some bugs at PLC side which affects communication at Karabo. Patch for Karabo was prepared and will be deployed today at 13:00 in SA1-2-3 the moving loops; downtime few
minutes.
- Hardware intervention done by IT in replacing a mother board at SA3 which runns PLC loops. Takes longer (30 mins) to replace and restart Karabo.
DA:
- Calibration team will migrate offline calibration to different nodes. No interruption of services. DET is informed.
EEE:
- Bug of PLC and work around by CTRL: bug was there before and now that makes issues (see CTRL).
- MID: update of master loop as wished from MID.
- Next Monday busy with bring up days.
VAC:
- No impact of power glitch.
XPD:
- ZZ successful at XTD9 yesterday with Bassler cameras..
DET:
- Nothing to report.
ITDM:
- Message from Luis: update of myMDC takes place.
- Power glitch: around 03:00-03:30 the cooling failed. At 604 around 20 servers switched OFF as cooling water went down (as prevention mechanism to mitigate temperature raises). Tomasz. P. is working to
bring servers up.
- Tobias C. also confirms TS sees the 8C water was affected (which is used for the 20C water).
XRO:
- Nothing to report.
LAS:
- SA3 laser was shut down yesterday due to Lasermet work. Pouneh S. says additional Lasermet works for today were then cancelled and no more work will happen in this maintenance.
UNSYS:
- 4 Quadrupoles are not moving and are under investigation.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- All fine.
HED:
- Not present.
SCS:
- Not present.
SQS:
- Nothing to report.
SXP:
- Not present. |
435
|
17 Jul 2023, 09:53 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 17.07.2023 |
General Announcement:
- SEPS interlock tests are taking place today. An e-mail was sent to xo-info last Friday containing information about the order of the tests.
CTRL:
- Start with bring up days (SCS and SXP today) the whole week.
- Workaround solution for PLC/Karabo was deployed but needs to be redone tomorrow at 8AM tomorrow. An e-mail will be sent to xo-info.
DA:
- Nothing to report.
EEE:
- Bring up days and support of SEPS.
VAC:
- Nothing to report.
- As PRC, Martin D. says on Saturday morning issues with AIBS blocked beam operation. Solved by DRC and EEE/PLC OCD.
XPD:
- Nothing to report.
DET:
- Nothing to report.
ITDM:
- Data analysis of experiments (past ones): to be migrated on next week over three days. E-mail to be sent to xo-info.
XRO:
- Involved in the SEPS interlock today.
- ZZ for HED Mono was successful by power cycling device Not yet clear if camera is broken or a Karabo/PLC issue.
LAS:
- Nothing to report.
UNSYS:
- All fine.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Nothing to report.
HED:
- PP laser work this week and then setting up beamtime for cw31.
SCS:
- Not present.
SQS:
- Nothing to report.
SXP:
- Not present. |
436
|
19 Jul 2023, 10:25 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 19.07.2023 |
General Announcement:
- SEPS interlock tests are done in SA1 and SA2. It continues tomorrow in SA3 during the morning.
- Machine accelerator startup is ongoing. Photons are being produced and operators are working hard to improve conditions.
- When summer maintenance started in mid-June some instruments' staff were entirely gone and none were available to open hutches, which delayed some tasks.
XO will have a look next maintenance into this issue.
CTRL:
- Proceeding with bring up days.
DA:
- Working on the bring up days.
EEE:
- Work with bring up days and support SEPS tests.
VAC:
- All normal with VAC systems.
XPD:
- Not present.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Starting today with ramp up.
- ZZ for M6 mirror in SA3 tomorrow morning.
LAS:
- Received report from Pouneh S. on the Lasermet events.
UNSYS:
- All fine.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- All fine at MID.
HED:
- Nothing to report.
SCS:
- Busy at beamline.
SQS:
- Not present.
SXP:
- Not present. |
437
|
24 Jul 2023, 09:49 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 24.07.2023 |
General Announcement:
- Machine still works in improving performance. News will be posted at xo-info.
- Mikako M. And Idoia F. presented the summary of photon commissioning activities for SA1, SA2 and SA3:
SA1: https://in.xfel.eu/elog/XRO+Beam+Commissioning/206
SA2: https://in.xfel.eu/elog/XRO+Beam+Commissioning/186
SA3: https://in.xfel.eu/elog/XRO+Beam+Commissioning/201
CTRL:
- Last week prioritized work with bring up days at instrument workplaces.
DA:
- Bring up days last week; everything went smooth.
EEE:
- Bring up days went well.
VAC:
- Nothing to report.
XPD:
- Everything is fine with Diagnostics.
DET:
- Bring up days went fine.
ITDM:
- Dcache update taking place tomorrow for two days (old data affected). GPFS storage works as usual.
XRO:
- Handover to be explained after the meeting (see above).
LAS:
- Nothing to report.
UNSYS:
- Everything is fine.
SPB:
- Bring up days went well and some minor issues are being worked on.
FXE:
- Nothing to report.
MID:
- All fine at MID.
HED:
- Quick ZZ today to XTD6 to verify camera motors.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Not present. |
440
|
02 Aug 2023, 10:12 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 02.08.2023 |
General Announcement:
- Status of machine: a lot of work was done last days on laser injector and RF jitter; all seems to be OK and improvement were done.
Concluded that electron beam shape is now Gaussian and right now SASE is being tuned. Aim is to have about 2mJ in SA1 and 1mJ in SA2 at 9keV. Beam to be handled to the instruments on Thursday morning, as
planned.
- MID HED asked on use of HXRSS this week/next week. To be discussed at the daily Ops meeting today.
CTRL:
- Investigating the MOV loop in SA3 watchdog.
DA:
- Nothing to report.
EEE:
- Routine checks in rack for instruments (currents for ground lines). One rack presented an issue and today there is a plan to disconnect the wrong wiring line and a second power will be installed.
No impact shall be seen on motors using 40V at SQS.
VAC:
- Nothing to report.
XPD:
- If HED or MID goes to use seeding then N. Kujala has to setup the Hirex.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Progress in SA1 PP-L going well.
UNSYS:
- Error in gap of SA1 cell 10 undulator. It seems to be fine now.
SPB:
- Preparing for the shifts from tomorrow.
FXE:
- Nothing to report.
MID:
- Will discuss with HED on the using of beam for next 2-3 weeks.
HED:
- Nothing to add.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Nothing to report. |
441
|
07 Aug 2023, 09:50 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 07.08.2023 |
General Announcement:
- A. Silenzi is the DRC for this week and will send an e-mail to instrument local contacts for the jour-fixe meeting this afternoon.
CTRL:
- Movement issue in SA3 watchdog: after splitting the system it became more stable and will monitor it for next days.
- HED reported a bug with Jungfrau and will deploy the fix today.
DA:
- Nothing to report.
EEE:
- Follow up of a problem in SQS rack room (grounding): it was corrected in Thursday and now everything is fine.
VAC:
- Nothing to report.
XPD:
- Solution from CTRL to watchdog in SA3 is working fine.
DET:
- AGIPD at SPB: the chiller for cooling electronics (not the Julabo) failed. Investigation shows that interlock worked well and further investigation to be done after replacement.
- Jungfrau module2 needs work at HED and will be followed up.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Sunday morning issue with SA1 PP-laser that shutdown the AMPHOS. It was brought up by OCD at the same day.
- Need to coordinate with EEE on crate work for SA2.
UNSYS:
- All fine with Undulators systems.
SPB:
- Issue with AGIPB: there is a spare to be changed and further investigations to be done.
FXE:
- Nothing to report.
MID:
- Nothing to report.
HED:
- All fine with beam last week.
- Ticket is ongoing to solve issue on Jungfrau for next week. Wajid E. says a deployment will be made this week.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Nothing to report. |
442
|
09 Aug 2023, 09:43 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 09.08.2023 |
General Announcement:
- Starting this semester, a jitter report will be started at the UOM on the blue weeks.
Instruments are kindly requested to make a 2-3 mins report on experience with jitter for the previous few weeks, especially in relation to instabilities with beam.
CTRL:
- Issue at HED with Jungfrau detector module2: workaround is put in please and seem to work fine. Also issue looks like to be specific only to this module / detector controller.
Once experts return from holidays more investigation will be done.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- All running fine.
XPD:
- Everything fine from XPD.
DET:
- Nothing to add.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Problem with SA3/cell9 got stuck (third time). Not clear if it is a mechanical or control system issue. Further investigation during the winter maintenance period.
Error is currently fixed and undulator is running.
SPB:
- Nothing to report. PCS experiment is running well.
FXE:
- Nothing to report.
MID:
- All fine at MID.
HED:
- Will get in touch with CTRL to discuss Detector fix.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Nothing to report. |
446
|
23 Aug 2023, 09:47 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 23.08.2023 |
General Announcement:
- Next Monday, 28.08 there is no XO Integration Meeting due to agenda unavailability of XO.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Not present.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Tri way valve SA2 cell4 causing warm water in vacuum chamber which drifts the gap for few um. To be fixed in a shutdown.
- Cell25/SA2 feedback of temperature sensors is solved.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Waiting for beam. Tuning under way.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Not present. |
447
|
30 Aug 2023, 10:55 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 30.08.2023 |
General Announcement:
- Facility development week is ongoing.
- After the cryo incident last weekend the machine colleagues have an impression everything is in good shape and next tuning day can be made on Monday.
- Next UOM will have an invitation for instruments to briefly report on jitter issues (pointing, temporal, etc) that impact performance from last weeks.
If you have found any jitter issues please mention it during the UOM for discussion with machine colleagues.
CTRL:
- Today 13:00 new version for the DOOCS devices will be deployed for undulator controls. It takes 2-3 minutes.
- Follow up of PP-ODL devices: Wajid E. will right a ticket to DOOCS colleagues.
DA:
- Nothing to report.
EEE:
- Yesterday a SA2 SEPS interlock was updated and tested with XRO group and it went fine.
VAC:
- V0 closed after the SA2 loop update which was promptly re-opened.
XPD:
- Nothing to report.
DET:
- Not present.
ITDM:
- Upgrade of GPFS was finished yesterday successfully.
XRO:
- SEPS interlock update in SA2 worked well.
- One issue: communication to DOOCS was unstable in the early afternoon. Late on the day it was better.
LAS:
- Nothing to report. All fine.
UNSYS:
- At 15:30 a ZZ will be done to fix 3-way valve at SA2/cell4.
- Issues with MARWIN robot in SA3 and A. Leuschner will make a ZZ for fixing it.
SPB:
- Small issues on last THU and MON related to middle layer device and macros. DOC is informed and have recognized it. Further investigation in the particular macro that may can cause the issue.
FXE:
- Preparing for next week.
MID:
- All fine at MID.
HED:
- Nothing special; preparing next experiment.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Not present. |
448
|
04 Sep 2023, 09:46 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 04.09.2023 |
General Announcement:
- Last week was a facility development week / blue week. Another cyo issue happened on Friday morning which was recovered quickly.
- As DRC, Steffen H. reminds on the Jour-Fixe today at 13:00 to instruments communicate needed data allocation; also reminds on calling DOC may have a quicker help than sending a ticket.
CTRL:
- DOOCS clone devices disappearing from topics was fixed on Friday and since then works nicely.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Not present.
XPD:
- XPD involved in facility development since Friday. All activities went fine and more studies for Harmonic lasing and Aspect will be continued.
- J. Gruenert mentions for cw38 XPD and XRO colleagues are in Trieste for a conference and only limited staff is available for support. Basic functionality of devices is guaranteed.
- REXX servers were down last week.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Message to SA2 instruments: last week the Euler calibration was done for M1 and M2 and instruments can make use. The link is on the main mirror scene and the SA2 scene will be updated.
LAS:
- Nothing to report.
UNSYS:
- Issue with cell 25/SA2 terminal: undulator could not be used for harmonic lasing. Needs access for fixing.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- During the blue week it repaired the long arm for detector angular motion.
HED:
- Issues with IDC on rear module; EEE is working on it.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Not present. |
449
|
06 Sep 2023, 10:03 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 06.09.2023 |
General Announcement:
-
CTRL:
- Issues with cameras are being addressed.
DA:
- Nothing to report.
EEE:
- Yesterday issue with SEPS/HED being investigated/addressed.
VAC:
- Not present.
XPD:
- Nothing to report.
- As PRC, Naresh K. reports 1: new SEPS interlock at HED not properly working; 2. FXE reported crosstalk from SA3 pulses being lasing at SA1; also when switching 0 pulses in SA1 the SA3 went to zero also.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Mikako M. is aware of HED SEPS issue and is helping EEE.
LAS:
- Nothing to report.
UNSYS:
- SA2/cell25 fixed (terminal replaced) and is functioning.
SPB:
- Nothing to report.
FXE:
- Crosstalk SA1/SA3.
MID:
- Not present.
HED:
- Issues with ADC fixed by EEE (rear module) - seems a recurring feature with a 1 week frequency.
SCS:
- Motors at ending station solved by DOC;
- Issue with bunch/crosstalk.
SQS:
- Nothing to report.
SXP:
- Not present. |
450
|
11 Sep 2023, 09:47 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 11.09.2023 |
General Announcement:
- A Cryo issue taking place in the night Fri-SAT lead to beam not being delivered since them. The failure was at a magnetic bearing with a misreading of motor position.
In parallel motors 3 and 4 were exchanged as preventive measure during this downtime. Cryo OK came back on Sunday afternoon and efforts are focusing in bringing machine up for cw37.
CTRL:
- Pulse on demand for FXE (main scene) will be updated today.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Not present.
DET:
- Nothing to report.
ITDM:
- Not present.
XRO:
- Change middle layer for the bunch patter at SA1 for fixing today.
LAS:
- Not present.
UNSYS:
- Nothing to report.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Nothing to report.
HED:
- Between Synchronization issue and cryo failure only a total of 6-7hs of data were collected.
SCS:
- Beamtime affected by cryo failure.
SQS:
- Nothing to report.
SXP:
- Nothing to report. |
451
|
13 Sep 2023, 09:49 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 13.09.2023 |
General Announcement:
- The current week is the long delivery beam week for instruments. Operation continues up to next Wednesday 20.09 at 7AM. The DRC, PRC and RC will change as usual on Monday 18.09.
CTRL:
- Nothing to report.
DA:
- Not present.
EEE:
- Different problems with digitizers being addressed: 1. At XPD/ PES (card not working, it is then disabled); 2. FXE with jitter scene; 3. SQS is solved (undulator panel).
VAC:
- Everything good.
XPD:
- Apart of issues with PES nothing to report.
DET:
- Nothing to report.
ITDM:
- At a retreat.
XRO:
- Nothing major to report.
LAS:
- Not present.
UNSYS:
- Not present.
SPB:
- Everything fine.
FXE:
- Not present.
MID:
- Nothing to report.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Not present. |
452
|
18 Sep 2023, 10:01 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 18.09.2023 |
General Announcement:
- Differently from standard Mondays, today is a beam delivery day. The machine tuning day for cw38 takes place on Wednesday.
CTRL:
- Nothing to report.
DA:
- Quiet OCD weekend with two calls.
EEE:
- Nothing to report.
VAC:
- Friday afternoon SPB found pressure increase at SA1 solid attenuator which triggered vacuum signal. Investigation showed a bellow of one attenuator arm (arm 5; 1.2mm diamond) leaking and organized tunnel
access;
- VAC will discuss with XRO how to monitor possible damage and possibly modify middle-layer device.
XPD:
- Involved on Friday with XGM triggering alarm due to vacuum issue in XTD2.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to add to the solid attenuator issue.
LAS:
- Nothing to report.
UNSYS:
- Nothing to report.
SPB:
- Not present.
FXE:
- Preparing for beamtime.
MID:
- Nothing to report.
HED:
- Troubles with sample stages being addressed with EEE.
SCS:
- Not present.
SQS:
- Not present.
SXP:
- Not present. |
453
|
20 Sep 2023, 10:04 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 20.09.2023 |
General Announcement:
- The access and maintenance day is taking place today. Current ZZs at the accelerator are scheduled until 10AM and then it follows the tuning for cw38 beam delivery that starts from tomorrow at 7AM.
- Martin Doering announces that next Monday 25.09 a short presentation on the 2023 Winter Maintenance will be made in this meeting series.
Please a representative from every group is welcome to be present. The Maintenance task list is going to be send to instrument and support groups next week.
- As PRC, Jan reports so far the week looks good. Downtime from Mond-Tue of 1:15 which is being looked today at tunnel (LINAC);
- J. Gruenert (as PRC) reminds instruments to post at least once a day a report at the Operations e-log; In addition, for SA1, it is introduced a power limit of 5W to protect CRL1 lens when operating low
photon energies (this will be discussed with SPB/SFX).
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report;
- Reminder that forbidden XGM ranges (SPB/SFX): in SA1 the two XGMs do not have the same gas. Upstream has Kr to be able to run 5.5keV; SPB/SFX can run with 7keV at this XGM and their own on with Xe
instead;
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Not present.
UNSYS:
- Quad issue in SA2/cell28: ZZ is done and successfully fixed.
SPB:
- Nothing to report.
FXE:
- Drop SASE Intensity when changing photon energy from FXE side: current understanding is that in some way the phase shifters are not set properly if the wavelength change is triggered from the Karabo.
- Wavelength changes/scans work properly when done by BKR.
- Test this afternoon from both sides (BKR and FXE) to understand and solve the issue. Possible workaround is to ask BKR to operate the undulators.
MID:
- Not present.
HED:
- Calibration for Jungfrau had problems with MyMDC starting only Thursday. Needs to discuss with DATA department on this scheduling issue.
- Status of sample stages with EEE: being further investigated.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Not present. |
454
|
25 Sep 2023, 12:31 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 25.09.2023 |
General Announcement:
- Two severe power glitches happened on the weekend (SAT and SUN around 7AM). We thank all colleagues that were in shift, OCD or volunteered to help assessing the situation and restoring their operation
ability.
- The presentation of the Winter Maintenance planning period is postponed due to time issues.
- Janusz S. reminds on cyber security: tomorrow the next step to secure Zimbra mail client takes place. A detailed e-mail will be sent.
- XO will create a report on the power issues.
CTRL:
- Issues with power glitch: restoring parameters, Karabo, SEPS, SA2 started from scratch.
DA:
- Nothing to report; end of chain regarding the power issues.
EEE:
- Same issues with power.
- Will follow up the SCS train ID.
VAC:
- No effect on VAC; no access to SA2 then it was blind / no Karabo, but at the end was fine.
- Issue with HED and MID Monochromator - to be investigated. Access is needed today.
XPD:
- As PRC Jan thanks all people who helped to restore SA and SUN (two power glitches around 7AM); Also Desy/MKK confirmed that machines were briefly affected. EuXFEL accelerator recovered quickly.
- Consequences on SAT: 1. Power went away of whole SA3 area; 2. Tunnels mostly not affected but water cooling pumps were switched OFF (affecting also balcony rooms).
One auxiliary pump did not turned ON and several hours later (10-11AM this was too hot) and brought down services. SA2 tunnel cooling water temperature raised but on SUN it was faster to act/recover (MKK;
30mins).
- XGM at XTD6 went down.
- MKK does not see the temperature but only humidity. To be implemented in the coming days.
- SA2 Karabo totally down.
- Interlocks went down (EEE, D3, SRP) and had to unblock radiation alarms. A Desy mTCA crate went down and had to be manually power cycled (due to cooling water lack at balcony room).
- PP-laser in SA3 went down. SA3 experiment at SCS was cancelled at the point.
DET:
- No specific feedback for DET; shall be OK.
- Broker pile up was not fully understood. SA2 was restarted and back to normal; MID restarted yesterday and will get in touch with them.
ITDM:
- Cooling issue: discovered by alarm (Kimon who informed MKK); it needs a better procedure to inform MKK.
- After 30 mins called from MKK saying T was going down, but due to open doors.
- Steffen H. says a report has to be done with goal oriented to avoid this in future.
XRO:
- Looking into systems: in tunnels, if power is OFF then mean cooling shall be OFF. Cameras do not work remotely (connection to servers; only one is accessible).
LAS:
- SA3 is down: problems with main amplifier;
- Checking for damage after second power glitch.
UNSYS:
- All fine.
SPB:
- Nothing to report. User got data.
FXE:
- SA1 was also affected by power glitch (interlock crashed): had to be reset by D3; AIBS went to error and be reset by DOC.m
- Prevented continued with user (lost 6+4 hours of beamtime). User went well and collected data.
- On Saturday the AMPHOS laser broke down in FXE. SRP and Management took measure to bypass. Not sure if caused by electricity. To be followed up this week.
MID:
- Nothing to report. Request: in such severe power glitches an e-mail to MID has also to be sent.
HED:
- Could recover after SA2 Karabo broke down. Recovery faster than expected.
- Continued with user program. Data was successful.
SCS:
- Stopped running experiment.
- Interlock problem with laser helped by Pouneh (thanks!).
SQS:
- Nothing to report.
SXP:
- Challenging communication. Manuel gave phone number to PRC and will give to XO.
- Power glitch: Desy colleagues do not have access to SXP hutches. |
455
|
27 Sep 2023, 10:30 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 27.09.2023 |
General Announcement:
- The Winter Maintenance Task list will be sent to all groups in the course of today. Next Wednesday 04.10 Martin Doering will make a quick presentation on the list itself and take questions.
Two minor upgrades are done on it: 1. Location places are not divided in up to 3 sub-sections/levels; 2. Services needed and blocked are now condensed in two columns with color code related to the chosen
service.
The deadline for submission is October 20th.
CTRL:
- Nothing to report.
DA:
- No relevant updates.
EEE:
- Continued with work recovering systems after power cut.
- Patrick G. points that a clear information on the infrastructure of e.g. power, water, has to be made available to groups.
VAC:
- Not present.
XPD:
- Nothing to report.
- As last PRC, J. Gruenert adds that to protect CRLs in SA1 a watchdog is being tested. F. Wolff-Fabris adds that XO will discuss it with instruments.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Mikako M. says that the test of the SA1 CRL watchdog was done yesterday night; some bugs were found and a fix needed to be done. Watchdog is in principle ready but should be still tested. However, it is
not yet a fail proof device.
- Steffen H. points that safety relevant devices that are not fail proof shall not be deployed. In addition, for longer use: one may be worthwhile to implement a dedicated device.
- XO states that the tested watchdog is not going to be placed in operation now.
LAS:
- Not present.
UNSYS:
- All fine.
SPB:
- Issue with PP-laser affected preparations.
FXE:
- Issues with AMPHOS shutter were fixed yesterday and now it runs properly.
- Progress was done with night work.
MID:
- AGIPD detector issue is in DET hands.
HED:
- Nothing to report.
SCS:
- Pointing had issues (drift) being root cause for instabilities last night. Right now beam performance looks better than the last night, after BKR intervention.
SQS:
- All good.
SXP:
- Not present. |
456
|
02 Oct 2023, 09:59 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 02.10.2023 |
General Announcement:
-
CTRL:
- Deploy dataloggers fix today; it takes around 5 minutes.
DA:
- Couple of OCD calls over the week that could be all addressed.
EEE:
- PLC team checking motors in SQS to test issue.
- uTCA team: 1. replace a CPU at MID (possible); 2. digitizers at HED will be replaced by uTCA team to improves setup;
- Also install power supply at SQS.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Last week started bumpy but operation stabilized later on.
ITDM:
- Nothing to report.
XRO:
- XTD1 attenuator: one arm is blocked; after speaking with SA1 instruments the decision is to wait to fix during the winter maintenance. In case of urgencies to use it please contact Maurizio V.
LAS:
- Issue with FXE laser: will need CTRL to help debug;
- AMPHOS issues at PP-lasers are solved.
UNSYS:
- Nothing to report; all fine.
SPB:
- Test with machine tomorrow / taking beam.
FXE:
- Nothing to report.
MID:
- Nothing to report.
HED:
- Nothing to report.
SCS:
- Not present.
SQS:
- Nothing to report.
SXP:
- Nothing to report. |
457
|
04 Oct 2023, 09:52 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 04.10.2023 |
General Announcement:
- A. Galler reminds this is a facility development week and for the next UOM on Friday a report of stability/jitter shall be done by the instruments.
- Martin Doering announces the Winter maintenance period 23/24:
1. The new maintenance task list was sent to the groups last week;
2. Deadline to submitting back the filled list is Oct.20th; release takes place on Nov. 16th and maintenance period starts on Nov.30th.
3. Locations can be chosen with dropdown menu in a 3-step process;
4. Services needed and blocked are condensed in two dropdown fields;
5. Submit task list to xo-maintenance@xfel.eu; for questions please address xo-planning@xfel.eu.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- PLC team working in SQS Elmo motor drive.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Nothing to be reported.
ITDM:
- Nothing to report.
XRO:
- Nothing major to report.
LAS:
- Nothing to report; PP-lasers survived the power glitches.
UNSYS:
- Nothing to report.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Question to EEE on the timing crate: it seems to be fine now.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Issue with 1-bunch digitizer (uTCA in rack room): SQS will contact EEE.
SXP:
- Not present. |
459
|
11 Oct 2023, 10:15 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 11.10.2023 |
General Announcement:
- Reminder that next week, on 20.10, is the deadline to submit the Winter maintenance task lists to XO. In case of help, please contact Martin Doering at xo-planning@xfel.eu.
CTRL:
- Nothing to report.
DA:
- Nothing to report; all quiet at the DOC also.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- A Silicon oil leak at DET cleaning room happened: leak comes from Jungfrau cooling loop (built by external company); status is area was cleaned and leak point is being searched;
- AGIPD setup at SPB got a failure of Julabo chiller; it does not work properly and it does not cool as expected. Detector is interlocked and there was no problems for itself; to be further investigated.
ITDM:
- Gitlab was down for about 30 mins and now is available again.
XRO:
- Nothing to report.
LAS:
- Status of SA3 PP-laser: still investigating and in contact with AMPHOS. Still to be seen.
- Smaract in SA2 is fixed.
UNSYS:
- All good.
SPB:
- Issue with AGIPD chiller. Installed yesterday mid-day and was cooling slowly but not reaching base temperature.
FXE:
- Handover laser to SQS.
MID:
- Nothing to report.
HED:
- Not present.
SCS:
- In contact with EEE (Bernard) regarding the commissioning of our additional local box for the THz experiment.
SQS:
- Not present.
SXP:
- Nothing to report; backlash of M6 seems to be fine. |