ID |
Date |
Author |
Location |
Severity |
Subject |
471
|
22 Nov 2023, 10:39 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 22.11.2023 |
General Announcement:
-
CTRL:
- Nothing to report.
DA:
- Issue reported on darks processing failure for data processing (Maxwell processes); ITDM and Desy is following.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Nothing to report.
ITDM:
- Nothing to report;
- Afternoon DOC member will look into ITDM reported issue.
XRO:
- Nothing to report; As PRC Maurizio says all fine.
LAS:
- Michael P. says a meeting with company is taking place and news will be brought to LAS.
UNSYS:
- SA3 cell 2 power terminal distribution has to be exchanged in ZZ;
- SA2 cell 37 problem with quadrupole mover in error state and top girder driver.
SPB:
- Thanks to SCS for helping with the chiller.
FXE:
- Nothing to report.
MID:
- Not present.
HED:
- Chamber contamination issue reported on Monday is closed. No contamination happened.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Not present. |
470
|
20 Nov 2023, 10:01 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 20.11.2023 |
General Announcement:
-
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- AGIPD SPB still offline with issues on Julabo chiller.
- Adam R. adds the AGIPD is not mandatory for the coming week of operation. Jola S.-D. adds they are in touch with SPB and the issue will be further investigated after run is ended.
In addition, AGIPD is let out of beam for this week's operation.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Fire alarm on Friday brought down all Lasermet systems (SA3 PP-laser and E.0 systems). Needs further investigation. PP-L is not needed for the current week.
- Fire alarm was due to smoke detectors triggering it accidentally
UNSYS:
- Nothing to report.
SPB:
- Nothing to report.
- Adam R. adds AGIPD detector may not be needed this week but it is good for checking.
FXE:
- Affected by fire alarm (timing sync down).
MID:
- Nothing to report.
HED:
- Experiment with PP-laser on Sat night faced a drop in pp-laser energy (not clear from LAS why);
- Users using hazards samples let contamination at chamber. Needs to be addressed with SRP.
- Covid-19 impacted personnel.
SCS:
- Nothing to report.
SQS:
- Since laser went down, the R&D was impacted.
SXP:
- Not present. |
469
|
15 Nov 2023, 09:48 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 15.11.2023 |
General Announcement:
- As PRC, A. Galler informs the machine runs without major issues but kicker adaptation has difficulties with 4.5MHz. For the next 20 mins machine colleagues will introduce stronger kickers and during
this time beam will be jittery at SA1/SA3. SA2 shall be unaffected.
During the meeting PRC received a call from BKR informing the intervention was done at 09:45 and stable beam is currently available.
- Martin Doering announces on the Winter Maintenance period:
1. The release of Maintenance Task List will happen tomorrow 16.11 and maintenance starts on 30.11;
2. Major topics:
Karabo (4-7.12 and 13.12),
DAQ and CTRL network interruptions (4-8.12),
Cooling water has no big interruptions planned apart of the regular maintenance with fluctuations in flow and temperature,
Electrical maintenance in XHQ (4-22.12) with rooms affected in different days,
electrical maintenance at XHEXP1 in 27.11-22.12,
Timing missing, smoke detector tests, laser interlock upgrades,
interlock test "light" (17.01);
3. Please check for precise dates on above at the Maintenance Task List.
CTRL:
- No announcements for CTRL.
DA:
- Nothing to report.
EEE:
- Nothing to mention.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Nothing for DET and for DRC (Monica T.).
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Nothing to report.
SPB:
- Nothing to report apart of the Kicker issues.
FXE:
- Issues with Kicker as mentioned by the PRC.
MID:
- Nothing to report.
HED:
- Nothing to report. Tomorrow the switch to 24keV will take place and time has to be allocated for tuning.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Not present. |
468
|
13 Nov 2023, 10:16 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 13.11.2023 |
General Announcement:
- This and next week of operation are the last 2 delivery weeks, which are done with 4.5MHz base rep rate.
CTRL:
- Announcement: today IT will make an intervention in one of the hosts (204) for the DOOCS topic; downtime is expected and colleagues using cloning devices in this host (DOOCS requester to clone DOOCS)
have to be refreshed and restarted.
DA:
- Nothing to report.
EEE:
- Incident that SCS motor not correctly moving during the weekend. PLC-OCD could not be reach due to phone issues; issue with motor was solved
VAC:
- ZZ on Thursday in XTD2 to exchange turbo pump was successful.
XPD:
- SA1 studies with 1nC was done in SA3 and not in SA1;
- 30keV went well on Wednesday but with Nepal-X no lasing was found on Friday.
DET:
- Nothing to report.
ITDM:
- Nothing to add apart of CTRL report. An e-mail will be sent when it is done.
XRO:
- A. Trapp says the issue with SA2 motors moving slowly was fixed now after a patch was deployed.
LAS:
- Nothing to report.
UNSYS:
- ZZ last week for SA1 cell20 PS motor being replaced (damaged encoder). Many spare encoders are in house.
SPB:
- Nothing to report.
FXE:
- Preparing for in-house experiment.
MID:
- All fine at MID.
HED:
- 2 experiments this week at 18 and 24keV.
SCS:
- Not present.
SQS:
- Nothing to report.
SXP:
- Preparing experiments at 4.5MHz. |
467
|
08 Nov 2023, 09:55 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 08.11.2023 |
General Announcement:
-
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- As PRC, Raul V. informs on a small issue at Hirex-Gotthard with number of pulses yesterday. Promptly solved in few minutes.
XPD:
- No issues and supporting the ongoing blue week with 30keV in SA2.
- Access happened on Monday in XTD2 to fix the solid attenuator.
DET:
- Nothing to report.
ITDM:
- Monday intervention (announced in e-mail to xo-info) went OK and systems are recovered.
XRO:
- Solid Attenuator in SA1 temporarily solved: middle layer device calculating transmission shall be working; the automatic interlock is not yet working and a new interlock logic will be deployed.
Please instruments in SA1 do that manually. Arm 5 and Arm 4 are currently disabled.
LAS:
- Issue with 4.5MHz operation in SA3 but it was fixed.
UNSYS:
- SA1/cell20 phase shifter shows a problem and a ZZ will happen tomorrow morning.
SPB:
- Laser interlock issue happening yesterday evening was solved (signal of a power glitch happened yesterday evening).
FXE:
- Nothing to report.
MID:
- Seems that short pulses studies at MID were successful.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Nothing to report. |
466
|
06 Nov 2023, 09:53 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 06.11.2023 |
General Announcement:
- As PRC, Jan G. reports that on Saturday morning no beam permission at the North branch happened. An issue with XTD2 solid attenuator vacuum happened and together with VAC-OCD the threshold was changed
to allow continuation of beam delivery. A ZZ will be needed today to fix the vacuum leak;
On Sunday around 09:20 an issue with accelerator quadrupole magnet in A21 caused downtime initially until noon (SA2) and around 16:00 (North branch); also D3 had to come onsite to reset burn through
monitors in the aftermath.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- MID had couple of uTCA cards offline that needs to be rebooted. This will be discussed with MID offline.
VAC:
- Preparing for the needed ZZs in XTD2.
XPD:
- Nothing to report.
DET:
- Couple of tasks for the DSSC (SQS) to reconfigure the detector;
- DAQ of DSSC was affected by failure of GPFS servers.
ITDM:
- GPFS issue happened last Friday and was fine over the weekend.
- An update of the clients will happen today in all SASEs. This was agreed with instruments.
XRO:
- Nothing to add to what was reported by the PRC.
LAS:
- SA2 issue with pulse on demand will be discussed with MID.
UNSYS:
- Nothing to report.
SPB:
- User beamtime went OK even with downtimes during weekend.
FXE:
- Nothing to report; last night shift (Sunday night) cancelled as program was finished on Saturday.
MID:
- Supporting today the short pulses activities; from tomorrow interventions at MID can be scheduled.
- Regarding laser issue: when changing sequence for using Laser showed trains with pulses and trains with missing pulses. Running macro was changed for working to 1 pulse per second.
- Users were happy and the program was accomplished in full.
HED:
- Not present.
SCS:
- Detector tests to be done by end of the week.
SQS:
- Affected by beam losses but overall a lot of useful data was acquired.
SXP:
- Nothing to report. |
465
|
01 Nov 2023, 10:04 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 01.11.2023 |
General Announcement:
- Jan G. as PRC comments: 1. reminds SQS and FXE instruments to place entries at the Operations e-log;
2. machine is running well and ZZ accesses happened on Monday (hindered by issues with tunnel entrance) but all done;
3. the XTD2 issue with upstream differential pump close to the XGM: one pump is defective and VAC will exchange if a ZZ is allowed next week.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- FSSS motor in MID: OCD received a call yesterday and device was restarted and came back to operation.
VAC:
- Not present.
XPD:
- Nothing to report from XPD.
DET:
- Nothing to be reported.
ITDM:
- Nothing to report.
XRO:
- On the XTD10 ZZ last Monday: V-slit was checked and a faulty cable for the encoder was found. For the time being this was fixed by tapping it and will continue monitoring.
LAS:
- Nothing to report.
UNSYS:
- All fine with Undulator.
SPB:
- Nothing to report.
FXE:
- Beam was not stable the last night / down for few times. To be investigated internally at FXE and reported to XO.
MID:
- Roll motor for M2 mirror at tunnel was moving very slowly; to be seen with XRO.
HED:
- Nothing to report.
SCS:
- Nothing to report; working in hutch preparations.
SQS:
- Nothing to report.
SXP:
- Not present. |
464
|
30 Oct 2023, 09:47 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 30.10.2023 |
General Announcement:
-
CTRL:
- No announcements for today.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Issue with turbo pump in XTD2 (happened 22:30 yesterday). Getting in touch the PLC team. Maybe a ZZ is needed today.
XPD:
- Nothing to report.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Open points from last week: vertical slit SA3 bottom (error); SRA in SA1 is currently in error. To be followed up if fixed or not.
LAS:
- Nothing to report.
UNSYS:
- Yesterday night problems in SA1 (XTD2): cell 2 motor drive problems; cell 11 power distribution terminal in cabinet; cell 14 bridge terminal problem; errors from cell#31 to downstream segments.
- Yesterday night possible to bring system operational at night; ZZ is ongoing to address above points. Extension of time for ZZ is unclear.
- Problems with central control node also observed.
SPB:
- Nothing to report.
FXE:
- Not present.
MID:
- Nothing to report.
HED:
- Beamtime good.
SCS:
- Nothing to report.
SQS:
- Nothing to report.
SXP:
- Saturday lost synchronization with both lasers: promptly solved by PRC and LAS-OCD. |
463
|
25 Oct 2023, 09:26 |
Andreas Galler | General | Information | Notes XO Integration Meeting 25.10.2023 |
General Announcement:
CTRL:
- nothing major --> a couple of people on vacation, i.e. issue repsonse time may be a bit longer this week
DA:
EEE:
VAC:
XPD:
DET:
ITDM:
XRO:
LAS:
UNSYS:
SPB/SFX:
- SA3 background affecting focussing of x-rays
FXE:
- smooth, beam instability yesterday
- horizontal jitter entagled with PopIn measurements --> around lunch time
- beam was clipping and LPD was seeing poweder scattering from the collimator pipe
MID:
HED:
- Yesterday evening unexpected venting in the CRL3 (HED-Optics Hutch) --> X-ray Operation cancelled at least for Wed night
SCS:
- beam instability in pointing over night
- this cuased non optimal beam transport --> pulse energy decreased on sample
SQS:
SXP:
DRC
PRC
- SA1 pulse energy will be lower than possible, because of XGM settings that need to cater 7.8 keV and 12.5 keV
|
462
|
23 Oct 2023, 09:26 |
Andreas Galler | General | Information | Notes XO Integration Meeting 23.10.2023 |
General Announcement:
CTRL:
DA:
EEE:
VAC:
XPD:
DET:
ITDM:
XRO:
LAS:
- SA3 PPLaser up and running
- SA1 PPLaser went down from time to time
- Windows PC crashed
- instabilities in NOPA
UNSYS:
SPB/SFX:
FXE:
MID:
HED:
SCS:
SQS:
SXP:
DRC
PRC
- 2 long down times over the weekend
- from 7 to 12:30 fast valve in BC
- Sun 10:30 - 19:00 broken network switch
|
461
|
18 Oct 2023, 10:01 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 18.10.2023 |
General Announcement:
- Next Friday 20th Oct. is the deadline for submitting the Winter maintenance task list back to XO.
CTRL:
- Nothing to report.
DA:
- Nothing major to report.
EEE:
- Nothing to report.
VAC:
- Nothing from VAC.
- As PRC, Martin D. informs on the issue with the Self-seeding monochromator: X-axis is not operational. A ZZ is needed today by UNSYS. To be seen the HXRSS program.
XPD:
- Not present.
DET:
- Nothing to be reported.
- Jungfrau calibration and DAQ: there is nothing more from DET to be done; P. Schmidt adds that operation modes have to be adapted and there is work on progress in long term.
ITDM:
- Nothing to add.
XRO:
- Nothing to report.
LAS:
- Technician from AMPHOS arrived yesterday and work in ongoing with SA3 PP-laser; no final conclusion yet.
UNSYS:
- All day working with the SA2 monochromator yesterday (X-axis and roll axis). It looks the X-axis is stuck or broken and in a ZZ an attempt to recover with a manipulator will be done.
Chances are not high for a fix and if the worst-case scenario, the HXRSS for the remaining of the year may be cancelled.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Busy at beamline and worried with the HXRSS program.
HED:
- All fine at HED.
SCS:
- No outcome form PP-laser; getting a temporary laser from SXP (1030nm) and working to make it for the beamtime (no THz pump probe experiment will be possible).
SQS:
- Issue with digitizers (Smaract and PES); Being followed by EEE as was informed by DOC. For the PES it was discussed with XPD and needs further follow-up with SQS and XPD.
In general, please contact DOC in case of operational issue or write EEE ticket if not an operational issue.
- High voltage power supply not responding at Karabo; being investigated by CTRL and will be coordinated with SQS for a fix.
SXP:
- Not present. |
460
|
16 Oct 2023, 09:46 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 16.10.2023 |
General Announcement:
- Last X-ray delivery week went normal.
- Friendly reminder for the submission of the maintenance task list on 20.10, next Friday. In case of questions please do not hesitate in contacting XO.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- ZZ on XTD2 today at 13:00 to exchange XGMD amplifiers.
DET:
- Investigating the settings of detector with FXE.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- SA3 AMPHOS engineer is at XFEL today. Intense work will be made to have PP-laser back by the end of the week - to be seen in coming hours/days.
- In SA1 PP-laser was recovered on Friday.
UNSYS:
- Nothing to report.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Nothing to report.
- Naresh K. will interact with MID to align Hirex for the HXRSS operation this week.
HED:
- Nothing to report.
SCS:
- Needs PP-laser for the beamtime starting on THU.
SQS:
- SQS broke interlock at 6AM and took away beam permission.
SXP:
- Takes beam tomorrow and Wednesday; it will use another laser if SA3 PP-laser is not available. |
459
|
11 Oct 2023, 10:15 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 11.10.2023 |
General Announcement:
- Reminder that next week, on 20.10, is the deadline to submit the Winter maintenance task lists to XO. In case of help, please contact Martin Doering at xo-planning@xfel.eu.
CTRL:
- Nothing to report.
DA:
- Nothing to report; all quiet at the DOC also.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- A Silicon oil leak at DET cleaning room happened: leak comes from Jungfrau cooling loop (built by external company); status is area was cleaned and leak point is being searched;
- AGIPD setup at SPB got a failure of Julabo chiller; it does not work properly and it does not cool as expected. Detector is interlocked and there was no problems for itself; to be further investigated.
ITDM:
- Gitlab was down for about 30 mins and now is available again.
XRO:
- Nothing to report.
LAS:
- Status of SA3 PP-laser: still investigating and in contact with AMPHOS. Still to be seen.
- Smaract in SA2 is fixed.
UNSYS:
- All good.
SPB:
- Issue with AGIPD chiller. Installed yesterday mid-day and was cooling slowly but not reaching base temperature.
FXE:
- Handover laser to SQS.
MID:
- Nothing to report.
HED:
- Not present.
SCS:
- In contact with EEE (Bernard) regarding the commissioning of our additional local box for the THz experiment.
SQS:
- Not present.
SXP:
- Nothing to report; backlash of M6 seems to be fine. |
458
|
09 Oct 2023, 09:23 |
Andreas Galler | General | Information | Notes XO Integration Meeting 09.10.2023 |
General Announcement:
CTRL:
DA:
EEE:
VAC:
XPD:
DET:
ITDM:
XRO:
- miror setup tuned for grazing incidence with high photon energy.
- higher backlash at M6, XRO is monitoring this.
LAS:
- SA2 treacy compressor smaract stage not operational --> EEE informed
- SA3 AMPHOS still suffering from low puse energy
UNSYS:
SPB/SFX:
FXE:
MID:
HED:
SCS:
SQS:
- uTCA in rack room --> EEE will be informed
SXP:
DRC
PRC
|
457
|
04 Oct 2023, 09:52 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 04.10.2023 |
General Announcement:
- A. Galler reminds this is a facility development week and for the next UOM on Friday a report of stability/jitter shall be done by the instruments.
- Martin Doering announces the Winter maintenance period 23/24:
1. The new maintenance task list was sent to the groups last week;
2. Deadline to submitting back the filled list is Oct.20th; release takes place on Nov. 16th and maintenance period starts on Nov.30th.
3. Locations can be chosen with dropdown menu in a 3-step process;
4. Services needed and blocked are condensed in two dropdown fields;
5. Submit task list to xo-maintenance@xfel.eu; for questions please address xo-planning@xfel.eu.
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- PLC team working in SQS Elmo motor drive.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Nothing to be reported.
ITDM:
- Nothing to report.
XRO:
- Nothing major to report.
LAS:
- Nothing to report; PP-lasers survived the power glitches.
UNSYS:
- Nothing to report.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Question to EEE on the timing crate: it seems to be fine now.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Issue with 1-bunch digitizer (uTCA in rack room): SQS will contact EEE.
SXP:
- Not present. |
456
|
02 Oct 2023, 09:59 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 02.10.2023 |
General Announcement:
-
CTRL:
- Deploy dataloggers fix today; it takes around 5 minutes.
DA:
- Couple of OCD calls over the week that could be all addressed.
EEE:
- PLC team checking motors in SQS to test issue.
- uTCA team: 1. replace a CPU at MID (possible); 2. digitizers at HED will be replaced by uTCA team to improves setup;
- Also install power supply at SQS.
VAC:
- Nothing to report.
XPD:
- Nothing to report.
DET:
- Last week started bumpy but operation stabilized later on.
ITDM:
- Nothing to report.
XRO:
- XTD1 attenuator: one arm is blocked; after speaking with SA1 instruments the decision is to wait to fix during the winter maintenance. In case of urgencies to use it please contact Maurizio V.
LAS:
- Issue with FXE laser: will need CTRL to help debug;
- AMPHOS issues at PP-lasers are solved.
UNSYS:
- Nothing to report; all fine.
SPB:
- Test with machine tomorrow / taking beam.
FXE:
- Nothing to report.
MID:
- Nothing to report.
HED:
- Nothing to report.
SCS:
- Not present.
SQS:
- Nothing to report.
SXP:
- Nothing to report. |
455
|
27 Sep 2023, 10:30 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 27.09.2023 |
General Announcement:
- The Winter Maintenance Task list will be sent to all groups in the course of today. Next Wednesday 04.10 Martin Doering will make a quick presentation on the list itself and take questions.
Two minor upgrades are done on it: 1. Location places are not divided in up to 3 sub-sections/levels; 2. Services needed and blocked are now condensed in two columns with color code related to the chosen
service.
The deadline for submission is October 20th.
CTRL:
- Nothing to report.
DA:
- No relevant updates.
EEE:
- Continued with work recovering systems after power cut.
- Patrick G. points that a clear information on the infrastructure of e.g. power, water, has to be made available to groups.
VAC:
- Not present.
XPD:
- Nothing to report.
- As last PRC, J. Gruenert adds that to protect CRLs in SA1 a watchdog is being tested. F. Wolff-Fabris adds that XO will discuss it with instruments.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Mikako M. says that the test of the SA1 CRL watchdog was done yesterday night; some bugs were found and a fix needed to be done. Watchdog is in principle ready but should be still tested. However, it is
not yet a fail proof device.
- Steffen H. points that safety relevant devices that are not fail proof shall not be deployed. In addition, for longer use: one may be worthwhile to implement a dedicated device.
- XO states that the tested watchdog is not going to be placed in operation now.
LAS:
- Not present.
UNSYS:
- All fine.
SPB:
- Issue with PP-laser affected preparations.
FXE:
- Issues with AMPHOS shutter were fixed yesterday and now it runs properly.
- Progress was done with night work.
MID:
- AGIPD detector issue is in DET hands.
HED:
- Nothing to report.
SCS:
- Pointing had issues (drift) being root cause for instabilities last night. Right now beam performance looks better than the last night, after BKR intervention.
SQS:
- All good.
SXP:
- Not present. |
454
|
25 Sep 2023, 12:31 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 25.09.2023 |
General Announcement:
- Two severe power glitches happened on the weekend (SAT and SUN around 7AM). We thank all colleagues that were in shift, OCD or volunteered to help assessing the situation and restoring their operation
ability.
- The presentation of the Winter Maintenance planning period is postponed due to time issues.
- Janusz S. reminds on cyber security: tomorrow the next step to secure Zimbra mail client takes place. A detailed e-mail will be sent.
- XO will create a report on the power issues.
CTRL:
- Issues with power glitch: restoring parameters, Karabo, SEPS, SA2 started from scratch.
DA:
- Nothing to report; end of chain regarding the power issues.
EEE:
- Same issues with power.
- Will follow up the SCS train ID.
VAC:
- No effect on VAC; no access to SA2 then it was blind / no Karabo, but at the end was fine.
- Issue with HED and MID Monochromator - to be investigated. Access is needed today.
XPD:
- As PRC Jan thanks all people who helped to restore SA and SUN (two power glitches around 7AM); Also Desy/MKK confirmed that machines were briefly affected. EuXFEL accelerator recovered quickly.
- Consequences on SAT: 1. Power went away of whole SA3 area; 2. Tunnels mostly not affected but water cooling pumps were switched OFF (affecting also balcony rooms).
One auxiliary pump did not turned ON and several hours later (10-11AM this was too hot) and brought down services. SA2 tunnel cooling water temperature raised but on SUN it was faster to act/recover (MKK;
30mins).
- XGM at XTD6 went down.
- MKK does not see the temperature but only humidity. To be implemented in the coming days.
- SA2 Karabo totally down.
- Interlocks went down (EEE, D3, SRP) and had to unblock radiation alarms. A Desy mTCA crate went down and had to be manually power cycled (due to cooling water lack at balcony room).
- PP-laser in SA3 went down. SA3 experiment at SCS was cancelled at the point.
DET:
- No specific feedback for DET; shall be OK.
- Broker pile up was not fully understood. SA2 was restarted and back to normal; MID restarted yesterday and will get in touch with them.
ITDM:
- Cooling issue: discovered by alarm (Kimon who informed MKK); it needs a better procedure to inform MKK.
- After 30 mins called from MKK saying T was going down, but due to open doors.
- Steffen H. says a report has to be done with goal oriented to avoid this in future.
XRO:
- Looking into systems: in tunnels, if power is OFF then mean cooling shall be OFF. Cameras do not work remotely (connection to servers; only one is accessible).
LAS:
- SA3 is down: problems with main amplifier;
- Checking for damage after second power glitch.
UNSYS:
- All fine.
SPB:
- Nothing to report. User got data.
FXE:
- SA1 was also affected by power glitch (interlock crashed): had to be reset by D3; AIBS went to error and be reset by DOC.m
- Prevented continued with user (lost 6+4 hours of beamtime). User went well and collected data.
- On Saturday the AMPHOS laser broke down in FXE. SRP and Management took measure to bypass. Not sure if caused by electricity. To be followed up this week.
MID:
- Nothing to report. Request: in such severe power glitches an e-mail to MID has also to be sent.
HED:
- Could recover after SA2 Karabo broke down. Recovery faster than expected.
- Continued with user program. Data was successful.
SCS:
- Stopped running experiment.
- Interlock problem with laser helped by Pouneh (thanks!).
SQS:
- Nothing to report.
SXP:
- Challenging communication. Manuel gave phone number to PRC and will give to XO.
- Power glitch: Desy colleagues do not have access to SXP hutches. |
453
|
20 Sep 2023, 10:04 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 20.09.2023 |
General Announcement:
- The access and maintenance day is taking place today. Current ZZs at the accelerator are scheduled until 10AM and then it follows the tuning for cw38 beam delivery that starts from tomorrow at 7AM.
- Martin Doering announces that next Monday 25.09 a short presentation on the 2023 Winter Maintenance will be made in this meeting series.
Please a representative from every group is welcome to be present. The Maintenance task list is going to be send to instrument and support groups next week.
- As PRC, Jan reports so far the week looks good. Downtime from Mond-Tue of 1:15 which is being looked today at tunnel (LINAC);
- J. Gruenert (as PRC) reminds instruments to post at least once a day a report at the Operations e-log; In addition, for SA1, it is introduced a power limit of 5W to protect CRL1 lens when operating low
photon energies (this will be discussed with SPB/SFX).
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Nothing to report;
- Reminder that forbidden XGM ranges (SPB/SFX): in SA1 the two XGMs do not have the same gas. Upstream has Kr to be able to run 5.5keV; SPB/SFX can run with 7keV at this XGM and their own on with Xe
instead;
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Not present.
UNSYS:
- Quad issue in SA2/cell28: ZZ is done and successfully fixed.
SPB:
- Nothing to report.
FXE:
- Drop SASE Intensity when changing photon energy from FXE side: current understanding is that in some way the phase shifters are not set properly if the wavelength change is triggered from the Karabo.
- Wavelength changes/scans work properly when done by BKR.
- Test this afternoon from both sides (BKR and FXE) to understand and solve the issue. Possible workaround is to ask BKR to operate the undulators.
MID:
- Not present.
HED:
- Calibration for Jungfrau had problems with MyMDC starting only Thursday. Needs to discuss with DATA department on this scheduling issue.
- Status of sample stages with EEE: being further investigated.
SCS:
- Nothing to report.
SQS:
- Not present.
SXP:
- Not present. |
452
|
18 Sep 2023, 10:01 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 18.09.2023 |
General Announcement:
- Differently from standard Mondays, today is a beam delivery day. The machine tuning day for cw38 takes place on Wednesday.
CTRL:
- Nothing to report.
DA:
- Quiet OCD weekend with two calls.
EEE:
- Nothing to report.
VAC:
- Friday afternoon SPB found pressure increase at SA1 solid attenuator which triggered vacuum signal. Investigation showed a bellow of one attenuator arm (arm 5; 1.2mm diamond) leaking and organized tunnel
access;
- VAC will discuss with XRO how to monitor possible damage and possibly modify middle-layer device.
XPD:
- Involved on Friday with XGM triggering alarm due to vacuum issue in XTD2.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to add to the solid attenuator issue.
LAS:
- Nothing to report.
UNSYS:
- Nothing to report.
SPB:
- Not present.
FXE:
- Preparing for beamtime.
MID:
- Nothing to report.
HED:
- Troubles with sample stages being addressed with EEE.
SCS:
- Not present.
SQS:
- Not present.
SXP:
- Not present. |