ID |
Date |
Author |
Location |
Severity |
Subject |
297
|
02 Feb 2022, 08:39 |
Andreas Galler | General | Information | Notes XO Integration Meeting 02.02.2022 |
General Announcement:
CTRL:
- observation from DOC
- macros some times after the upgrade to Python 3.8 --> please contact CTRL?
DA:
- significant upgrade to Offline Analysis -->
- proc data won't inlcude a copy of raw data any longer
EEE:
VAC:
XPD:
- no incidents
- 2 tunnel accesses on Monday --> succesful
DET:
- DET commissioning at SPB (AGIPD JUNGFRAU 1M), MID (AGIPD), HED (ePIX, JUNGFRAU)
ITDM:
- E0.604 rack room, Wed next week
- fixing power distribution units in every rack --> TS and external company will need a full day
- affected services are: Lab DAQ, LAB control servers (EEE; CTRL, DET), GitLab runners, one machine for data analysis, PLC database, karabo project database (?)
- Yesterday short interruption with influx reading note --> test requested for next Monday causing a 1 hour interruption
XRO:
- SA1 / SA2 M3 PBLMs have been equipped with diamond plates and can be used for beam stability measurements
- Backlash Controls implemented last week --> values
- report of lost positions of vertical slit in SA3
- in case of similar issues please contact DOC prior to self fixing.
LAS:
- SA2 PPL is good
- SA3 PPL planned to be up today
- SA1 PPL being investigated as agreed with SA1 instruments
UNSYS:
- transparency test for APPLE X went successful
SPB/SFX:
- SPB/SFX aligned since yesterday, no issues
FXE:
- blown fuses in crates --> solved with EEE
MID:
HED:
- Com yesterday
- outstanding issue: fastADC not working EEE aware of that
SCS:
SQS:
- GATT always worked but very slowly responding
- OCD contacted
- Raul comments that this is a known issue and should be communicated to CTRL
- SH nesures that Raul creates the ticket
- XTD10_PES
- digitizer display frozen almost all time (connected to 2 DAQs) --> reported to DOC
- SH comments that when connecting to two DAQs (tunnel and SQS), should be connected to only one DAQ.
|
296
|
31 Jan 2022, 10:03 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 31.01.2022 |
General Announcement:
- On this current run, beam delivery to instruments will start on Tuesday at 07:00AM.
CTRL:
- New version of DeviceClone and Doocsml device will be updated today which contains a bug fix;
- Minor update on Karabo framework version on the datalogger machines will be performed today
- CTRL will proceed with the update today at 13:00 if they didn't hear any objection till 12:00. Intervention will not take longer than half an hour. No need to check afterwards.
DA:
- Nothing to report.
EEE:
- Nothing to report.
VAC:
- Notification on power interruptions on the weekend: it seems to have no impact but please check the systems.
- Zunaira A. says the AIBS looses connections to network during glitches and has to be restarted if not masked. She will take a look.
XPD:
- Nothing to report.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- No big issues during the weekend; smaller ones were solved.
LAS:
- SA2 PP-laser is ON;
- SA3 PP-laser will need tomorrow to fix final things.
- SA1 PP-laser: LAS has negotiated with instruments to further investigate more.
UNSYS:
- SA1 issue on Saturday: problem with CPU;
- Ongoing tests with Apple-X this morning.
SPB:
- Aligning instrument to take beam tomorrow.
FXE:
- Nothing to report.
MID:
- All fine at MID.
- MID did not look yet in the PP-laser but will do so tomorrow or Wednesday.
HED:
- Waiting for DAQ rollout. In contact already with DA.
- Zunaira A. reminds on the reply for scheduling the hutch search training. HED will inform after their afternoon group meeting.
SCS:
- Nothing to report. SCS will also make a schedule and will contact Zunaira A.
SQS:
- Nothing to report. |
295
|
26 Jan 2022, 10:46 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 26.01.2021 |
General Announcement:
- The cold compressor failed this morning and beam operation is currently stopped. We expect the beam operation for the alignment of beam systems will resume tomorrow morning (Thursday).
CTRL:
- Nothing to report.
DA:
- Nothing to report.
EEE:
- Closing last tasks for the maintenance.
VAC:
- All fine.
XPD:
- Issues with PES were solved by CTRL.
- Jan G. asks when beam would be aligned to M1 and M2 in SA1 and SA2? (see XRO report).
DET:
- Nothing to report.
ITDM:
- Work ongoing with company regarding Monday's report on offline storage system.
- Jan asks for the DAQ Karabo test with XPD. Janusz S. suggested to contact Partha who is on charge for this.
XRO:
- Ahead of schedule with alignment until this morning: SA1 and SA2 beam established in all experiments. SA1 beam on mirror but it needs to finish Bender calibration (nice beam at the end of the SA1
tunnel).
- Currently cryo is down and pushes back next days plans: Today planned schedule of tuning 17.2 keV in SA1 as well as Apple-X in SA3 is shifted to tomorrow.
If beam comes tomorrow morning SA2 is available for alignment. By end of the week one starts with SA3.
LAS:
- SA2 almost ready: it shall be finished by end of the day.
- SA3 Amphos maintenance is finished: maybe ready end of the week or start next week.
- SA1 is making progress: 1.1MHz operation shall be OK but 280kHz can be have some difficulties.
UNSYS:
- Power failure in SA1 central control node prevented communication to undulators during the evening. It was solved after power cycle.
SPB:
- Busy with aligning beamline.
FXE:
- Same as SPB; nothing else to report.
MID:
- All fine.
HED:
- Not present.
SCS:
- All fine.
SQS:
- Nothing to report. |
294
|
24 Jan 2022, 11:18 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 24.01.2022 |
General Announcement:
- Steffen H. states DOC starts in special operation mode 09:00-17:00 for the weekdays. Jour-fixe also starts today at 13:00. Instruments are welcome to join.
- Regarding DATA OCD numbers, DRC started last Saturday: it did not work in some cases: please do not call directly any other OCD number, just the DOC or DRC number;
- Over the weekend an issue the bunch number pattern happened. It is ongoing and has to be tested.
CTRL:
- Nothing to report.
DA:
- Tomorrow DA members are in satellite workshop of their group.
EEE:
- Two OCD calls for ELMO drivers which lost configurations: solved.
VAC:
- No issues.
XPD:
- Commissioning work during the weekend on SA3 tunnel (mainly): bunch pattern could not be changed in any SASE.
- Tickets were sent and people responded promptly.
- One MPOD crate was in error in SA3: a power cycle in XTD10 was done (helped by XRO) yesterday night and fixed.
- Commissioning of Imagers, PES in SASE3; hard X-ray PES in SA1 (missing some DAQ) started; MCP-detector; and Shimadzu in SA2.
DET:
- Nothing to report.
ITDM:
- Relating to last Wednesday maintenance: a re-cabling at Desy side occurs and after that it observed unstable behaviors on the network.
Maybe today or tomorrow one needs to make some tests leading to a break of GPFS operation (short interruption of migrating data from online to offline).
XRO:
- SA3 proceeding well and for SQS several mirror positions are saved.
- On Sunday there was a problem preventing to complete the SCS (control issues of translation stages of pre-mirrors). It was solved but happened a couple of times.
Please experiments in SASE3 be aware that current translation stages positions may differ by 1mm.
- XRO implemented two-steps backlash corrections in SA3 (as done before in SA1): to solve the different positions after moving mirror.
Maurizio V. informs that instruments shall not be surprised if the motor overshoot and then return to correct position.
- SA2 beam transport: SRA calibrated; beam arriving to HED Pop-in instrument; PBLM was commissioned.
- SA1: starts today. This week focus goes to SA1 and SA2 realignment after the SESP interlock tests.
LAS:
- SA2 PP-laser: issues with ADQ card and smarAct: with help from EEE and CTRL, is fine now.
- SA3 PP-laser: AMPHOS maintenance this week; it shall be ready for user run.
- SA1 PP-laser: updating this one; LAS is working hard.
UNSYS:
- Beam trough in SA3; Working to correct some kicks in Apple-X.
- BKR requested to give a feedback that system is giving signals to BKR. Working on it.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Not present.
HED:
- All fine.
SCS:
- Not present.
SQS:
- Nothing to report. |
293
|
19 Jan 2022, 11:00 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 19.01.2022 |
General Announcement:
- Machine status: SA2 is currently lasing @ 9keV and 2.5mJ. North Branch was closed yesterday and after a quick ZZ in XTD10 this morning the BBA and tuning will resume today and tomorrow.
- X-ray delivery tables for cw05 have been sent to the instruments and we thank to all that have already sent us a feedback.
- DOC will start next Monday. From MON-FRI from 9Am to 5PM. For the next weekend the photon staff in shift work shall send a ticket or only in case of urgent/critical matters call the DRC phone. The PLCs
are providing OCDs.
CTRL:
- Bassler cameras were migrated to Aravis library.
DA:
- Nothing to report.
EEE:
- ZZ is ongoing to check a faulty switch signal in XTD10.
VAC:
- Benoit R. is currently in the tunnel on the ZZ;
- VAC systems are all fine and ready to take beam.
XPD:
- Aravis library installed and tested; all went fine.
- XTD6 ongoing tasks are under progress and may be finished already at the end of the day. If progress is made accordingly to the plan, XTD6 can be closed tomorrow.
The new M-port crate was connected yesterday (upgrade of MCP). Almost all is finished; Wolfgang F. finishes small things.
DET:
- Nothing to report.
ITDM:
- Offline storage system update scheduled for today (Dcache affected). E-mail will be send when finished by end of the day.
XRO:
- Nothing to report; all maintenance activities completed.
LAS:
- As aftermath of cooling water and air conditioning issues during maintenance: problems with misalignment of all lasers; LAS has troubles to align them.
- For SA2, LAS is confident will be fine with alignment. The SA3 is a bit behind of SA2;
- SA1 presents the biggest misalignment after T and H shock - long pump pulse. The 280kHz test is now in secondary priority. LAS will report day-by-day and is not so confident will be ready for instruments
in February.
UNSYS:
- Finished to install air coils in Apple-X.
- SA2 phase shifter solved.
SPB:
- Nothing to report.
FXE:
- Interlock issues were fixed.
MID:
- All fine at MID.
HED:
- All fine at HED.
SCS:
- Nothing to report.
SQS:
- All good at SQS. |
292
|
17 Jan 2022, 10:41 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 17.01.2022 |
General Announcement:
- SEPS interlock tests are scheduled for this Friday January 21st; Zunaira A. needs XTD6 closed by the SEPS interlock at 09:00.
Martin D. will evaluate if it is possible to close XTD6 before Friday pending to finish the work (without closed vacuum we won't have beam in future).
- Machine is tuning SA1 and SA3 this week and XTD2/4 tunnels are opening today and tomorrow for finishing Apple-X works.
CTRL:
- Nothing to report.
DA:
- Preparing for start of operation; nothing out of ordinary.
EEE:
- Nothing to report.
VAC:
- Vacuum all is fine.
- Tunnels closures: XTD9 tomorrow afternoon; XTD6 on Friday or Thursday afternoon (see general announcement).
XPD:
- Camera library tests/migration together with CTRL (Aravis library) from today afternoon. Some cameras won't work for the update (5mins or bit more).
Plan is to do all cameras - please keep in touch with A. Koch if cameras are needed.
DET:
- Nothing major to report: preparing detectors for start of the run.
ITDM:
- Reminder on the storage system (offline side) update on 19.01: dcache won't be available for the whole day; connection between online and offline won't work also. An e-mail will be sent when activity I
done.
XRO:
- Next week it starts the photon beamline alignment. Next Friday one needs the SCS or SQS hutch interlocked over the weekend.
LAS:
- Nothing to report.
UNSYS:
- Issue with phase shifter in SA2: BKR forgot to use the right operation mode.
- Apple-X: waiting to enter tunnel today at 12:00 to install air coils and finalize hardware commissioning.
SPB:
- Controller was affected by power glitch; fix ongoing.
FXE:
- Interlock system problem was detected and under fix.
MID:
- All fine at MID.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Nothing to report. |
291
|
12 Jan 2022, 10:06 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 12.01.2022 |
General Announcement:
- SEPS interlock tests are scheduled for January 21st: starting early morning from 08:00 to 16:00 across all the SASEs;
In parallel D-3 tests the interlocks in all hutches including SXP. An e-mail to instruments was sent and a reply has to be directed to Zunaira A.
- Disabling SEPS interlock yesterday night was not necessary. Further discussion with A. Galler will follow.
CTRL:
- No planned updates.
DA:
- No updates to report.
EEE:
- Yesterday the SA1 vacuum update took place.
- MPS contacted Nerea J. on the SEPS interlocks issue: Nerea J. to be included in further discussions.
VAC:
- Nothing to report.
XPD:
- Preparing for tunnel closure: finishing Hard X-ray PES tests; works in XTD9 are completed.
- Works in XTD6 finishing EPS switches and installing upgrade M-ports in MCP (crate coming on Thursday and update interlock connection).
DET:
- Nothing to report.
ITDM:
- Remind on the Wednesday maintenance on the dCache and GPFS; Access to XFEL related data from Maxwell compute cluster is not guaranteed during the intervention. Please contact ITDM in case of conflicting
with it.
XRO:
- Not present.
LAS:
- Failed to upgrade the SASE1 front ends during Winter Maintenance (company did not come due to Corona regulations).
- 8D report for the Seeder shutdown issue at the start of the maintenance period: A. Galler informs that by the end of this week it shall be ready and distributed.
- Power supply connection: Max L. asks for XO support to follow up with TS.
UNSYS:
- Apple-X undulators are aligned and currently the intersection is being aligned together with air coils. Vacuum shall also be connected permitting to close tunnel tomorrow.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- Thanks to SPB group for helping with providing a flange.
HED:
- Apart from the SEPS all is fine.
SCS:
- Not present.
SQS:
- All fine. |
290
|
10 Jan 2022, 09:22 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 10.01.2022 |
General Announcement:
- The "Readiness" mailing list will be discontinued and substituted by the "xo-info@xfel.eu" list. This new list is owned by XO and please use it in future for sharing information to a broad audience.
- Next News on the Covid regulations will be communicated on Wednesday 12.01 or for the next Thursday's staff meeting.
- Please take attention to Zunaira A. e-mail on the radiation safety training. It shall be done until the end of January to avoid disruption in access rights.
CTRL:
- No major updates.
DA:
- Nothing to report.
EEE:
- SASE1 VAC loop will be update tomorrow including the PES system.
VAC:
- No issues to report. Continuing with small tasks being done at XTD2 and XTD6.
XPD:
- All continued tasks are fine, mainly XTD2, XTD6 and XTD9.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Carry on with maintenance works - SASE1 and SASE2 are in vacuum;
- XTD6 PBLM diamond to be exchanged this week as well as screens.
LAS:
- Not present.
UNSYS:
- Alignment for the Apple-X to be done this week and vacuum will be closed until Thursday.
SPB:
- Small issues with things moved into new racks; being followed with EEE.
FXE:
- Continuing with tasks.
MID:
- MID looks for a CL flange with SmartAct connection. Please contact MID or Joerg H. if you have one to help MID.
HED:
- All fine at HED.
SCS:
- Nothing to report.
SQS:
- Nothing to report. |
289
|
05 Jan 2022, 09:29 |
Andreas Galler | General | Information | Notes XO Integration Meeting 05.01.2022 |
General Announcement:
CTRL:
DA:
EEE:
- 1 MID plC dead
- cable work planned for next week
VAC:
- In general excellent progress
- closing of M2 XTD2 (SA1) in progress, sealing issues mitigated
XPD:
- XTD10, XTD6
- improvements, commissioning ongoing
- clean room tent water leak reported on 23.12.
DET:
ITDM:
- Jan 19: dCachce maintenance
XRO:
- progress as planned
- changes to existing components will be posted to respective elogs
LAS:
UNSYS:
SPB/SFX:
FXE:
MID:
- PLC outage on Jan 1 st caused VAC issues (reported by EEE)
- no access to MID optics hutch due to interlock test
HED:
- everything on schedule
- leak on a hose to humidifier device in A12. This leaked a few litres on the floor.
SCS:
SQS:
|
288
|
20 Dec 2021, 09:25 |
Andreas Galler | General | Information | Notes XO integration meeting 20.12.2021 |
General Announcement:
CTRL:
DA:
EEE:
VAC:
XPD:
- nothing special, shtudown tasks are in schedule, HXPES / XTED2 cable work ongoing
DET:
ITDM:
XRO:
LAS:
UNSYS:
- APPLEXv inv place, VAC to be closed next year
SPB/SFX:
- working on mirrors
- not clear what is the status with the movement of racks wrt SXP (SPB/SFX optics hutch racks currently powered down)
- electrical installation?
- PG comments: Brad won't allow operation until 230 V power distribution is calrified, once Brad gives the OK, the racks can be switched on
racks are being finalized by EEE, but some checks need electrical power
- RB: SPB/SFX were promised that the move of the racks were finished before X-Mas, but it seems the electrical work is not up to satandards
FXE:
- shutdown task in schedule
MID:
- all fine
- some loop updates planned for Today
HED:
SCS:
SQS:
AOB:
- light OCD over X-Mas for XPD in case of floowings, power outages.lists will be
|
287
|
15 Dec 2021, 10:26 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 15.12.2021 |
General Announcement:
- Severe power cut affected Schenefeld campus yesterday: it seems to be associated to human error but investigations are still ongoing. As soon as information will be available, it will be distributed via
e-mail either from TS or from XO.
- Regarding access regulations coming into effect Thursday evening (16.12): those who shared vaccination status with SRP will be able to enter via side gates or without showing vaccination certificates to
the gate keepers.
Without showing to SRP one can enter only via the main gate.
CTRL:
- Due to power cut some devices went to error state; a bunch reset helped to bring them up. If errors appear again please write tickets to CTRL.
DA:
- Nothing to report.
EEE:
- Issues with power cut: lost electronics in instruments; lost parts in tunnels (but recovered). Please check systems if they need to be reset.
- XGM in SA1 and SA3 were also affected.
- uTCAs are back and running after power cut.
- Work with cabling is ongoing accordingly to the maintenance planning.
VAC:
- PLC in rack room were affected and recovered with help from CTRL and EEE. So far nothing severe seems to be appeared but investigations still ongoing.
XPD:
- XGM is under investigation by Theo M.
- HIREX in SA1 has a motor not moving. Today on-site investigation will be done together with Bernard B,
DET:
- Nothing to report.
ITDM:
- Power cut: problems with office switches had to be rebooted/recovered. It got stuck during the reboot process causing problems that switches had to be manually power cycled (maybe due to a second short
power failure).
XRO:
- Nothing to report.
LAS:
- Affected by power cut; PLCs were off but running again after help from EEE.
- Seeder seems to do not suffer major issues.
UNSYS:
- Problems with electricity at XHU due to power outage were recovered later in the afternoon.
- Two Apple-X undulators were installed in SA3 yesterday.
SPB:
- Could recover the essential systems quickly enough; no big damages so far.
FXE:
- Working in tunnel this morning. Investigations related to the power outage ongoing.
MID:
- Issues were recovered with EEE.
HED:
- Could recover vacuum systems; Laser systems were turned OFF.
- SA2 gas alarm controller was damaged due to the power outage. So there is no operation with liquid nitrogen and other gases from now. TS need to discuss with HED as tasks must be performed by latest mid-
January (lasers and/or cryo activities).
SCS:
- Not present.
SQS:
- Damage in the air conditioning (electronic and physical) in room 509 due to the power outage. To be discussed between SQS and TS. |
286
|
13 Dec 2021, 10:51 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 13.12.2021 |
General Announcement:
- The cooling water various types shall be working/restored and it seems that temperatures are back to normal. In case of finding deviations in temperatures let TS or XO know. Racks and racks rooms can in
principle be closed.
- Tobias H. comments on the open rack doors: in general if racks have to be opened, please keep in mind the corridors in between the labs are fire safety doors and this has to be discussed with SRP or
Michael Prollius prior taking place.
The comment serves as a precaution for the future.
- An e-mail from SRP will be sent concerning disclosing the vaccination status and registration at the entrance gate.
CTRL:
- Upgrade/migration of all tunnel cameras to Aravis was concluded in tunnels.
DA:
- Nothing to report.
EEE:
- All runs smoothly at the moment.
VAC:
- Maintenance activities going accordingly with the plan.
XPD:
- Proceeding with shutdown tasks. Today alignment activities and vacuum activities at XTD2.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- This week resume the work in lab after restoring cooling water.
- XRO reminds the need to resume discussion with the ramp-up activities to divide commissioning days - a plan has to be done due to the high number of updated devices.
A date for recommissioning of these devices shall be fixed. XO will discuss with XRO.
LAS:
- Restarting the lasers and proceeding maintenance.
UNSYS:
- Plan to install Apple-X from today on.
SPB:
- Installing granites in experimental hutch.
- Intervention in optics hutch today: please refrain to access it for staff other than the ones performing task.
FXE:
- Access XTD2 to check CRL; commissioning motors in hutch.
MID:
- All fine at MID.
HED:
- Laser interlock upgrade last week was successful with Klaus Witt.
SCS:
- Activities are going accordingly to the plan.
SQS:
- Nothing to report. |
285
|
08 Dec 2021, 11:02 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 08.12.2021 |
General Announcement:
- The temperatures in the Hall are warming and accordingly to Tobias C. this is related to cooling water as ventilation is also affected.
- A. Galler reminds on an e-mail sent yesterday related to check-in at XFEL today: one has to show vaccination status at the gate. For next week likely one can register the Covid status at the badge reader
and further communication will be sent.
- On next Saturday 11.Dec tunnels between Osdorf-Born and XHEXP1 will also be closed in the context of the fire extinguisher maintenance.
It is also reminded that all Schenefeld campus buildings are inaccessible and only SRP, TS and external company are allowed in the site.
CTRL:
- After the control server network update, Karabo is up again in all the topics.
DA:
- Nothing to report also including calibration team.
EEE:
- All maintenance tasks progressing in tunnels and instruments.
VAC:
- Not present due to tunnel works.
XPD:
- Progress yesterday was done by bringing hard Xray PES in XTD2 which was installed in the beamline with help from VAC and MEA groups
- Loop update with EEE. Bernard B. says SA3 motion2 PLC will be updated/finished today.
DET:
- Nothing to report.
ITDM:
- Commissioning proposals: interested people got an e-mail on the closing/reviewing of the current proposals. ITDM will eventually close it this week.
- Request for changes on new proposals have to be sent today to ITDM, before its creation. After being created, remains valid for the next half year. New proposals will be available for use from Monday.
- Maintenance tasks: online storage was finished yesterday evening. Currently everything online is available again. DAQ will be brought up today and cross-checked specially for SA1 where servers were
exchanged (new clean installation).
- Exchange of switches for transferring data between Desy and Schenefeld was done.
- Early morning network maintenance: just two routers need further investigation by NOC (using old firmware). Action shall be unnoticed as there is a redundant system.
XRO:
- Proceeding with maintenance planned tasks.
- Reminder on the delayed installation of the XTD2 new bender for SA1 because of problems with cooling/ air conditioning at the labs.
- XTD9 distribution mirror M3 for SA1: XRO noticed the soft limits were changed and position of one mirror was different.
Changes in soft limits may be related to power cut; XRO has changed back to the proper values but this has to be checked again during re-start of operation.
LAS:
- Nothing to report.
UNSYS:
- SA1 cell#33 undulator and rack have been shifted to cell#2.
- Apple-X small hiccups but optimistic that will be good for next week.
SPB:
- Nothing different from Monday.
FXE:
- Usual activities ongoing; tests with robot software library and middle layer devices for movements to be done today.
MID:
- All fine at MID.
HED:
- Nothing to report.
SCS:
- Nothing to report. All fine with transportation activities.
SQS:
- Nothing to report. |
284
|
06 Dec 2021, 11:54 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 06.12.2021 |
General Announcement:
- Cooling water situation at XHQ: initially there is no cooling water until possibly 13.12. To the best of XO knowledge all groups are informed. If this is not the case please contact XO.
As soon as water comes back to normal an e-mail will be sent.
- Jorg H. asks on the safety trainings for employees (relates to the sexual harassment traning from an e-mail of Sven M. / Desy this morning).
Sandra Plett comments that this is not mandatory to XFEL colleagues but it will be clarified today via an e-mail. The Basic training and hygiene trainings are the ones mandatory to all the staff.
CTRL:
- Last week Karabo deployment done except of the Lasers (to be done this week). If any issue is found please contact CTRL.
- Control network maintenance: control server maintenance takes place on Wednesday between 07:00 and 09:00.
DA:
- Online data analysis service is also down due to maintenance tasks.
EEE:
- Start this week with tunnel loop updates.
- Ongoing HED installation cables; also cabling work at SPB ongoing.
VAC:
- Continuing with shutdown activities (ongoing accordingly to the schedule).
- Power cut in XTD2 last Friday is not yet understood.
- Broken power supply that affected fire extinguisher system was resolved.
XPD:
- Continuing with maintenance tasks: XTD2 differential pumping systems; transport of hard X-ray PES taking place tomorrow at XTD2.
DET:
- Nothing to report.
ITDM:
- Wednesday early 7:00 to 9:00 no control network due to switches upgrade.
- Today and tomorrow online storage system will be upgraded in offline mode by being disconnected from the clients (therefore it takes shorter time).
- Planned for today and tomorrow the exchange of switches to Desy.
XRO:
- Continuing the list of activities: due to the stop of cooling water the preparation of the re-installation of the bender in XTD2 is delayed.
Lab is currently at 28C and when temperature is stable activities will resume. Only then will evaluate if intervention will be feasible or not.
- Tasks with EEE and motors will start next days.
LAS:
- Confusion with cooling water on lab area that was lost around 15:00 on Friday. Apart of that all is fine. XO will look into it when cooling water is back.
UNSYS:
- Two Apple-X undulators are in the shaft building for connecting vacuum pipes.
- Undulator SA#1-cell#33 to be moved to cell#2. Collimator/Pb shield was moved to downstream of Diagnostic Undulator.
SPB:
- Installations of hardware planned for this week.
- Working with EEE and others to get hardware moved to the new rack room.
- Needs to negotiate to keep vacuum running.
FXE:
- Not present.
MID:
- All fine at MID with maintenance works.
HED:
- Laser interlock update at HED: laser interlock not functional this week;
- Tunnel activities by HED is done.
SCS:
- Activities are ongoing accordingly to the plan.
- Experimental chamber is moving to SCS hutch today.
- Contacted TS in order to run the air flow in the laser run (due to SXP installations): discussed and solved with Alexander K. (TS). About the rack room the 20C is running but rack rooms must be kept
open.
- Alarm of the Draeger cabinets appeared this morning but not yet understood.
SQS:
- Nothing to report. |
283
|
01 Dec 2021, 09:49 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 01.12.2021 |
General Announcement:
- Info that today the 20C cooling water will be switched to the towers was sent. Please also open rack doors to help until further notice. Tomorrow we will understand if cooling water tower is working and
if not, then A. Block and XO will communicate.
- Fire alarm happened this morning but not affecting people working at XHQ.
- Steffen H. says that DOC is not operational until middle January.
- Next Friday there won't be an UOM.
CTRL:
- Start with deployment today.
DA:
- Nothing to report.
EEE:
- Fully busy with preparation works and starting updates today.
VAC:
- Maintenance period activities been prepared. Tunnels are opened - if staff access rights are expired, please contact M. Dommach.
XPD:
- Not present.
DET:
- Nothing to report.
ITDM:
- Starting the maintenance activities today; On-line processes are unavailable until end of the week.
XRO:
- Last Monday: ramp down activities (stored motor positions for instruments with proper beam trajectories; XPD also worked with MCP detectors for all 3 SASEs (not done in SCS);
- XRO activities will be reported during this meeting series.
- Calibration SASE2 mirrors FD proposal was submitted for starting 2022.
- One PC at BKR (middle one) is extremely slow and shall be investigated during the maintenance time.
LAS:
- Maintenance on the PP-lasers are started.
UNSYS:
- Not present.
SPB:
- Not present.
FXE:
- Nothing to report.
MID:
- All fine at MID. It will investigate the water leak that happened this morning with chiller.
HED:
- Preparing XTD6 activity (exchange screen in monochromator) next week.
SCS:
- Nothing to report.
SQS:
- Nothing to report. |
282
|
29 Nov 2021, 11:50 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 29.11.2021 |
General Announcement:
- Today and tomorrow are the two last days of operation with ramp down of tunnels components. On Wednesday many activities are planned for the start of the maintenance period.
- Sandra Plett comments there are 1000 packages o Self-tests at EuXFEL with 5 tests each. Sandra P. also mentions that FFP2 masks can be taken from reception (few amounts) or warehouse (larger amounts).
CTRL:
- Karabo deployment will start accordingly to the maintenance planning.
DA:
- Nothing to report.
EEE:
- Preparing for maintenance tasks.
VAC:
- Nothing to report.
XPD:
- Naresh K. mentions the issue with emergency light test carried by TS. HED planned to make measurements with a dark room (all lights OFF).
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Not present.
SPB:
- Nothing to report.
FXE:
- Cooling issue in cw49 (opening rack doors): A. Galler says that for the experiments does not need to open racks. On 02.12 XO will inform if the cooling backup system for 20C water works.
MID:
- Training to operate air compression in the Hall was done. Normal state for MID compressor will be OFF and only switched ON when this is needed.
- Jan-Torben D. also asks for the training. This has to be scheduled with TS and XO can assist with it.
HED:
- Emergency light problem occurring on Friday: A. Galler says that this test is being done remotely every week on Mondays; The referred test was a maintenance task and TS was not aware of the presence of
users.
SCS:
- Nothing to report.
SQS:
- Re-arrangement of the beamline starting today.
- SQS is also interested in the training for compressors also. |
281
|
24 Nov 2021, 09:56 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 24.11.2021 |
General Announcement:
- Discussions on 3G rules access controlling are ongoing by the Covid task force. However, there is nothing else apart of what was recently announced.
- Next Monday and Tuesday (29-30.11) is dedicated to the ramp-down of the accelerator and photon systems and from Wednesday 01.12 it is the start of the Maintenance period.
CTRL:
- Nothing to report.
DA:
- P. Schmidt informs that instruments noticed early in the morning some issues with calibration.
DA is catching up to make it stable again (it relates to the new measure/state for the expected high data on this week). Please contact DOC if issues can't be solved by staff instrument.
EEE:
- Nothing to report.
VAC:
- Not present.
XPD:
- SA2 Filter interlock is put back (EPS limit of 300 bunches).
DET:
- DSSC cooling system issue (reported at SCS operation e-log): chiller had an alarm due to low level of coolant. The detector was stopped; chiller was refilled and brought back to operation; it seems to be
fine.
DET is investigating why the level had dropped. SCS is also investigating if it relates to a leak or low temperatures. Monica T. is in contact with Jan-Torben D.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Nothing to report.
SPB:
- Confirm that data will be taken also during the night.
FXE:
- Nothing to report; happy with beam performance.
MID:
- Organizing to get a training for the compressor to be able to switch it off when this is not needed.
HED:
- User onsite at the end of the week.
SCS:
- PLC-1 crashed on Monday taking time away for the experiments. A fan went off. Patrick G. says this is recognized that the fan went off and some cooling had to be switched down.
The maintenance of cooling systems is done by TS and it relates to a closed/sealed rack that can't exchange heat.
A solution permitting to spot/alarm when issues with cooling system appears in the racks must be put in place.
SQS:
- Nothing to report. Users are expected this week. |
280
|
22 Nov 2021, 10:13 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 22.11.2021 |
General Announcement:
- On Accessing EuXFEL premises at Schenefeld: it is no longer necessary to present the test result at the Gate; it is necessary to present the QR code for vaccination/recovery as well as the DACHS card at
the new terminal. For non-vaccinated people an official test certificate is mandatory.
CTRL:
- No major updates this time.
DA:
- Nothing special to report.
EEE:
- On Saturday EEE had to exchange a fuse at a MID motor (motor burned).
VAC:
- Nothing to report.
XPD:
- Nothing to report.
- A. Galler reported that for SA2 the interlock conditions for the upstream filters in SA2 are temporarily lifted allowing to go to 350 bunches (limit is 300 bunches). This is an exception that is valid
until tomorrow.
DET:
- Nothing to report; IT group is working to increase space for data.
ITDM:
- It collected 2.2 Petabytes of raw data for one user (MID) plus the same amount of calibrated data and there is still one day of beam delivery until tomorrow. Also SPB produced about another 1 Petabyte.
- If such requirements are needed in future please communicate to ITDM well in advance, largely more than one week before.
- A. Madsen says that the reason such big amount of data relates to an exceptionally well run, so it is difficult to predict how many data will be collected. K. Wrona says the communication went very well
this week. M. Turcato also says that when many detectors are running in parallel.
- Further discussions on this topic will take place.
XRO:
- Nothing to report; next Monday XRO plan to have the ramp down activities to have the current positions for the recovery.
LAS:
- From tomorrow on only SA1 will use the PP-laser.
UNSYS:
- Nothing to report.
SPB:
- Issue with calibration pipeline with AGIPD is under investigation (performance of the pipeline is not the expected one). P. Schmidt says that used calibration can not be fully supported on the weekend so
a fix can't be delivered in short time. In such cases it is better to use the old version.
- K. Wrona say one can plan for the maximum expected of acquired data but needs to have full resources for it. The current week was an exception. If one needs similar conditions like the current week it is
a big challenge.
- M. Sikorski completes that about 2-3 weeks before they can have an idea of how much data will be generated.
FXE:
- Nothing to report. M. Biednov says that they work at 9.3 keV that machine performance was poor and that would need to retune on Tuesday (about 1.5mJ).
MID:
- Nothing to report; all fine at MID.
HED:
- Nothing to report.
SCS:
- Experiment running so far good with minor issues.
SQS:
- Nothing to report. |
279
|
17 Nov 2021, 11:20 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 17.11.2021 |
General Announcement:
- XO will publish on Friday the full maintenance task list. Everyone is invited to also look at the Infrastructure calendar which contains main dates on this topic.
Please contact XO in case you have questions. The task list will be published via e-mail containing also the file location in an Alfresco folder.
- A power glitch was observed today at around 07:40AM. Machine was not affected. Please check your systems.
- Jan G. asks on Zoom operation issues seen yesterday (link of PBS meeting was shared with non-XFEL staff).
K. Wrona says that this particular case the link was posted at the website including the password and everyone from the world could access. ITDM will prepare an e-mail describing good working practices.
CTRL:
- No major updates.
DA:
- No major updates.
EEE:
- Nothing to report.
VAC:
- Nothing to report.
XPD:
- Systems were checked after this morning's power glitch and they seem fine.
- The transmissive imager (Diamond screens) in SA1 and SA2 are still in. To remove them the instruments shall contact Andreas Koch.
DET:
- Nothing to report.
ITDM:
- ITDM expects this week a lot of data for the user experiments that may overflow systems. ITDM is preparing contingency space for it.
XRO:
- Nothing to report.
LAS:
- No effects were seen from the power glitch; lasers are running.
UNSYS:
- Last Monday the ZZ access in SA3 (cell#22) was successful and the Phase Shifter motor was exchanged. Currently discussions are taking place to evaluate counter-measures for the case it is related to
radiation damage.
SPB:
- Nothing to report.
FXE:
- Nothing to report.
MID:
- All fine at MID.
HED:
- Nothing to report.
SCS:
- Nothing to report.
SQS:
- Nothing to report. |
278
|
15 Nov 2021, 09:57 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 15.06.2021 |
General Announcement:
-
CTRL:
- Nothing to report.
DA:
- Interruption during the weekend on the automatic data processing: was disrupted on Saturday night (proposal that was finished and treated as same priority). A change of priorities had to take place for
HED having the data analyzed.
EEE:
- Faulty sensor at ATD broker - EEE will take a look.
VAC:
- No issues to report.
XPD:
- Nothing to report for XPD.
- Jan G. reports the machine overall performance was quite good without major incidents. Several issues were reported on the Operation e-log concerning the calibration pipeline and DAQ.
DET:
- Nothing to report.
ITDM:
- On the prioritization (refers to DA): major reorganization of priorities took place. This results in the fact jobs that are submitted to the partition, (many jobs for a current beamtime) are killed.
If instrument has beamtime please request reservations for the proper amount of jobs.
XRO:
- Nothing to report.
LAS:
- Nothing to report.
UNSYS:
- Planned access to exchange motor of PS21/SA3 this morning.
SPB:
- AGIPD data issues were seen and is in contact to DET and DOC. Issue relates to memory modules not sending data from time to time. Jolanta S.-D. says it can happen for 1-2 modules a train may miss and the
DAQ will discard the data.
This is not unusual and one has to look at the frequency which happens - there is nothing wrong with Detector; data looks as usual. Further discussions will take place.
FXE:
- Nothing to report. User beamtime seems to have run well.
MID:
- Restriction of priorities: MID jobs went through. Experiment jobs have priority and for this week one has to put back in priorities.
- K. Wrona says a system will prioritize the user beamtimes for each week, getting out of the system automatically.
HED:
- Two user experiments using AGIPD went nicely but a bit stuck due to the priorities list (slow data processing).
- Beam instabilities occasionally: appearing and disappearing from time to time (maybe related to mirrors). It was reported to the stability task force.
Harald Sinn mentions about a crosstalk between SA2 with SA3. This HED case will be further studied.
SCS:
- Nothing to report.
SQS:
- Nothing to report. |