ID |
Date |
Author |
Location |
Severity |
Subject |
391
|
01 Feb 2023, 10:56 |
Frederik Wolff-Fabris | General | Information | Notes XO Integration Meeting 01.02.2023 |
General Announcement:
- A. Galler reminds instruments to use the Operation-2023 e-log for daily reports.
CTRL:
- Issue with Karabo dataloggers connection to newly available Proxies (not connecting). Problem is understood and a fix in under made. Then one shall apply to topics (takes 5-10 minutes downtime).
DA:
- Nothing to report.
EEE:
- Nothing major to report; working on a received ticket.
VAC:
- Last night FXE reported issues with SA1 ion pump. Vacuum was OK but the status of Karabo was wrong. It's under further investigation and Wajid E. comments a ticket was sent and is being addressed.
XPD:
- Nothing to report.
DET:
- Detector installed back to HED (ePix) after successfully intervention.
ITDM:
- Updates of Maxwell cluster was done yesterday. If something unexpected happens please get back to ITDM.
XRO:
- Instruments taking beam at hutches and no major problems have seen so far.
LAS:
- SA1 and SA2 PPL's are in operation.
- SA3 PP-laser is under preparation.
UNSYS:
- Issue with cell24/SA1 undulator motion happened and was fixed on Monday.
- Gap of exchanged undulator in SA2 seems not be correct and it is needed a ZZ for the gap calibration, possibly next Monday (tbc).
SPB:
- Camera devices goes to error and needs power cycle.
- After power cut some motor lost positions.
FXE:
- He gas supply: A. Galler informs that currently there is no shortage of He for the next two weeks.
MID:
- Making good progress by setting things up at instrument.
- One sample stage has problems and EEE will work on Friday.
HED:
- Preparing for taking beam on Friday and currently working on Dipole laser.
SCS:
- Problem with new Interlock and BKR had to overwrite it. Beam is now at the instrument.
SQS:
- Not present. |
195
|
01 Feb 2021, 09:06 |
Arvid Gebhardt | General | Information | Notes XO Integration Meeting 01.02.2021 |
General Announcement:
- The test of the emergency automatic switch of power bridge is scheduled for Tuesday, 2nd of February 2021, 09:00. This will affect orange outlets in XHQ and green and orange outlets in XHEXP1. A short interruption in the power supply lines will happen. We expect all groups have taken appropriate measures to place devices in safe mode or connected to white power outlets.
CTRL:
- not much to report, things are running
- Read the docs is operational again. A given feedback to CTRL is welcome.
DA:
- Nothing special to report.
EEE:
- Tomorrow update sase1 motion loop at 13:00: an email will be sent to readiness mailing list.
VAC:
- Nothing to report.
XPD:
- Maintenance work ongoing: Imagers in XTD6; Work with EEE in XTD6 and XTD9.
DET:
- AGIPD: new control with new gain is tested. DAQ issue with data is fixed (K. Wrona informs it was related to a wrong hidden parameter setting).
ITDM:
- Downtime on Offline calibration system / GPFS: update to be made in about two weeks from now. K. Wrona says that ITDM would like to have one day of GPFS maintenance; it is important to switch to the new scratch files before we start operation. Data taking shall not be affected by this.
- Jan G. asks when would be required by the earliest. K. Wrona says they need to copy a lot of data before the update and it is estimated in two weeks (+-2 days), to take place in the second half of February. As photon activities take place in the last two weeks of February, one needs coordination to find a suitable date.
XRO:
- everything fine.
- Today closing of soft mono chamber in XTD10.
LAS:
- Upgrade in SA1 uTCA: information will be given to LAS this afternoon.
- Two Desy colleagues needs access in SA1 hutches. Use of FFP2 masks is mandatory.
UNSYS:
- ZZ in XTD4 is planned for Wednesday (for cleaning undulator area).
- Suren K. asks if the balcony rooms are effected by test emergency power switch. This will be discussed offline.
SPB:
- Nothing special to report.
FXE:
- Nothing special to report.
MID:
- all fine
HED:
- Nothing special to report.
SCS:
- Nothing special to report.
SQS:
- Nothing to report. |
204
|
03 Mar 2021, 09:23 |
Arvid Gebhardt | General | Information | Notes XO Integration Meeting 03.03.2021 |
General Announcement:
- Hick-ups in the beam operation are happening since last night. There is no obvious indications on the origin of the problem and MXL is investigating to restore a normal beam delivery condition.
- There is an issue with “Operation-2021” e-log where some SPB staff can’t login to create entries. This will be addressed by XO/PRC and ITDM (see ITDM entry).
CTRL:
- SA1 DAQ was dropping some trains (the late ones) in the XGM. CTRL and ITDM are following up and will test the XGM late this week or next week. CTRL will contact XPD for deployment first in SA2 and SA3.
DA:
- News from the DOC: deployment of major updates ongoing/to take place.
EEE:
- nothing to report
VAC:
- not present
XPD:
- Commissioning and setting up for SA3 HIREX
DET:
- preparing for commissioning Detectors
ITDM:
- Janusz S. says the configuration for the “Operation-2021” e-log uses the same configuration as previously set for 2020. F. Wolff-Fabris likens the issue with possible new users and asks SPB to provide the name of users who can’t login into the e-log.
Janusz S. says one can use the copy function from instruments e-log to place a copy into the Operations-2021 e-log. Jan G. says that login is necessary to avoid the entry being as “Anonymous”.
XO and ITDM will address it offline.
XRO:
- not present
LAS:
- In SA3 PP-laser one had exchange of power supply for AMPHOS due to communication errors.
- In SA1 PP-laser one observes a loss of 1/4 of power in AMPHOS. LAS is not sure if there is enough power for users.
- Two problems with triggers on Monday: 1. a faulty board in SA3; 2. MCS group update timing system caused trigger loss in all hutches (due to a bug in their server).
UNSYS:
- nothing to report
SPB:
- Couple of Helps asked to DOC and SPB/SFX are thankful for that.
- Login at Operation-2021 e-log does not work for 4 people.
FXE:
- Prepared for AIBS. Few PLC problems remain but not related to AIBS.
- Next week between Tuesday/Friday there are no access to the hutch due to floor work ongoing.
MID:
- all fine
HED:
- all fine
SCS:
- sliding door will probably be repaired today with an external company.
SQS:
- nothing to report |
309
|
16 Mar 2022, 10:39 |
Frederik Wolff-Fabris | General | Information | Notes XO Int. Meeting 16.03.2022 |
General Announcement:
- The Summer maintenance planning period is approaching and next Wednesday at the end of the XO Int. Meeting a presentation of the changes/improvements will be made.
Please communicate to the colleagues who are responsible for your group’s maintenance planning to join. This will be done after the regular round of discussions.
CTRL:
- Not present.
DA:
- Nothing to report.
EEE:
- Continuing with support tasks to the groups.
VAC:
- Not present.
XPD:
- XGMD was fixed (ADC broken down twice last week): Desy people is also investigating; root cause not yet found.
- FEL Imager (YAG screen) in SA3 is found to be damaged. It was moved to a different spot for the time being.
- Ventilation in all tunnels caused beam jitter for 30-40 minutes yesterday.
It was caused by outside humidity (not common, 2-3 times per year) that filled up the filters and clogged it with water droplets leading to overrunning of the regulation schemes of pumps; then pressure
changed in tunnels.
DET:
- Nothing to report.
ITDM:
- InfluxDB: large issues yesterday (cluster crashed on MON evening); it took all day yesterday to brought up and data is now visible also at Karabo.
- ITDM has a root cause now and DAQ matrix devices were switched OFF yesterday which provides a fix. Last crash there was a lot of data that had to be synchronized and lacked resources.
- ITDM found a way to start without data from Monday evening to Tuesday 21:00. This data will be injected to InfluxDB / cluster in the coming days. In the meanwhile, one needs to use Graphana to look at
this data, not Karabo.
- Discussing future measures to trigger alarm to avoid the crash.
- Please don't start DAQ without the GO from ITDM via e-mail.
XRO:
- FXE jitter last week: not yet clear if it was the clean tent or the cooling water for M1/M2 the real cause; hard to find as cooling water was refilled and the clean tent was switched ON accidently, both
in parallel.
LAS:
- Issues with Sync system in SA1 persists: spurious 10Hz signal. PP-laser still do not have optical signal. LAS has a rough idea of the cause and is in contact with instruments.
UNSYS:
- Phase shifter in cell11/SA3 the motor was replaced and more 10 motors were ordered as spare parts.
- Quad mover in SA1#cell 18 was fixed.
- Quad mover in cell25/SA3 (wrong movement direction) is still to be addressed. MXL will work together with UNSYS.
- Tables for Phase shifters of Apple-X still to be uploaded.
- One new PS on the Apple-X is not working; Frank Brinker is informed and a ZZ is needed before the next blue week.
SPB:
- Influx issue but could use Graphana.
- Pulse energy is low this week: 2mJ while last week was 3.5mJ;
- Crosstalk with SA2: SA1 energy gets worse when changes happen in SA2 but recovers shortly after.
- SPB did not see any vibration on Tuesday 15.03. Also last week beam was stable.
FXE:
- Nothing to add.
MID:
- All fine at MID.
HED:
- Not present.
SCS:
- This morning a filter of RIXS spectrometer broken and has to be exchanged (filter gauge valve). VAC group is helping with the exchange.
SQS:
- Nothing to report. |
308
|
14 Mar 2022, 10:41 |
Frederik Wolff-Fabris | General | Information | Notes XO Int. Meeting 14.03.2022 |
General Announcement:
- On the Summer-22 maintenance period: In about 2 weeks (by end of March) the call for the summer/22 maintenance period will be send to the groups.
CTRL:
- Deployment of new framework in order to fix an issue (this week): instruments shall agree with CTRL contacts on a suitable time. It affects cameras and/or similar devices that are installed.
DA:
- Report on last week issue with processing on Maxwell including offline calibration: people were not able to open files that were there; small selected notes were not present and were corrected on Friday.
Since then it seems no issues recurred.
EEE:
- Starting equipment installation of SXP racks in rack rooms.
- PLC side calls from SQS on weekend: issues with motors to be followed up; train-IDs not getting proper.
VAC:
- Not present.
XPD:
- XGMD in FXE hutch under investigation: new issue happened at the end of the week and may be again related to last Monday's one. People are working on it (XPD, FXE and Desy).
DET:
- One comment to DA: Jolanta S-D was on DOC and issue with offline for ePIX detector still occurring after Friday. Phillip S. says it was a different problem and it is fixed.
ITDM:
- Issue (2x) with Influx): still not understood; Data was not lost on last occurrence. Hick-ups and recovered. So far under investigation.
XRO:
- Jitter at SA1 in FXE last week: mostly likely the cooling in M1 and M2 or clean tent.
- No big issues after that.
LAS:
- Degradation in SA1 PP-laser with trigger software; maybe there is damage to optics. Being investigated today and can't guarantee when it will be working again.
- SA3 change of setpoint for SQS taking one-two days. Not sure if ready today.
- SA2 is running well.
UNSYS:
- Phase shifter cell11/S3: a ZZ is ongoing also fix with remotely procedure.
- Issue with quad mover cell18/SA1 is fixed remotely.
- Issue with Quad cell25/SA3: the ZZ is needed.
SPB:
- Issue with Agipd detector: fixed (one module not sending data).
- Agipd online preview devices: fixed;
- Laser timing issue: data was not collected.
FXE:
- Jitter problem already mentioned.
- XGMD issue under investigation.
- Laser problem: stop measuring on Saturday. Not sure if data was taken during the weekend.
MID:
- Difficult week.
- Broker pileup problem on Monday and Friday: Gabriele says it may be related to a memory leak - under investigation.
On Friday there were there sources identified to help getting rid of problems (Jolanta S.D); also GUI was a second source. Not fully understood the main source of Friday. It came back stable.
- Seeding beam had to be tuned on weekend as energy decreased by factor 5 when comparing to the best lasing at the week start. No scientific output could be achieved but feasibility with setup was
demonstrated.
HED:
- Wednesday optics hutch ADC was not working. Now it works fine.
- Full power with RELAX was tested and is being analyzed. Bassler cameras acquiring wrong train IDs. It shall be fixed.
SCS:
- Not present.
SQS:
- Successful week;
- Problem with motor being fixed by SQS, ME, EEE.
- Tool for online analysis was slow
- Heat load in the soft monochromator was seen and it has to be fixed in long term. |
97
|
10 Feb 2020, 10:12 |
Frederik Wolff-Fabris | General | Information | Notes Dispatch meeting Feb 10th, 2020 |
SPB: Nothing to report;
FXE: Everything ongoing; nothing special to be reported;
MID: Finalize motors of split and delay line; MID has to test interlock for vacuum loop update.
HED: Commissioning of laser heat table. It may need assistance from EEE and CTRL during the week.
SCS: Nothing to report - Last week's mounted setup will be disassembled in order to mount the standard setup for users;
EEE: OCD was called from XRO during beamline alignment of SA2: 1 BMPI motor had a problem and is under investigation - EEE reminds on the new homing position for
negative direction motion.
CTRL: Following reset of train-ID and timing system for today.
XPD: Shifted Beam Pointing Studies from A. Koch to Tuesday.
DET: Nothing to report
*machine is currently not running due to filter clean in the cold compressor. Shall be re-started this afternoon and by tomorrow must be lasing.
*Master time reset and train-ID reset this morning. Then EEE (check systems), followed by CTRL (karabo servers re-start) and ITDM (check DAQ operation). LAS and DET are
informed.
*Wednesday: Network stress test with Cameras by CTRL on Wednesday 12:00-14:30. |
111
|
20 Apr 2020, 10:59 |
Frederik Wolff-Fabris | General | Information | Notes Dispatch Meeting 20.04.2020 |
General:
- Desy informed a power glitch occurred on Thursday 16th at 04:55AM. TS has not seen any traces of it in the EuXFEL internal GLT.
- From next week the Dispatch meeting will start at 09:30AM.
- An additional Friday's Integration Meeting will happen at 11:30AM. Proper information and invitation will be send soon.
CTRL:
- The notification service for the reduced operation mode is ready for start-up and an e-mail will be send around with instructions.
- Groups/staff will be requested to subscribe to the lists.
- Training will be conducted on next Wednesday at 15:00 and/or next week. One member of each group is required to attend it.
DA:
- Nothing to report. Working busily from home.
EEE:
- No issues to report
VAC:
- Two pumps located in a gas attenuator were OFF due to the power glitch of last Thursday 16/04 at 04:55AM.
XPD
- No issues related to the power glitch were observed.
- XPD has received devices and they were successfully stored in vacuum / clean rooms. Another devices shall arrive this week.
DET:
- Nothing to report.
ITDM:
- Nothing to report.
XRO:
- Nothing to report.
LAS:
- Not present.
SPB:
- Nothing to report.
FXE:
- Not present.
MID:
- Everything is fine.
HED:
- All is fine, all is running.
SCS:
- Not present.
SQS:
- Not present. |
102
|
16 Mar 2020, 09:36 |
Frederik Wolff-Fabris | General | Information | Notes Dipatch Meeting March 16th, 2020 |
FXE:
- Beamline is in safe state;
- Minor Jungfrau calibration issues over last week. In touch with DET and EEE;
SPB:
- Good exercise: experiment run with only 2 users. Successful beamtime with data taken: to be analysed by users;
MID:
- Successful beamtime with good beam conditions and data has been taken;
- Beamline is in safe state and most of staff is in telecommuting;
EEE:
- Start collecting a list of broad activities to take place in case of a long shutdown;
- No major issues to be fixed;
CTRL:
- Few cameras and detectors hiccups – e.g. DAQ re-start bring system back;
- Most of people will run from home office / telecommuting;
- Tests together with EEE/FXE still to be performed this week - to be scheduled;
DET:
- DET is placing a plan to warmed up detectors and be ready for shutdown;
- AGIPD, DSSC are already in safe mode; |
9
|
18 Feb 2019, 10:09 |
Adriano Violante | General | Information | Minutes of the dispatch meeting 09.30 |
SPB/SFX
- Data aggregators were often crashing during the WE and needed to be re initialized.
- Puls picker commissiong - Scan tool was used and worked. Issues concerning motors position and triggers ==> To be worked out with AE.
- DET group is helping setting up the Jungfrau detector.
FXE
- Mono commisisong ongoing. Beam was passing through both monos at the end of last shift.
- New beamstop mounted and being used.
SQS
SCS
- Parallel operation with SA1 did not generate major issues
- Karabo did not store the bunch pattern info
- Leds on the beamline are needed. Loop4 is the most important and will be updated today.
MID
- Water connections for the mono chiller will probably be reworked with the vac group.
- MIddle layer tests will be performed by CAS (CY)
- Aligment of the optics in XTD6.
- Tomorrow: exapod commissiong. PC with windows needed. ITDM+CAS
- Issues with scenes being stuck, probabl;y due to a HW error to be fixed with manual on-off operation ot the component switch. Being checked with CAS.
HED
OPT Lasers
XRO
XPD
- New trigger system needed for a gated camera in SASE3 - BF on it.
VAC
AE
- Timing server has been updated
- A PLC in HED has to be replaced because of some beckhoff license that expired (?)
ITDM
DET
- SCS took data with the fast CCD. Data has to be analyzed but seems ok.
- SPB agipd OK
- FXE LPD had issues being followed up
EET
CAS
AoB
A questionnaire to gather the activity of the next shutdown was sento to the XFEL and DESY group and operation leaders. The planning is going to be done together with DESY using the same templates and planning tools. |
12
|
25 Feb 2019, 09:52 |
Adriano Violante | General | Information | Minutes of the dispatch meeting 09.30 |
SPB/SFX
- Change to the triggers for cameras is being discussed/implemented with BF.
- Preparation for the AGIPD intervention planned in the next mini shutdown is being discussed with AE.
- On Wednesday the user period start. No more commissioning possible.
FXE
- Issues with digitizers for the OGT1. AE will investigate.
SQS
- Issues with digitizers that crash regularly (issue concerning data aggregators may also depend on it) - Cyril on it.
- Mpods commissioning - Also here CAS (Cyril and CY providing support)
SCS
MID
- Commissioning ongoing. (Agipd "saw" beam last WE)
- This week the interlock definition in Loop4 (VAC) should be updated. Before performing the update MID needs to be informed
HED
OPT Lasers
XRO
XPD
VAC
AE
- When HS and ZA the requirements for the SEPS updates have to be discussed and put in written before the loop update. Could happen on Thursday.
- On Friday Loop 13 in SCS was updated (Train id) and after that MS noted some strange behavior. To be checked with MTeichmann.
ITDM
DET
- SPB test ok
- Today: work on LPD and Jungfrau SPB.
EET
CAS
- Kickoff meeting with ITDM to start to evaluate InfluxDB, Grafana and Elastic Search as an alternative to the current Karabo DataLogger implementation (LA and VAC represents use cases to be addressed/solved). However, "nothing" will impact operation in the next months. ==> Hotfix 2.3.5. has been released. Mandatory to solve a GUI issue currently present.
- SPB Loop 14 needs to be deployed (Karabo) ==> Today
AoB
|
15
|
04 Mar 2019, 10:38 |
Adriano Violante | General | Information | Minutes of the dispatch meeting 09.30 |
SPB/SFX
- Ongoing discussions with the support groups to organize the shutdown activities.Update of the shutdown task list could eventually be provided.
- X-Ray interlock system had issues in the WE and needed to be re set. Not clear why this happened.
- Problem with Karabo freezing. CAS contact person ihas been involved and is working on the issue. This is related to the calibration running.
FXE
SQS
SCS
MID
- Fast diodes needed on the WE. HW and Software support required and ongoing. ==> Successful
- CY called for a meeting on monday to collect all the requirements for the epix det to work. The det will be then tested in the common lase room. Karabo interface change is required. It will take 3 weeks.==> Epix has been moved back to the common laser room for proper testing and commisisoning. Several tasks have to be accomplished to try to make the detector work for the April's run. CY is basically organizing the technical commissioning.
HED
OPT Lasers
XRO
XPD
VAC
- A pump for the GATT in SASE 3 had to be treplaced (borrowed from SPB).
- Solid ATT in XTD 1 has issues with control valves. Being fixed now with a ZZ access.
AE
- Probably the timing update planned for this morning will not be needed anymore. BF will send a notification mail.
- there are many activities planned for the next shutdown and priorities will need to be set . PSPO agreed on sending the preliminary task list to the support groups so that they can start having an overview.
ITDM
- Failure of the logbook during the night: Also one partition failed. Not clear why. Everything has been fixed and is back to normal.
- Simultaneous operation of all the instruments generates competition of the resources. Priorities to be set.
- Calibration pipeline: Only 1 server in place. POssible issues when operationg MID and SPB agipd at the same time... DET should look into that.
- SPB requested to restart the DAQ before every shift. This can be done but it requires experts from DET and SPB to be present and to perfome some checks. This is now not always the case. To be discussed ho to find a short term solution.
DET
EET
CAS
- Test Karabo 2.4.0. ongoing. Ongoing
- List of loop updates for PLCs has to be issues by AE to CAS as soon as the activities are known.
AoB
|
17
|
08 Mar 2019, 13:01 |
Adriano Violante | General | Information | Minutes of the dispatch meeting 09.00 |
SPB/SFX
- Only minor issues being addressed with the support groups.
FXE
SQS
SCS
MID
- Epix planning is advancing. List of activity being maintained and followed up by CY. Interface box for cables being modified by KS, CRs being implemented by AE. Ongoing. Goal date is 10.04.2019 to have the detector completely commissioned
HED
OPT Lasers
XRO
XPD
VAC
AE
- for SQS there are issue with the Karabo/digitizers communication. Workaround exists, being fixed for good (AE + CAS).
- Given the preliminary shutdown task list sent to readiness this week, it is clear that AE does not have the resources to cope witl all the requests. The tasks do not have priorities right now and this is a problem. In general, priorities and tasks that could be dropped are needed. Instruments, Support groups and PSPO cannot decide. PSPO will address the issue to HS/TT.
ITDM
DET
- Calibration pipeline stability test will be performed with SPB (during shifts). CAS/DET/SPB on it.
EET
CAS
- Hotfix 2.3.6 (GUI server, data loggers bugs fixed) deployed in SA1,2, in SA3 not yet.
AoB
|
74
|
31 Jul 2019, 10:46 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 31.07.19 |
XO
FXE
SPB/SFX
- Loop 12 has to be updated ==> Done.
- SASE1 M3 issues with the pitch motor lead to FXE losing beamtime. EEE to follow it up. ==> Basically fixed. Some remaining issues will be fixed during the shutdown week in August.
- Some issues reading the encoders of the JJ XRay CSLIT. EEE checks whether similar issues are also present in other instruments.
SCS
SQS
- Commissioning of smaract stages ongoing.
MID
- By the end of the week MID should send the requirements for the vacuum interlock to EEE. ==> tests are ongoing. AS will organize a meeting with CAS, VAC and EEE to define a test procedure to be followed by the instruments when testing the vacuum interlocks. ==> to be done.
HED
- New update of Loop 6 will be done propably today after new interlock definitions have been delivered.
- Hexapod: Loop 9 has been updated. Loop 7 to follow.
Optical lasers
VAC
XRO
XPD
DET
EEE
- Crash of PLCs because of not a number values. The fix on the PLC side requires the update of the loops framework and can be done only in the August shutdown. By the end of the week a filtering tool will be availbale in Karabo.==> Karabo client has been deployed in the tunnels and FXE. The other instruments will get it during the August shutdown.
- XPD reported some issues with digitizers for MCPs. It should be fixed already by now.
CAS
- CAS received host alerts regarding sa2-br-sys-con-1, sa2-br-sys-con-2, sa2-br-sys-con-5 resources usage. To avoid the worst issues during experiments, we will do maintenance on these machines today from 12:00 to 14:00.
- CAS/SQS working on karabo scenes to display the LaserMet laser isnterlock status.
ITDM
SRP
TS
AoB
- RBean reported that by the end of the week a macro will be run by HSinn to decouple machine interlocks in SASE1 and SASE3. For more details AGaller should be addressed.
|
1
|
30 Jan 2019, 11:26 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 30.01.19 |
SPB/SFX
- Planned for the next 2 weeks
- AGIPD commissioning and tests. DET group heavily involved.
- Screens are being installed and commisisoned in the hutch.
- CSS component commissioning. AE involved and informed.
- Loop 14 (Jungfrau) is needed for next week to start the commissioning. AE and EET main players for now. Issues are now Fuses boxes delivery (AE) and ITDM connections ready (CAT cables measurements not compliant with XFEL standard, AV/ITDM are trying to solve the issue with the companies)
- CRL implementation in the hutch will happen in April. AE and EET involved.
FXE
SQS
- REMI chamber was installed. Currently vacuum connections are being done and tests have to be performed (next couple of days)
- CYoungmann is commissioning the Mpots.
- The chain digitizers-triggers-software is being commissioned to allow online analysis. AE, CAS required and involved.
- New requirement document will be submitted today for a new slit system component (not for this SD).
- exhaust line for explosive gases has not been tested so far and no information is available to SQS. Urgent. SRP should comment.
SCS
- PLCs have been all installed. Firmware deployment has to be pushed. AE is on it.
- For the microscope a in-house fast solution has been put in place.
- Concerning the smaract stages a technician is today here. If anybody is interested, please contact JTDelitz.
- Detector group is working on the fastCCD.
MID
- Julabo chiller came back from repair and it is being connected back and tested in this days.
- Issue with the remote control of the ion pumps. It is not possible to switch on and off the pumps remotely due to the fact that a wrong terminal was used . It seems this was also the same issue in SASE1 and 3. AE and EET are on it to fix it and update the doucumentation.
- Fa. Schulz did not manage to pull all the MID EXP cables by Tue afternoon as requested. Fine tuning of whoch cables are needed to start the loop commissioning and whoch ones can be postponed has to be done and driven by EET/MID (M. Flade/JHallman)
- On Monday afternoon (17:00) a power cut caused by works on the SASE emergency power off caused the PLC to be switched off and additioanl efforts to bring them on again. This intervention was not known to MID and not planned in the winter schutdown list of activities. Technical Services is/will be reminded that proper communication is needed.
HED
- Loop 2 needed for commissioning. Should be ready this week. Modules for the attenuator are expected to be delivered today/tomorrow. AE required and on it.
- Tomorrow the commisisoning of the monochromator in the tunnel and CRL in the optics hutch will start.
OPT Lasers
- LOPA is being commisisoned in SASE1. Test by the end of the week. No special support required.
- In SASE3 commisioning isd ongoing and there are no concerns about having the laser ready in July
- In SASE2 commissioning of the seeder started.
- Issue with the door is being discussed with TS.
XRO
- In SASE3 additional safety shutters issue: Hardware will be ready by middle of Feb. On the 24 and 25. the installation and the modification of the EPS loop should take place if D3, SRP agree on the change request proposed by HS and MD. AE is involved and has allocated resources. It is not clear to whoch extent CAS is involved. This should be clarified, also for proper CAS resourced allocation.
- For this friday SRP schedule a test of the GATT. SQS/SCS informed and part of the activity.
- Durng the installation of mirrors for SASE3 with InGa an issue occured. While pumping down (slowly) foam was generated and it required baking of the system to be able to pump it and bring it to ion pumps vacuum level. Tests are needed to finalize the proper procedure. In general, all the instruments dealing with mirrors should contact XRO (M.Vannoni) for proper venting and pumping down of the system to avoid surprises.
XPD
VAC
- Mono in XTD6 commissioning. AE and XRO involved.
AE
- DESY will reboot the timing system next monday. BF circulated a mail about that.
- On 19.02 AE and CAS will present the new timing system for the SASEs and related Karabo functions the PBS meeting
ITDM
- Switch of the storage in SASE1 was succesful. Testing ios ongoing. OK so far. In the next days other tests with detectors will be done. (IBM technician also at XFEL today)
- DAQ In HED will be udpated today
EET
CAS
- No information received on allocation of CAS resources for the B4C implementation os SASE3 (see XRO first item)
AoB |
49
|
29 May 2019, 10:30 |
Gerd Wellenreuther | General | Information | Minutes of the Technical integration meeting 29.05.19 |
FXE
SPB/SFX
- XTD2 attenuator appears to be damaged. ==> Request to VAC: is it possible to exchange the attenuators in the summer shutdown? LBatchelor might have spares.
- Digitizers have been installed
- Bunch number is reported to have jumped from 1 --> 150, report to be provided by SPB/SFX so CAS can investigate.
- Top priority: Testing new camera + DAQ interface along with motor positions recording @ 10Hz
SCS
- Currently running + supporting the user experiment
SQS
MID
- Integration of the LDM: Safety document is needed. ==> This is needed ASAP!!!
- Further commissioning is ongoing
HED
- Loop 6 being commissioned. ==> done
VAC
XRO
XPD
DET
EEE
- Loop priorities for HED and MID are needed.
- AE reminds everybody to contact TSchoen if power is required during the middle of June, e.g. in order to keep the vacuum running.
CAS
- Supported / tested measurements of gain curves in SASE1
- Next week: Testing of correlator
- Karabo Hotfix 2.4.2 was released, new week testing of 2.5. commences
- Progress regarding InfluxDB etc. was achieved and reported, e.g. to readiness mailing list
ITDM
SRP
TS
AoB
- No Dispatch on Friday due to the bridge day!
|
83
|
28 Aug 2019, 10:33 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 28.08.2019 |
XO
FXE
- Leak in the LPD cooling system has been found. DET on it. Ongoing
- Laser went out of synch several time during the last weeks. Being addressed with the synch. group.
- Mono middle layer is not working properly. Calibration parameters are not reloaded correctly after restart. CAS on it. Ongoing.
- SA1 MOV loop has been updated. Minor issues with the CRL encoders are being addressed.
SPB/SFX
- CSLITS - discrepancies between the behavior of normal motors vs MC2 motors. CAS on it. ==> will be fixed once Karabo 2.6.1. has been deployed.
SCS
SQS
MID
- During last WE big issues with the AGIPD were experienced. Probably calibration pipeline related. DET on it.
- Laser hutch A.07 is not supposed to be accessible in the next weeks because of the laser operation. SRP points out that they are not sure whether the interlocks are properly working and active. To be clarified offline between MID, HED and SRP.
HED
- Access to the experiment hutch, Optics and Rackroom will be restricted this week due to laser shots tests.
Optical lasers
VAC
XRO
- Change of middle layer devices for the attenuator in SASE2 will be discussed offline with CAS
XPD
DET
EEE
- Loops are being updated in coordination with the instruments.
CAS
- 2.6.1. ready to be deployed. Deployment schedule has been agreed with individual groups.
ITDM
- Maxwell cluster maintenance on 2-3. 09.
- Control PCs wil be deployed during the shutdown week
- Claoud service (DESY platform) will be put into operation. An official announcement will details will follow.
SRP
TS
AoB |
24
|
27 Mar 2019, 10:48 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 27.03.19 |
SPB/SFX
- CRL implementation in the hutch will happen in April. AE and EET involved. ==> Connectors on the beckhoff frames need to be re-shuffled according to the newest version of the EPLAN documentation. EET will take care of that, so that the loop can be started as soon as possible.
- CSS crate is ready. Loop 7 bridged (same loop as CRL). Test can start tomorrow.
- AGIPD motion. Prelminary tests are being performed to be ready for the final tests in the march shutdown. ==> Some issues with cabling of limit switches were found. SPB will fix it today/tomorrow. ==> Test starting today. From the Karabo side seems ok.
- AGIPD chiller has been shipper for repair. SPB/SFX are trying to find alternative solutions for the next users run.
FXE
- 4 cables are needed to control some stages and were requested to AE/EET. After checking by NC some rewiring from FXE side on the module side is needed to make the electronic solution proposed usable. NC is in contact wit TKorsh.
SQS
- Loop 8 and 11 HW will be prepared by AE this week and can run by next week.
- Interlock updates ongoing
- Meeting organized by PGessler to review the recurring issues with the digitizers
SCS
- When operating the machine in the train pattern (not 10 Hz) mode and pulse on demand mode the data from the XGMs in XTD10 and SCS is not recorded. ==> That's still the case and it is caused by a Doocs update not agreed with CAS. ==> Ongoing. ==> Meeting took place yesterday. Issue seeems to be closed.
- The dangling cable of the sliding door has to be fixed. This is supposed to be done by ARathjen during the March shutdown.
- A Change in the storage of raw data of the XGM is required. SCS is discussing with XPD (Theo). Later ITDM and CAS will be also involved.
- Loop 13 being prepared. As well as 11. Interlock updates ongoing.
MID
- Smaract stages do not work propery and reliably. High priority. Has to be clarified with AE since it is not clear whether this is the same issue that SCS had and for whoch they developed a workaround. ==> BB will contact the Smaract company to address the issue.
- Loop 9 has been updated and the broncos can now be controlled.
HED
- Cables and loops for HED instrumentation. LW is monitoring closely the installation of cables to give the go ahead to AE for the starting of tyhe loops. Loop 5 is still the priority. High priority cables for Loops 5, 7,8 should be all installed by the end of this week.
OPT Lasers
- Modules for the time stamp have to be installed by AE. How long does it take? Will be discussed offiline with PG. ==> Installation dates: SASE1 in summer shutdown, SASE3 being worked out, SASE2 will follow. Workaround with time stamp is being investigated. ==> PLCs have to be updated. HW ready.==> SASE3 OK. SASE2 crates being changed now.
XRO
XPD
- Hirex 2 will be shipped from Cologne tomorrow and will be brought into the VAC lab. Crates ready, cables ready. AE can support local tests outside the shutdown week.==> Hirex will be installed in the shutdown and probably a couple of testt will need support from AE. These can be scheduled for the 01/02.04. Vac loop is affected.==> AE only supporting crates installation and powering.
VAC
DET
- 3rd branch SA3 - one ion pump has to be included in the VAC loop (update tomorrow)
AE
- Timing update foressen for today will rpobably happen a little bit later than expected in SASE2.
- Waiting for the finalization of the MID LDM risk assessment from SRP to allow the usage of the remote control of the device.
ITDM
EET
CAS
SRP
TS
AoB |
13
|
27 Feb 2019, 11:27 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 27.02.19 |
SPB/SFX
- CSS component commissioning. Concept and implementation finalized. EPLAN is being modified. PLC modification will follow.
- Loop 14 (Jungfrau) commissionig is ongoing. Issues concerning the simultaneous operation of more modules. SPB and DET on it. The detector is needed functional within 3 weeks.
- CRL implementation in the hutch will happen in April. AE and EET involved. ==> ZA needs the documentation of the CRL. before April. MS will ask the responsbile person in SPB.Ongoing.
- Changes concerning modules for the AGIPD motion are being discussed. Intervention on the modules planned for the next shutdown. AE, DET, CAS, SPB involved.
- Issue Basler camera <==> Loop. Will be solved offline with AE.
- Refurbished beamstop is needed by Mid. April. XRO has been informed.
FXE
SQS
- exhaust line for explosive gases has not been tested so far and no information is available to SQS. Urgent. SRP should comment.==> MMeyer is following up the issue. - NO update.
SCS
MID
- Issue with the remote control of the ion pumps. It is not possible to switch on and off the pumps remotely due to the fact that a wrong terminal was used . It seems this was also the same issue in SASE1 and 3. AE and EET are on it to fix it and update the doucumentation. ==> AE needs to perform a test at MID before deploying the solution. Since there are other priorities now the test has not happened yet. ==> Test has been shifted to HED. ==> Test will be synchronized with planned loop updates in MID. Will be coordinated between AE and MID.
- An automatic chain of communicaton is needed (TS?) to notify relevant peopole when the power is down. ==> UBr will follow up the automatic notification setup.==> Ongoing.
HED
- Si bounce monochromators - Chiller leaks. Will be repaired in the next shutdown. ==> Waiting for delivery of parts. Not clear whether can be fixed in the next shutdown.
- hirex mono water connections will be reworked in March.
- Cables for the experiment hutch are being installed but some quality issues occured. Last pallets of cables have been sent back for proper confectioning. It is possible that there will be some delays. KA will discuss with LW how to prioritize cables to eventually minimize the impact on the commissioning.
- Si- mono commissioning ongoing
- CRL2 in XTD6 - issue wit motors losing steps. Being fixed
- DAQ and signal processing is neede int he OPT hutch. Ongoing with ITDM and CAS.
OPT Lasers
- Modules for the time stamp have to be installed by AE. How long does it take? Will be discussed offiline with PG. ==> Installation dates: SASE1 in summer shutdown, SASE3 in the next 2 weeks, SASE3 being worked out.
XRO
- In SASE3 additional safety shutters issue: Hardware will be ready by middle of Feb. On the 24 and 25. the installation and the modification of the EPS loop should take place if D3, SRP agree on the change request proposed by HS and MD. ==> Still no official decision from D3 and SRP. AE would like to get as many info as possible in advance to start preparing the loop update. According to MDommach NJardon is supposed to be already in the loop. ==> Concept in written has not yet been communicated to SRP and AE. ZA will talk to HS. ==> HW is installed. Loop update can happen monday or tuesday. The proposed additional radiation safety measures for SASE3 were discussed with Zunaira Ansari. A test is foreseen for 5 March. Implementation in the SEPS system will be directly coordinated between Zunaira and Nerea.
XPD
VAC
DET
AE
- Correlation test between MCP detector and XGM last friday highlighted issues at the Karabo doocs interface concerning train id. Meeting AE, CAS, ITDM, XPD to follow up the issue.
ITDM
EET
CAS
- Who is responsible for the bunch pattern recording for the different instruments? CAS will clarify with ITDM and AE.
SRP
- The self rescue training is supposed to become part of the access requirement in the DACHS system for the tunnel. This caused a lot of concerns during the meeting (visitors? Access to components only in the very firrt part of the tunnel ). PSPO will discuss with SRP .==> SRP is trying to find a way to minimize the impact of this training on the XFEL employees. Discussion with DESY needed.
AoB
|
60
|
26 Jun 2019, 10:22 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 26.06.19 |
XO
From now on A. Galler will join the technical integration meeting. Therefore the XO field has been added.
FXE
SPB/SFX
- Detectors has been moved in the new position. MOV tests have been done. Today the detector modules will be swapped.
- EEE started the deployment of the updated Loop4 which now supports the MC2 motors.
- Loop 5 has been updated yesterday.
SCS
SQS
- AVC3000 leak controller from Pfeiffer. EEE will release the controls today. ==> Ongoing.
MID
- Integration of the LDM: next steps of the risk assessment: SRP is reviewing it, AMadsen has to sign it, then JHallman will call for a meeting with tin involved parties (e.g. EEE) for the implementation.
- Quadrant motion system test revealed some issue with the cables shielding. Issue is being tackled.
HED
- Loop 11 being set up by EEE. Start Today or tomorrow.
- Loop8 commissioning ongoing.
Optical lasers
VAC
- Requested to see the first pump + valve of SPB/SFX ==> definitions still needed by EEE == Ongoing.
XRO
- M1 in XTD6 is closed and pumped (10^-7 mbar range).
- XTD2 SA1 M1n closed and pumping. Leak test next week.
- Self seeding mono in XTD1 is now closed and pumping but the vacuum level is not yet satisfactory.
- After the last Karabo restart some devices are not working and this should be fixed before the end of the shutdown. Robert is the contact person in CAS and will follow the issue up.
- CRL in XTD2 maintenance has been completed.
XPD
DET
EEE
- The timing uTCA will be switched off on the 24.06 between 9 am and 5 pm. ==> there are some issues with different versions of power supplies. EEE on it.
- Electrical safety regulation. From the beginning of July the new electrical safety regulation is supposed to be in place. According to this regulation the work at the back of the racks in the XHEXP1 and in the entire racks in the tunnels can only be done by certified EFK (Elektrofachkraefte) or by EUPs with an additional training. The additional training can only be provided after summer. The nomination of XFEL EFKs is suffering a delay due to the fact that is not clear who should nominate them (TS, UBR, HR?). PG reported the problem and also the fact that no official statements were made in the past concerning the topic. THaas is trying to clarify the status, and come up with a statement and proposal in the Operations meeting.
CAS
- Release cycle for the next Karabo version in progress.
ITDM
- At 10 today the doors of the balcony rooms and racks will be opened due to short interruptions (20 minutes per tunnel) of the cooling water from DESY. This should not cause any issues. If the pomps intervention will require more time, the ITDM racks will be switched off.
SRP
- From SBP/SFX: Currently, it is not clear what to do if one of the Pandoras goes off. This apparently happened @ FXE on the night from Sat.-Sunday, and it took 12 hours to fix it. Update from SRP: Occasionally Pandoras trigger radiation alarms. It is not clear why. One possibility is that this is caused by electronic noise. Discussions with D3 are ongoing. Right now an expert is needed to reboot the system since it is not a simple off/on procedure, but data has to be analysed. There is a a document in preparation by SRP/D3 that will allow more people to be able to perform this procedure, but it is not ready yet. A side note is that there is no OCD agreement in place for these experts and the reboot is done on the best effort level. ==> Z.Ansari will be also able soon to reboot the system. ==> The issue will eventually be put up for discussion in the operation board. Statement from SRP is that there are nnot enough people to ensure an OCD service.
- Racks in HED outside the rackroom. They are too high and the top cannot be reached. SRP/JE/SRP will follow the issue up. ==> A solution has been found: TS will modify the existing movable stairs.
TS
AoB |
71
|
24 Jul 2019, 10:54 |
Adriano Violante | General | Information | Minutes of the Technical integration meeting 24.07.19 |
XO
FXE
- DAQ issue reported (data not being recorded). Comment from ITDM is that this was a miro issue solved with the re initialization of the proposal. More details will eb discussed offline.==> issue fixed (it was due to a misconfiguration in the DAQ server)
SPB/SFX
- Loop 12 has to be updated
- SASE1 M3 issues with the pitch motor lead to FXE losing beamtime. EEE to follow it up.
SCS
SQS
MID
- By the end of the week MID should send the requirements for the vacuum interlock to EEE. ==> tests are ongoing. AS will organize a meeting with CAS, VAC and EEE to define a test procedure to be followed by the instruments when testing the vacuum interlocks.
- Calibration server for mid has memory problems and needs to be replaced. Calibration pipeline has to be moved to a different servers. Will be discussed offline with MID.==>
HED
- Yesterday the loop 6 was updated. There were issue with pumps switching off and causing overhead for HED. PG will investigate why this happened... ==> the procedure was clear to EEE and foresaw the switching off of the pumps at some point. This was supposed to be clear also to the instruments buy clearly it was not the case.
- Nanocube PLC has been integrated.
Optical lasers
VAC
XRO
XPD
DET
EEE
- Crash of PLCs because of not a number values. The fix on the PLC side requires the update of the loops framework and can be done only in the August shutdown. By the end of the week a filtering tool will be availbale in Karabo.
CAS
- Karabo 2.5.1. hotfix is being prepared (Fixing 2 bugs in the framework and several GUI issues). ==> Done
- Tomorrow at 14:00 there will be a test of the pulse on demand feature (conditional triggers) in FXE. (AS/BF) ==> Done
ITDM
- Calibration server for mid has memory problems and needs to be replaced. Calibration pipeline has to be moved to a different servers. Will be discussed offline with MID.==> It is ongoing. In general this issue led to a loss of beamtime for MID. Alessandro suggests that in case of hardware failure there should be a joint ITDM-CAS decision/effort on what to do.
- Data migration: it was slow in the past days due to some script not being used for safety reasons. This is being fixed. Hovewer, due to the queue of old data to be migrated it will take some more time until full performance will be reached.
SRP
TS
AoB |