All trigger problems in XTD2 and XTD9 solved except for one: spb_xtd9_imgpi, probably a cabling issue on the camera side.

IP address problem for camera of sa1_xd2_imgpii45 solved.

Dear All,

We have just changed the time that the Radiation Protection Measurements will take place on Monday. This will now begin at 15:00. The time has been pushed back so that the DESY Radiation Protection group will be able to also be there.

I will inform you if anything changes.

I have deployed the Karabo master (the HEAD of our development) in the control network for the Vacuum loop device server.

During last weeks Karabo was not able to cope with the data coming from the PLC for this loop (our biggest to date)
and would become unresponsive after 5 to 10 minutes.

The current system is online since 3.5 hours and is still responsive.

Please note:
This is a partial release and the GUI might show unwanted behaviour.
Please leave the vacuum loop untouched during the weekend.

In attachment you can find an example of unwanted behaviour and the DPS gui

Thank you very much and have a nice weekend!

Alessandro

This is the list of data coming directly from the PLC via its TCP client connected to the EPS PLC.

Format:

deviceName  temperature  status errorCode

XTD2

==================================================

SA1_XS3_SHUT/TSENS/T1 0.4000000059604645 0xa020 0x0
SA1_XS3_SHUT/TSENS/T2 0.4000000059604645 0xa020 0x0
SA1_XS3_SHUT/TSENS/T3 0.4000000059604645 0xa020 0x0
SA1_XS3_SHUT/TSENS/T4 0.4000000059604645 0xa020 0x0
SA1_XTD2_ABS/TSENS/T1 0.0 0x20 0x0
SA1_XTD2_ABS/TSENS/T2 0.0 0x20 0x0
SA1_XTD2_ATT/TSENS/A1 30.739999771118164 0x20 0x0
SA1_XTD2_ATT/TSENS/A2 1372.0 0x15020 0x0
SA1_XTD2_ATT/TSENS/A3 30.380002975463867 0x20 0x0
SA1_XTD2_ATT/TSENS/A4 31.05000114440918 0x20 0x0
SA1_XTD2_ATT/TSENS/A6 31.130002975463867 0x20 0x0
SA1_XTD2_ATT/TSENS/A7 30.78000259399414 0x20 0x0
SA1_XTD2_ATT/TSENS/A8 30.98000144958496 0x20 0x0
SA1_XTD2_ATT/TSENS/A9 33.5 0x20 0x0
SA1_XTD2_COLA/TSENS/T1 0.0 0x1020 0x0
SA1_XTD2_COLA/TSENS/T2 0.0 0x1020 0x0
SA1_XTD2_COLB/TSENS/T1 0.0 0x20 0x0
SA1_XTD2_COLB/TSENS/T2 0.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_10_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_10_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_1_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_1_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_2_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_2_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_3_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_3_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_4_T1 33.60000228881836 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_4_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_5_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_5_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_6_T1 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_6_T2 85.0 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_7_T1 33.60000228881836 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_7_T2 33.5 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_8_T1 33.60000228881836 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_8_T2 33.5 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_9_T1 33.60000228881836 0x1020 0x0
SA1_XTD2_CRL/TSENS/LENS_9_T2 33.5 0x1020 0x0
SA1_XTD2_DPS1/TSENS/FLOW 2.0300002098083496 0x20 0x0
SA1_XTD2_DPS1/TSENS/RETURN1 33.5 0x20 0x0
SA1_XTD2_DPS1/TSENS/RETURN2 2.0200002193450928 0x20 0x0
SA1_XTD2_DPS3/TSENS/FLOW 33.5 0x20 0x0
SA1_XTD2_DPS3/TSENS/RETURN1 4.160000324249268 0x20 0x0
SA1_XTD2_DPS3/TSENS/RETURN2 4.420000076293945 0x20 0x0
SA1_XTD2_PBLM/TSENS/BLADE_LEFT 32.900001525878906 0x20 0x0
SA1_XTD2_PBLM/TSENS/BLADE_RIGHT 33.5 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_BOTTOM_B4C 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_BOTTOM_W 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_LEFT_B4C 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_LEFT_W 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_RIGHT_B4C 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_RIGHT_W 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_TOP_B4C 0.0 0x20 0x0
SA1_XTD2_PSLIT/TSENS/BLADE_TOP_W 0.0 0x20 0x0
 

XTD9

============================================================

SA1_XTD9_PBLM/TSENS/BLADE_LEFT 24.31999969482422 0x20 0x0
SA1_XTD9_PBLM/TSENS/BLADE_RIGHT 24.810001373291016 0x20 0x0
SPB_XTD9_COLB/TSENS/T1 0.0 0x20 0x0
SPB_XTD9_COLB/TSENS/T2 0.0 0x1020 0x0
SPB_XTD9_DPS1/TSENS/FLOW 33.5 0x1020 0x0
SPB_XTD9_DPS1/TSENS/RETURN1 85.0 0x1020 0x0
SPB_XTD9_DPS1/TSENS/RETURN2 85.0 0x1020 0x0
SPB_XTD9_DPS2/TSENS/FLOW 85.0 0x1020 0x0
SPB_XTD9_DPS2/TSENS/RETURN1 85.0 0x1020 0x0
SPB_XTD9_DPS2/TSENS/RETURN2 85.0 0x1020 0x0
SPB_XTD9_PBLM/TSENS/BLADE_BOTTOM 23.05000114440918 0x20 0x0
SPB_XTD9_PBLM/TSENS/BLADE_LEFT 22.860000610351562 0x20 0x0
SPB_XTD9_PBLM/TSENS/BLADE_RIGHT 23.05000114440918 0x20 0x0
SPB_XTD9_PBLM/TSENS/BLADE_TOP 23.05000114440918 0x20 0x0
SPB_XTD9_SHUT/TSENS/T1 0.0 0x1020 0x0
SPB_XTD9_SHUT/TSENS/T2 0.0 0x1020 0x0
SPB_XTD9_SHUT/TSENS/T3 0.0 0x1020 0x0
SPB_XTD9_SHUT/TSENS/T4 0.0 0x1020 0x0
FXE_XTD9_COLB/TSENS/T1 0.0 0x20 0x0
FXE_XTD9_COLB/TSENS/T2 0.0 0x20 0x0
FXE_XTD9_PBLM/TSENS/BLADE_BOTTOM 22.780000686645508 0x20 0x0
FXE_XTD9_PBLM/TSENS/BLADE_LEFT 22.8700008392334 0x20 0x0
FXE_XTD9_PBLM/TSENS/BLADE_RIGHT 22.850000381469727 0x20 0x0
FXE_XTD9_PBLM/TSENS/BLADE_TOP 22.93000030517578 0x20 0x0
 

 Now the loop is accessible to Karabo as well with the project SA1_EPS

Dear all,

the Tunnel Access Schedule has been update - potential access has been shifted from Wednesday the 19.7. to Tuesday the 18.7. Should you need this access, please inform Harald Sinn!

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

Dear all,

I updated the tunnel access schedule - as you can see there will be no access tomorrow. Any access requests regarding next weeks Tuesday as usual to M. Dommach, cc to Harald Sinn and T. Tschentscher.

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules&listViewLinkBack=true

Cheers, Gerd

Dear all,

I updated the tunnel access schedule to reflect the access to the XTDs tomorrow.

Cheers, Gerd

Dear all,

I updated the tunnel access schedule to reflect the access to the XTDs tomorrow.

Cheers, Gerd

SPB

• Test on the mirrors peltier showed some issues due to the fact that the test boxes were different from the actual stuff. R.Bean will clarify it with FMB.
• Loops 12,13 update to be done.
• Magnetic turbos control: the serial communication has some issues. Being checked with AE and CAS

SQS

• Loop1 cabled up. Some names haad to be corrected in the firmware. As soon as the export is delivered by EET to AE the formware can be uploaded, Probably today.
• Upstream turbo pumps running using controlers in the rackrooms but still no PLC control.

SCS

• Some issue with the movement of the motors found. AE is on it.
• 1 crate does not have enough power to power up all the terminals. Will be solved today.

​AE

• The valves will be included in the interlock systems (SASE1 issue) - Loop ready to be updated - The MPS has to be disabled before updating the loop to avoid tripping of the beam.
• FXE loops update started

DET

• Hotfix for AGIPD ready to be deployed. Can start today. Details will be sorted out with SPB.

CAS

• SASE3 Soft Mono is now technically commissioned. PBLMs next in the line.
• XTD1 CRLs will be integrated iin the control system in KW 22
• XTD6 priority on the MID beamline. 

General

It is not clear if and when the experiment in SASE1 could get beam during this week. Has to be checked with the PRC.

SQS

• Loop1 still being worked by AE. Not yet up and running. 
• Loop 3 is needed to be up and running in 2 weeks since the mirror motors ar eincluded in this loop and they are needed for the final check with FMB.

SCS

• Bernard has been added to the SCS operation logbook and he is getting al the updates concerning commissioning.
• Testing of the Loop 1 motors ongoing., 2 out of 20 motors have problems. Being investigated.
• Some connectors names are reported not to be consistent with the EPLANS. Martin Teichmann will check that with EET.

​​AE

• SCS:
  • loop 2 running. Bernard has recommended that you start with commissioning of KBS_VDM, _HFM, _BFL, VFM motors because they are using MC2MOTOR soft device class (controller is Elmo).
  • Crate BKF00157 can be removed and refresh terminal will be added without interrupting the PLC - only equipment connected to this crate will not be available during this period of change.

• SASE3:
  • -Request from Andreas Koch about interlocks for MCP and LED Power in SQS/SCS (this installation is unique in SASE3). For that purpose we have to update EPS, MOV2 and Master loop in SASE3. PRC will be triggered by E-Mail

• SASE2:
  • -We want to update SASE2 in order to get beam permission interlocks properly. Therefore we need to update, EPS, Vacuum loop and Master loop. PRC will be triggered by E-Mail

• SPB:
  • -Today we will deploy loop 12 and 13 and Master loop. In the Master loop we will add beam permission conditions, so we want to test it and therefore we will need to open all the valves in SPB as well as the last ones in the Tunnel. Has to be decided when it is possible with the SPB people.

• FXE:
  • -All loops updated past week. Waiting for answer from FXE about when we can test the same beam permission conditions as in SPB. To be agreed with FXE.

DET

• Working of the optimization of the python piece of code allowing the display of online images. Right now 30 images per second rate is possible.

ITDM

• GPFS server fixed.

CAS

• XTD1 CRLs will be integrated iin the control system in KW 22
• 20.07 is the goal date for beam in MID and HED optics Hutch.  

​​AE

• SASE3:
  • -Request from Andreas Koch about interlocks for MCP and LED Power in SQS/SCS (this installation is unique in SASE3). For that purpose we have to update EPS, MOV2 and Master loop in SASE3. PRC will be triggered by E-Mail ==> AE updated the loop. Nerea is making sure that the MPS signals are enabled in the BKR 
  • Stepper motor for the mirror bender was found to be miswired. Has been already fixed. Similar issue (inverse wiring) was found in SASE1. Cannot be fixed now. It will be done in the next shutdown. For the time being one has to drive the morot the opposite way.

• SASE2:
  • -We want to update SASE2 in order to get beam permission interlocks properly. Therefore we need to update, EPS, Vacuum loop and Master loop. PRC will be triggered by E-Mail ==> DONE. Not clear if some motor parameters for XRO devices were saved before. D. La Civita and CAS are checking that

• SPB:
  • -Today we will deploy loop 12 and 13 and Master loop. In the Master loop we will add beam permission conditions, so we want to test it and therefore we will need to open all the valves in SPB as well as the last ones in the Tunnel. Has to be decided when it is possible with the SPB people. ==> DONE. 

• FXE:
  • -All loops updated past week. Waiting for answer from FXE about when we can test the same beam permission conditions as in SPB. To be agreed with FXE.

ITDM

• DAQ has to be upgraded to thelast verison of Karabo (just hotfix). Will be checked with Gero what is the best way to go.

SPB/SFX:

• Mirror installation next week

FXE:

• Replaced broken diamond-window

SCS:

• Installation of alignment laser
• Mirror tank installation next week

SQS:

• Not present / no issues.

SASE tunnels:

• C. Youngman is coordinating the works in the SASE1, 2 and 3 tunnels, daily meetings are at 9:30 every morning.
• Works have been scheduled and planned, most work to happen in SASE3 and SASE2.
• Staff should be aware that works e.g. regarding the vacuum loops have been scheduled and are ongoing, in doubt contact C. Youngman, N. Jardon or V. Bondar

CAS:

• Release test with Karabo 2.2.3 are concluding probably today, including test with PLC firmwares 1.26 and 1.27.
• This would lead to a release for ITDM & Detectors by this evening. A timeslot is required to test the new Karabo with ITDM / DAQ.
• Deployment at the beamline could potentially be in two weeks, if the test with ITDM and Detectors will conclude fine.

XPD:

• Email to xfel-op-allhands was received, indicating that there could have been a shortage of cooling water for the magnets and racks supplied from XS4 and also XS2 (should be SASE2-branch only). If you have electronic racks in this area, check the temperature, email will be forwarded to readiness.

Detectors:

• A single supermodule of the LPD-detector is flickering, is being investigated.

ITDM:

• A rise of the temperature in the balcony room from 27° to 39° was observed in the morning from ca. 6:00 until 6:30.
• This lead to damage at two power supplies.
• There was no indication by MKK or TS about any kind of shortage or shutdown of the cooling water supply.

SPB:

• Ion pumps running in the Mirror chamber.
• Starting test with mirrors together with CAS this week.
• In general everything in good position to receive beam.
• Primary limits for coupled motion not included in E-plan, safety switches are implemented but SPB prefers to have the primary limits included. This will suppose a hardware modification.

AE:

• SASE3: train IDs in loop2 are working, the problem was fixed last Friday.

ITDM:

• Measurements for commissioning continues.

CAS:

• H.Santos inform that CAS has not resources to run tasks for SASE2, this might affect the date for first lasing in SASE2. Priorities are SASE1 operation and SASE3 commissioning.
• G. Wellenreuther will bring this topic to the PSO meeting.

FXE/VAC/Detector/XPD: nobody attended

Tunnels (J. Eidam, J. Gruenert)

• SASE3:
  • ​1st section is not recovered yet, ongoing.
  • Diff. pumping XGM is being installed
  • Leak tests ongoing
  • MCP manipulator XTD2: company got the training - repair ongoing 
  • Absorber was moved ==> has to be realigned by MEA
  • A. Koch is commissioning the imagers. Some bugs were found and will be solved with the new release of Karabo.
  • MCP, FEL-Imager: Crates are there, AE still has to understand and define the priorities of their work
  • No access to the tunnels with visitor cards! (Access to tunnels requires rebreather training, knowledge about procedures, warnings, dosimetry etc.)
  • Exchange of gotthard detector in XTD 9 ongoing.
• SASE1:
  • ​XGM Inihbit system will be brought up tomorrow. VAC loop needs to be brought down . Will be notified with a message in the logbook.

Electronics (P. Gessler)

• Responsibles
  • SASE1: Bernard Baranasic
  • SASE3: Jan Tolkiehn
  • Instruments SASE1: Tobias Freyermuth
• Fast valve terminals exchanged today. 
• Documentation by WP74 + Vacuum is not consistent ==> AE will provide an Excel-sheet to be used, which should give a consistent picture of all terminals, connections etc. Ongoing

ITDM (K. Wrona)

• Both online & offline-filesystems have to be updated. Postponed to TOMORROW (16.10.): Online-FS will be down, offline-FS will be worked on continuously, should not be down, but at reduced performance, since update will be server by server.
• DAQ-Upgrade: still running Karabo 2.1.12 on detectors, which requires lots of restarts. UPGRADE: TOMORROW 13==>16 to Karabo v.2.1.16

Software (H. Santos, A. SIlenzi):

• Karabo 2.1.18 being tested. Test to continue for a couple of days before deployment.
• S. Brockhauser will report on the status of the scan tool this Friday.

General:

• Shutdown started.

Instruments

• SPB:
  • AGIPD prepared for exchange modules intervention
  • AE is in contact with SPB for crates issues fixing
  • The schedule for next year, especially when the experiment has to be ready before the users run has to be known as soon as possible.
• FXE
  • Starting loops update in the next days. Reason for the small delay is that FXE is checking all the default parameters to be able to restore them after the update.
  • In order to solve some DAQ issues, FXE requested to have their own topic. AE agrees, ITDM agrees, CAS is trying to find out how to do it and to do it before the next users run. Technical details have to be discussed offline with FXE.

ITDM

• in KW 51 (18-22 Dec.) the DAQ and online cluster will not be available due to recabling works in the balcony rooms.
• Update of the servers in the experiments has to be done. ITDM requested to know from CAS when the major update of Karabo is taking place to coordinate the effort and minimize the downtime. CAS will present a schedule concerning Karabo this Friday.
• 700 Tb of data are stored on the online cluster. It has to be communicated by the experiments what can be moved from there, since there is almost no space left.

Diagnostics

• Implementation of Gotthard detector in Karabo is still a problem. S.Hauf will invite the main groups involved for a meeting to find a solution.
• The data correlation with time stamps when using the Karabo DAQ has to be done carefully and contacting WP74 since there are still issues present. 

AE

• Activities on the Hirex in SASE1 will need the MOV loop to be brought down in the afternoon. An entry in the ELOG will be written.
• SPB loop 8 prepared. 
• Full list of new crates belonging to the motion loop in SASE3 tunnels has to be delivered to AE
• Full list of new crates belonging to the IPS loop in SASE3 tunnels has to be delivered to AE in order to be able to close the loop.
• Technosoft controller issues in SASE3 will be attacked once the SASE1 activities will allow it.

VAC

• VAC testing ongoing.
• 2 mirrors will be installed before Christmas
• Motion crates for the motion lopps and test of the VAC interlock will happen this week.

SPB:
- The exchange of the mc and mfpga boards has been done and some problems were found. The boards are now being debugged at CFEL.
- Sample enviroment's group is testing the liquid jet this week.
- Wait for the Loop 5 to be updated by AE

FXE:
not represented
Vacuum:
not represented
XPD:
not represented

Laser:
not represented

Detectors:
not represented

AE:

- Timing devices update done. One crate in XTD2 has a problem and cannot be operated remotely. Fixing ongoing.
- The clock, VETO and Start signals for the AGIPD could not be brought correctly from the utca in the rackroom to the patch panel in the hutch due to the excessive cable length. Workaround found using a signal replicator at the patch panel position. The problem seems to be solved.


ITDM:
 - LPD recabled with Infiniband connection. Test succesful.
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company is comeing back next week to check the 
cabling. Interventions have to be planned accordingly since the Beckhoff 
loops are running! ==> no news.

CAS:
- not represented

SPB:
- one mfpga board was found malfunctioning and was replaced with the old one. Concerning the mc the new version can be used.
- Liquind jet test ongoing.
- Waiting for the Loop 5 to be updated by AE
- Old version of Karabo was sluggish as of yesterday.
- 2.2.1 Karabo verion being rolled out for SPB - test ongoing.

FXE:
not represented
Vacuum:
- SASE1 vacuum in the tunnels closed (CRL chamber closed).
- KB mirror chamber in D.02 (SPB optics hutch) will be opened at the beginning of February
- In SASE3 the gas supply is now decoupled from the vacuum loop
- Soft XRay mono vacuum to be closed today.
XPD:
not represented

Detectors:
not represented

AE:

-XTD 2 CRL to be tested tomorrow
- Update of loops ongoing
- Emergency off button test in FXE showed that the relative controlling PLC was not configured correctly. In addition, communication problems between TS and FXE made things more difficult. PSPO will talk to TS and decide how to proceed. In the meanwhile the test foreseen in SPB for next monday HAS BEEN CANCELLED. GWellenreuther has informed Torsten Schoen.



ITDM:
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company solved the problem without bringing down the loops.

CAS:
- 2.2.1 Karabo was deployed in FXE experiment and tests are ongoing.

- 2.2.1. being rolled out in SPB exp and Laser

- SASE3 tunnels  tunnels will get the new Karabo version next week. SASE1 tunnels will follow (probably in the same week) 

General remarks:

• Stable beam only to be expected starting from 1.03. for SASE1 instruments commissioning
• There is no guarantee for parasitic beam delivery to the experiments. If it happens at all it  will most probably be during the last week of Feb. This has to be agreed on the fly  with the BKR and H.S/ T.T.

SPB/SFX:

• Preparing D.02 for mirrors installation (next week)
• Karabo being updated

Advanced Electronics:

• Trigger delivered by DESY should be back and stable at 10Hz. 
• ATM for timing has to be replaced in the balcony rooms. Bruno Fernandes is trying to find a way to reduce the downtime down to 30 minutes. Will be scheduled in the next days.

ITDM:

• 1 line card has to be replaced. Probably happens on Wednesday this week during the DAQ tests.
• Tuesday afternoon the online storage units will me remounted with the same path of the offline ones.
• This week a proposal for the in-house commissioning is going to be circulated.

SPB/SFX:

• Preparing D.02 for mirrors installation (next 2 weeks no access to D.02)
• Karabo being updated - Detector being tested, seems fine so far.
• DAQ tests foreseen for today
• NKB movement (Loop3) should be tested to make sure that the controls work fine before interfacing the MKB with real mirrors inside.
• AE will check the status of the CSS crate. If ready it will be mounted and tested after the MKB.

Advanced Electronics:

• ATM for timing has to be replaced in the balcony rooms.  downtime expected to be  15 minutes. Will happen Today at 16:00

CAS:

• Deploy SASE1 topic in the tunnel this week.Tomorrow at 13:oo CAS would like to make the switch. A.Silenzi will write an entry in the logbook to make sure everyone is on board with that.
• Release of the version 2.2.2. being prepared. This would mainly improve the GUI performance.

XPO

• FXE Mono local test performed and OK. To be tested through Karabo in Feb.

CAS SASE3 (C. Youngman):

• Gas attenuator controls being completed and tested.
• Soft MOno local test done and only small problems found
• EPS loop in good state - interlock test tomorrow.