Photon Run Coordinator (N. G. Kujala):

• SASE1
  • Machine delivery 300 pulses upto XTD2 without M1 mirror in beam on Wednesday evening.
  • On Thursday 300 pulses deliver to FXE for radiation test.
  • Collected data using DAQ with 100, 200 and 300 pulses.
  • Large area Detector40 x 40 mm was set-up for radiation test at FXE for .
  • Damage test with 300pulses in FXE hutch with 0.5mm steel, 0.2mm copper and B4C block   
  • Vacuum at XTD2 absorber was stable and temp was also stable for 300 pulses
  • Harald setup piezo motors for M3 of SASE1.
  • Yesterday night beam delivery to SPB with single bunch.
  • ZZ for XTD9 on Tuesday by SPB for CRL test/check.
  • Vacuum on FXE branch in XTD9 was tripped on Tuesday early at 1am, which lost beam permission. Manually I have reset interlock.
• SASE3
  • SASE3 commissioning is going on.
  • Commissioning the XGM and IMG correlations with Gas ATT and without Gas ATT;
  • SoftMono mirrors is commissioning
  • Gas attenuator commissioned and Raul made look-up table for pressure setting
• 30bunches were operated in SASE3 with Gas attenuator

SPB

• Ready for first commissionig
• AGIPD power and cooling procedures OK
• Feedback problems between Karabo and AGIPD. Being worked out

SQS:

• Hardware installation starting Mid. March after  cleaning of the hutch
• FMB will install the mirror tanks in April. Has to be synchronized with the remaning work in the hutches
• Meetings with AE soon to strat planning the commissioning

FXE (P. Zalden / A. Galler):

• Ready to accept first beam
• Radiation tests done

SCS (J.T. Delitz)

• Removing of wood cover: 05.02.2018

• Cleaning: end of week 11.

• End of march beginning of the installation of the first components (ALAS and TIM)

• Waiting of the exact date for Interlock test to specify more exact the dates for the KB mirror system installation.

PP-Laser (M. Lederer):

• LAS is now commissioning 3-stage NOPAs for both FXE and SPB.
  ​so far all in plan to finish in March

Vacuuum (R. Villanueva):

• SASE1 
  • Nothing remarkable: the whole vacuum system works in a very stable way. We only had one incident on the output of 1-2 ion pumps that trigger once the interlock. For that we are investigating the causes .
• SASE 3.
  • The whole vacuum loop (as required for the first lasing) down to the instrument front-end is under control and no special issues have appeared yet.
  • The SASE3 Gas Attenuator is under manually operation and most of the Photon Commissioning Members are familiar with it. Next steps are aligned with the deployment of the Pressure Feedback Control PID control available at Karabo level and to continue with the software-related tasks.
• SASE2.
  • XTD1------->The complete vacuum string down to the front-end in the shaft building is under vacuum but in local mode operation (including XGM subsystem)
  •          ------->During this almost past week we already initiated the PLC local tests (i.e. US & DS Differential Pumping are already done), and in KW10 we aim to move on with the rest of the system (ion pumps and vacuum sector gate valves).
  •          ------->Absorber, V0 and Fast Valve wiring are in good progress and we think they could be ready for local tests by the beginning of next week.
  •          ------->The interlock tests are planned also for KW10.
  •          ------->Depending of the progresses of AE and CAS, we may even be able to deploy the first control scenes in Karabo.
  •  
  • XTD6 ------->The assembly works of the whole beamline there progress adequately and without remarkable "show-stoppers" that couldn´t be managed.
  •          ------->Vacuum Group and Optics group (N. Kohlstrunk) are having also special attention and dedication to the SDU vacuum chamber conditioning and preparation works for its upcoming in-vacuum mechanics installation.

XPD (T. Maltezopoulos ):

• XGM in XTD1 is connected to DOOCS (together with DESY)
• Synchronization of XGM@XTD1 microTCA next week
• Local tests at XTD1 XGM were finished (together with AE)
• XTD2, XTD9, and XTD10 XGMs are running 24/7
• Change to 300 bunches went without problems
• XTD9 XGM need SRG recalibration ... done meanwhile
• Wajid, Joakim, Tommaso, and Theo will repeat  SASE3 DAQ Imager/XGM correlation today (need single shot operation)

Advanced Electronics (P. Gessler):

• Broken terminal between SASE3 MOV2 PLC Loop and Master Loop were replaced last Friday without any side effects. The Train ID is now also provided from the MOV2 loop.
•  So far, no PLC related export from the SASE2 Tunnel eplans of the vacuum, beam transport and diagnostics PLC crates are possible. Therefore we will build up the XLS file for the start up of the VAC and XGM loops next week. However, the related group plus AE and EETF should meet to evaluate, if an export is possible until end of March in order to use it for further installation work. Otherwise XLS files have to be crates manually again with support from all involved groups. Also consistent documentation is required between diagnostics on one side and vacuum and beam transport eplans.

SASE2 Tunnels (C. Youngman)

• Table with the set priorities and deadlines available in Indico.

ITDM (K. Wrona):

• Run Configuration Groups required for the user experiments for FXE and SPB/SFX have been defined and tested
• Discussion after the meeting: Since the Karabo-DOOCS interface for XGM will be changed and it will affect also how the data arrive at DAQ data aggregators we need to modify and (re)test the BEAM_COND group.

Detectors (J. Sztuk-Dambietz, P. Lang):

• AGIPD will operate with the old firmware due to problems with the new one
• No calibration data available as of today
• AGIPD is in a safe state

CAS (S. Brockhauser):

• Gotthard for HIREX being operated
• Progress on GATT commissioning by Valerii

SPB

• Last night the beam disappeared between 4 and 8. 
• AGIPD scene showed many problems, both concerning Karabo and Data preview. Detector is not operable.  To solve the issue a joint effort of all the groups involved, as well as the physical presence of experts during the shift is requested. 
• CSS has to be moved out of the beam. As of now there are problems with cabling that need to be solved
• Smaract stages are misbehaving. More support from AE (Mahadi) is requested
• Status of the coordinated motion test was asked. Should be ongoing but the experts were not there.

FXE:

• Testing fast intensity monitors and LPD 
• DAQ worked very well
• Scan tool must me tested
• Correlation between LPD and fast intensity monitor is problematic due to train id problems. There is a new device to be deloyed on the XGM. Today or tomorrow will be seen whether this solves the issues.
• BKR reported that the beam tripping was due to some FXE device close to the M3, but FXE is not aware of that. Info will be asked to Harald today during the PSO meeting.

Advanced Electronics (P. Gessler):

• MCP HV problem in SASE3 - the power supply failed. The fixing shoud be done according to the priorities, Photon run coordinator decides.

ITDM (K. Wrona):

• FXE slow data DAQ issues: these will be solved splitting the LPD slow data from the rest. small update of the software needed. 
• Simultaneous acquisition from LPD and cameras is now available and working well.

Detectors (J. Sztuk-Dambietz, P. Lang):

• AGIPD will operate with the old firmware due to problems with the new one
• No calibration data available as of today
• AGIPD is in a safe state

Below some clarifications from R. Bean on the issues reported in today's dispatch meeting:

- AGIPD suffered many problems which affected the operability (not really a problem of the scene). It was just operable, although not at all efficiently. Problems with the pipeline / DAQ and Karabo controls for the AGIPD suggest that some reasonable amount of work is required to find and solve the problems which may not be independent from each other

- CAS OCD point already addressed with previous email: They were available, just we needed them rather more than we should have done

- The CSS is the large rail that carries the detector. We cannot move it via the integrated system because of a misunderstanding in the implementation of limit / safety switches into the eplan design of the crate. The cabling between rack room / device seems to be fine (but of course not really tested properly because the crate is not correct). We have to use the test crate to move it right now, which is rather tedious since the encoders / limits cannot be handled properly by that system. I am available to discuss the proper implementation of switches / buttons such that we don’t need a fourth attempt at this crate

- For the smaracts we still have axes that are not controllable via the PLC. It is these that would be beneficial if they are fully integrated 

SPB:

• Monday morning Karabo failed. It was not possible to operate anything. Aftrer restart it took 2 hrs. to have the tunnel loop and Lopp 5 again operable . Other issues connected to that. CAS is on it.
• AGIPD can be operated with some workaroung concerning the control system. No final fix available yet.
• CSS can be operated locally.
• Coordinated motion test probably on 14.03.

FXE:

• IPM/XGM correlation data process ongoing. 
• DAQ worked reliably
• Scan tool operational for almost every component. Data storage works. Test ongoing.
• PG points aout that no MPS definitions were given and no MPS is available for FXE yet. To be checked.

SQS

• Corrdination of installations in the hutch with the TUV tests for Rad. Interlock needed.

SCS

• Nothing ton report

ITDM:

• FXE LPD slow data separation will be done today after the meeting
• Some devices are setting the train id as "0". CAS is informed.

SASE3

MCP HV - Crate restarted. Now ok. There was a misconfiguration of Power/Karabo settings. This meeds to be avoided increasing the communication between people performin operations on the XPD components and the technical support groups.

AE

• uTCA crate in SASE2 for timing being prepared, as well as PLC for VAC loop
• Replacement of a uTCA crate in the balcony room for SASE2 needed

Photon Run Coordinator (L. Samoylova):

Commissioning during machine studies Tue and Wed and parasitically during SA1 user operation at 
 SASE3 on Thursday and Friday 

SASE1

SPB

User operation, current status (Adrian): got few data from a test system on Saturday. The users’ 
sample delivery technology now seems to deliver nice & stable droplets, so if SPB 
get beam early enough today to set up for a measurement, a full data set could be collected. 
 

FXE

• Instrument Commissioning
• Measure beamline transmission (unfocused 550 µJ per pulse at the sample position, with lenses 
  115 uJ)

 Machine studies

• Wed afternoon: a set of K-mono FEL spectra taken

https://ttfinfo.desy.de/XFELelog/show.jsp?dir=/2018/10/07.03_a&pos=2018-03-09T17:56:20 
 SASE3

 Thu, Friday - a first experience of the SASE3 commissioning in 30 bunch mode during user operation: 

• minimal number of undulators (12) for lasing, low intense SASE (XGM shows 10 uJ, linear 
  regime), SRA closed to 2 mm
• MCP2 @ SASE3 - signal from 30 SASE pulses, consistently decrease with closing the slit down 
  to 0.5 mm

https://ttfinfo.desy.de/elog/XMLlist?file=/XFELelog/data/2018/10/08.03_a&xsl=/elogbook/xsl/elog.xsl&picture=true#2018-03-08T18:24:14 

• Fri afternoon GATT calibration with XGM and MCP ongoing, in 30b mode, optimizing SASE3 lasing 
  for these measurements needed - too weak and not homogenous signal in the center, see 
  https://ttfinfo.desy.de/XFELelog/show.jsp?dir=/2018/10/08.03_a&pos=2018-03-08T18:26:50

  Problems:

 'unexpected' from photon side closing XTD1 two working days earlier, explicit schedule 

• for the south branch tunnels access would be appreciated
• power off in XHPSC on Thursday at 21:15, no cooling water in experimental hall that led to 
  increasing the temperature in the rack with the MicorTCA crate, the CPUs of the two MicroTCA 
  crates in SPB and one of the two crates in FXE were automatically switched off. SPB crate 
  1 did not come automatically back afterwards, as the result: no train IDs for SPB PLC 
  loops, no MPS for SPB, no Clock and Control for AGIPD, no triggers for SPB systems on 
  Friday
• Saturday morning the MPS interlock takes into account the signal from one of three fast valve 
  on SPB shutter although these valves are disconnected to SPB EPS interlock. The MPS 
  bit had to be masked

Correction from SPB [12.03.2018 10:10]: This issue was not related to the XTD9 shutter, but to the 
optics hutch shutter. The fast valve in XTD9 is connected to the EPS, the fast valve 
in the OH is not yet connected to the EPS. After the restart of the EPS/MPS on Thursday 
/ Friday after the cooling water issue, the condition for the fast valve in the optics 
hutch was wrongly included in the MPS. 

SPB

1) Thursday 08.03.2018 (8 am - 8 pm) - SPB/SFX commisioning shift

    Notes:

          - everything running

          - RAW data overview very responsive with  data filtering as suggested by Steffen

          - Corrected data preview started very nicely and seem to provide only limited update rate after a few hours running

          - karabo DAQ deices (dataaggregators) need several rests, resulting in a full DAQ restart around 7pm

          - sometimes updates in KARABO were lost

          - preparation for users experiment - over night

2) Thursday 08.03.2018 (around 9 pm) - "cooling water" incident

     Notes:

         - vacuum pumps affected

         - AGIPD warmed up

         - uTCAs have tripped/shutdown (some infrastructure needs to replaced)

3) Friday 09.03.2018 - morning (User beamtime)

         - damaged uTCAs equipment was replaced but trigger setting needs to be input/ adjusted =  loss of the user beamtime

         - User experiment progresses

SQS:

Hutch cleaning Mid. March

KB tanks installation in parallel with SCS

9-14 air filter will be installed. After that date only clean works have to be carried out

Collision with KB tanks installation and other activities like Rad. Interlock test to verify

FXE (P. Zalden / A. Galler):

Setting up the experiment

IPM and XGM data manualy correlated. Would be nice to have the train id working to have the "live" correlation

SCS (J.T. Delitz)

• cleaning of hutch next week (wk11)
• we will start moving first components wk12-13, ie. ALAS and DPS
• KB installation (positioning only, no AC+filter required) beginning April-depends on commissioning of precision climate zone (currently discussed with PI)

PP-Laser (M. Lederer):

SASE1 commisisonin gonoing.

Some misalignment due to the cooling water accident. To be investigated 

09.04 SASE2/3 laser tables will be installed.

Vacuuum (R. Villanueva):

SASE2--------->Good progresses on XTD1 control system deployment: Differential Pumping sectors are ready for first beam // First five vacuum sectors under vacuum and locally tested (already with Karabo). XTD6 tunnel installation phase progresses at good pace.

SASE1--------->In general, nothing additional to remark. We are working in a more robust interlock definition to prevent false triggering due of ion pump control signals.

SASE3--------->Complete vacuum system is working normally. Gas Attenuator is now under routinary operation up to 2-3 mbar with a UHV interface pressure after its DPS sector < 1*10e-9 mbar.

XRO 

SQS/SCS/XRO collaboration for the mirror coating ongoing

Installation of M5 in preparation

XPD (T. Maltezopoulos ):

XGM@XTD2, @XTD9, and @XTD10 are online.

XGM@XTD1 is in the loop and accessible from Karabo

(together with AE+CAS).

Currently I test XGM@XTD1 components from DOOCS.

XGM@XTD9 shows slightly higher energy then in XTD2.

I calibrate all XGM components twice, but still it remained.

Note that the beam pass the XTD9 XGM with -7 mm offset

(in beam direction 7 mm to the left, to the south).

With this high offset absolute calibration is not guaranteed.

Advanced Electronics (P. Gessler):

1 CPU for SPB broken due to the cooling water accident. Already replaced. Being configured

FXE grating integrated in the vac loop

SASE2 Tunnels (C. Youngman)

More coordination is needed, especially concerning the access to the tunnels. For the technical details, please see slides on the indico page https://indico.desy.de/indico/event/20115/

ITDM (K. Wrona):

No problems with overrheating

Problems with Train ID reliability after the heating. Should be checked and monitored online

Documentation of single devices have to be improved. + there is need for a tool to make the sanity check

Detectors (J. Sztuk-Dambietz, P. Lang):

LPD commissioning OK + DAQ working. Now LPD off because of cooling water problems.

CAS (H. Santos):

• Sandor is in a meeting with the Detector group to align requirements regarding AGIPD and DA
• We started to prioritize and address the (agile) missions of SPB and FXE. As part of this effort:
  • Alessandro started to develop the scenes link on the GUI. This feature is really required and has been present in several SRDs (For example, to link vac sections/loops).
  • We implemented the GUI combo box to select different GUI Server - it is part of the SPB Mission (Leonce)
• Hamed and Sergey made good progress on the Digitizer. They will work on a demo/talk for the next PBS Meeting.
• Wajid, Sergey, and Dennis fixed the X2Timer issue reported by FXE (hash/data inconsistency). It is already deployed and running.
• Regarding the cooling incident. Wajid was the first contacted yesterday. After that Alessandro at 01:00. Today, during the morning, we reinstall Karabo at SPB-RR-SYS-UTC-1.

SPB (Weinhausen):

• During the night there was an interlock issue. The fast shutter was triggering the MPS interlock system. After several calls and request of support from SPB to the photon run coordinator and the machine run coordinator the problem was found and the interlock signal bypassed. This was due to some incomplete cabling of the fast valve. While this was knwe, after the last updated the correct configuration was not set up in the controlling PLC.
• In general one has to note that the experiments do not have access to the MPS panels and there is no well established procedure when MPS interlocks are concerned. This will be reported in the PSO meeting.

FXE (Gawelda):

• Before the beam shutdown FXE observed that the beam was very unstable.
• 2 ion pumps went in error and the interlock was triggered. Again in this case is not clear who is responsible for the re setting of the interlock and the pumps.
• Servers had to be restarted due to wrong train id after the cooling water accident
• 0.5 MHz operation of the LPD successfully tested. 

SASE2 Tunnels (Youngman)

• It is reported that the technical coordination of the necessary activities is missing.
• A list of activities was sent by CYoungman to TTschentscher concerning SASE2 tunnels. In total 10 working days are needed to complete these tasks. This includes 1 week implementation and 1 week commissioning. Carefully planning with DESY is needed to avoid that the necessary 10 working days are scheduled around Easter, which will result in less effective days.

CAS (Santos)

• HSantos is preparing a deployment plan for Karabo whoch includes the requirements and deadlines to be fulfilled by the other support groups (ITDM, DET, AE). The idea is to have this plan presented on Friday for discussion.

SPB (Weinhausen/Round/Mancuso):

• Again on the fast valve was triggering the MPS interlock system. For XRO was not completely clear what happened. for AE everything is clear and it depends on the wrong MPS configuration with respect to missingl cables. The will discuss this offline and eventually bring back the issue. Linked to this, it is reported that the MPS panel is difficult to decipher for the experiments...

• FXE (Galler):

• 2 ion pumps went in error and the interlock was triggered. Issue clarified.

SASE2 Tunnels (Youngman)

• Access schedule sorted out. Works in the SASE2 tunnels will be carried out in the period 4-19.04 plus the week before Easter. Discussion with AE on the status of the loops ongoing.

AE (Jardon)

• It is reported by SPB (March Messerschmidt) that people in the BKR are confused about MPS signal that they are getting from XFEL. The different lables assigned to the signals have to be sorted out and communicated. Nerea and Marc will try to start this clarification offiline.
• GATT Loop in SASE3 will be updated today
• Check of the inficon gauges ongoing- eventually update required.

ITDM (Wrona)

• Lots of data (14 Tb) collected during the last SPB shift.
• Issue concerning speed of offline data transfer reported by SPB
• For the user runs in May data will need to be deleted from the offline cluster. A procedure has to be established to clearly communicate with GLs and User groups.
• SASE3 IT installations have been handed over to XFEL. IT setup ongoing. Network will be available in the experiments by the end of the month.

Photon Run Coordinator (H. Sinn):

1.) SASE1: User experiment SPB could be finished sucussfully after several problems on the weekend (thank to FXE for providing night shift). Big progress in improving throughput of FXE insrument. Next D3 material test with 300 bunches on April 3, 9am. 

2.) SASE2: Transmission of electron beam to T5D dump (SASE2 beamline) this week sucessful. Switching between electron beamlines 

works nicely (minor problem witth dark current during switching).

Now air coil measurement going on. BBA needs to done before first lasiing starting on May 2nd. 

To be clarified today: To we need access to XTD in week 12/13.

3.) SASE3: Photon commissioning (with 24/7 shifts) finished sucessfully. Now specific tasks are done parasitically to SASE1 operation 

during regular working hours (organized by photon run coordinator). 

4.) General (stimulated by DESY operators): EPS system can lead to shutdown of DESY accelerator through MPS by many different channels (PLC in tunnels, Fast valves, PBLM etc). 

On DESY side, MPS provides overview of inputs, but structure difficult to understand and labels are not always conclusive, 

On XFEL side, no general overview panel for all SASEs+Instruments (scene) exists, but it required for facility operation. 

SPB

No issues/no report 

SQS:

- Cleaning of the hutches is ongoing. From 19.3. on, access will be limited. Furthermore, shoe covers have to be worn in E.06 and E.07.

- Electricity in E.22 is delayed. AE and ITDM will be informed about the status by PSPO.

- Vacuum will start the installation of the transfer pipe between the tunnel and E.07 on 19.3.

- SQS will start the installation of their instrumentation on 19.3.

- Some of our beamline components are currently stored at Hera South and must be transported to Schenefeld next week. However, there is an issue with the crane at Hera South. Kai-Erik has to look into this!

FXE (P. Zalden / A. Galler):

No issues/no report 

SCS (J.T. Delitz)

No issues/no report 

PP-Laser:

"The laser group has nothing particularly noteworthy to report, only that installation is still going as planned"

Vacuuum:

Installations in XTD6 are ongoing. Damaged vacuum pipes have been 

exchanged/ordered. Vacuum in XTD1 is closed. Control System in XTD1 is 

working and locally tested, Interlock-tests will happen after Easter.

XPD:

No issues/no report 

Advanced Electronics:

No issues/no report 

SASE2 Tunnels (C. Youngman)

Going according to plan. 1 week after Easter the movement and EPS loops are supposed to be brought up. 

ITDM (K. Wrona):

No issues/no report 

Detectors (J. Sztuk-Dambietz, P. Lang):

A module for AGIPD was replaced. 

No issues/no report

CAS (H. Santos):

DEMO for the coordinated motion was done during the week. Further tests needed. 

Schedule and discussion about the Karabo rollout process. See presentation bz H. Santos given in the technical meeting.

No new issues to be discussed where reported by any groups.

The status and issues reported on friday are still valid. 

SPB

On monday there was an issue with a detector related PLC due to misconnection of one EtherCAT cable. The issue has been solved and it "just" resulted in 3 of hours delay in  the AGIPD calibration activity.

FXE, SCS, SQS, XPD, XRO,ITDM no issues reported

AE related

uTCA in XTD 6 are installed 

VAC-Related

Marc Messerchmidt reported  abouit an entry in the DESY Logbook about a pump that triggered the interlock in the SASE1 tunnels. According to Nerea this this seems to be related to the old issue already known. VAC group is working on that. 

 CAS

CYoungman is working on the priority list for the components in SASE2 tunnels. Double checking with HSinn happening soon.

Access for Tue/Wed next week to XTD1 will be requested by CY through HS

Beam is in SPB/FXE this week due to the HERCULES project.

General

Due to the triggering of the emergency power off in FXE caused by electrical works being done on Sunday, FXE and SPB lost beam and electronics was switched off. In general the efficiency of the communication between groups has to be impoved. An important first step is the publication to a easy accessible site of the OCDs phone numbers list. PSPO will organize that.

Comment by PGessler

"It was clear to all, that currently the impact on operation due to emergency power off is severe, since the Racks with the PLC CPUs as well as the MicroTCA systems will be also switched off, what is not what was defined/specified. Therefore it was agreed, that no work on the power distribution/emergency off will be done until the defined date (for FXE 16th April). That work was done there nevertheless is extremely careless. The fact, that this was done on a Sunday and also during user time makes it even much worse then that.  It has to be completely followed up and made clear, that this is not acceptable and will not be repeated again!"

SPB

A run of 14 hrs acquiring data was a sort of stress test for the DAQ. It worked. ITDM would like to remove the data. Should be ok.

Due to the triggering of the emergency power off also SPB lost the beam. This was sort of strange since the power off signal was triggered in FXE. AE reported that this behavior is to be expected with the current configuration. This has to be discussed.

FXE

DAQ is recovering from the power off. 

Tripping of the interlock caused again by control unit of ion pumps in the week end. During the shutdown the responsible units will be replaced.

In general is is agreed to prepare an overview interlcok panel in the BKR. AE is on boat, CAS to be checked.

500 kHz operation is problematic for the machine due to heavy losses. They are only able to deliver 1 bunch at this rate. FXE will discuss this directly with the PRC (this week is Jan Gruenert)

AE related

Follow up to the issue of re-setting pumps after interlock is triggered: VAC and EPS loops need to be updated. HSinn was pushing to have it done as soon as possible without waiting for the shutdown. This is not riskless for the experiments. This will be discussed in a small round (AE, PRC, SPB, FXE)

ITDM

Due to the power outage many devices do not have proper train id. CAS should check it. Time servers also need to be checked.

FXE needs to use digitizers for the upcoming measurements session. These were not implemented in the DAQ. ITDM/FXE/CAS meeting needed to clarify and solve the issue .

 DET

LPD calibration had a problem but should be ok now. 

More supermodules coming.

Communication with upcoming userve concerning data format and other issue has to be improved according to the DET group. SPB suggests to tackle the issue offline.

SPB

• Sample delivery installed. Tonight will be commissioned. Experiment set-up starts tomorrow night

FXE

• Digitizers were already implemented in the DAQ but there is an issue related to the Karabo device. CAS already looking into that.

AE

• Follow up to the issue of re-setting pumps after interlock is triggered: loops updated. 
• SASE2 MOV loop patched
• Imagers in XTD1 have to be tested with Karabo

ITDM

• On monday afternoon problems with the storage servers were reported. The issue is a defective board, whoch is meanwhile been deactivated. IBM is working on that. The system seems to be working in this state.

CAS

• The next Karabo release will be able to handle better big loops. ASilenzi points out that the next update is really important since it adresses problems like the restart of time servers issue.

Readiness Meeting Issue list

• Item 2 (XGM SASE3): After restarting of the CPP servers the XGM turbo pumps stopped, valves closed and it required a manual intervention to restore them. ==> Firmware updated. CLOSED

Operation Meeting Issuel List

• Item 1 (Karabo rollout process): Testing steps of a new Karabo release have to be discussed with AE-ITDM-DET ==> Meeting took place. Closed

Due to some delay in the installation of some necessary cables, the emergency power off test foreseen for today 06.04. in the SPB experiment will not take place.

The details will be reported by Ulf Brueggmann today in the dispatch meeting at 13:00. The needed cabling should be done this week, so that the test in FXE planned for Monday, 16.04 CAN take place.

TS will confirm the FXE date and propose the nex date for SPB this Friday in the Readiness meeting.

General

Emergency off test in SPB is shifted. Date to be agreed on Friday. FXE test stays on 16.04. TS will confirm that.

MKK will shut down the cooling water for Tunnels and XHEXP1 on Thursday startin from 11:00 for maintenance purposes.

1. Based on the information we have the intervention could last up to 2 hrs. UBrueggmann is trying to check whether is possible to limit the duration of the intervention to bunches of 45 min. This could avoid the shut down of the Turbo pumps.
2. UBrueggmann is taking care of opening the rack doors in the tunnels (all of them!) and close them again when the water is back. 
3. The experiments should take care of opening the doors of the racks in the rackrooms. Also the main door.
4. AV will speak with the important groups involved and then send a summary e-mail to staff today afternoon.

SPB

1. Detector z motion would be needed. AE checks the status of the crate
2. Quadrant motion - Suggestion from AE is to perform tests and implement it in the next shutdown.
3. MIrrors installation ongoing. NO ACCESS to D.02!!!!
4. Laser Interlock/Rad. Saf. interlock issue still being investigated.  No news from SRP.

SASE2 -CYoungman

Goal is to stop the HW changes by the end of Wed. Thursday final loops and firmware will be updated. Both in SASE2 and 3.

FYI

Below the E-Mail from Hugo Santos with the Karabo deployment dates.

Dear All,

I would like to call your attention to the fact that we made Karabo 2.2.3 available last Monday (04/04, and the changelog 06/04). From the last Readiness Meeting, I got informed that we agreed and are allowed to proceed with the Karabo rollout planning on SASE3 and SASE2. (SASE1 still needs agreement):

1. SASE3
   1. PRE-CONDITION: (AE) PLC version and (ITDM) DAQ should be interface-compatible, otherwise, the deployment will not be able to be performed. (No Karabo-DAQ and Karabo-PLC interface issues)
   2. Date: 13/04/2018 (Friday)
   3. Should be the first one because of the completeness of this system: VAC, PES, XGM and GAT loops, for instance, that have to be checked and run properly afterward.
   4. Downtime expected: 9:00 to 14:00. 
   5. (AE) PLCs version and (ITDM) DAQ should be interface-compatible, otherwise, the deployment will not be able to be performed.
   6. No major change regarding DET systems
   7. CAS Members for this specific task: Chris and Wajid.
2. SASE2
   1. PRE-CONDITION: SASE3 Deployment task must be done and Karabo 2.2.3 running with PLC and DAQ (No Karabo-DAQ and Karabo-PLC interface issues)
   2. Date: 16/04/2018 (Monday)
   3. Downtime expected: 11:00 to 13:00.
   4. (AE) PLCs version and (ITDM) DAQ should be compatible, otherwise, the deployment will not be able to be performed.
   5. No major change regarding DET system.
   6. CAS Members for this specific task: Chris and Wajid.
3. SASE1
   1. PRE-CONDITION: SASE2 and SASE3 Deployment tasks must be done and Karabo 2.2.3 running with PLC and DAQ (No Karabo-DAQ and Karabo-PLC interface issues).
   2. Date: (17/04/2018 - Tuesday - to be agreed)
   3. Downtime expected: to be defined. 
   4. (AE) PLCs version and (ITDM) DAQ should be compatible, otherwise, the deployment will not be able to be performed.
   5. No major change regarding DET systems.
   6. CAS Members for this specific task: Alessandro and Wajid

Finally, please, also see that as soon as we have the Karabo 2.2.3 running during the shutdown, we will have the most comfortable time to perform the setup, configurations and stabilize it, still, during the shutdown. This way we can handover the control system properly to the operators.

Please, if you do not agree or are not comfortable with something in this message, let me know as soon as possible. Otherwise, I will assume that it does not contain any showstopper and we (CAS) can proceed.

H.S.

SPB

1. Detector z motion would be needed. AE checks the status of the crate. still pending
2. MIrrors installation ongoing. NO ACCESS to D.02!!!!
3. Laser Interlock change of logic by K.Witt will be done on monday at 9:00
4. Karabo update installation dates being agreed with CAS. Probably early next week.

Die Arbeiten an der Kühlwassersteuerung konnten innerhalb des genannten

Zeitraums abgeschlossen werden,

so dass die Kühlwasserversorgung nun schon einige Zeit wieder geregelt

läuft.

Großer Dank an alle Beteiligten, die den Ablauf der Abschaltung meines

Wissens auswirkungsfrei gestalten konnten.

Wir bitten über kurze Rückmeldungen zur Optimierung, sollten

Schwierigkeiten im Zusammenhang mit der Abschaltung entstanden sein.

Mit freundlichen Grüßen

Jörg Hanik

Hi all, for your information below a message from Chris Youngman concerning Karabo 2.2.3 installations in SASE3.

"preparation work has concluded and we will definitely try to deploy Karabo 2.2.3 on SA3 machines TODAY 13.04.. SA3 s/w will not work from 8:00 on. Hopefully we will finish during the morning.

SA1 and SA2 installations should continue as normal."

Emergency power test will be performed at the following dates:

FXE - Mon. 16.04.

SPB - Tue. 17.04. 

Schedule

• 10:00 Power will go off in SASE1 - Only redundant power line will stay ON
• Reprogramming of the PLC of the Emergency power off system - Estimated 1.5 Hrs.
• Power ON again ==> Emergency OFF test will be carried out. (info from AE: AE PLCs will not go off. uTCA crates will be shut down before)
• About at 15:00 everything should be back to normal.

For technical questions contact T. Schoen and U. Brueggmann.

Interlock systems SASE3 - TÜV test

The pre test by D3 will be held on 20.06.

On 05.07.The TÜV Test for SASE3 will be done together with the SASE2 test and the repetition of the SASE1 test