SPB

• Loops update: 1-4+8 this week. 5-7 Wed. next week. 11-13 end next week.
• Smaract motors for PAM are needed and Nerea is on it.
• It is not clear whether the AGIPD firmware has to be updated. To clarify with Jola when she is back. The info is important foe ITDM to plan the DAQ tests. Jola sick until 1st week of May. To be seen then.
• During the last users run it happened that a valve in XTD9 was closed but the beam did not trip. It is not clear why. Will be clarified with AE. Interlock definitions  have to be checked next week when HSinn is back. 
• During the switch on of the detector many Karabo devices had a problem even if no error messages were displayed. The problems disappeared restarting the devices. The errors could not be reproduced. Ot top of that it is possible to collect data only from 15 modules out of 16. The one missing changes every time the device is restarted. ASilenzi on it. 
• Technofoft motor with magnetic switches do not work. In the tunnel the same configuration works, Will be investigated by AE/CAS/SPB
• Motors upstream move sometimes by themselves. Discussion ongoing between AE SPB and CAS. To be adressed offline.

XPD

• Access  of XTD1 today @14:00 to fix some EPS switches
• SASE2 firsta lasing should be tried this week end + monday. CAS support requested. ASIlenzi checks with CAS whether this is possible.

AE

• SCS exp: first loop running. 2 creates missing but loop is up.
• MPS connection tunnels SASE2 done

ITDM

• Online storage update patch to be installed today. The system should remain online but in some cases one could have some downtime (1-2 hrs)
• MPS connection tunnels SASE2 done.

General

On Wednesday the meeting will be held at 13:00. Starting from next monday it will go back to 9.30

I will update the SASE3 MOV1 loop in 30 minutes. Please let me know if you would like me to postpone the update.

Best regards,

Marijan

SPB

• Timing tool for optical laser installed
• Sample chamber's PLC will be updated next week due to ongoing tests until monday. Pending.
• It is not clear whether the AGIPD firmware has to be updated. To clarify with Jola when she is back. The info is important foe ITDM to plan the DAQ tests. Pending. 
• During the last users run it happened that a valve in XTD9 was closed but the beam did not trip. It is not clear why. Will be clarified with AE. Pending.

AE

• SASE2 EPS Loop running
• Updating PLCs for MPS in the tunnels. The update was stopped last friday due to a problem with the on/off funcionality of some ion pumps. The problem was found and the fixing requires an update of the VAC loops whoch will be done in the next hours/days. In case the loop update is not possible due to specific requirements the problem can be bypassed in Karabo. 

General

On Wednesday the meeting will be held at 13:00. Starting from next monday it will go back to 9.30

Dear all,

below please find the hottest and most up-to-date times for the closing of the tunnels after the shutdown:

Cheers, Gerd

SPB

• Karabo has been deployed  without major  problems! 
• Now PLCs update follows. Sample chamber's PLC will be updated next week due to ongoing tests until monday.
• It is not clear whether the AGIPD firmware has to be updated. To clarify with Jola when she is back. The info is important foe ITDM to plan the DAQ tests. 
• During the last users run it happened that a valve in XTD9 was closed but the beam did not trip. It is not clear why. Will be clarified with AE

AE

• Timing servers update - this Thurdsay, starting at 14:00 for 1 hour the trigger and train id signals will be not available.
• Updating PLCs for MPS in the tunnels 

ITDM

• Network stability problems in SASE3 are understood and a workaroung is being implemented.

General

Closing time of the tunnels on monday still not clear. To be clarified by PSPO.

Dear all,

 the emergency power off test has been successfully performed in SPB.

Nobody can instantly cut the power to non hazardous racks (see racklist.xlsx managed by Ulf Brueggmann) by pushing the emergency off button anymore. The outlets in the experiment hutch do not go off. This is due to some missing cables. These will be installed in the next weeks and a new test will be planned for next shutdown.

Furthermore three racks for D.02 (SPB optics) and two racks for D.13 (SPB laser) cannot be powered off anymore by pushing the emergency off in D.09. The missing two emergency power off switching devices will be mounted in UV-XHEXP1-01-13-AV in the next shutdown. Until then the power to those five racks (D03-UH1 - D03-UH3, D13-UH1, D13-UH2) cannot be cut by emergency off buttons in D.02 or D.13.

Summary of Karabo 2.2.3 deployment dates:

• Karabo 2.2.3 deployed in SASE3 tunnels last friday ==> no major issues. DAQ can be deployed
• Karabo in SASE2 Tunnels to be deployed today at 14:00
• Karabo in SASE1 Tunnels to be deployed tomorrow at 08:00
• Karabo in SPB/SFX experiment to be deployed today and tomorrow
• Karabo in FXE experiment to be deployed on Wednesday

SPB

• By tomorrow the mirror installation will be done.
• It is not clear whether the AGIPD firmware has to be updated. To clarify with Jola when she is back. The info is important foe ITDM to plan the DAQ tests.
• During the last users run it happened that a valve in XTD9 was closed but the beam did not trip. It is not clear why. Will be clarified with AE

AE

• Timing servers update - this Thurdsay, starting at 14:00 for 1 hour the trigger and train id signals will be not available.

ITDM

• Infiniband port fixing ongoing
• GPFS bug to be fixed with a patch that should be issued on 24.04. To be checked whether it is still OK to install the patch or it is too late.

General

• PSPO will ask HSinn for a detailed schedule concerning closing of the tunnels to fine tune the activities going on this week in the SASE2 and 3 tunnels

Dear all,

I was just informed by Ulf Brueggmann that the emergency power off test was successfully performed in FXE.

The  reality is now reflecting the planned logic, exeption being the power plugs in the experiment hutch. They do not go off. This is due to some missing cables. These will be installed in the next weeks and a new test will be planned for next shutdown.

The tests will continue tomorrow in SPB/SFX.

This entry is on behalve of Frank Babies, if you have any doubt about it, contact him:

On 16.Apr.2018 09:45am till Noon I have to shutdown the MicroTCA4 Crates Rackroom from FXE
"fxe-rr-sys-utc-1"
"fxe-rr-sys-utc-2"
because of the Power Down issue on Monday.

On 17.Apr.2018 09:45am till Noon I have to shutdown the MicroTCA4 Crates Rackroom from SPB
"spb-rr-sys-utc-1"
"spb-rr-sys-utc-2"
because of the Power Down issue on Tuesday.

This will affect timing system and MPS communication.

Interlock systems SASE3 - TÜV test

The pre test by D3 will be held on 20.06.

On 05.07.The TÜV Test for SASE3 will be done together with the SASE2 test and the repetition of the SASE1 test

Emergency power test will be performed at the following dates:

FXE - Mon. 16.04.

SPB - Tue. 17.04. 

Schedule

• 10:00 Power will go off in SASE1 - Only redundant power line will stay ON
• Reprogramming of the PLC of the Emergency power off system - Estimated 1.5 Hrs.
• Power ON again ==> Emergency OFF test will be carried out. (info from AE: AE PLCs will not go off. uTCA crates will be shut down before)
• About at 15:00 everything should be back to normal.

For technical questions contact T. Schoen and U. Brueggmann.

changes according to the racklist on the electrical power distribution cabinets for rackrooms E.21 and E.22 are done. Power supply should be stable now. Please note that there will be emergency power off tests prior to the final acceptance test of SASE3 electrical installations around mid of May. You will be notified about the exact time and date.

Hi all, for your information below a message from Chris Youngman concerning Karabo 2.2.3 installations in SASE3.

"preparation work has concluded and we will definitely try to deploy Karabo 2.2.3 on SA3 machines TODAY 13.04.. SA3 s/w will not work from 8:00 on. Hopefully we will finish during the morning.

SA1 and SA2 installations should continue as normal."

Die Arbeiten an der Kühlwassersteuerung konnten innerhalb des genannten

Zeitraums abgeschlossen werden,

so dass die Kühlwasserversorgung nun schon einige Zeit wieder geregelt

läuft.

Großer Dank an alle Beteiligten, die den Ablauf der Abschaltung meines

Wissens auswirkungsfrei gestalten konnten.

Wir bitten über kurze Rückmeldungen zur Optimierung, sollten

Schwierigkeiten im Zusammenhang mit der Abschaltung entstanden sein.

Mit freundlichen Grüßen

Jörg Hanik

SPB

1. Detector z motion would be needed. AE checks the status of the crate. still pending
2. MIrrors installation ongoing. NO ACCESS to D.02!!!!
3. Laser Interlock change of logic by K.Witt will be done on monday at 9:00
4. Karabo update installation dates being agreed with CAS. Probably early next week.

FYI

Below the E-Mail from Hugo Santos with the Karabo deployment dates.

Dear All,

I would like to call your attention to the fact that we made Karabo 2.2.3 available last Monday (04/04, and the changelog 06/04). From the last Readiness Meeting, I got informed that we agreed and are allowed to proceed with the Karabo rollout planning on SASE3 and SASE2. (SASE1 still needs agreement):

1. SASE3
   1. PRE-CONDITION: (AE) PLC version and (ITDM) DAQ should be interface-compatible, otherwise, the deployment will not be able to be performed. (No Karabo-DAQ and Karabo-PLC interface issues)
   2. Date: 13/04/2018 (Friday)
   3. Should be the first one because of the completeness of this system: VAC, PES, XGM and GAT loops, for instance, that have to be checked and run properly afterward.
   4. Downtime expected: 9:00 to 14:00. 
   5. (AE) PLCs version and (ITDM) DAQ should be interface-compatible, otherwise, the deployment will not be able to be performed.
   6. No major change regarding DET systems
   7. CAS Members for this specific task: Chris and Wajid.
2. SASE2
   1. PRE-CONDITION: SASE3 Deployment task must be done and Karabo 2.2.3 running with PLC and DAQ (No Karabo-DAQ and Karabo-PLC interface issues)
   2. Date: 16/04/2018 (Monday)
   3. Downtime expected: 11:00 to 13:00.
   4. (AE) PLCs version and (ITDM) DAQ should be compatible, otherwise, the deployment will not be able to be performed.
   5. No major change regarding DET system.
   6. CAS Members for this specific task: Chris and Wajid.
3. SASE1
   1. PRE-CONDITION: SASE2 and SASE3 Deployment tasks must be done and Karabo 2.2.3 running with PLC and DAQ (No Karabo-DAQ and Karabo-PLC interface issues).
   2. Date: (17/04/2018 - Tuesday - to be agreed)
   3. Downtime expected: to be defined. 
   4. (AE) PLCs version and (ITDM) DAQ should be compatible, otherwise, the deployment will not be able to be performed.
   5. No major change regarding DET systems.
   6. CAS Members for this specific task: Alessandro and Wajid

Finally, please, also see that as soon as we have the Karabo 2.2.3 running during the shutdown, we will have the most comfortable time to perform the setup, configurations and stabilize it, still, during the shutdown. This way we can handover the control system properly to the operators.

Please, if you do not agree or are not comfortable with something in this message, let me know as soon as possible. Otherwise, I will assume that it does not contain any showstopper and we (CAS) can proceed.

H.S.

General

Emergency off test in SPB is shifted. Date to be agreed on Friday. FXE test stays on 16.04. TS will confirm that.

MKK will shut down the cooling water for Tunnels and XHEXP1 on Thursday startin from 11:00 for maintenance purposes.

1. Based on the information we have the intervention could last up to 2 hrs. UBrueggmann is trying to check whether is possible to limit the duration of the intervention to bunches of 45 min. This could avoid the shut down of the Turbo pumps.
2. UBrueggmann is taking care of opening the rack doors in the tunnels (all of them!) and close them again when the water is back. 
3. The experiments should take care of opening the doors of the racks in the rackrooms. Also the main door.
4. AV will speak with the important groups involved and then send a summary e-mail to staff today afternoon.

SPB

1. Detector z motion would be needed. AE checks the status of the crate
2. Quadrant motion - Suggestion from AE is to perform tests and implement it in the next shutdown.
3. MIrrors installation ongoing. NO ACCESS to D.02!!!!
4. Laser Interlock/Rad. Saf. interlock issue still being investigated.  No news from SRP.

SASE2 -CYoungman

Goal is to stop the HW changes by the end of Wed. Thursday final loops and firmware will be updated. Both in SASE2 and 3.

Due to some delay in the installation of some necessary cables, the emergency power off test foreseen for today 06.04. in the SPB experiment will not take place.

The details will be reported by Ulf Brueggmann today in the dispatch meeting at 13:00. The needed cabling should be done this week, so that the test in FXE planned for Monday, 16.04 CAN take place.

TS will confirm the FXE date and propose the nex date for SPB this Friday in the Readiness meeting.

On Friday, 6.04, there will be a maintenance of the online storage system. This will tackle the issues we had just before the last user beamtimes, when we decided to apply workaround to keep the system working. The availability of the storage system is not guaranteed, especially when faulty h/w parts needs to be exchanged.

Best, 

Janusz

Tomorrow, 5.04, between 9:30 and 12:00 there will be urgent maintenance of one of the two redundant main switches for the control network. The network connectivity during that time is not guaranteed, but also any major interruptions are not expected. For the two switches in sase1 tunnels with only one uplink, there could be short (~1min) interruption.

Best, 

Janusz