Dear all,

We need to update the Vacuum loop in order to correct and add some configuration to the interlock conditions.

It would be running in few minutes again.

Thank you!

Dear all,

We have to make some changes again into the Motion loop. Therefore I will do an update at 14.45h.

I will keep you inform when it is running again.

Thanks!

 After some minor issues, the first test has been done today and the interlock wasn´t triggered despite the conditions were met (also visible in Karabo).

This requires intervention and revision from AE

The next try-out should be done as soon as possible during this week.

Regards.

Raúl, Benoit and Valerii

Dear all.

Motion loop is running again.

Thank you

XTDs Info:

Tunnel XTD2 will be accessible this Thursday, 27.04. between 9 and 12. Dosimeter and General radiation training required. Access requests should go to T.Haas.

XTD10 will be closed starting from this Thursday, 27.04. at 12:00

Provided that the authorities give the "go ahead", DESY plans to put beam in XTD2 on Thursday's evening.

Karabo:

• System running  but performance of the GUI still sluggish
• There appears to be a resource leak. Second server will be set up tomorrow as workaround. Experts are working on it.
• In parallel experts are working on a definitive solution
• Deployment today on a new beckhoff device for the decodification of PLC error messages.

Vacuum:

• Systematic tests of ion pumps started again

PBLM/Motion Loop: 

• There is a resource problem when trying to read out the cameras. This problem will be reconsidered once the general issue with the GUI server is solved.
• ASilenzi will ask Natalia for acceptance test definition for PBLM
• Motion loop will be down until 10:30 for update. An entry will be written when the loop is again up. 

SPB Tunnel components

• The Renishaw encoders can be read and positions are reliable. An error message is coming from the encoder and is probably due to a wrong voltage level delivered to the encoder. The encoders can be operated in this condition.
• T. Freyermuth is in contact with the encoder company tu understand the problem and work on a fix
• The same problem also affects FXE components
• The cameras operated at 1Hz deliver delayed images. 1/3 Hz operation (almost live mode) is OK - probably related to GUI memory issues

FXE

• Encoders of Loop1 have the same error message as SPB - the encoder positions are reliable - local components test can go on (e.g. OGT1)
• Encoders of loop2 are not powered - AE to investigate

Tasks for today:

• Identify a fix/workaround for the GUI server resource problem
• Deployment of a beckhoff device for the decodification of the error messages.
• Find a solution for the Renishaw encoder problem SPB Tunnel/ FXE Loop1
• Investigation of the FXE Loop2 encoders power
• Continue testing of ion pumps in XTD9
• Test of gas section interlock in XTD9

Tasks for tomorrow:

• Setup of the second server for karabo GUI

In case of any problems, please, call the CAS personnel on 9-6443, 9-5630, or 9-6840.

Have a nice week-end, Sandor

The EPS-loop is now complete again.

The interface from EPS to MPS is working now. One broken wire in the cable between EPS and MPS was found. We are using now other pairs for the signals "SA1_SYS_EPS2MPS/DCTRL/MODE1", "SA1_SYS_EPS2MPS/DCTRL/MODES" and "SA1_SYS_EPS2MPS/DCTRL/MODEM".

Cheers,

    Tobias

Together with Nerea and Wajid and Alessandro

10:00 closed vatterflies at XTD9 and XTD2 from Karabo
Florian is at XTD9 next to the XGM 
XTD9: 
LOOP re-cycle
All valve closed and scroll off (turbos in unknown state; Florian in tunnel sees 990 Hz)
Scroll and turbos switched ON
Angle valve opened over Karabo
Problem to open gate-valve
Set Karabo offline
Nerea switched turbos at XTD2 ON (XTD9 were on) by PLC
Karabo is reconnected to PLC (brought online again)
XTD2: (with Alessandro) 
Scene optimized: Turbo speed, new prevac display, and preliminary scroll icon
Open KF angle and gate valve (follow "XGM vacuum incident procedure")
XTD9: (with Alssandro) 
Scene optimized: Turbo speed, new prevac display, and scroll icon
Open angle and gate valve (follow "XGM vacuum incident procedure")
New project created "SASE1-XGM" which contains both XTD2 and XTD9 XGM vacuum projects, thus scenes from both can be opened from one GUI simultaneously.

11:55 XTD9 vatterflies OPEN 
12:00 XTD2 vatterflies OPEN  
      (V10120V could only be opened after shutdown of device and re initialization) 
 
Maybe upper pressure probes need new calibration -> Check formula 
Prevacuum pressures: 
XTD9: 6.4E-4 mbar (G11081P)
XTD2: 2.5E-2 mbar (G10100P) and 1.2E-3 mbar (G10101P)

Summary : both XGMs in XTD2 and XTD9 are now back in normal pumping operation, and they were never exposed to more than a few 10E-6mbar. Good. But it took 2 hours and 4 experts.

Dear all,

The service of On-call Duty will be available from AE group during weekend in this number: 6442

Please make a post with the OCD numbers from CAS, WP74 and WP73.

Thank you and bests.

We have to disconnect the EPS crate in RACK 1.14 in order to check the interface to the MPS.

I will keep you updated

exflplcgwSA1 seems not reachable. This inhibits support for the PLC systems.

 Due to problems and lack of support of CAS for accessing the devices in XTD9 tunnel through KARABO we couldn´t run the test. The system has been returned to manual operation and will remain like that until next notice.

As soon we received the request we worked on that one.  After analysing the issue proper response was sent.

​​​----- Forwarded Message -----

​​​​From: "Wajid Ehsan" <wajid.ehsan@xfel.eu>

To: "Raul Villanueva" <raul.villanueva@xfel.eu>

Cc: "Alessandro Silenzi" <alessandro.silenzi@xfel.eu>, "Nerea Jardon" <nerea.jardon@xfel.eu>

Sent: Friday, 21 April, 2017 12:56:14

Subject: Re: Karabo problems in XTD9 - Urgent!

​​​​​Hi Rahul,

we checked all the devices that are in error. The issue is realted with hardware.

Error description is mentioned below:

0x1001 FlagCtrl Error flag from controller

Regards,

Wajid Ehsan 

Here is the list of devices that are in error:
 

SA1_XTD9_VAC/VALVE/V11090A  Error 3 -> valve stayed between open and closed for too long time (3s set on the valve) (the reset command is sent to the PLC and it cannot clear the error)
SA1_XTD9_VAC/VALVE/V11100A  Error 3 -> valve stayed between open and closed for too long time (3s set on the valve)
SA1_XTD9_VAC/VALVE/V11110A  Error 3 -> valve stayed between open and closed for too long time (3s set on the valve)
SA1_XTD9_VAC/TPUMP/P11220T Error 0x1001 -> err_FlagCtrl -> Error flag from controller (need manual reset on the controller) after reset error is now cleared.
SA1_XTD9_VAC/TPUMP/P11280T Error 0x1001 -> err_FlagCtrl -> Error flag from controller (need manual reset on the controller) after reset error is now cleared.
SA1_XTD9_VAC/VALVE/V11210G Error 0x10001 ->  _err_InterlockDevNotFound  -> Interlock device not found. (the valves depends from a SA1_XTD9_VAC/SWITCH/DP_XGM_DP_ERROR which does not exist. a SA1_XTD9_VAC/SWITCH/DP_XGM_DP_OK is present in the list though, likely a PLC configuration error).
SA1_XTD9_VAC/VALVE/V11220G Error 0x10001 -> _err_InterlockDevNotFound -> Interlock device not found.
 

For the last 2 errors, which are the only one that have a clear PLC problem, I created a bug report in the redmine project of AE.
https://in.xfel.eu/redmine/issues/16510

Cheers

Alessandro Silenzi

XGM status 21.4.2017

During Easter break, the two XGMs in the control system (XTD2+9) had no issues.
XGM@XTD10 however had an issue: tuesday morning it was found that all pumps had stopped. The scroll pump showed an error. UPS apparently had excess load sunday evening 19h30 but couldn't send email (network error). Possibly vented with tunnel air up to 10mbar since that time. Investigation is ongoing.

Thursday enabled DOOCS pressure reading of XGM in XTD10, UPS rechecked

System surveillance on coming weekend: Jan and Theo 1 day

Other remarks:
- WP74 staff have received their dosimeters for ZZ access if required
- some automatic monitoring and info system is needed soon, e.g. karabo to NAGIOS connection

 In KW17 there will be NO maintenance day on Tuesday (25.04.) 

 
On Monday (24.04.) morning up to about 11:00 we will have the TÜV approval of CS8. During that time 
 NO access of "other people" to the XTL tunnel is possible. After the TÜV approval 
 up to 13:00 there will be the possibility for URGENT access to the XTL. Access requests 
 have to written to the logbook/access requests entry. 
 
The tunnel XTL/XTIN will stay in interlock, access will be possible by ZZ only. 
 
The northern branch XTD2->XS3->XTD4->XDU2 will stay closed, in order not to interrupt the 
 magnet commissioning. 
 
This elogbook entry was sent to following experts: 
xfel-opl@desy.de RC mailing list  

 Tunnels XTD2 and XTD4 are closed, XTD9/10 remain open

Vacuum:

• PLC/Karabo Running stably
• Local tests of ion pumps in XTD9 being done by J. Eidam
• Work continues on M3 in XTD9 and on the vacuum in XTD10 
• DPS in XTD9 should be integrated into the vacuum loop. This requires a cycling of the loop. Timing should be coordinated with WP74 because of XGM in XTD2. Unfortunately nobody from WP74 attended the meeting. Hence it was decided to perform the work tomorrow morning.

EPS Loop:

• Checks for the signal from the undulators and for the fast valve will be done with DESY MPS today

PBLM/Motion Loop:

• Work on the PBLM motors (ELMO controller) continues.
• Yesterday it was possible to align the screens 
• There is a resource problem when trying to read out the cameras. For this reason an additional server needs to be set up dedicated to acquiring the images from the cameras.

SPB Tunnel components

• Crate that was checked in the lab yesterday was moved back into the tunnel
• The unstable support of the imager was fixed
• B. Weinhausen and M. Messerschmidt are testing
• All modules necessary for loops 1 & 2 were received from Poland
• Waiting for the "Freigabe" of the cables for SPB Optics

FXE

• Loops 1,2 and 4 are waiting for input from EETF
• Loops 3&5 are running
• No visible testing activity from FXE team

Karabo:

• Cleanup continues

Tasks for today:

Note: All loops should be left running and no changes to Karabo.

• Prepare mirror M3 installation
• Continue testing of PBLMs in XTD9
• Continue local tests of ion pumps in XTD9
• Continue testing of SPB tunnel loop

Tasks for tomorrow:

• Cycle vacuum loop for inclusion of DPS in XTD9

Other topics

• Weekly SASE1 Readiness meeting tomorrow at 09:00 in XHQ 1.173
• L. Samoylova requests access to XTD2 to install another webcam
• Magnet current tests will likely continue through Wednesday in XTD2. Maintenance day to be shifted accordingly probably to Thursday/Friday. More information later

Tunnels XTD2 and XTD4 are closed, XTD9/10 remain open

Vacuum:

• Running stably
• Work continues on M3 in XTD9 and on the vacuum in XTD10 

EPS Loop:

• Input from undulator still missing. This requires  fixing a cable in balcony rooms and then testing

PBLM/Motion Loop:

• Work on the PBLM motors (ELMO controller) continues.
• It appears that the uneven movement is due to improper setting of motor parameters

Imagers

• Trigger problem reported yesterday was due to a cabling issue. Apparently during work on the relevant rack cables were unplugged and then wrongly put back
• Network issue with pop-in (245) in XTD9 solved

SPB Tunnel components

• Crate with "problem on signal pins" was brought to the lab and checked. Everything was ok. Problem was due to spurious measurement.

Karabo:

• PSLIT was tested successfully yesterday
• Behaviour of Karabo when a loop goes down needs to be made more graceful. 
• Cleanup continues

Tasks for today:

Note: All loops should be left running and no changes to Karabo.

• Prepare mirror M3 installation
• Continue testing of PBLMs in XTD9
• Continue testing of SPB tunnel loop

Other topics

• None
   

Dear all,

Today from 13:30h  Mahadi and Natalia will be doing local tests in the PBLM in Rack 1.14 of XTD9, therefore, if someone will be working in the following racks of XTD9, will loose connection to the Motion loop.

Thank you and best regards,

All trigger problems in XTD2 and XTD9 solved except for one: spb_xtd9_imgpi, probably a cabling issue on the camera side.

IP address problem for camera of sa1_xd2_imgpii45 solved.

 Author: Decking

Goals:
- Operation with 10 GeV, 500 pC, 5 kA, 30 bunch
- Increase gradient limits in RF stations
- Technical comissioning of north branch 

XTIN closed all week
XTL open till Tue, 11:00
XTD2 open till Tue, 15:00
XTD4 open till Tue, 15:00
XTD9&XTD10 open all week


|17/04 Mo M |  Easter shutdown
|17/04 Mo A |  Easter shutdown
|17/04 Mo N |  Easter shutdown
|18/04 Tu M |  Interlock pre-test CS8, set XTL interlock at 11:00, Injector beam operation
|18/04 Tu A |  A2 calibration after phase shifter change, Beam operation  to TLD 10 GeV, set interlock XTD2-XTD4
|18/04 Tu N |  Beam operation  to TLD 10 GeV, 30 bunches, 0.5 nC
|19/04 We M |  Slice emittance in BC2; Magnet PS test XTD2-XTD4 
|19/04 We A |  BCM, BAM EO, commissioning; Magnet PS test XTD2-XTD4 
|19/04 We N |  BCM, BAM EO, commissioning; Magnet PS test XTD2-XTD4 
|20/04 Th M |  Collimation set-up; Magnet PS test XTD2-XTD4 
|20/04 Th A |  BCM, BAM EO, commissioning; Magnet PS test XTD2-XTD4 
|20/04 Th N |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC; Magnet PS test XTD2-XTD4 
|21/04 Fr M |  Collimation set-up with 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|21/04 Fr A |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|21/04 Fr N |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|22/04 Sa M |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|22/04 Sa A |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|22/04 Sa N |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|23/04 Su M |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|23/04 Su A |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|23/04 Su N |  Beam operation  to TLD 12 GeV, 30 bunches, 0.5 nC, 5 kA; Magnet PS test XTD2-XTD4 
|24/04 Mo M |  Beam set-up 12 GeV, 1 bunches, 0.1 nC, 1 Hz
|24/04 Mo A |  Beam operation  to TLD 12 GeV, 1 bunches, 0.1 nC, 1 Hz
|24/04 Mo N |  Beam operation  to TLD 12 GeV, 1 bunches, 0.1 nC, 1 Hz