Information posted on behalf of Torben Falk (WP74 cabling)

Wir haben nun alle Kabel für den XGM in XTD2 am Beckhoff angeschlossen.

6.3.2017, Karabo tests in XTD2 with Wajid, Andreas

1. Karabo all devices, all scenes working, with restrictions.

2. Power supply, motor, LED working.

3. Scenes tested: Motor scene and camera scene need additional parameters. AK will edit a list of suggestions, improvements and bugs.

4. Bugs, observations, details in improvement list.:

- Homing not working, tried also old parameter set from prototype crate.

- Encoder position not correct, dragging behind.

- Parameter for braking switch missing “ALowSpeed”.

As discussed with Wajid, I am availalbe tomorrow Wednesday the whole day to continue testing Karabo, motion loop. Wajid checked with Narea. I understand that there may be interruptions and that I have not first priority. Thanks, Andreas

End switch testing: the in-position end switch stopped devices working, also stopped gateway connection. Stroke of motor is not correct, parameter list to be checked. More details in "Improvements / bug list".

Here the link to the detailed list of issues: https://docs.xfel.eu/share/page/site/xfelwp74/document-details?nodeRef=workspace://SpacesStore/1733b825-e62e-47af-abba-fdef1ff9485d

The XTD2 CRL is now aligned and cabled.

Motor cabling is complete and ready for local testing. Encoder and PT100 cabling at rack to be completed by the middle of week 11.

XTD2, IMAG.2694.T9, Pop-In II-45: Acceptance test of motor control under Karabo, done. Camera device needs further explanations and testing.

Trigger signal now available at imager. First Karabo tests of camera device with trigger successful. Detailed
tests continue.

-

XTD2, FEL imager with Wajid

Screen, motor, switches - ok, minor braking switch issue.

LED, camera power – ok.

Filter, motor, parameter issue, should be solved now, to be tested again today.

EPS tests and SR imager tests, to be done.

XTD9 cameras, no power, will check today with Bernard.

Has anyone an idea why the motion loop is not working? Thanks Andreas

Hello Andreas,
please use redmine for bug reports. I opened a redmine issue for this:

https://in.xfel.eu/redmine/issues/16265

Please log in and give more details on what you experienced.

In case of emergency contact your main contact person Wajid.

Cheers

Alessandro

 Thanks, Alessandro,

Both XGMs were and are on manual control (but the remote cables into the turbo pump controllers remained plugged in as usual) = both XGMs are "out of the VAC loop". 

In XTD2, I saw around 13h that both turbos were off (0 Hz) and the pressure was at 10E-4 mbar !

The green light above the ON-button was flickering. 
I switched parameter 023 (Motor TMP) to ON and restarted both pumps. The green light above the ON-button 
 stops flickering and goes steady green. 
 
---going to XTD9 
 
In XTD9, I saw that the upstream turbo was off (0 Hz) and the pressure was in the lower 10E-6mbar. 
The downstream turbo in XTD9 was still operating. 
Parameter 023 to ON, and restart pump. 
 
---going to XTD2 
 
Error codes 6, 6, 98, 2, 37 on one pump, error codes 6,6,6 on the other (XTD2).
I unplugged the REMOTE cables in XTD2 and XTD9 to avoid further trouble as long as we are on "manual". 
This info was also sent by email to the involved experts.

 The interlock definition for complete isolation of the whole section based on high pressure signal has been successfully tested. 

DPS (Upstream and Downstream) are now fully "controlled" by Karabo.

XGM has been reported by J. Grünert that it will be re-integrated during the evening.

Still some problems to call the remote operation of the turbopumps from Karabo, but it doesn´t compromise any of the interlock definitions (only is troublesome if the one needs to work with the pump, for instance after the interlock has been triggered).

 The Fast Valve test has been positively tested. 

It has been consider that the signal on the EPS to open again, should be also available in the vacuum loop. Still this need to be further discussed.

 The Interlock conditions for these two components has been positively tested but not exhaustively done:

The triggering of both via the actuation on the Fast Valve----------------->DONE and OK

Their triggering via HV down of each of the 6 ion pumps------------------>Partially DONE and OK: Partially means that only one of the pumps randomly chosen has been deactivated. The agreement is that the other remaining 5 can be done later, also from the office since the correspondence between the devices and the physical components has been confirmed. 

Also, the individual actuation of both, Beam Stopper and V0 valve has been proof showing a consistent behaviour regarding the interlock definition.

Finally the XFEL Vacuum OK signal is available but AE is currently improving it. By the moment is not automatically sent to DESY, but instead it needs to be activated manually. According to AE feedback this should be corrected in the following days.

Both SASE1 XGMs (in XTD2 and XTD9) are now again in the Vacuum Control Loop, their vacuum being controlled by Beckhoff/karabo. This works fine.

Therefore: DO NOT FOOL AROUND WITH THE VACUUM LOOP !

Since we all cannot enter the tunnel XTD2 in the next two days, it's agreed that the Vacuum Loop is NOT going to be touched at all, since  we cannot intervene in the tunnel if something goes wrong.

Together with Nerea and Wajid and Alessandro

10:00 closed vatterflies at XTD9 and XTD2 from Karabo
Florian is at XTD9 next to the XGM 
XTD9: 
LOOP re-cycle
All valve closed and scroll off (turbos in unknown state; Florian in tunnel sees 990 Hz)
Scroll and turbos switched ON
Angle valve opened over Karabo
Problem to open gate-valve
Set Karabo offline
Nerea switched turbos at XTD2 ON (XTD9 were on) by PLC
Karabo is reconnected to PLC (brought online again)
XTD2: (with Alessandro) 
Scene optimized: Turbo speed, new prevac display, and preliminary scroll icon
Open KF angle and gate valve (follow "XGM vacuum incident procedure")
XTD9: (with Alssandro) 
Scene optimized: Turbo speed, new prevac display, and scroll icon
Open angle and gate valve (follow "XGM vacuum incident procedure")
New project created "SASE1-XGM" which contains both XTD2 and XTD9 XGM vacuum projects, thus scenes from both can be opened from one GUI simultaneously.

11:55 XTD9 vatterflies OPEN 
12:00 XTD2 vatterflies OPEN  
      (V10120V could only be opened after shutdown of device and re initialization) 
 
Maybe upper pressure probes need new calibration -> Check formula 
Prevacuum pressures: 
XTD9: 6.4E-4 mbar (G11081P)
XTD2: 2.5E-2 mbar (G10100P) and 1.2E-3 mbar (G10101P)

Summary : both XGMs in XTD2 and XTD9 are now back in normal pumping operation, and they were never exposed to more than a few 10E-6mbar. Good. But it took 2 hours and 4 experts.

Problems described in detail below (this entire procedure takes way too much of our time, effectively still nearly 2 hours!):

Nerea/AE requested yesterday afternoon to shutdown and restart the SASE1 VAC PLC loop once again. We start today morning (wednesday) at 8h30. 
8h30
karabo is not showing status of all devices, see three screenshots (attached). karabo-GUI issue.
These devices have to be shutdown and reinstantiated to show a status and to operate them. 
Several valves and gauges and scroll pumps affected both in XGM of XTD2 and XTD9. 
 
9h00 finally I can close all four vatterfly valves 
 
9h15 Nerea shuts down and restarts the SASE1 VAC PLC loop 
 
9h25 still waiting for karabo to reconnect to Beckhoff PLC (I cannot do anything in karabo, and 
 no current status updates of anything) 
 
9h40 SPB_XTD9_XGM scene has loaded (seems like at least). I start the reactivation procedure (switch 
 on scroll, open valves etc.). Need to reinstantiate several devices. 
 
10h00 SPB_XTD9_XGM: reinstantiate both turbo pumps, CRemotePriority ON, COn, CPump (if CRemotePriority 
 not visible, again reinstantiate) 
      READ ADriveCurrent, ARPMIn several times, both should change from time to time, don't 
 worry about "0" every second read time 
10h05 SA1_XTD2_XGM: reinstantiate both turbo pumps, StartPumpingStation 
      READ ADriveCurrent, ARPMIn several times, both should change from time to time, don't 
 worry about "0" every second read time 
10h10 READ the actual value of the gauges in configurator (not to trust value shown in scene) 
10h15 observe DOOCS RVC300 UHV pressures, open vatterfly valves 
 
10h20 end of intervention