Group Selection Page HELP DOC Controls Group Data Analysis Group ITDM WP76 DET Vacuum XRO EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Photon Commissioning Team Operation PSPO XO TS
Technical integration and coordination SASE1 Status and SASE3 Readiness
  Joint Operation and Readiness Logbook, Contact: T.Haas, A. Violante, G. Wellenreuther, Page 11 of 30  Not logged in ELOG logo
ID Date Author Locationdown Severity Subject
  119   07 Apr 2017, 17:54 Tobias FreyermuthGeneralFeedbackHotfix for all loops (VAC, MOV, EPS) deployed

Tobias Freyermuth wrote:

Dear collogues,

 

all loops are up and running after deploying the hotfix.

 

Have a nice weekend,

    Tobias

 

 Confirmed from Karabo side.

  121   10 Apr 2017, 09:36 NEREA JARDONGeneralInformationVacuum Loop

Dear all,

I will update the Vacuum loop in order to add the XGM XTD9 into it. It will be running in few minutes again.

Thank you.

  122   10 Apr 2017, 10:09 NEREA JARDONGeneralInformationVacuum Loop

Dear all,

The Vacuum loop is running again, now with XGM XTD9 included. We will be testing interlocks in XGM XTD2, so could be needed another upload.

Thank you,

  123   10 Apr 2017, 10:47 Tobias HaasGeneralInformationSchedule after 09:30 Meeting

Tunnel Open.

Mirrors:

Status:

  • Both mirrors M1 & M2 installed
  • Vacuum being closed in M2 today should be ready for ion pumps tomorrow
  • Surveying work going on (see email by M. Schlösser below)

Vacuum Loop:

Status:

  • Needs reconfig for XGM in XTD9
  • Interlock tests on XGM in XTD2 should go on. 
  • 2 Turbopumps still have a communication problem (J. Tolkiehn is working on this)
  • 6 Ion pumps in XTD9 still have communication problems (list was given by AS to JE)
  • Systematic testing according to the list by J. Eidam with V. Bondar starting this afternoon

EPS Loop:

Status:

  • One terminal needs to be exchanged for FEL Imager 
  • Description of EPS interlock still missing. It is planned to start with just one imager in for first beam commissioning
  • One fiber connection from Balcony room to DESY still to be done by DESY. AFter this testing of communication should start

Motion Loop:

Status:

  • Quick fix for motors not stopping was deployed on Friday. Now TF is working on a better fix to be deployed this evening

Imagers

Status:

  • EPS switch on FEL imager should be checked after exchange of terminal
  • Photonics camera driver still not working but not critical for tunnel closure
  • Work should now concentrate on SR imager

Karabo:

Status:

  • New version with TCP fix needs to be deployed. This should also be done to the local software run by users.

 

Tasks for today:

 

Note: Loop reconfiguration and updates of Karabo to be done in the morning. Testing can resume after lunch. Status updates should please be recorded in the log

  • Reconfigure vacuum and motion loops (updates for XGM interlock, motors not stopping)
  • Replace broken terminal in EPS loop for FEL imager 
  • Continue test XGM interlocks (N. Jardon, M. Planas, T. Matsopoulos)
  • Go through systematic test of the vacuum components (J. Eidam, V. Bondar)
  • Continue tests with PBLM in the tunnels (M. Jidda)
  • Test motion system of the imagers in the tunnels (A. Koch and W. Ehsan)
  • Monitor stability and responsiveness of GUI Server and Log Servers (A. Silenzi)
  • Set up monitoring of temperatures and pressures overnight (A. Silenzi)
  • Test communication between XFEL and DESY MPS once fiber connection from Balcony rooms to MPS is available (B. Fernandes)

Tasks for tomorrow:

  • Put XGM into the vacuum interlock
  • Continues testing of all interlock conditions for the vacuum

Other topics

  • Surveying is going on for mirror chambers M1 and M2 in XTD2
  • We need a well defined startup configuration of PLC, Karabo, Parameters, List of devices (A. Silenzi)

Email by M. Schlösser re alignment

Hi Tobias, hi Markus, Hi Winni,

Tobias, we discussed briefly the schedule for the T9 and T10 beamlines for the next few days, when we met in the tunnel yesterday.

From what i understand it is impossible to finish T10 until April 11th from the alignment and vacuum side. And probably more important, this also holds true for T9. There is at least one major problem with the alignment of the mirror chambers of T9 in XTD2, which will need some additional days to solve.
From my understanding at least the T9 beamline should be closed and aligned before putting SASE1 in operation. While it will of course be possible to send electrons down to the XSDU2 dump, it will not be possible to send photons to the experimental hall with the current state of the beamline.

Since it is not possible to work in XTD10 while the SASE1 electron beamline is in operation, i would propose to also finish this beamline before SASE1 electron startup. Since there is major work to do in XTD10 to finish the beamline, there is no hope to  do the remaining work during maintenance days. Finishing the beamline would require a major shutdown of the SASE1 branch later.

Since there will be no alternative to send the beam to (e.g. SASE2) for the next year (?), it does not make sense to stop work in XTD10 now, just to continue later then, without any overall benefit for the machine.

Of course i don't have a complete overview of all the machine schedule, but from my point of view it would make more sense to finish work in the northern branch, at least to align all components and to close the vacuum system, even if that means some more weeks before the SASE1 startup.

cu
Markus

  124   10 Apr 2017, 12:31 Alessandro SilenziGeneralInformationKARABO Update maintainance

The karabo services will be down for an upgrade for an hour.

The access will resume at 13:30.

  125   10 Apr 2017, 18:13 NEREA JARDONGeneralInformationVacuum Loop

Dear all,

Right now both XGMs are into the loop and would be running. The loop must run continuously, in case you should do any work, just let me know in advanced.

Thank you,

  126   10 Apr 2017, 19:19 Tobias FreyermuthGeneralInformationupdate of the EPS loop

Dear cllougues,

 

we will run an update of the EPS-loop, in order to have a proper solution for the latest hotfix, as it was announced this morning in the standup meeting.

 

Cheers,

    Tobias

  127   10 Apr 2017, 20:05 NEREA JARDONGeneralInformationEPS & Motion

Dear all,

EPS and Motion loop have been updated to the last framework version with the last hotfixed. Vacuum loop would be done tomorrow in order to let the XGM work the hole night and part of the morning tomorrow.

Best Regards,

  128   11 Apr 2017, 11:48 NEREA JARDONGeneralInformationVacuum Loop

Dear all,

Vacuum loop is up and running, we would wait till Raul check the interlock of the "OK" signal for the Vacuum and then, if everything is correct, we would put again the XGM controlled by the Vacuum PLC.

Thank you!

  129   11 Apr 2017, 12:51 Tobias HaasGeneralInformationStatus after 0930 meeting

Tunnel Open.

Mirrors:

Status:

  • Vacuum on M2 should be good enough to switch on ion pumps
  • Surveying work going on 

Vacuum Loop:

Status:

  • Several vacuum sections successfully tested in XTD9 yesterday
  • Tests need to proceed in XTD2 today
  • The protection switch on the ion pumps cannot be actuated remotely 
  • Loop needs an update for the XGM
  • XGM was in the vacuum interlock overnight. Will be taken out during the day but should be put back at the end of the day
  • Interlock of High Voltage with vacuum in XGM not working yet
  • V0 valve needs to be exercised in collaboration with DESY 

EPS Loop:

Status:

  • The terminal exchanged for the FEL imager was not the broken one. Another terminal needs to be exchanged today
  • The MCP needs to be moved out of the beam manually. This will be checked using the EPS switch. For this the EPS loop is needed

Motion Loop:

Status:

  • An improved fix for the motors not stopping was deployed yesterday
  • The loop needs to be reconfigured to replace the SRA motors 
  • One broken motor was exchanged on the mirror chambers and needs to be tested

Imagers

Status:

  • All motion tests in XTD2 done. Now continue in XTD9
  • Photonics camera driver still not working but not critical for tunnel closure (there is also a workaround by the name of ebus player)
  • EPS switch on FEL imager needs testing  after terminal replacement

Karabo:

Status:

  • New version with TCP was deployed. Experience overnight is good

 

Tasks for today:

 

Note: Loop reconfiguration and updates of Karabo to be done in the morning. Testing can resume after lunch. Status updates should please be recorded in the log

  • Reconfigure vacuum, motion loop (updates for XGM interlock, SRA motors)
  • Replace broken terminal in EPS loop for FEL imager 
  • Move MCP out of the beam manually and check using EPS
  • Continue systematic test of the vacuum components (J. Eidam, V. Bondar)
  • Test EPS switch on FEL imager after terminal exchange (A. Koch and W. Ehsan)
  • Monitor stability and responsiveness of GUI Server and Log Servers (A. Silenzi)
  • Set up monitoring of temperatures and pressures overnight (A. Silenzi)
  • Test V0 and absorber

Tasks for tomorrow:

  • Tunnels closed
  • All loops should run stable
  • No or only minimal interference with Karabo
  • Continue tests from remote and check stability and robustness of systems

Other topics

  • Tunnels open over the Easter weekend and also during the day on Tuesday 18 April
  • There will be one more standup 09:30 meeting tomorrow (Wednesday)
  • There will be a SASE Readiness Meeting at 09:00 on Thursday
  130   11 Apr 2017, 13:24 NEREA JARDONGeneralInformationMotion Loop

Dear all,

Motion loop is running again with the last update (names for PSLIT and LVDT have been changed).

Thank you,

  131   11 Apr 2017, 13:32 NEREA JARDONGeneralInformationMotion loop

Dear all,

Sorry for disturbing again, I will do a small update to put one missing coupler in the hardware configuration in rack 1.18, the loop would be running again in few minutes.

Thank you!

  132   11 Apr 2017, 13:48 Alessandro SilenziGeneralInformationScript to install the latest karabo

Please find in attachment the dowload bash script for the latest Karabo.

if your system has a "karabo" directory, this script will not overwrite it, but the update will fail.
In this case, remove or rename the karabo directory.

To use it:
1. download the script in a directory of your choice.

2. open a terminal and execute it with
   $ bash PATH_TO/download.sh

Where PATH_TO is the path where the download script was saved.

Attachment 1: download.sh
#!/bin/bash

if [ -z ${outfilename} ]
then
    outfilename=.karabo.sh
fi
echo $outfilename
git clone https://xfel.redmine:K9Sss5CmWtQ_nbx3gyt2@git.xfel.eu/gitlab/Karabo/commissioning-guidelines .temp-commissioning --depth=1
cp -f .temp-commissioning/project_generation/install.sh .install.sh
rm -fr .temp-commissioning
. .install.sh
rm ${outfilename}
rm .install.sh
  134   11 Apr 2017, 15:38 NEREA JARDONGeneralInformationEPS loop

Dear all,

The EPS loop would be stopped, in order to correct some issues with the hardware in Rack 1.1 and Rack 1.5 in XTD2. We will upload it as soon as possible and I will post a new entry here when it is running again.

Thank you,

  139   12 Apr 2017, 09:16 Andreas KochGeneralInformationImager status

Imager Status, 12.4.17

-          All imagers tested, motors and cameras, motion loop, EPS loop, for 3 imagers XTD2, and 3 imagers XTD9.

-          All blocking issues concerning tunnel closure solved.

-          Remaining issues:

o   Camera trigger, tbd. for 4 imagers.

o   Camera driver for SR imager, Photonic Science, tbd.

o   Camera of sa1-xtd9-imgpii45-cam-beamview not reachable via IP address, tbd..

o   Camera display is now at around 2 Hz, not yet full speed.

o   Store image function, tbd.

o   Firmware update to operate braking switches for homing procedure, tbd.

  140   12 Apr 2017, 12:52 Tobias HaasGeneralInformationStatus and plans after 09:30 meeting

 Tunnel Closed.

Mirrors:

Status:

  • No information on status of mirrors and PBLM since Optics group was not at the meeting

Vacuum Loop:

Status:

  • All vacuum sections were successfully tested in XTD2 yesterday (work ended very late)
  • DPS was tested successfully and all interlocks and remote controls were working
  • The protection switch on the ion pumps can still not be actuated remotely 
  • XGMs in XTD2 and XTD9 fully in the interlock 
  • V0 valve and absorber were moved and interlock tested. Communication with DESY was also tested and is ok.

EPS Loop:

Status:

  • No explicit information today (see remark above)

Motion Loop:

Status:

  • No explicit information today (see remark above)

Imagers

Status:

  • All imagers in XTD2 are ok
  • Trigger is not working on one of three imagers. 
  • Testing proceeded in XTD9 yesterday
  • One camera in one pop-in can not be reached (Network problem?)

Karabo:

Status:

  • Running stably
  • Work required now:
  • Photonics Camera driver
  • Clean up the project database
  • Clean up master equipment list

 

Tasks for today:

 

Note: All loops should be left running and no changes to Karabo.

  • Proceed with remote testing
  • Check out adequacy of scenes
  • Monitor stability and responsiveness of GUI Server and Log Servers 
  • Set up monitoring of temperatures and pressures overnight (A. Silenzi)
  • Cleanup Karabo

Tasks for tomorrow:

  • Tunnels should re-open tomorrow
  • Continue tests from remote and check stability and robustness of systems

Other topics

  • Tunnels open over the Easter weekend and also during the day on Tuesday 18 April
  • There will be a SASE Readiness Meeting at 09:00 on Thursday in XHQ E1.173

 

  141   13 Apr 2017, 10:15 Alessandro SilenziGeneralProblemEtherCAT loop broken for Vacuum loop

The EtherCAT loop for the vacuum PLC is interrupted at crate 1.6 (XTD2).

This triggered all Beckhoff Terminals  default behaviour from this point on (i.e. Valves Closed, Digital Outputs low, etc.).

 

 

  142   13 Apr 2017, 12:07 Tobias HaasGeneralInformationStatus and Plans after 09:30 meeting

 Tunnel Closed until approx 1300, Open over the Easter Weekend until Tuesday 1500.

Vacuum:

  • All valves and pumps in XTD2 are working and can be operated remotely
  • The vacuum system including DPS and XGM is interlocked. Interlocks have been tested and are working properly
  • The fast valve and the absorber at the interface to the electron beam vacuum are working and interlocked. The communication with DESY has been successfully tested
  • Some issue exist when switching turbo pumps on/off in Karabo. Then erratic behaviour is observed. Doing the same remotely communicating via TCP directly works ok
  • More testing needs to be done in XTD9

Mirrors, PBLMs, SRA, Attenuator

  • M1 and M2 are installed and under vacuum. M3 will be installed after easter
  • 3 out of 5 motors can be operated in M1 and M2. The mirrors have been moved into a safe position
  • PBLM motors (ELMO driver) can be operated (started and stopped) but operation is still problematic (motors stop in wrong position, movement not linear)
  • On the SRA the connection to two motors is interchanged
  • On the attenuator certain motors occasionally get stuck
  • Slits have been moved out so the beam line is safe for first operation

Imagers:

  •  
  • All imagers tested, motors and cameras, motion loop, EPS loop, for 3 imagers XTD2, and 3 imagers XTD9.
  • All blocking issues concerning tunnel closure solved.
  • Some remaining points that can be treated remotely: Trigger, software delays, others.
  • The Photonics Camera driver is now working in Karabo.
  • Cameras can currently be read out at 3 Hz which is too slow but ok for first beam commissioning

XGM:

  •  
  • XGM inserted into vacuum loop and interlock (in XTD2 and XTD9)
  •  
  • Grounding completed in XTD2+XTD9 (power cycle of rack was needed)
  • Gases ready to be used in XTD2
  • XGM @ XTD2 ready for beam         (need DOOCS middle-layer servers activated) 

FXE

  • Testing finished for all devices in loop1
  • Now waiting for loops 2 and beyond
  •  

SPB

  • SPB tunnel loop deployed. Testing can start next week
  • For the Optics hutch some Beckhoff crates are still missing 
  •  

PLC

  • Fixes to MC2 library (ELMO and TECHNOSOFT motors) will deployed next week and then tested in XTD9
  • FXE loop 2 was modified and loops 3&4 are deployed
  • SPB loop 1 to be deployed today. Working on loop 2

 

KARABO

  • Slow response appears to be solved
  • There are other not so severe bugs on the backlog
  • Currently all high priority components are supported
  • No requires cleanup and day-one component list
  • Monitor stability and robustness

 

Tasks for today:

Note: All loops should be left running and no changes to Karabo.

  • Fix the issue with the disconnected vacuum loop after tunnel reopens

Tasks for the weekend:

  • Relax and look for Easter Eggs
  • AE and CAS will provide contacts on a good will/best effort basis in case an intervention should be necessary

 

 

  143   13 Apr 2017, 13:19 Tobias HaasGeneralInformationStatus Interlock Tests Today

Radiation Interlock Tests are finished. Tunnels XTD9 and XTD10 up to the fence at the dump can be accessed right now. Magnet current interlock tests still takes a bit of time (ca. 1h). Once this is also done, I shall post a note 

  144   13 Apr 2017, 14:09 Tobias HaasGeneralInformationInterlock tests are finished in Northern Fan

All northern tunnels XTD2/4/9/10 are fully accessible again! 

ELOG V3.1.4-7c3fd00