SASE1 / SASE3 readiness --- WP74

2017-07-14

SASE1: 

• commissioning of Kmono started yesterday at 10h for some hours
• Kmono will be used for undulator commissioning as soon as Kmono is commissioned itself, and machine is ready for this
• Support to SASE tuning given to machine (ongoing)
• Maybe today: support during SASE gain studies
• During last week (service week) : GLT-Anbindung XTD2 established

SASE3:

• During last week shutdown : 
  • progress on DP installation, XGM cabling almost done, much imager and timing cabling done
• When XTD10 is closed (as now) we currently cannot do much because there is no PLC loop in SASE3.
• Non-Beckhoff technical commissioning started (e.g. some progress on MPOD HV-power supply)
• WP74 must do a lot of technical commissioning in XTD10. Our work is by far not done when all our vacuum screws are tightened. There are different WP74 components in XTD10 than in SASE1 (not just a simple copy). WP74 requires the Beckhoff loops for tech. commissioning.
• Alignment of DP „islands“ to be done in next shutdown.
• Planned for next shutdown: installation of PES in XTD10, finishing of cabling

Dear all,

the Tunnel Access Schedule has been update - potential access has been shifted from Wednesday the 19.7. to Tuesday the 18.7. Should you need this access, please inform Harald Sinn!

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

General

• From now on progress with the SASE3 tunnel will be monitored in this meeting

FXE (A. Galler)

• Work is going on to be ready for the start of instrument commissioning of Sunday
• Currently the LPD movement is being tested
• Today the firmware for the vacuum interlock system is being deployed

SPB (M. Messerschmidt)

• All inputs needed for the vacuum interlocks can now be read out
• Karabo was migrated to version 2.1.12. This appears to have some fixes for reading out cameras, but the GUI becomes unusable after ca. 10 min. Hence the version had to be rolled back to 2.1.11. This generated a lot of extra work
• The test of the emergency off system was done this week. In preparation the PLCs were shut down to avoid damage. Bringing the system back was a significant effort. This should be getting less as more and more parameters are migrated to the firmware
• A partial test of the MPOD power supplies was done in preparation for the AGIPD commissioning. Unfortunately some Wiener power supplies still have an older firmware that appears to produce voltage spikes when ramping up. This could damage the detector. Hence a firmware upgrade is needed. But no Windows PC was available on the controls network. The respective card was dismounted and taken to the lab for upgrading by K.-E. Ballack

Tunnel Installations SASE3 (M. Dommach)

• A lot of installation work was carried out during the service period last week
  • All Day 1 devices are in place
  • A lot of missing connections were made (water, pressurized air)
  • Calbing was done for the imagers
  • The Solid Attenuator was assembled
  • There are still two vacuum leaks, one of them is on top of the dump. This may have to wait for the next service period in August
• Focus during the next service period in August:
  • Install missing connection pieces and bellows in order to close the entire vacuum
  • Start bringing up Beckhoff loops with support from AE
  • Bring up as many components as possible 
• For the comments made by WP74/J. Grünert, see the respective log book entry today

Laser

• After a number of fixes and adjustments the AC parameters (Humidity/Temperature) are now according to specs

LPD

• LPD is now fully installed in FXE
• Work on the DAQ system starts next week
• Next step for AGIPD in HERA-S is to get the DAQ system working. After this the detector will be brought to SPB
• K. Wrona would like to cross check the configuration in HERA-S before this happens

AE

• Work focussed this week on getting the remaining SPB loops up, in particular loops 12 and 5
• For loop 12 a number of modules had to be taken out, modified and brought back. This relates to the operation of so-called SMARAGD motors
• Originally it was planned to test the vacuum interlock system in SPB first. This requires inter-loop communication and there are still unused loops in SPB
• Due to the problems with loop 12, it was decided to do the tests in FXE.
• The firmware for the FXE interlock should be deplyed today
• A meeting was held with MPS to clarify the interface between XFEL and MPS sind the MPS system cannot distinguish between the branch beam lines. This can now be handled by logic inside the PLC
• AE is preparing for the support of SASE3 during the next service period

CAS

• For more details see the detailed slides presented by S. Brockhauser during the meeting

AE on call service:

    Contact person:

        Nicola Coppola

    Phone

        9 - 8040

Dear all,

an updated TAS is provided: https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

There is potential access to the XTDs on next Tuesday + Wednesday (25+26.7.), as well as on Monday - Wednesday (31.7.-2.8.). Access will be organized by M. Dommach, since H. Sinn will be on vacation - so please send Martin your access request in time.

Cheers, Gerd

General

• The latency observed by SPB in the GUI client could be reproduced by CAS. Also a  potential reason was identified. As soon as this has been sufficiently tested it will be released
• DAQ tests in FXE with LPD worked for the first time yesterday for 1/16 of the detector
• FXE and SPB will show 1-2 slides about progress and ongoing work during the TC meeting on Friday morning

XTDs

• Fast valve in XTD9 still has issues with noise and will probably need to remain disabled

FXE

• Interlock for loop 1 was deployed. Still need to do loop 2. Work per loop requires ca. 3-4 h.

SPB

• Firmware for the AGIPD hv power supply was updated. The spikes during ramping are gone
• Next items to get going are the SMARAGD motors
• Loop 7 needs to be brought back. Crate 51 has been fixed.
• Work on in vacuum interlock in SPB will need to be carefully timed and coordinated after finishing FXE. Due to impending beam probably not before next week

Dear all,

according to the current planning on next Tuesday (25.7.) both XTD9 and XTD10 will be accessible from 7-14:00:

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules&listViewLinkBack=true

Cheers, Gerd

Dear all,

access to XTD9 and XTD10 will be available from tomorrow Tuesday the 25.7. 7:00 (Vacuum group will make sure the interlock is broken), until Wednesday 26.7. 14:00:

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Please send any additional access requests to M. Dommach and T. Tschentscher.

Cheers, Gerd

General (T. Tschentscher/G. Wellenreuther)

• Further tuning of the machine might require some time which was initially dedicated to photon/experiment commissioning.
• Since Harald Sinn will be on vacation: If you need any kind of access please address T. Tschentscher or M. Dommach. TT will organize coordination of groups requiring access to the tunnels vs. requirements of the machine vs. the demand of the instruments for commissioning.
• Lasing with 30 bunches was achieved this week. Consequently the radiation protection measurements have to be undertaken as soon as possible.

FXE (C. Bressler)

• yesterday no beam at all
• attenuators are being tested
• all hardware installed

SPB (R. Bean/A. Mancuso)

• Implementation of loops required for focal characterization — mainly loops 12 and 13. Also loops required for vacuum interlock.
• Still more loops required to be brought up—working with AE down a clear priority list.
• Clear plan for AGIPD to come to SPB/SFX instrument. Requires DAQ and more.
• CAS reacted quickly to undesirable behavior (freezing) in Karabo, which is now largely solved.

Tunnel Installations SASE3 (J. Eidam, L. Samoylova, N. Kujala)

• SASE3: No access – no progress
• SASE2: work as planned is ongoing
• SASE1: There are still issues with the fast valve.
• During this and last week maintenance days, the Newton PICO motors of FXE monochromator were successfully tested in the tunnel together with Nicola. All four motors for crystal roll and pitch can be moved in vacuum. However a slow motion, necessary for fine tuning, seems still not available. Further motor parameters adjustment is needed, which will be done in the lab with test environment.
• To fulfill the technical commissioning and make the FXE mono available for photon commissioning and operation, the firmware for Alio linear stage should be created. The work is on-going. May be available for tests before mid-August, but the schedule and priorities should be clarified.
• Status of HIREX spectrometer in XTD9 tunnel: The HIREX saw FEL beam for the first time yesterday. All motors were commissioned with Karabo software and ready for commissioning with FEL beam. It is in discussion with SPB and FXE to have parasitic beam for commissioning in the coming week.

Laser (Daniel Kane)

• The pump-probe laser group reported that, with the stabilization of the AC system, we are now continuing with production of the laser system. We are still missing the functionality some components of our control system, but we have enough at the moment that it is not an impediment to us building the laser for now, however it is something that needs keeping an eye on.

Detectors (J. Sztuk-Dambietz)

• LPD: K. Wrona reported that they could observe on all channels but they are still working on storing the data. Loss of device settings induced by glitches in Karabo cost a lot of time to overcome.
• AGIPD: ten out of 16 modules have been successfully tested but there is still a lot of work to do; still testing procedures and electronics, etc. before AGIPD is moving  in to SPB in August

AE (P. Gessler)

• Focus is on loop 12 this week, continued to get remaining parts for loop 12 running
• The firmware based interlock for FXE was implemented, certain conditions are disabled but otherwise it is running.
• Plans for shut down week of SASE3 in tunnel in August are currently being made

CAS (S. Brockhauser)

• As reported previously, a release candidate for 2.1.13 has been prepared last Friday
• During the last week-end CAS could systematically reproduce the GUI delay problem SPB was reporting
• In the beginning of the week CAS could identify the source of the GUI delay issue (which was in an old code of the GUI client and got triggered only under special circumstances) and fix it.
• Solution was patched into the new 2.1.13 release resulting in a new release candidate which CAS finished testing now.
• Note that the new release will also fix most of the annoying issues (especially the configuration persistence problems) experienced recently during LDP integration to FXE
• As soon as the new release becomes finalized its deployment in the different production systems will be done under CAS coordination after consulting with all relevant WPs involved.
• The newly introduced quick release cycles with the attached deployment process (as shown on the meeting last week) will help provide access to improved releases while guaranteeing that no surprises and irreversible regressions apply.
• On the ScanTool thread opened by Christian: CAS has started to design a ScanTool around the SRD from SPB which is now being crosschecked against requirements coming from others, too, like FXE, before starting its implementation.
• Thomas, Gerd, and Harald suggested to disseminate its design, so others can also give feedback. Hence, we agreed, that Sandor Brockhauser provide this information and organize a meeting next week.

General

• The latency observed by SPB in the GUI client has been almost completely fixed. Still some "freezing" from time to time remains.
• DAQ tests in FXE with LPD worked for 12/16 of the detector. Trigger's signal stability issue being addressed by AE.
• XTD 9 and 10 will remain open from 25.07. at 7:00 to 26.07. at 14:00  
• On Wednesday afternoon the machine will be prepare to lase with 30 bunches in order to allow the radiation measurements in SASE1 Exp. Hall on Thursday and Friday.

XTDs

• Fast valve in XTD9 still has issues with noise. Issue will be addressed tomorrow with AE support.

SPB

• Integration of the turbo pumps in  the VAC loop ongoing.
• Chiller control being included in Loop 5
• Firmware for the AGIPD hv power supply was updated. The spikes during ramping are gone
• SMARACT motors being implemented. (depend on Loop 13)
• Work on in vacuum interlock in SPB will probably start this week.
• Some problems with Karabo (While measuring profiles the level scaling does not work, Region of interest values wrong) have been reported to CAS.

Dear all,

I had some minor updates of the TAS during the last days, please check again.

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules&listViewLinkBack=true

Be reminded that there will be virtually no access to XTD2, XTD4 and XTD10 before the 14.8. shutdown, so plan your work accordingly. Access requests please to M. Dommach, cc T. Tschentscher.

Cheers, Gerd

General

• Today the machine is being prepared to lase with 30 bunches in order to allow the radiation measurements in SASE1 Exp. Hall on Thursday and Friday.
• Today GUI will be down for a couple of minutes at 11.30 to allow modifications of the servers for bugs fixing.
• Specific SASE1 areas will be off limits during the test, scheduled for    SPB OPT 27.07.17   12:00-16:00 ,  SPB EXP 28.07.17 9:00-15:00   and  FXE 03.08.17  9:00-15:00.  Please address Eric Boyd's presentation for details. A mail reminder will be also sent to notify the beginning and ending of the tests.
• The machine will be running with 30 bunches during the test and with 1 bunch during the night. Since no EPS is implemented, screens and sensitive components need to be taken out during the 30 bunches operation.
• New version of Karabo ready to be deployed. The time slot when this will happen will be discussed on Friday,
• Screens controlled by Karabo cannot be read from DOOCS. The Karabo/DOOCS interface problems will be be addressed with high priority by CAS.

XTDs

• Fast valve in XTD9 still has issues with noise. Issue will be addressed tomorrow with AE support.

FXE

• The detector group is updating the firmware for the LPD to fix the trigger signal problems. A test on just 1 quadrant will be done today together with ITDM
• Interlock implementation in Loop 1 ongoing.

SPB

• Integration of the turbo pumps in  the VAC loop ongoing. Interlock possible.
• Chiller control being included in Loop 5 - Crate has been modified this morning. Loop will be updated checking that this will not stop the turbo pumps.
• SMARACT motors being implemented. First motor is ready to be tested today after hardware problems have been fixed.
• Work on in vacuum interlock in SPB ongoing.
• The ELMO controllers for the motors of the liquid jet manipulator in Loop 13 should be brought and ready by CW 32. AE working on that. 

SPB Loop 5 is back online:

- but without crate BKF00095-SPB_IRU_CHL (there is missmatch of hardware and documentation (investigation is ongoin)),

- pumps: PUMP_TURBO4, PUMP_TURBO5, PUMP_TURBO8, PUMP_ROOT_1, PUMP_ROOT_2 ,PUMP_ROOT_3 and IRU_VAC/PUMP_ROOT_1  are without serial connection

Interlock is inlcuded and ready for testing.

Best Regards,

Bernard

Dear all,

the problem with the fast valves XTD9 tripping randomly is still not fully understood but we exchanged the controllers for both valves and they now seem to be stable. The valves are set back to active mode again with 1*10^-6 mbar trigger level while vacuum pressure at the sensors is around 1*10^-8 mbar.

No radiation measurements have taken place today (27.07) due to insufficent energy of the machine. We think we can achive the desired energy by tomorrow morning at which time the SRP group will attemtp to commission both the SPB OPT and SPB EXP hutch. The attached presentation shows the new expected time schedule.

A log book entry will be made when area restriction goes into affect and when it is lifted.

Desired parameters where reached for radiation measurments of the SPB OPT and SPB EXP hutches. Radiaiton could not be detected inside the hutch with the beam shutters open and beam verified hitting the target, therefore it was not deemed necessary to take radiation measurments outside the hutches. As a result there are no additional access restrictions in SASE 1.

General

• The radiation test in SASE1 could not take place yesterday since the machine was not able to deliver proper light intensity with 30 bunches. Another tentative is happening now.

• Instrument commissioning during the weekend

• Monday - Wednesday next week will be machine development time. XTD 9 will be accessible on request. Please contact M. Dommach. XTD 10 is foreseen to be close.

   

  SPB

• After tests Smaragd motors are not working. Under investigation with AE/CAS

• Cameras trigger for the sample environment is not working. AE has been made aware of that during the meeting. Ongoing checks.

• Vacuum interlock for Loop 5 (AGIPD) is being implemented. This has to be done before the detector installation (foreseen beginning of August).

• Freezing and delays of the GUI are being experienced. CAS ticket has been sent. This has to be addressed as soon as possible in view of the instrument commissioning time during the week end.

   

  Tunnels

• Fast valve fake triggering has not been solved. After tests the suspect was on the controller side. Controller has been replaced but the issue has not been completely fixed. Investigation ongoing. Valves are ACTIVE regardless.

• In case of fake triggering of the fast valves, there is now a procedure to open them again remotely. Please check the related entry posted by M. Dommach.

• XTD10 – progress in the last 2 days have been achieved concerning closing of vacuum gaps and leaks testing

• XTD6 – Vacuum connections ongoing.

   

  AE

• FXE Interlock deployed and under tests.

• Working on Alio Stages integration for Mono in the tunnel. The goal is to have it ready by the end of August.

• SPB Loop 12 – Smaragd motors and ELMO controllers implementation ongoing

• SPB Loop 5 is up including the AGIPD related systems. AGIPD interlock will be added once low level tests done

• SPB Loop 13 - Phase 2 cables for some modules are at different positions compared to eplans. SPB wants to leave it and rather swap module positions and hand in a change request.

• Cables for interface to Person Interlock laid. Connections to the MPS system to be done next week.

• SASE3 Tunnel - Work started on PLC projects. There is the need for accurate schematics and a list of priorities in order to plan and prepare the shutdown week.

• SASE3 Instruments – AE will start the installation of components ion the racks soon (Mid Aug. after network cables are finished in the PLC related racks).

• General remark  - Do not leave cables in tunnels disconnected after having worked!

   

  DET

• LPD – Trigger problems solved. DAQ tests to continue

• AGIPD – Detector being tested in Hera south. As soon as the important issues concerning DAQ have been solved in Hera south (time allocated next week for that). The decision to move it to the SPB hutch can be taken. Schedule may be slightly affected, depending of the test results in HERA.

   

  CAS

• 2.1.13 Karabo version has been presented yesterday

• Scan tool measurement presentation today at the Mock-up control room

Next week NO SASE1/3 Readiness meeting!

Dear all,

on Tuesday, XTD9 will be accessible from 7:00 - 14:00, due to requests by Naresh and Konrad. I updated the TAS:

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

General

• SRP was not able to measure any radiation even inside D.09 and D.02. This Thursday the radiation test in D.05 (FXE) will be performed. 

• Tomorrow XTD9 open 7:00 ==>14:00

FXE

• Gotthard detectors to be tested in Karabo

SPB

• Problem with Smaract motors has been identified. Loop to be updated. 

• Cameras trigger for the sample environment is now working. Software issues.  CAS is addressing that. 

• Vacuum interlock for Loop 5 (AGIPD) is being implemented. There will be a meeting soon (today?) with AE to check the interlock definitions for AGIPD and continue work on this loop.

• Focus characterization with CRLs ongoing. Beam size reached: <10 um

• Tunnels

• Fast valve fake triggering has not been reported in the last week end. 

• In case of fake triggering of the fast valves, there is now a procedure to open them again remotely. Please check the related entry posted by M. Dommach.

• Some problems with the cameras that stopped working have been reported by A. Koch. That could be just a matter of restarting the GUI. To be checked.

​AE

• FXE Interlock definitions have been changed and this reflects in the update of loop 1 and 2. Ongoing.

• Working on Alio Stages integration for Mono in the tunnel. To start this week. 

• SPB Loop 12 – Smaract motors and ELMO controllers implementation ongoing

• SPB Loop 5 is up including the AGIPD related systems. AGIPD interlock will be added once low level tests done

• SPB Loop 13 - Phase 2 cables for some modules are at different positions compared to eplans. SPB wants to leave it and rather swap module positions and hand in a change request.

• Cables for interface to Person Interlock laid. Connections to the MPS system to be done this week.

​

• DET

• LPD – Time server was set up. Images collected from 1 quadrant. Firmware upgrade for the other quadrants ongoing. 

• AGIPD – Detector being tested in Hera south. KWrona will start the tests of DAQ in Hera today or tomorrow.

   

  CAS

• 2.1.13 Karabo version update is being agreed with the interested parties. It takes approx. 1-3 hrs. 

General

• This Thursday the radiation test in D.05 (FXE) will be performed (from approximately 0900 to 1500), access to all areas adjacent to the hutch is only allowed by those that are assisting with the measurements.

• Meeting on Friday is cancelled. We will meet again on Monday 7-08-2017.

FXE

• Gotthard detectors to be tested in Karabo. On going.
• Finished commissioning of OGT2_CRL and OGT1_ATT. The laser to be used in September installed and connected to electronics.

SPB

• Problem with Smaract motors has been identified. Loop 12 updated, full test still on going.

• Cameras trigger for the sample environment is now working. Software issues, CAS is addressing that. 

• Vacuum interlock for Loop 5 (AGIPD) is being implemented. Interlock definitions for AGIPD under discussion with AE to continue work on this loop.

• Working in the commissioning of Loop13.

Tunnels

• In case of fake triggering of the fast valves, there is now a procedure to open them again remotely. Please check the related entry posted by M. Dommach.

• Two issues with photonic cameras already solved by CAS.

• Images are displayed later, this issue will be discussed off-line with CAS.

​AE

• FXE Interlock definitions have been changed and this reflects in the update of loop 1 and 2. Done.

• Working on Alio Stages integration for Mono in the tunnel. Developing. 

• SPB Loop 12 – Smaract motors and ELMO controllers implementation done.

• SPB Loop 5 thre are some issues with Julabo Chiller, AE contacted the manufacturer for information.

• SPB Loop 13 - Phase 2 cables for some modules are at different positions compared to eplans. Change request submitted  to swap module postions. 

• Cables for interface to Person Interlock laid. Connections to the MPS system to be done this week.

• Loop 8 should be the next, EET  to clarify if the export is done and provide to AE.

DET

• LPD – Synchronization is still showing problems, this is delaying the tests.

• AGIPD – Detector being tested in Hera south. KWrona started the tests of DAQ, some minor issues arised but in general it works.

CAS

• 2.1.13 Karabo version update is being agreed with the interested parties. Implementation will be next Tuesday 8.08.2017 in the afernoon. It takes approx. 3 hrs. 

• CAS will send around a list with the tasks foreseen to be solved during the next weeks.