Group Selection Page HELP DOC Controls Group Data Analysis Group ITDM WP76 DET Vacuum XRO EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Photon Commissioning Team Operation PSPO XO TS
Technical integration and coordination SASE1 Status and SASE3 Readiness
  Joint Operation and Readiness Logbook, Contact: T.Haas, A. Violante, G. Wellenreuther, Page 8 of 30  Not logged in ELOG logo
New entries since:Thu Jan 1 01:00:00 1970
ID Date Author Location Severity Subject
  457   22 Jan 2018, 15:12 Bruno FernandesGeneralInformationRe: Status report after dispatch meeting 13:00

Hi everyone,

there was some miscommunication between us and DESY: only the Timing firmware and doocs server need to be updated, the RTM was already been replaced. This require update is similar to the one perform two weeks ago (see entry #449).

The time slot for this update is next Wednesday, 23rd January, between 16h and 17h. The update will affect all Timing Devices in SASE1. This operation should not take more than 10 minutes; during this time no timing signals will be generate (clocks/triggers/bunch patterns/etc.). Naturally, problems may occur during this update which will lead to increase down time.

If somebody sees a problem with the current schedule, please contact me.

 

Adriano Violante wrote:

General remarks:

  • Stable beam only to be expected starting from 1.03. for SASE1 instruments commissioning
  • There is no guarantee for parasitic beam delivery to the experiments. If it happens at all it  will most probably be during the last week of Feb. This has to be agreed on the fly  with the BKR and H.S/ T.T.

SPB/SFX:

  • Preparing D.02 for mirrors installation (next week)
  • Karabo being updated

Advanced Electronics:

  • Trigger delivered by DESY should be back and stable at 10Hz. 
  • ATM for timing has to be replaced in the balcony rooms. Bruno Fernandes is trying to find a way to reduce the downtime down to 30 minutes. Will be scheduled in the next days.

ITDM:

  • 1 line card has to be replaced. Probably happens on Wednesday this week during the DAQ tests.
  • Tuesday afternoon the online storage units will me remounted with the same path of the offline ones.
  • This week a proposal for the in-house commissioning is going to be circulated.

 

 

  456   22 Jan 2018, 13:28 Adriano ViolanteGeneralInformationStatus report after dispatch meeting 13:00

General remarks:

  • Stable beam only to be expected starting from 1.03. for SASE1 instruments commissioning
  • There is no guarantee for parasitic beam delivery to the experiments. If it happens at all it  will most probably be during the last week of Feb. This has to be agreed on the fly  with the BKR and H.S/ T.T.

SPB/SFX:

  • Preparing D.02 for mirrors installation (next week)
  • Karabo being updated

Advanced Electronics:

  • Trigger delivered by DESY should be back and stable at 10Hz. 
  • ATM for timing has to be replaced in the balcony rooms. Bruno Fernandes is trying to find a way to reduce the downtime down to 30 minutes. Will be scheduled in the next days.

ITDM:

  • 1 line card has to be replaced. Probably happens on Wednesday this week during the DAQ tests.
  • Tuesday afternoon the online storage units will me remounted with the same path of the offline ones.
  • This week a proposal for the in-house commissioning is going to be circulated.

 

  455   22 Jan 2018, 10:36 Gerd WellenreutherGeneralInformationStatus and plans after Readiness Meeting

SPB/SFX:

  • 2nd AGIPD intervention will be finished today.
  • SE worked on a reliable procedure for nozzle exchange and trained the instrument scientists
  • SmarAct motors are in error, Mahadi / AE is informed. These motors are needed for user operation / commissioning.
  • loop 5 update still outstanding
  • Karabo update for loop 1-4 was done, seems to be ok. Full update is planned for beginning of next week.

FXE:

  • JJ X-rays finalized installation of the last components at FXE:
    • Diamond grating (in XTD9)
    • Spectrum analyzers
    • Timing tool
    • repositioning of the sample manipulator and working on better cabling
    • minor work on cables of FXE CRL to allow full movement along the beam direction
    • Johann spectrometer
  • FXE got the information from CAS that the Karabo update was successful and it’s ready for testing
  • FXE decided not not pursue the further commissioning of the new equipment due to the emergency power off tests
  • FXE is working on a more universal sample environment allowing faster and more reliable changes from solid to liquid experiments.

Vacuum:

SASE1:

  • vacuum system closed,
  • PLC loop update done,
  • ready for restart

SASE2:

  • XTD1 all devices in the tunnel and aligned

SASE3:

  • Soft Mono chamber closed yesterday, pumpdown, leaktest later today
  • GasAttenuator comissioning ongoing
  • Karabo scene developement

XPD:

  • SASE3 MCP detector technical commissioning:
    • HV tested OK (MPOD operational), today ADC/signal test
    • motors: rewiring done --> correct directions and limit switches
    • motors were intermediately ok, but are not stable (now there are again issues)
    • several PLC / karabo issues were sorted out, ok
  • SASE3 XGM:
    • test with Kr gas supply yesterday/today
    • test with gas plus HV
  • SASE3 PES:
    • ordered grounding works (cabling) to be done similar to what was done for the XGM
  • SASE2 XGM:
    • vacuum incident: vacuum operation of the XGM in XTD6 was unauthorized interrupted for some time, The rack had been powered down without prior information to XPD.
    • Reminder to all that rack near the XGMs shall be powered down only after informing and coordinating with XPD.

Laser:

 

  • SASE1 PP-laser commissioning and upgrade is continuing according to our plan.
  • Karabo upgrade to 2.2.1 was rather uneventful and successful.
  • Hutch readiness milestones for laser hutches in SASE 3 and 2 seem to consolidate:  15. March. If this continues to hold, we will start our Phase II installation of PP-lasers in E.04 and A.04 with laser tables at the start of April. Laser tables for the ILHs will be installed at the same time.

Detectors: Did not attend.

Advanced Electronics:

  • Replacement of the microTCA for the PES was done
  • microTCA update of the timing system was done
  • local test of the CRL can be done after the cables and the hardware have been fixed by the Vacuum group
  • All loops of FXE updated (firmware etc.)

ITDM:

  • Network ports for SASE2 in the racks have been fixed
  • DAQ in SASE1 needs fixing of hardware, foreseen for CW 4
  • Test of DAQ with new Karabo release is on its way, should be done by Wednesday 24.1.

CAS:

 

  • Karabo 2.2.1 has been deployed on FXE, LAS-LAB, LAS-PPL, and 3 SPB loops without any real problems
  • migration continues next week on SPB
  • BKR will be equiped with a 3rd Karabo Control PCs (we work with Tim already on it)

 

  454   19 Jan 2018, 16:06 Bruno FernandesGeneralInformationEvent #16 back (for good) to 10Hz

Event trigger #16 is now back to 10Hz and it will still that way.

  453   17 Jan 2018, 16:53 Bruno FernandesGeneralInformationEvent #16 back to 1Hz

Dear all,

I communicated today on the 13h meeting that the Trigger #16 was back to 10Hz; this is no longer the case, the trigger signal is back to 1Hz.

 

  452   17 Jan 2018, 15:55 Bruno FernandesXTD2InformationRack 1.2 Timing Update

Dear all, 

following up from the message #449, the MicroTCA Crate in rack 1.2 has been rebooted. Timing system hardware is now running firmware v1.24.0.

  451   17 Jan 2018, 13:54 Adriano ViolanteGeneralInformationStatus report after dispatch meeting 13:00
SPB:
- one mfpga board was found malfunctioning and was replaced with the old one. Concerning the mc the new version can be used.
- Liquind jet test ongoing.
- Waiting for the Loop 5 to be updated by AE
- Old version of Karabo was sluggish as of yesterday.
- 2.2.1 Karabo verion being rolled out for SPB - test ongoing.

FXE:
not represented
Vacuum:
- SASE1 vacuum in the tunnels closed (CRL chamber closed).
- KB mirror chamber in D.02 (SPB optics hutch) will be opened at the beginning of February
- In SASE3 the gas supply is now decoupled from the vacuum loop
- Soft XRay mono vacuum to be closed today.
XPD:
not represented

Detectors:
not represented

AE:

-XTD 2 CRL to be tested tomorrow
- Update of loops ongoing
- Emergency off button test in FXE showed that the relative controlling PLC was not configured correctly. In addition, communication problems between TS and FXE made things more difficult. PSPO will talk to TS and decide how to proceed. In the meanwhile the test foreseen in SPB for next monday HAS BEEN CANCELLED. GWellenreuther has informed Torsten Schoen.



ITDM:
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company solved the problem without bringing down the loops.

CAS:
- 2.2.1 Karabo was deployed in FXE experiment and tests are ongoing.

- 2.2.1. being rolled out in SPB exp and Laser

- SASE3 tunnels  tunnels will get the new Karabo version next week. SASE1 tunnels will follow (probably in the same week) 

  450   15 Jan 2018, 13:56 Adriano ViolanteGeneralInformationStatus report after dispatch meeting 13:00
SPB:
- The exchange of the mc and mfpga boards has been done and some problems were found. The boards are now being debugged at CFEL.
- Sample enviroment's group is testing the liquid jet this week.
- Wait for the Loop 5 to be updated by AE

FXE:
not represented
Vacuum:
not represented
XPD:
not represented

Laser:
not represented

Detectors:
not represented

AE:

- Timing devices update done. One crate in XTD2 has a problem and cannot be operated remotely. Fixing ongoing.
- The clock, VETO and Start signals for the AGIPD could not be brought correctly from the utca in the rackroom to the patch panel in the hutch due to the excessive cable length. Workaround found using a signal replicator at the patch panel position. The problem seems to be solved.


ITDM:
 - LPD recabled with Infiniband connection. Test succesful.
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company is comeing back next week to check the 
cabling. Interventions have to be planned accordingly since the Beckhoff 
loops are running! ==> no news.

CAS:
- not represented
  449   12 Jan 2018, 14:34 Bruno FernandesGeneralInformationUpdate Timing Firmware/Doocs Server

Dear all,

with the exception of one board, all boards in SASE1, SASE3, FXE, SPB and PPL have been successfully updated. All Timing boards are now running firmware v1.24.0. The doocs-x2timer-server has also been updated in all CPUs; running version is 7.8.

The problem occur in the Timing System of rack 1.2 located at XTD2. The MCH board is not reachable via ssh/telnet/web interface; thus I am not able to power cycle the timing board, which would make the hardware load the latest version. The board is currently running (i.e. generating all the timing signals) but with an older hardware version. During the next week this problem will be tackle this problem with a quick visit to the rack.

Cheers.

  448   12 Jan 2018, 11:10 Gerd WellenreutherGeneralInformationStatus and plans after Readiness Meeting
SPB:
- AGIPD was tested with old mc board to verify that it was working fine
after the in-vacuum intervention in Dec 2017. The detector is working
and also the vacuum is ok.
- The exchange of the mc and mfpga boards is ongoing and will continue
next week (according to planning)

FXE:
- Diffraction Grating is being installed by JJ-XRay. End to end tests to 
follow.
- Karabo being updated to the 2.2.1. version.

Vacuum:
SASE1:
- CRL XTD2 closed yesterday, seems to be leak tight
- PLC VAC loop update done yesterday
SASE3:
- soft mono chamber still open, XRO is installing mirrors
- MCP connected to beamline again, pumped down yesterday
- VAC - GATT loop separation ongoing, loop restart next Monday

XPD:
SASE1:
- rebooted vacuum loop yesterday, XGM vacuum gymnastics
SASE3:
- MCP technical comm.:  motor tests, some rewiring & enabling control of HV
- Imagers: all imagers in motion Loop 1 and 2 - some small mistakes
- Photonic Science camera issue (16bit) is solved
SASE2:
- MCP transported into tunnel XTD6
- now ALL vacuum chambers of XPD are in the TUNNELS (few more for the hall)

Laser:
- the timing specification has been extended, 6 laser patterns available
for SASE1/2/3 and the laser lab. 4 others are reserver for SASE4/5. In
the next months desy will upgrade doocs servers and firmware.
- SASE1: pp-laser upgrade and remaining commissioning progress as per
plan (finish end of March). No commitments to experiments till then.
- SASE3 and 2: Plenums are being installed. Plans for laser table
installations: currently still for April.

Detectors:
- did not attend

AE:
SASE3
- Gas attenuator is in separate loop and loop running
- MOV loop 1 updated (incl. MCP related change)
- Plan to update EPS loop today
- VAC loop update planned for Monday
- MicroTCA crate for PES replaced - Timing back for this crate and also
for XGM and KMONO
General 
- Timing changes - Machine switched to 1Hz and they reset the train IDs to avoid soon issues with the machine DAQ system

ITDM:
 - LPD recabled with Infiniband connection. Test ongoing.
 - Meeting next week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company is comeing back next week to check the 
cabling. Interventions have to be planned accordingly since the Beckhoff 
loops are running!

CAS:
- Karabo 2.2.1 has been created and its roll out for hw tests on real
field has been started
- We have started at FXE. Until now only a single problem has been found
with LPD FEM.
- Laser Laboratory has also been updated without any problem detected
until now
- Next Monday Lasers group's PPL installation will be upgraded
- Next week, SPB will be upgraded also step-by-step
- Finally, SASE1 will also be upgraded.

  447   11 Jan 2018, 15:17 Bruno FernandesGeneralInformationFXE/SPB/PPL Timing Update

Hi everyone,

after talking with the responsabile people, here's the timetable for the Timing firmware/server update:

FXE/SPB/PPL x2timer update timetable
Machine Experiment # Devices Time
FXE-RR-SYS-UTC-1 FXE 2 12.01.2018 @ 13H00
FXE-RR-SYS-UTC-2 FXE 2 12.01.2018 @ 13H00
SPB-RR-SYS-UTC-1 SPB 2 12.01.2018 @ 08H30
SPB-RR-SYS-UTC-2 SPB 2 12.01.2018 @ 08H30
LA1-LAS-SYS-UTC-1 PPL 2 12.01.2018 @ 14H00
LA3-LAS-SYS-UTC-2 PPL 2 12.01.2018 @ 14H00

 

In the meantime, all of SASE1 and SASE3 machines have been updated (with exception of the MicroTCA in the SASE1 Balcony room) with no problems.

 

  446   11 Jan 2018, 11:04 Bruno FernandesGeneralInformationUpdate Timing Firmware/Doocs Server

Dear all,

an update of the Timing boards firmware and doocs server is require. This operation should not take more than 10 minutes per device; during this time no timing signals will be generate (clocks/triggers/bunch patterns/etc.). Naturally, problems may occur during this update which will lead to increase down time.

I will perform this necessary update in the devices locates in SASE1 and SASE3 tunnels today at 11H30. If somebody sees a problem please contact me.

Concerning the devices in the Experiment Rack Rooms, I will contact a responsible person to schedule a time slot where this update can be perform. If are one of these people contact me.

  445   10 Jan 2018, 17:35 Bruno FernandesXTD10InformationMicroTCA in rack 3.4/XTD10

The MicroTCA crate used in rack 3.4 (XTD10) has been replaced. The system is up and running. Timing information is properly distributed from this system to the MicroTCA system in rack 3.2 and XGMXTD10.

  444   10 Jan 2018, 17:20 Bruno FernandesGeneralInformation1Hz Triggers

Hello everyone,

The main Timing System at DESY, which distributes all timing information to our systems, is right now configured to generate 1Hz trigger signals on Event #16. This is the trigger that is deliver to most detectors/cameras on our site.

DESY is testing a new Gun and wanted to use a slower trigger frequency during this evaluation.

We were not inform in advance about this reconfiguration. After a few colleagues from SPB approach us reporting this problem, we contacted DESY and got the update. We will take steps to make sure that future Timing changes done by DESY are communicated to XFEL ahead of time.

  443   10 Jan 2018, 14:54 Dennis GoeriesGeneralInformationFXE Mono

Hi,

the FXE Mono's won't be available via Karabo from 15.30 onwards due to an update from CAS (karabo) and ITDM (server).

Best, Dennis

  442   10 Jan 2018, 14:53 Hugo Silva SantosGeneralInformationRe: Status and plans after Readiness meeting

Dear Monica,

Just one small correction at "Karabo 2.2.0 being deploy today for FXE." The version is, actually, Karabo 2.2.1 that we released yesterday.

Hugo

Monica Aresté wrote:

PSPO:

- G.Wellenreuther inform that MKK is planning to test the Umschaltautomatik, this will affect XHE1, XHE3, XHPSC, XHEXP, XHVAC. Oleg Engelmann will send a mail with detailed information.

 

SPB:

- AGIPD is fully occupied again and restarted this morning as scheduled. Everything works well so far and it is expected to finish the test on time. Further test with new Microcontroller are scheduled for next week.

- Most of the loops that required modifications are updated or on going.

- Module BKF00128 will be modified this week, update of Loop 5 will folllow.

- Loop 12 is ready to be deployed, to confirm if modifications in the modules are finished and reorganization on rack UH2.12 is ready.

- It is necessary to correct an error in module BKF00220,  AE will agree with SPB  when this will be possible.

 

FXE:

- New Vacuum device in operation.

- Karabo 2.2.0 being deploy today for FXE.

 

AE:

- Replace one Micro-TC in SASE3, a cooling unit is failing and one module will be replaced by a new one today. This is providing timing for XGM and K-mono. AE will inform by the logbook before to start the replacement.

 

 

 

  441   10 Jan 2018, 13:22 Monica ArestéGeneralInformationStatus and plans after Readiness meeting

PSPO:

- G.Wellenreuther inform that MKK is planning to test the Umschaltautomatik, this will affect XHE1, XHE3, XHPSC, XHEXP, XHVAC. Oleg Engelmann will send a mail with detailed information.

 

SPB:

- AGIPD is fully occupied again and restarted this morning as scheduled. Everything works well so far and it is expected to finish the test on time. Further test with new Microcontroller are scheduled for next week.

- Most of the loops that required modifications are updated or on going.

- Module BKF00128 will be modified this week, update of Loop 5 will folllow.

- Loop 12 is ready to be deployed, to confirm if modifications in the modules are finished and reorganization on rack UH2.12 is ready.

- It is necessary to correct an error in module BKF00220,  AE will agree with SPB  when this will be possible.

 

FXE:

- New Vacuum device in operation.

- Karabo 2.2.0 being deploy today for FXE.

 

AE:

- Replace one Micro-TC in SASE3, a cooling unit is failing and one module will be replaced by a new one today. This is providing timing for XGM and K-mono. AE will inform by the logbook before to start the replacement.

 

 

  440   09 Jan 2018, 10:32 Martin DommachGeneralProblemShutdown of compressed air distribution in XTDs today

Dear all,

we got the information from MKK, that they have to fix a leak in the main compressed air line. Therefore they will close the valves to the tunnels for approximately one hour starting today at 1 p.m.

Please do not use the compresed air system in the tunnels during that time with air driven tools or air guns, as the volume of the distribution systems acts as a buffer.

Regards,

Martin

 

  439   09 Jan 2018, 10:14 Gerd WellenreutherGeneralInformationAccess to XTDs

Dear all,

according to H. Sinn the closing of the XTDs 2, 4 and 10 will be delayed until the 30th of January.

Cheers, Gerd

  438   08 Jan 2018, 13:47 Monica ArestéGeneralInformationStatus and plans after Readiness Meeting

PSPO:

- G. Wellenreuther inform that DESY is commissioning a new gun in the injector that could take longer than expected. Harald Sinn is in a meeting today to disccuss this issue, any change in the scheduled will be reported.

 

SPB:

- Lost motor positions in the tunnel, CRL in defined state. Good state in general no major problems expected.

- Detector was clabled again.

- Loop updates still pending, SPB request to have loops ready during this week.

 

FXE: No one attended

 

Vacuum:

- SASE1:

  • Some works in CRL for XTD1.
  • Working in FXE grating chamber, necessary to update Vacuum loop.

- SASE3:

  • M1 and M2 mirrors were installed before X-mas. Mirrors installation in K-mono to be finished next week.
  • Created new loop this affects Vacuum loop, update necessary.
  • XGM loop need to be updated.
  • Motion loop 1 is working, some works in MCP motors on going.

 

Diagnostics:

- Technical commissioning of MCP on going together with the Russian collegues.

- MCP SASE2 will be transported to XTD6 on Wednesday afternoon.

 

AE:

- Working to implement redundancy into FXE loops.

- To agree a date with FXE to work in the grating chamber at the end of XTD10.

- Motion loop for SASE1 will be interrupted this afternoon at 14:00 to add some terminals for mirror 2.

 

Software:

- Deployment of Karabo 2.2.0 will start this Wednesday in FXE.

- Next dates for Karabo deployment are: 15th January for the Tunnels,  22nd January for SPB.

 

TS:

- Torsten Schoen inform that it is necessary to carry out an 'Emergency off button' test during the Shutdown.

- SPB agreed to perfom the test the 22nd January in the morning.

- For FXE the proposal is to perfom the test the 17th January, to be confirmed with AE and FXE.

 

 

 

 

 

 

 

 

 

ELOG V3.1.4-7c3fd00