General

"In the beginning of this meeting SM congratulated the whole staff of the European XFEL and in particular the commissioning team with the very important achievements 
- the first SASE3 lasing / bi-lasing at SASE3/SASE1. It happened yesterday late evening. More information will be given during our PBS meeting next week."

Photon Run Coordinator Report to Readiness Meeting

• parasitic operation in SASE3 (closed 3 undulator segments) for photon commissioning with SR-beam
  • V0 opened, vacuum open down to XHEXP-shutters
  • photon beam detected down to FEL imager, yesterday until SR absorber beyond mirrors M1/M2 and MCP
• photon systems supported tuning/search for SASE1 lasing
• Progress in SASE3: 
  • First Lasing in SASE3/XTD10 
  • simultaneous lasing SASE1 and SASE3

SPB

• During operation Karabo crashed. The problem is not clear yet. Det. loop was affetcted ==> AGIPD tests stopped!
• 2 mirrors in D.02 are being installed.
• Test of the mirrors movements is ongoing
• Motors issues are being checked by AE (Mahadi)

FXE

• Gotthard det. and digitizers back to work
• Leak close to the diamond window being fixed

Laser:

• Timing system has been upgraded - currently we can have any repetitive laser pattern for each laser, number of laser pulses in not limited to number of electron bunches, also we can have lasing when there is no x-ray (electron bunch)

•  Installation according to the plan, no major delays. No commitment to experiments for laser availability in March.

Vacuum

• SASE2:

  • XTD1 under vacuum up to XS2 shutter

• SASE3: 

  • interlock problems solved, runs without problems since yesterday

  • GasAttenuator loop needs update. when is the best time to do that? to be agreed with PRC

Optics

• issues mainly in the XPD section 

XPD

• SASE1 : photon systems supported tuning/search for SASE1 lasing
  • tests of DAQ in SASE1 ongoing
• parasitic operation in SASE3 (closed 3 undulator segments) for photon commissioning with SR-beam
  • V0 opened, vacuum open down to XHEXP-shutters
  • photon beam detected down to FEL imager, yesterday until SR absorber beyond mirrors M1/M2 and MCP
• karabo update to 2.2.x in SASE3 tunnel scheduled for next tuesday morning
• SASE3 DAQ urgently required
• issues with in-vacuum goniometer motor control of KMONO
• issues with IMGFEL to be understood - found a scintillator which works when we have FEL radiation

AE

•  The need to properly plan the activities close to the tunnels closing is pointed put.
• The need of a specific technical coordinator as in SASE3 for the SASE2 tunnels is pointed out

ITDM

•  Sase3 DAQ not ready since Karabo is not yet up-to-date
•  Slow data acquisition issue affecting cameras: being worked out.

CAS

•  In SASE3, deployment of new (2.2.1) Karabo versionwas  delayed because of many people getting the flu.
•  2.2.2 release test cycle ongoing

SPB/SFX

- An AGIPD crate providing power broke down and has been fixed already by Kai-Eric Ballack. This delayed the data acquisition test planned by Jola. These are being done today.

 -Synch motion of mirrors will be tested with fake mirrors in D.09 as soon as the control is ready. Work is ongoing.

-First mirron in the SPB optics hutch will be installed today. 

- New Karabo release working smoothly so far.

FXE

- CRL in the tunnel being commissioned

 -Leak close to the diamond window being fixed. If everything goes fine ready for parasitic beam on Wed next week.

VAC

-Vacuum interlock tripped twice in the last 12 hours. Issue has to be clarified with CAS.

AE

-Reconfiguration of the SmarACT controllers between rotary and linear motion is being carried out. Marcin and Mahadi are the main players here.

ITDM

-Major safety update of the cluster has to be done. ITDM proposal is to do it next week. SPB and FXE opted for the early morning (between 8 and 10). Exact date will be announced by ITDM as soon as possible.

SPB

- MIrrors installation (2 horizontal mirrors) in D.02 has started 

 -Synch motion of mirrors will be tested with fake mirrors in D.09 as soon as the control is ready.

SASE3 - CYoungman

- After SASE3 tunnels were closed it was discovered that was not pobbile to set values of the massflow (GAT loop). This required a remote intervention (firmware) that was supposed to be fast and easy  but it was not, and this required work until friday evening. Current status is: The gas attenuator secion closed and in static vacuum since the shutdown of a turbo pump was required. The system is in asafe status.The issues are being analized and solved with experts right now.

-It is important to establish a procedure to set priorities for ZZ access. The issue will be brought up in the PSO meeting

ITDM

-AGIPD p[roject is OK. Jolanta will start taking data already today

SPB

- Loops update OK

- Issue with SmarACT controllers. AE is on it (see below).

- Synch motion of mirrors motion being worked out with AE.

FXE

- Loops update OK

- DAQ updated

- CRL in the tunnel have two encoders with problems. Can be disabled and are not a problem for operation.

- A leak before the diamond window was found. Being closed now.

- Preparation for timing tool commissioning ongoing

- The new user chicane will be probably cut next week.

Laser:

- SASE 1 commissioning and upgrade moves according to plan

- SASE 2 and 3:  so far we still have green light concerning access control, laser safety installations, network and sync for installation of laser tables starting at the beginning of April.

Vacuum

- SASE1 und SASE3 vacuum systems are ready for restart of beam operation

- update of SASE1 vacuum loop on Wednesday caused many problems (all ion pumps in error state)

Optics

- Test piezo motors XTD9 today

XPD

- XTD10: K-Mono, XGM, PES, filter chamber, imagers operational except:

- Karabo tests for Exit slit imager.

- Camera cables to be exchanged for 2 imagers, ZZ access 1h.

- PES, issues with digitiser and MPOD scenes.

- XTD2 and 9: All components operational, Hirex also including Gotthard.

- Next steps

- Progress on scenes and projects implementations.

- Compatibility with new Karabo, Karabo GUI.

- Test and improve usage of DAQ.

AE

- Coordinated motion of MC2 Based Motors were implemented and presented to first users in SPB and XPO as well as to CAS

- The issue with the SmarACT motor controllers observed in the past was due to an issue in the controller. It was solved with an firmware update and requird configuration with dip switches. For this the later usage (e.g. rotary, linear, etc stages) has to be defined. The modifications for SPB is ongoing.

- AE observed, that in some cases PLC related electrical work was carried out until the very last day of open tunnel, which require a PLC firmware update and successive test. Since there is also a risk in the firmware update as it could be, that during tests issues might be observed, there is no access anymore to the actual system and thus could not be solved without ZZ or new access time. We had issues due to this, where at least temporary workarounds were possible - but it could also be different. This is a high risk, which should be know to everyone involved and decided on, if we want to continue like this in the future. AE would prefer to stop with any electrical PLC changes at least some days before tunnel closure and allow firmware updates (possible Karabo changes) and full tests by the users of the components with Karabo in order to identify and solve possible issues.

- Another issue took place, where a change request for a PLC change was issued, but instead of waiting for the finally modified designs to then do the related hardware modification, the modification on hardware were done without any new designs. This creates many problems: no CAD design according to hardware exists, the current revision state of the module is not defined, no firmware structure could be exported and generated for the current design, etc. Therefore modifications without new CAD design and consulting AE is not allowed!

ITDM

- Sase3 tunnels: cabling issues solved.

- DAQ so far stable. Has to be tested at full load now with the new Karabo.

- DAQ update for Tunnels in SASE1 and 3 ongoing. Today and next Monday will be test days.

- Gateway update done: small issue with SSH connections, easily solvable==> the procedure will be posted in the logbook

- The Maxwell cluster will need to be shut down for 2 days next week due to upgrades.

CAS

-  2.2.1 has been deployed without real problems on FXE, LAS, SPB, SA1.

-    In SASE3, it is still 2.1.18 and it will be updated early next week

-    2.2.2 release test cycle almost finished. The release will be published early next week - This release will not be mandatory, but recommended based on the new GUI benefits.

-    SASE3 Tunnel most critical tasks were done, including the gas attenuator by Valerii. It was a really complex task that Valerii went deeply and get it done for us all!

-    Sergey keeps working in Digitizer with Hamed (AE) but based on the deadline they decided to go for the old protocol.

-    Andrea (Alessandro and Steffen) has been working on the AGIPD Library to report when AGIPD will stop to acquire data

-    Gabriele started discussions to address the DAQ for LAS with ITDM.

-    XGM's DOOCS integration has been updated to address TrainID issues. We must allocate 2-3 days to test it with Beam.

Today late in the evening (~22:00), the exflgateway will be migrated to a new hardware. All users will be logged out before migration and home folders will be copied.

After migration is finished, you may experience ssh warning about ssh host key. For all centrally managed Linux this should be fixed the following morning. For all other, like Windows, or yellow desktops, you need to remove the old key and/or accept a changed one. 

Best 

Janusz

The migration has finished, to avoid temporary problem with ssh you can use

ssh exflgateway -o StrictHostKeyChecking=no

until it will be resolved in the morning ( the usual update of all hosts ssh keys).

Home folder has been changed to /data/<user_name> and it is slightly bigger then before...

Let me know if there is still anything missing.

best, Janusz

SPB/SFX:

• Fixing of Karabo scenes ongoing
• Loop 5 need to be updated after discussion with DET group (Jolanta Stuk)
• A meeting with AE will take place today to discuss the synchronous movement of the mirrors motor. 
• The test on the NKB chamber of the indipendent movement should take place. It has to be double checked that such motors are implemented in Karabo.

CAS SASE3 (C. Youngman):

• SASE3 tunnels closing today at 14:00. All the critical tasks were done. 
• The day before the first lasing, a ZZ access is strongly requested. The request will be discussed in the PSO meeting.

ITDM:

• Tomorrow the DAQ will be available for the AGIPD. LPD to follow. The goal is to have SASE1 ready next week as a whole.

SPB/SFX:

• Problems with data loggers ==> no statements about the past.
• Loop 5 upgraded
• Alle the other Loops are not at the 100% of their potential but sufficient for the users.
• Test on nkb mirror movement to be done.

XPO

• In SASE3, 4 cameras have trigger problems. AE is working on that.
• In SASE3, 4 cameras have accessibility problems. ITDM is working on that.

CAS SASE3 (C. Youngman):

• Interlock for VAC, GAT and XGM loops are tested and OK
• The rest is so far as it could go. For details refer to C. Youngman report.

ITDM:

• Broken Linecard  replaced .
• Update/switch for the gateway to the control network will happen in one late evening during this week. ITDM will write an entry in the logbook just before. 

Dear all,

the closing of the XTD10 tunnel is shifted to Wednesday (January 31st.) at 2 p.m.

Best regards,

Martin

Dear all,

please enter your access requests to the tunnels into the list provided by Winni Decking & Co.:

https://ttfinfo.desy.de/elog/XMLlist?file=/XFELelog/data/doc/Access-Requests&xsl=/elogbook/xsl/elog.xsl&picture=true

Can be also found through the usual XFEL-Logbook, and then clicking on "Access Request" on the left hand side.

Cheers, Gerd

SPB/SFX:

• Preparation for the mirror installation were ongoing as planned.
• The Karabo and server update went smooth, but scenes were broken a bit more then anticipated so we are still testing on that part.
• ITDM was working on DAQ update and Jola checked out the detector with the new Karabo successfully.

• did not attend / did not send any report

Vacuum:

• SASE1:

  • Vacuum system ready for restart

• SASE3:

  • loops were updated

  • final interlock test ongoing

  • GATT tests with Karabo

    • Vacuum possible

    • manual attenuation of the beam possible, no PID control

    • Controls of Ebara pump needs commissioning.

• SASE2:

  • XTD1: under vacuum, leak test done, all pumps running

Dear all,

the update to the SASE1 Timing system has been perform with no problems. The Timing board is now running firmware 1.24.0 and the doocs server was updated to version 7.8. I also updated the MCH firmware to version 2.19
 

SPB/SFX:

• Preparing D.02 for mirrors installation (next 2 weeks no access to D.02)
• Karabo being updated - Detector being tested, seems fine so far.
• DAQ tests foreseen for today
• NKB movement (Loop3) should be tested to make sure that the controls work fine before interfacing the MKB with real mirrors inside.
• AE will check the status of the CSS crate. If ready it will be mounted and tested after the MKB.

Advanced Electronics:

• ATM for timing has to be replaced in the balcony rooms.  downtime expected to be  15 minutes. Will happen Today at 16:00

CAS:

• Deploy SASE1 topic in the tunnel this week.Tomorrow at 13:oo CAS would like to make the switch. A.Silenzi will write an entry in the logbook to make sure everyone is on board with that.
• Release of the version 2.2.2. being prepared. This would mainly improve the GUI performance.

XPO

• FXE Mono local test performed and OK. To be tested through Karabo in Feb.

CAS SASE3 (C. Youngman):

• Gas attenuator controls being completed and tested.
• Soft MOno local test done and only small problems found
• EPS loop in good state - interlock test tomorrow.

General remarks:

• Stable beam only to be expected starting from 1.03. for SASE1 instruments commissioning
• There is no guarantee for parasitic beam delivery to the experiments. If it happens at all it  will most probably be during the last week of Feb. This has to be agreed on the fly  with the BKR and H.S/ T.T.

SPB/SFX:

• Preparing D.02 for mirrors installation (next week)
• Karabo being updated

Advanced Electronics:

• Trigger delivered by DESY should be back and stable at 10Hz. 
• ATM for timing has to be replaced in the balcony rooms. Bruno Fernandes is trying to find a way to reduce the downtime down to 30 minutes. Will be scheduled in the next days.

ITDM:

• 1 line card has to be replaced. Probably happens on Wednesday this week during the DAQ tests.
• Tuesday afternoon the online storage units will me remounted with the same path of the offline ones.
• This week a proposal for the in-house commissioning is going to be circulated.

Hi everyone,

there was some miscommunication between us and DESY: only the Timing firmware and doocs server need to be updated, the RTM was already been replaced. This require update is similar to the one perform two weeks ago (see entry #449).

The time slot for this update is next Wednesday, 23rd January, between 16h and 17h. The update will affect all Timing Devices in SASE1. This operation should not take more than 10 minutes; during this time no timing signals will be generate (clocks/triggers/bunch patterns/etc.). Naturally, problems may occur during this update which will lead to increase down time.

If somebody sees a problem with the current schedule, please contact me.

SPB/SFX:

• 2nd AGIPD intervention will be finished today.
• SE worked on a reliable procedure for nozzle exchange and trained the instrument scientists
• SmarAct motors are in error, Mahadi / AE is informed. These motors are needed for user operation / commissioning.
• loop 5 update still outstanding
• Karabo update for loop 1-4 was done, seems to be ok. Full update is planned for beginning of next week.

FXE:

• JJ X-rays finalized installation of the last components at FXE:
  • Diamond grating (in XTD9)
  • Spectrum analyzers
  • Timing tool
  • repositioning of the sample manipulator and working on better cabling
  • minor work on cables of FXE CRL to allow full movement along the beam direction
  • Johann spectrometer
• FXE got the information from CAS that the Karabo update was successful and it’s ready for testing
• FXE decided not not pursue the further commissioning of the new equipment due to the emergency power off tests
• FXE is working on a more universal sample environment allowing faster and more reliable changes from solid to liquid experiments.

Vacuum:

SASE1:

• vacuum system closed,
• PLC loop update done,
• ready for restart

SASE2:

• XTD1 all devices in the tunnel and aligned

SASE3:

• Soft Mono chamber closed yesterday, pumpdown, leaktest later today
• GasAttenuator comissioning ongoing
• Karabo scene developement

XPD:

• SASE3 MCP detector technical commissioning:
  • HV tested OK (MPOD operational), today ADC/signal test
  • motors: rewiring done --> correct directions and limit switches
  • motors were intermediately ok, but are not stable (now there are again issues)
  • several PLC / karabo issues were sorted out, ok
• SASE3 XGM:
  • test with Kr gas supply yesterday/today
  • test with gas plus HV
• SASE3 PES:
  • ordered grounding works (cabling) to be done similar to what was done for the XGM
• SASE2 XGM:
  • vacuum incident: vacuum operation of the XGM in XTD6 was unauthorized interrupted for some time, The rack had been powered down without prior information to XPD.
  • Reminder to all that rack near the XGMs shall be powered down only after informing and coordinating with XPD.

Laser:

• SASE1 PP-laser commissioning and upgrade is continuing according to our plan.
• Karabo upgrade to 2.2.1 was rather uneventful and successful.
• Hutch readiness milestones for laser hutches in SASE 3 and 2 seem to consolidate:  15. March. If this continues to hold, we will start our Phase II installation of PP-lasers in E.04 and A.04 with laser tables at the start of April. Laser tables for the ILHs will be installed at the same time.

Detectors: Did not attend.

Advanced Electronics:

• Replacement of the microTCA for the PES was done
• microTCA update of the timing system was done
• local test of the CRL can be done after the cables and the hardware have been fixed by the Vacuum group
• All loops of FXE updated (firmware etc.)

ITDM:

• Network ports for SASE2 in the racks have been fixed
• DAQ in SASE1 needs fixing of hardware, foreseen for CW 4
• Test of DAQ with new Karabo release is on its way, should be done by Wednesday 24.1.

CAS:

• Karabo 2.2.1 has been deployed on FXE, LAS-LAB, LAS-PPL, and 3 SPB loops without any real problems
• migration continues next week on SPB
• BKR will be equiped with a 3rd Karabo Control PCs (we work with Tim already on it)

Event trigger #16 is now back to 10Hz and it will still that way.

Dear all,

I communicated today on the 13h meeting that the Trigger #16 was back to 10Hz; this is no longer the case, the trigger signal is back to 1Hz.

Dear all, 

following up from the message #449, the MicroTCA Crate in rack 1.2 has been rebooted. Timing system hardware is now running firmware v1.24.0.

SPB:
- one mfpga board was found malfunctioning and was replaced with the old one. Concerning the mc the new version can be used.
- Liquind jet test ongoing.
- Waiting for the Loop 5 to be updated by AE
- Old version of Karabo was sluggish as of yesterday.
- 2.2.1 Karabo verion being rolled out for SPB - test ongoing.

FXE:
not represented
Vacuum:
- SASE1 vacuum in the tunnels closed (CRL chamber closed).
- KB mirror chamber in D.02 (SPB optics hutch) will be opened at the beginning of February
- In SASE3 the gas supply is now decoupled from the vacuum loop
- Soft XRay mono vacuum to be closed today.
XPD:
not represented

Detectors:
not represented

AE:

-XTD 2 CRL to be tested tomorrow
- Update of loops ongoing
- Emergency off button test in FXE showed that the relative controlling PLC was not configured correctly. In addition, communication problems between TS and FXE made things more difficult. PSPO will talk to TS and decide how to proceed. In the meanwhile the test foreseen in SPB for next monday HAS BEEN CANCELLED. GWellenreuther has informed Torsten Schoen.



ITDM:
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company solved the problem without bringing down the loops.

CAS:
- 2.2.1 Karabo was deployed in FXE experiment and tests are ongoing.

- 2.2.1. being rolled out in SPB exp and Laser

- SASE3 tunnels  tunnels will get the new Karabo version next week. SASE1 tunnels will follow (probably in the same week) 

SPB:
- The exchange of the mc and mfpga boards has been done and some problems were found. The boards are now being debugged at CFEL.
- Sample enviroment's group is testing the liquid jet this week.
- Wait for the Loop 5 to be updated by AE

FXE:
not represented
Vacuum:
not represented
XPD:
not represented

Laser:
not represented

Detectors:
not represented

AE:

- Timing devices update done. One crate in XTD2 has a problem and cannot be operated remotely. Fixing ongoing.
- The clock, VETO and Start signals for the AGIPD could not be brought correctly from the utca in the rackroom to the patch panel in the hutch due to the excessive cable length. Workaround found using a signal replicator at the patch panel position. The problem seems to be solved.


ITDM:
 - LPD recabled with Infiniband connection. Test succesful.
 - Meeting this week to decide whether to use or not the train builder for 
AGIPD and LPD detectors
 - in 3 racks in XTD10 some network connection added last year do not work 
properly. The responsible company is comeing back next week to check the 
cabling. Interventions have to be planned accordingly since the Beckhoff 
loops are running! ==> no news.

CAS:
- not represented

Dear all,

with the exception of one board, all boards in SASE1, SASE3, FXE, SPB and PPL have been successfully updated. All Timing boards are now running firmware v1.24.0. The doocs-x2timer-server has also been updated in all CPUs; running version is 7.8.

The problem occur in the Timing System of rack 1.2 located at XTD2. The MCH board is not reachable via ssh/telnet/web interface; thus I am not able to power cycle the timing board, which would make the hardware load the latest version. The board is currently running (i.e. generating all the timing signals) but with an older hardware version. During the next week this problem will be tackle this problem with a quick visit to the rack.

Cheers.