General

• After discussion between XFEL Vacuum (J. Eidam) and DESY XFEL Run Coor (Matthias Scholz) it is agreed that XTD10 will only be closed on Thursday and that XFEL will search the tunnel between 15:00 and 17:00. (This information became available only after the meeting)
• This gives some extra time to finish activities in XTD10

Instruments

• FXE
  • Currently A. Parenti and FXE are hunting a spurious data loss in Gotthards
  • Otherwise the software is surprisingly stable
• SPB
  • Slow progress towards getting the Detector going again
  • The task to ease the acquistion needs help from the DESY developers

Tunnels

• XTD10:
  • A lot of progress was made by the team working in XTD10: Thank you very much!
  • The complete tunnel is under vacuum
  • Also the missing pieces between XGM/MCP and the rest were inserted 
  • Cleanup is done
  • Denis and Raul are still working on local tests of the gas attenuator system
  • Testing is somewhat slowed down because the GUI client no longer allows active manipulation of components at the moment. So the ikarabo interface is used
  • Work also continues on getting the mov1 loop going
  • Local tests are done on all imgers except on the exit slit imager
• XTD9
  • ​The FXE Mono is not finished. It needs a restart of the motion loop
  • AE will check when this is possible, but there are a lot of constraints from users and from resources
  • Since there has been a change in the interlock system, SPB needs to be in place when the shutter to XTD9 is opened. There is a risk of triggering the interlock and shutting down the machine.

AE

• Almost everything that was planned could be achieved in XTD10
• Some of the remaining work can be finished when the tunnel is closed.
  • Work is going on to finish the controls
• Even the interface to MPS that was originally not foreseen is now available

CAS

• The workaround that allows to modify parameters even though the protection does not allow this, is now in place. This should help with the trouble observed by the Vacuum Group.
• A new bug in the PLC was observed in this context. Changing a parameter whose name has changed leads to a crash. 

ITDM

• Currently the MAXWELL cluster is unavailable.  This is a serious software/hardware issued. The manufacturer is actually on site to fix the issue today

FXE:

• Pump-probe beam is now in the instrument hutch
• Currently working on setting up the remainder of beam steering and compression
• Gotthardt is working. Tests are going on a and some issues with the DAQ still need to be solved
• The sample stack needs to be moved up stream for the next user period

SPB

• Still working on AGIPD. Many cables needed to be disconnected and are being re-connected now
• AE finished all the scheduled modifications on the crates and are doing some more that were originally not scheduled
• A few details with the interlock need fixing
• The newest version of Beckhoff_Devices needs to be deployed
• The previously observed slowness of the GUI is probably fixed but needs more real life exposure. Final test are being done
• The full infrastructure should be back and operational Tuesday/Wednesday

Tunnels

• Work is going on on the gas attenuator
• Interlocks are ready for the WP74 DPS and the XGM
• Detailed steps to completion in XTD10 were re-iterated by C. Youngman. Additional coordination off line after the meeting
• L. Samoylova mentioned that the mirror encoders in XTD2 are currently in error. B. Baranasic will have a look

AE

• In the new version of KARABO, the rights for reading and writing parameters from and to the PLC are now being observed, differently from before.
• This causes problems. P. Gessler mentions that he is very concerned that such a major change is introduced so briefly before the next user period. This puts AE under great pressure and leaves almost no time for testing
• All the SASE1 loops were updated because of the new "feature" in KARABO
• A. Silenzi mentions that he can implement a workaround for this "feature" so that loops need no updating. However, this requires a new KARABO release

CAS

• KARABO2DOOCS: Meeting today with the DESY control people at 1430 in BKR
• ScanTool: Meeting foreseen Tuesday/Wednesday with the instruments
• The GUI slowness necessiated a hotfix which was verified within CAS. Now waiting for feedback from instruments

General

• SASE1 Instruments are asked to comment on progress with KARABO 2.1.18
• Tunnels XTD9/10 will be closed on 25 Oct at 1700

Instruments

• FXE (A. Galler):
  • KARABO 2.1.18 appears to be working ok. The device instantiation problem reported earlier in the week is no longer there. However, not all equipment could be tested yet
  • Work currently focusses on preparing for the next User Run and in particular on preparing the sample environment
• SPB (A. Mancuso)
  • A situation similar to the on in FXE is observed. Full proof of functionality of KARABO will only be possible once AGIPD is fully assembled again which is next week at the earliest
  • Currently, work is going on inserting the AGIPD cables into drag chains so that the detector can be moved on the CSS. This work should finish today

Tunnels (J. Eidam, R. Villanueva, T. Maltezopoulos)

• Leaky vacuum pipes were successfully welded
• Absorber section is closed and under vacuum
• MCP detector is connected
• DESY finished work on the upstream section in the dump
• The gas attenuator has very good vacuum (10^-9)
• Differential pumping is ready from the HW side
• Cabling for the gas attenuator section is still going on
• Alignment was done in view of the small apertures in the gas section
• Local tests can start now (This is 3 days behind the schedule made at the beginning of the week)
• PES is installed but will only act as a pipe.

Laser (M Lederer)

• NOPA Beam is now going to FXE
• Next week it will be brought to SPB
• The KARABO version used is several versions back with respect to the one in the instruments and there are still a few minor problems
• On Monday work on programming the laser burst patterns together with the respective people at DESY shall start
• Continue testing
• There appears to be a shortage of conversion crystals for the instruments. New ones have been ordered but are not there yet.

Detectors (P. Lang)

• AGIPD HW work is on track and finishes thie week. Software work starts next week
• LPD control system works ok with KARABO 2.1.18. 
• Some small issues with LPD calibrations

ITDM (K. Wrona)

• Two updates were done for the DAQ of LPD
• Some issues were found with the RC
• It was tried to install a new version. This was already done for AGIPD

AE (T. Freyermuth, B. Baranasic, J. Tolkiehn)

• SASE1 Tunnel:
  • Vacuum loop was updated without problems
  • The motion loop was updated a few times because there were problems with some motors. The work continues today
  • EPS loop was updated
  • On Monday/Tuesday the SPB loop will be updated to account for the terminal changes from 24V -> 48V
• SASE1 Instruments:
  • Loop 5 with the redundancy module is now ok
  • Loops 2, 4, 7 were updated
  • Next come loops 1 and 12
  • All modified modules (24V -> 48V) are installed but a few more might need to be changed
• SASE3 Tunnel:
  • Tried to get the vacuum loop up
  • But there are still many mistakes and inconsistencies with names
  • A big amount of work is still needed for the gas attenuator

CAS (S. Brockhauser)

• KARABO 2.1.18 was deployed
• Good progress was made with the Gotthard and GreatEyes Detectors
• For GreatEyes a specific driver on a specific computer will be used
• The problem with extra delays in the GUI client was resolved
• Scantool:
  • A plan of what to implement exists. This needs to be checked with the instruments
  • A first part of this tool is actually already implemented
  • A. Galler points out that he sees the problems predominantly in the integration with the rest of the environment
• DOOCS2KARABO
  • An example for pipelined data was sent to DESY
  • There is no showstopper seen, but the work is progressing according to priorities which are 1 Laser, 2 Undulators, 3 DOOCS2KARABO
  • Diagnostics points out that the will need both the scantool and DOOCS2KARABO for their commissioning period from 1 - 7 Nov. CAS should please provide information so that the tools can be tried

Dear all,

I updated the schedule for the tunnel access in the upcoming weeks based on the discussions in the PSO- and the Operation Board-meetings:

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

General

• Closing of XTD2/4/9/10 foreseen for Oct 23. Possibly XTD9/10 can be re-opened afterwards and stay open until 28 October

Instruments

• SPB:
  • A lot of cabling work was done. Cable drag chains were installed for AGIPD to move it to another posiiton
  • Roughing pipes were installed
  • Update of KARABO to 2.1.18 was not as smooth as expected. Currently it is not operational. An operational control system is needed soon
  • M. Messerschmidt sent another message with problems early today
• FXE
  • Final testing ongoing of Gotthardt with KARABO
  • Cable management and cable re-routing was done last week

Tunnels

• V0 valve was replaced
• The first vacuum section down stream from V0 was successfully closed and leak tested
• DESY Vacuum group is currently working on the respective section upstream of V0
• Welders are here today to work on the two pipes with leaks. They spent all morning on safety trainings
• The gas attenuator was leak tested on the "good" side (where vacuum section is currently ok/no leaks)
• Pumping has started on the diff pumping section by Vac
• Many small things (mostly cabling and electronics)_ remain to be done

Diagnostics

• Group was not represented by some infos from J. Tolkiehn
• More tests are needed for the Diff Pumping of the Diagnostics group.
• Diff Pumping and PES require more crates to be connected

AE

• SPB loop 5 was cycled.
• The issues with the turbo pumps is being worked on today (Pumps do not turn on after loop is cycled)
• Work is ongoing on the MKS gauge controller
• In the SASE1 tunnel the motion loop will be updated tomorrow and the vacuum loop on Wednesday 
• At this time the SPB loop in XTD9 will also be cycled
• FXE is asked to provide information on what works need to be done
• SASE3 Tunnel:
  • Interlock definitions arre prepared but need to be implemented
  • Many local tests still ongoing
  • There are still problems with inconsistent/wrong device names
  • Two motions loops (mov1/mov2) mov1 contains the most important components upstream
  • EPS not expected for some time

ITDM

• GPFS upgrade today
• redundancy data link might be done today
• Tomorrow: DAQ SW upgrade for LPD (Is still at KARABO 2.1.16.4)

CAS

• Long discussion happened on what the word "mandatory" means in conjunction with the most recent KARABO release
• CAS will look into the specific problems in SPB and FXE that have been brought up
• H. Santos discussed the scan tool with S. Brockhauser on Saturday

Dear Tobias,

Just to clarify that the last information that we got from SPB was the message below. From this information and the technical evaluation that we did the issue is not strong enough to classify Karabo 2.1.18 as "Currently it is not operational. An operational control system is needed soon". This statement is too strong and does not contain technical fundamental. 

Anyway, we already mentioned in the meeting that since Friday we have been working to solve the problem and we already have alternatives. We will publish a patch (Karabo 2.1.18.x) today or tomorrow.

Finally, I talked offline with Adrian and we agreed about some topics. Especially to review the "mandatory" word in the release process.

Hugo

From: "Marc Messerschmidt" <marc.messerschmidt@desy.de>
To: "Hugo Santos" <hugo.santos@xfel.eu>, "chen xu" <chen.xu@xfel.eu>, "cyril danilevski" <cyril.danilevski@xfel.eu>, "Marc Messerschmidt" <marc.messerschmidt@xfel.eu>, "Jolanta Sztuk-Dambietz" <jolanta.sztuk@xfel.eu>, "Britta Weinhausen" <britta.weinhausen@xfel.eu>, "Alessandro Silenzi" <alessandro.silenzi@xfel.eu>
Cc: "John Wiggins" <john.wiggins@xfel.eu>
Sent: Friday, 13 October, 2017 20:47:41
Subject: AW: GUI Responsiveness in SPB

I saw some issues today as well when instantiating devices, but nothing as bad as yesterday. The gui delay also seemed functional and went only up top around 30s.

Marc 

General

• A recent survey has shown that the experiment hall and the end of the tunnels XTD9 and XTDD10 have settled by as much as 6.5mm. If components need to be re-aligned a completely new network of points would need to be established. This is not foreseen before the end of the year. A plot is attached to these minutes

Instruments

• SPB: 
  • Recabling of many components is currently going on
  • AGIPD has been opened and some electronics are being exchanged
  • Preparations are being made to move AGIPD also to a more downstream position
  • Laser table for bringing the PP laser to the sample has been set up
• FXE:
  • Cables in the hutch have been re-routed and cleaned up
  • Preparations for the PP laser are well advanced as the Tangerine laser was already brought to the sample

Laser:

• Work is going on under pressure as the time available until first laser beam is short
• 3 Days were lost due to unannounced shut downs of the AC systems. 
• Both the SPB and the FXE NOPAs are running
• There are problems with triggering the ADC and hence the timing drift cannot be locked. The problem was reproduced in the lab and fixed. But the fix needs to be transferred to the laser hutch
• Tests with the machine timing are still outstanding and it seems that there are some resource issues on the DESY side
• Currently the optical sync system from DESY provides too little power and hence the laser cannot be synchronized by optical X-correlation. For the upcoming run period sync to the RF system will be used. This means that up to ps drifts are to be expected
• The laser is very sensitive to changes in the humidity. This has been traced to the construction of the laser tables which involves wooden side panels. The problem has been discussed with Newport and the laser tables for SASE2/3 will be modified to avoid this

Tunnels:

• Vacuum XTD10/SASE3:
  • There are currently two leaks in welded beam pipes. The welders are coming on Monday to fix this
  • After the leaks are fixed the MCP can be connected to the vacuum
  • And then finally this entire vacuum section can be closed
  • There are several leaks in the dump area. In particular the V0 valve is leaking and one flange. DESY will exchange V0 next week.
  • The pipe that hit in the vacuum incident in September appears not to be leaking
  • The absorber appears to have been damaged in the September incident and needs to be replaced. The SASE2 absorber could be used
  • Vacuum tests were done for the differential pumping downstream of the gas attenuator
  • Work is still being done one the differential pumping of the Diagnostics group
  • Interlock needs to be set up and tested
  • M. Petrich defined the interlock conditions and sent them to AE. This still needs  to be implemented
  • Cabling was done for the gas attenuator
• Diagnostics
  • Inhibits for XGMs in in XTD2/9/10 was implemented
  • Loops were successfully rebooted in XTD9
  • A lot of work done on HIREX (camera exchanged, Gotthard connected to Beckhoff and tested)
  • Today repairs will be done on the MCP in XTD2
  • For both MCPs in SASE1/3 additional Beckhoff terminals will be installed
  • Next week focus will be to finish in XTD10 and get plc control of installed devices
  • All 10 imagers in SASE3 were tested. 3 needs fixes/repairs

AE

• In order to close the loops in SASE3, focus should be given to the most relevant components
• Component lists were circulated. Please check!
• Work on KMONO is complete
• What should happen to the CRL of FXE in XTD2? Statement from A. Galler: Nothing for now as it can be operated for the upcoming user run
• Work on the instruments is ca 50% done
• In SASE3 most local test are done
• It will be tried to get the Vacuum and Motion 1 loop going next week
• The EPS loop is not so critical

ITDM

• The announced GPFS update was postponed several times but will now happen next Monday
• The DAQ for AGIPD was upgraded (The SW used so far was dating back to May)
• The time allocated was 3h. The upgrade was done after 1h, so the remaining 2h were used for tests.
• Longer runs could be taken without major problems
• People should be aware that during the first three days next week, the detectors cannot be operated due to the GPFS upgrade

Software

• Planning is being aligned between CAS (H. Sanchez) and PSPO (M. Areste)
• Progress was made for the laser. The software however will remain at version 2.13 since they do not need DAQ and the scan tool
• Yesterday version 2.18 was released. It is already deployed for SASE1
• Note that this version is not complatible with previous GUI clients
• Next week: Work with AE for SASE3
• There is no news on the scantool as S. Brockhauser was not present
• FXE should be updated to 2.18 today
• The DOOCS/KARABO interface is not considered a priority on the DESY side. They are working on the undulator controls

Tunnels (J. Eidam, J. Gruenert)

• SASE3:
  • ​1st section is not recovered yet, ongoing.
  • Diff. pumping XGM is being installed
  • Leak tests ongoing
  • MCP manipulator XTD2: company got the training - repair ongoing 
  • Absorber was moved ==> has to be realigned by MEA
  • A. Koch is commissioning the imagers. Some bugs were found and will be solved with the new release of Karabo.
  • MCP, FEL-Imager: Crates are there, AE still has to understand and define the priorities of their work
  • No access to the tunnels with visitor cards! (Access to tunnels requires rebreather training, knowledge about procedures, warnings, dosimetry etc.)
  • Exchange of gotthard detector in XTD 9 ongoing.
• SASE1:
  • ​XGM Inihbit system will be brought up tomorrow. VAC loop needs to be brought down . Will be notified with a message in the logbook.

Electronics (P. Gessler)

• Responsibles
  • SASE1: Bernard Baranasic
  • SASE3: Jan Tolkiehn
  • Instruments SASE1: Tobias Freyermuth
• Fast valve terminals exchanged today. 
• Documentation by WP74 + Vacuum is not consistent ==> AE will provide an Excel-sheet to be used, which should give a consistent picture of all terminals, connections etc. Ongoing

ITDM (K. Wrona)

• Both online & offline-filesystems have to be updated. Postponed to TOMORROW (16.10.): Online-FS will be down, offline-FS will be worked on continuously, should not be down, but at reduced performance, since update will be server by server.
• DAQ-Upgrade: still running Karabo 2.1.12 on detectors, which requires lots of restarts. UPGRADE: TOMORROW 13==>16 to Karabo v.2.1.16

Software (H. Santos, A. SIlenzi):

• Karabo 2.1.18 being tested. Test to continue for a couple of days before deployment.
• S. Brockhauser will report on the status of the scan tool this Friday.

We would like to take the oportunity of the shutdown period and update the online storage next week on Monday 9th Oct. This will take approx. half a day. During that time the online storage will not be available on the online computing cluster as well as on pclayer servers used for DAQ. In the same way, the nfs export for data logger is also affected.

Regarding data collected on the online cluster, although it should not have any influence, but as a precautionary measures, please make sure that all relevant data from all past proposals  are migrated to offline storage, which in practice means requesting migration via metadata catalog web interface. If there are still issues with particular runs not being able to be migrated, please contact ITDM.

regards,

Janusz 

After today's discussion with WP75 group, we decided to postpone the update of the online storage to the next Monday 16th October.

SPB (B. Weinhausen): 

• ​Users took good data
• Problems occured with the DAQ / control system
• Working on diamond @ beamstop
• Preparation for several updates / activities regarding AGIPD

FXE (A. Galler):

• ​User took good data, analysis is ongoing
• Will rework cable routing/connections
• Chiller was moved to the rackroom, redo the connections etc.

Tunnels (M. Dommach, H. Sinn, J. Gruenert)

• SASE3:
  • ​1st section is not recovered yet, DESY Vacuum has to replace the last valve here, works have not started yet
  • 2nd section with K-Mono, which was vented, is now pumped again.
  • Diff. pumping in gas section has been tested, upcoming are interlock tests
  • Gas attenuator still needs a lot of cabling, cooling water and compressed air
  • Leak test starts
  • MCP has to be connected next week
  • Leak @ absorber: has to be reweld by Trinos, or a blind flange has to put right after the absorber ==> should not hinder the 1st lasing attempt.
  • Absorber was moved ==> has to be realigned by MEA
  • A. Koch is ready to commission several imagers as soon as the motion loop is up&running
  • Next Monday AE, WP74, Vacuum etc. will finalize which components will be implemented in the motion loop during the October shutdown
  • MCP, FEL-Imager: Crates are there, AE still has to understand and define the priorities of their work
  • Repair of MCP: Access to XTD required, WP74 to clarify with SRP.
  • No access to the tunnels with visitor cards! (Access to tunnels requires rebreather training, knowledge about procedures, warnings, dosimetry etc.)
  • Hirex was calibrated @ 9keV, is ready to measure the bandwidth, was determined to be 20eV FWHM

Laser (M. Lederer)

• NOPA-Alignment progressing
• Some setbacks due to shutdown activities by MKK, which affected the humidity stability ==> tables bend
• Laser synchronization has to be implemented by WP18, is questionable, the responsible expert is going on vacation next Friday.
• Motors seem to move reliably now
• No process yet to let the users easily change the laser burst pattern via software. Currently, BKR is being informed about our needs, and they "drive" the laser through timing signals from BKR.
• Laser group will operate in a shift pattern:
  • 8-16h morning shift
  • 15-23h late shift

Detectors (M. Kuster)

• AGIPD:
  • Decided not to modify the vacuum
  • Working on internal calibration
  • Working on increase of the bias voltage to improve the performance
  • Electronics are being replaced partly
• LPD:
  • Upgraded the chiller
  • A leak occured due to the increased pressure, since the chiller was put into the rack room ==> pressure reducer is used now
  • Will exchange faulty super-modules
  • Working on the internal calibration
• Planning next beamtime & shift schedule as laser group has setup
• There still exists a risk to collide AGIPD-modules, meeting with the AGIPD-consortium to agree on changes for the next detectors
• Problem with additional frames which had to be taken during data acquisition: pattern upgrade / firmware upgrade required, will not come in October.

Electronics (P. Gessler)

• Responsibles
  • SASE1: Bernard Baranasic
  • SASE3: Jan Tolkiehn
  • Instruments SASE1: Tobias Freyermuth
• Lots of updates are required, could all be done in short succession ==> can the beamline be put in an insensitive state, to allow quick series of updates?
• Documentation by WP74 + Vacuum is not consistent ==> AE will provide an Excel-sheet to be used, which should give a consistent picture of all terminals, connections etc.

ITDM (K. Wrona)

• Both online & offline-filesystems have to be updated. Monday: Online-FS will be down, offline-FS will be worked on continuously, should not be down, but at reduced performance, since update will be server by server.
• How should ITDM communicate this to the user? Suggestion: Setup an email-list for users to subscribe to, so those individuals who actually work on the data get the updates, and not every user.
• Capacity of the offline FS has to be increased
• DAQ-Upgrade: still running Karabo 2.1.12 on detectors, which requires lots of restarts. Will upgrade to .13 first.
• Still there is no proper commissioning of the Gotthard. Testing would be appreciated to improve reliability ==> time required from Detector and Instrument

Software (S. Brockhauser):

• Karabo 2.1.18 to be released today, but not immediately to be installed. To be deployed after PLC upgrades.
• Karabo + DOOCS: A meeting took place with T. Wilksen (responsible for DOOCS). ==> interface for slow data / digits exists, but not for pipelined data / cameras / images
• Highest priority of the DOOCS-Team are the SASE3 undulators, which is not in line with the XFEL priorities (improving the stability / reliability of the SASE1 operation)
• C. Youngman to work on the master equipment list for SASE3
• A. Galler: Is the current situation really bearable for the instruments, can we actually call this upcoming beamtime "user time"? ==> S.Brockhauser: Lots of problems have been solved, new will arise...
• A. Galler: Will we be able to do a simple blade scan through a laser beam and get a plot?
  ==> S. Brockhauser: We are working on this. You can have plots showing motor positions, intensity, and then correlate this.
• H. Sinn: Basically, we still to not have a working scan tool, which is a big drawback. This is one of the very basic functionality every user will expect!

Priorities & Schedule (G. Wellenreuther, H.Sinn):

• Operation board should decide about updated schedule next week
• Users & instruments request time to with beam to set up before user time!
• OCD-contacts for TS were provided (again?)
• In general, interface to OCD should be harmonized for all groups (e.g. always have a fixed cellphone number)
• Activities for the shutdown have been gathered by PSPO, and are being provided to the support groups for crosschecking.
• No major collisions/ bottlenecks appeared apart from the high workload / demand for support by AE and CAS.
• AE and CAS will crosscheck and provide feedback to PSPO regarding resources & priorities if required.

General:

Reminder to all the groups to provide the shutdown task list today afternoon to PSPO. Find template attached.

SPB:

• ​In general was a succesful weekend. On Sunday there were some problems with control and DAQ already reported to CAS.
• To check if there were additional problems on Monday and report to CAS if necessary.
• To provide the shutdown task list to PSPO.

FXE :

• ​Was not present, so nothing was reported.

Tunnels:

• Nothing particular.
• List with the shutdown tasks to be provided to PSPO.

Laser:

• Motors were not working, this issue was partially solved in a couple of motors. Detected that the problem was a configuration issue in the PLC. WP-78 will test the solution in other motors of the same kind.
• Laser was running during thre weekend, data will be reviewed today to analyse laser behaviour.

Detectors:

• To investigate a problem with AGIPD, this will require the support from CAS.

Electronics

• Preparing for Oct. shutdown.

ITDM

• DAQ problems on Monday and Tuesday due to detector configuration, still working to find out the causes of the problem.

Software

• Working in the problems reported during the weekend.

30.09. from 08:00 until 01.10. 08:00 - Andreas Koch 9-6443

01.10. from 08:00 until 02.10. 08:00 - Joakim Laksman 9-5630  

The SASE1/SASE3 Readiness meeting next Monday is cancelled we will meet again on Wednesday the 3rd of October.

Correction: We will meet again on Wednesday the 4th of October.

SPB (A. Mancuso): 

• ​Users are in good shape
• Beam operation with ca. 600 microJ, but beam seems to be ring-shaped after M2?
• Preparing for first pump&probe experiments in November.
• Start of the focussing mirror controls should start after the user run

FXE ():

• ​Was not present at the beginning of the meeting, so nothing was reported.

Tunnels (M. Dommach):

• SASE1:
  • Nothing particular.
• SASE3:
  • ​Preparing for the works in XTD10 during the Oct. shutdown are ongoing.
  • PES: Will just be connected to vacuum & pumped, not functional.

Laser (M. Lederer)

• Sustained small damage during set-up
• Not much time left until user run requires PP-laser
• Issue regarding motors not coming back to their position reliably still persists / is not solved. ==> To be tackled by Gabriele (CAS) + Nerea / Nicola / Janna (AE)

Detectors (P. Lang)

• Detectors are in operation
• Works in Oct. shutdown mostly indepenent from other groups
• LPD & Gotthard run together, but Gotthard has not yet detected X-rays?

Electronics (P. Gessler)

• Preparing for Oct. shutdown
• Train-builder should be tested in the Winter shutdown, because the contract only runs until March.

ITDM (K. Wrona)

•  There are some issues with run-control, being investigated.

Software (S. Brockhauser):

• Daily meetings with FXE & SPB.
• C. Youngman is working on the Master Equipment List for SASE3. The basic assumption is to copy any configurations/parameters from SASE1 to SASE3.
• Focussing now more on the SASE1-issues (e.g. Karabo-DOOCS-Interface ==> will mirror devices from Karabo into DOOCS.)

Priorities & Schedule (T. Tschentscher, G. Wellenreuther):

• Performance of SASE1 has to be improved, in particular the performance of the accelerator. This has higher priority than continuing building up SASE3 or SASE2.
• For example, the measurement of gain curves requires the MCP, consequently its completion has highest priority.
• The control room also needs fast & direct access to more devices than just the XGM to properly tune lasing, consequently the Karabo-DOOCS interface has highest priority.
• For the overall planning of the next ca. 5 weeks all groups are expected to report back to T. Tschentscher & PSPO until Wednesday next week. PSPO will produce a Gantt-chart of all activities. Template for the feedback will be sent around still today.
• PSPO is tracking ALL major activities associated in all SASEs in the presentation shown. Missing items to be reported to G. Wellenreuther. Will be revisited on a ca. monthly timescale
• Planning for the next shutdown in December should commence in early November (= after the upcoming shutdown), to allow for proper planning and preparations.
• Despite the priority on SASE1, activities in SASE3 and SASE2 should progress. We still would like to attempt to do first lasing in SASE3 before the November user run!

General

• T. Tschentscher inform about Management Board priorities for the October shut down:
  • Management Board Priority 1 after the Oct shutdown will be the characterization for SASE1 performance. Any still necessary development for this (MCP, HIrex operation; two-way functional DOOCS-Karabo interface and data exchange; etc.) should get highest priority during the shutdown period. It would be good to achieve substantial progress with the DOOCS-Karabo interface before the MAC meeting to avoid general bashing of our controls. The instruments will provide a detailed list to permit support groups (ITDM, CAS, AE) to plan resources.
  • Priority 2: SASE3 completion. It is expected that the machine will anyway be able to reach lasing at SASE3, but this has lower priority at present.

SASE1 Instruments

• ​SPB : 
  • No major issues to report
• FXE:
  • In the next days FXE will work in the commissioning of Gothard & LPD in parallel operation to be ready to test with beam on Saturday.

Electronics

• MPS Interlock for SPB already switched on.
• No further issues to report.

ITDM

• Users have no access to on-line DAQ. They can use off-line cluster, the transfer is 5 min. after the end of the run.
• Out of the shift scientist should work with off-line cluster.

Safety

• There is  a new procedure for tunnel access. Accelerator areas will have to be measured by Safety group before tunnel access release.

General

• Beam was lost due to incident with electron beam hitting & damaging components downstream of the northern dump.
• Goal is to stablish proper conditions for the beam again, last week seemed much better. On Wednesday this week it is expected to have stable beam conditions and sufficient pulse energy for next experiments. 

SASE1 Instruments

• ​SPB : 
  • ​Several persistant problems with Karabo that affect DAQ and Detector. Detector failed on Saturday triggered by the Karabo issues, it took long to recover since a complete power cycle of the Detector was required (30 minutes).
  • CAS is working to fix the problems.

Tunnels

• SASE1:
  • Beam shutter closed during FXE shift, the reasons are not clear. M. Dommach and L. Samoylova to investigate to solve this issue.

Electronics

• MPS Interlock for SPB tested but not switched on. Agreed to switch on this Tuesday.

ITDM

• ITDM report that FXE experienced some problems with the DAQ but they did not inform ITDM . The Experiment team decided to continue and try to figure out by themselves. ITDM and FXE to discuss how to improve the communication for next time.
• Next experiments require Gotthard & LPD in parallel operation ==> to merge Gotthard & LPD to same run control ITDM needs some time to test. Agreed to carry out the tests Tuesday and Wednesday this week, also fine for SPB.

General

To stablish proper conditions for the beam as it was before. On Wednesday to have the mirrors in the correct configuration and machine back to the old conditions at the end of the week.

SASE 1 Instruments

• ​SPB (M. Messerschmidts): 
  • Reported several persistant problems with Karabo that affectd the DAQ and Detector. On Saturday detector falled over and took long time to recover. CAS wokring to fix this issue.
• FXE:
  •  

Tunnels (M. Dommach):

• SASE1:
  • Beam shutters closed during FXE shift. It is not clear the reasons why this happened, to investigate.

Electronics (P. Gessler)

• MPS Interlock tested but not swichted on. SPB agreed to switch on the MPS tomorrow.

ITDM (K. Wrona)

• FXE was runnig  experiments with problems in DAQ, the experiment this was not reported to ITDM
• Next experiments require Gotthard & LPD in parallel operation ==> to merge Gotthard & LPD to same run control ITDM needs some time to test, agreed to perform these tests Tuesday and Wednesday this week.

Software

• FXE:
  • Still daily meetings, prepare LPD & Gotthard
  • Working on GreatEyes, too
• SPB
  • Configure slow-data cameras
  • Laser project is running in CAS
• SASE3:
  • C. Youngman is the CAS-responsible, should collect requirements to compile master equipment list for SASE3 ==> to be compared against MEL SASE1 to identify additional work required.
• SASE1:
  • Loops & Karabo will be updated together.

General

• First users have done experiments - despite the current problems it appears as if everybody was able to take some data for future publications.

Short Report from SASE1 Instruments

• ​SPB (A. Mancuso): 
  • ​Beam was lost due to incident with electron beam hitting & damaging components downstream of the northern dump.
  • Currently only single bunch operation
  • Lots of work to be done in Oct. shutdown, Adrian will provide a list to Thomas Tschentscher
• FXE (T. Tschentscher):
  • ​Problem with the Tangerine-laser affects PP-experiment ==> to be fixed on Tuesday
  • Consider to switch to oscillator only until then ==> lower repetition rate, lower OL pulse energy
  • Next experiments require Gotthard & LPD in parallel operation ==> merge Gotthard & LPD to same run control next week before user run.
  • Today, test with DAQ and Gotthard was successful, but Gotthard has never seen XFEL X-rays so far. ==> Test with beam could happen potentially Monday night (instead of user experiment)

Tunnels (M. Dommach, J. Grünert):

• SASE1:
  • Few issues with XGM, have been or are being solved.
  • It was suggested to put the Hirex-grating into the beam while FXE is taking beam to measure / commission parasitically. But it appears next week neither Naresh nor Jan will be available.
• SASE3:
  • ​Due to the electron beam incident the first vacuum sections have been vented, there is most probably a hole in the V0, but only Rad. Safety to access the tunnels right now. ZZ/Access by anyone else only after clearance through SRP - working there is probably only possible in Octorb
  • Beam did hit the beam pipe apparently just upstream of the transmissive imager. Imager and further (vacuum) electronics seems to have been damaged in the upstream part of XTD10.
  • Due to this incident, it should be reconsidered if dosimeters are required in XTD10, too. SRP did unfortunately not participate, and thus was not able to comment.

Laser

• Working still on improving the laser
• Issue regarding motors coming back to their position is most probably a firmware/options issue

Detectors

• AGIPD:
  • ​According to Yolanta "operates better than expected"
• LPD
  • ​Also operates better than expected
  • Configuration of LPD is getting messed up by people working in parallel / unclear status of the detectors Gotthard & LPD ==> There should be a responsible for the detector, who is present in the control room and can allow remote access to work with the detectors / run configuration. ==> These steps should be well-documented in the FXE-Logbook.
  • K. Wrona: Next week will be better with just one run controller.

Electronics (P. Gessler)

• There is an issue with some motors & encoder @ SPB.
• AE will finish the work on the last loops in SASE1.
• WP79 needed & got support by AE.

ITDM (K. Wrona)

• DAQ for the tunnels is setup. ==> XGM-run control for the tunnel is to be tried.
• Slow data: It is an issue how and how fast to store slow data (lots of properties which do not change).

Software

• FXE:
  • Still daily meetings, prepare LPD & Gotthard
  • Working on GreatEyes, too
• SPB
  • Configure slow-data cameras
  • Laser project is running in CAS
• SASE3:
  • C. Youngman is the CAS-responsible, should collect requirements to compile master equipment list for SASE3 ==> to be compared against MEL SASE1 to identify additional work required.
• SASE1:
  • Loops & Karabo will be updated together.

October Shutdown:

• SASE1
  • No major items here currently, focus is on SASE3 / XTD10.
• SASE3
  • First priority is to get all items required for first lasing up & running ==> Vacuum system including anything up to the absorber.
  • CAS is starting to set up Karabo and all controls.
  • ITDM will also start to work, but does not require access to the tunnel.
  • C. Youngman to generate MEL from input from the groups ==> invites to a meeting if necessary.
  • Eplan for PES will not be available on time ==> has to be pumped in a provisional fashion.

General

• First user experiments ended on Monday evening/Tuesday morning
• They are considered a great success and overall the instruments worked better than expected
• The S1S/S3R meeting on Friday happens at 13:00 this week due to SAC

SPB

• Karabo is freezing up much less frequently than previously
• The latency of data being available for online analysis seems to be increasing
• Todos:
  • Update the laser loop (11) today
• Please keep in mind that updating the PLC loops requires a lot of hand work from the instrument afterwards and should not be done during beam time

FXE

• FXE was not represented
• The parallel operation of Gotthard and LPD was discussed. This is currently not possible. But this restriction was put deliberatly in order not to compromise LPD operation. It can be lifted whenever requested. However, the incoming user group will only need LPD. Hence this should be postponed for one week

AE

• Preparations are ongoing for work on SASE3 tunnel during October shutdown
• Prepartions are on done to connect the EPS in SPB to the DESY MPS system

CAS

• The second stage of the SPB-CAS and SPB-FXE projects has been started
• They include additional functionality that is required for user operation in November and also for the operation with the optical laser
• The next Karabo release is being prepared
• Next thing on the list is to implement the slow data acquisition for the tunnels

Dear all,

XTD9 will be accessible tomorrow until the afternoon, but access should be communicated to M. Dommach, H. Sinn and T. Tschentscher. Martin will organize a search team for the afternoon.

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

General

• User run is currently going well. Thanks to everybody!
• Preparations need to be made for the October shutdown. Information will be collected by PSPO and T. Tschentscher. Care must be taken that the relevant people are available. Priority will be given to XTD10!

Short Report from SASE1 Instruments

• ​SPB: 
  • ​First user shift last night
  • No hickup from accelerator
  • Focus is still too large (12 - 15 um) and beam pointing and beam position are unstable
  • Calibrations were done with AGIPD using powder. 
  • DAQ ran stably
• FXE:
  • ​No report as FXE is currently on shift

Tunnels

• Optics:
  • ​Reminder: The last two metal valves in the FXE and SPB beam lines are currently not in the interlock. Hence the beam could hit them if no attention is paid. This has been fixed in the firmware but will only become active after the next reboot. This will not happen before October
  • Switching between instruments is done using a macro by Harald. It worked without hickups so far. Nevertheless, one should attenuate the beam before moving the mirrors
  • It is important to put relevant info into the operations log book. In particular since all the user log books are closed
• Diagnostics:
  • ​XGM data is now available both via DOOCS and Karabo and can be put into the DAQ stream. However, this data needs to be included. This has not been done so far
  • The XGM data should also be stored for the purpose of optimizing the facility. However, if stored with the user data, access will be difficult
  • Parallel storing of XGM and user data is not possilble currently

Laser

• Work is still progressing in line with laser delivery in November. However, the time is getting tight and overtime requests have been filed for the whole group
• Drifts of the laser beam are being observed that do not correlate with temperature and humidity. A feed back is being worked on
• A fix has been prepared by CAS for the motor problem reported earlier. However, due to the high work load from user operation this could not be tested yet. It is scheduled for next week
• DOOCSGATE was updated to allow adjustment of synchronisation

Detectors

• AGIPD:
  • ​DAQ ran stably 
  • Users took calibration data last night that allows to calibrate the different gain stages
  • A. Mancuso showed nice data on his laptop with very clean rings from the glas in the beam
• LPD
  • ​Not used yesterday
  • Should take data today
  • The calibration pipeline on the Maxwell Cluster is now enabled allowing calibrations to be calculated for both detectors

Electronics

• All instruments loops were updated before the user experiments for DAQ and for failsafe operation with train id injection
• Additional signals were prepared for the MPS system (Vacuum ok, end switches, pulse picker beam pattern). This is not yet integrated
• ZZ happened last Tuesday in XTD2 for CRL and MCP. On CRL significant problems with the cabling were identified (wrong connections, sparks in connectors). The sparks are probably the reason for the broken Beckhoff terminals. Fix can only be done in October
• In October the remaining tunnel loops with also be updated

Software

• 90% of personnel is currently working supporting the user operation in SASE1
• A hot fix for the data logger was deployed. 
• Instrument user shifts will be supported by 2 CAS people at least until end of next week. 
• Remaining personnel will work on the backlog, e. g. fixing the motor problems in the laser
• FXE:
  • DAQ for Gotthardt was deployed yesterday
  • Scantool was updated
  • Work continues on the GreatEyes detector. Currently it only works in Win7 and not in LINUX
• SPB
  • Doocs bridge is operational and slow data can be recorded, e. g. e-beam energy, K param
  • Users also made first use of the ZeroMQ tool