Group Selection Page HELP DOC Controls Group Data Analysis Group ITDM WP76 DET Vacuum XRO EEE WP78 WP74 FXE SCS SPB MID HED SQS SXP Sample Environment Photon Commissioning Team Operation PSPO XO TS
Technical integration and coordination SASE1 Status and SASE3 Readiness
  Joint Operation and Readiness Logbook, Contact: T.Haas, A. Violante, G. Wellenreuther, Page 10 of 30  Not logged in ELOG logo
New entries since:Thu Jan 1 01:00:00 1970
ID Date Author Location Severity Subject
  417   09 Nov 2017, 13:21 Tobias HaasGeneralInformationStatus after 0930 meeting

General

  • There is currently no beam less than 24h before user operation is supposed to start in SPB
  • The affects everybody (Instruments, Detector, Laser 
  416   06 Nov 2017, 14:13 Tobias HaasGeneralInformationStatus after 0930 meeting

General

  • Recomissioning of the FEL has been bumpy and up to this point only very weak lasing could be achieved. 
  • So far SPB had only little beam and FXE had no beam at all. Both instruments state that the commissioning time for the up coming user run is insufficient

Instruments

  • FXE:
    • First SW and DAQ stress test on Friday went well.
    • Some problems appeared with the slow data that should be fixed by now
    • K. Wrona states that these problem originated from missing permissions due to change of userid
  • SPB 
    • Entire instruments needed re-alignment with beam since the CSS was moved during the shutdown
    • Very low intensity and not very stable beams were available on the Weekend. The beam could not be properly focussed
    • The bender for M2 cannot reach the flat condition. Is this the temperature problem again?
    • One pump was overheating
    • AGIPD tests with new bunch patterns started this morning
    • Significantly more time needed before user operation
    • There were some problems with the interlock system that trigger beam aborts

Tunnel

  • Commissioning program was very limited in the last few days
  • KMONO got some first interesting results
    • Rings and Kicks could be seen
    • Some wrong undulator settings were identified

Detectors

  • Firmware update was done to record all 30 bunches. However this still has problems
  • Need beam for calibrations
  • The planned update by ITDM of the DAQ was not yet done

Advanced Electronics

  • The mentioned interlock issues in SPB require an update to loop 7
  • In loop 4 need to reset the "all valves open" signal by hand if it trips
  • The SPB Master loop needs updating both for MPS and timing

Software

  • Tests of the scantool took place with hardware. These tests continue today
  • A new data model is being worked on for the scan tool but this is not available yet
  • SPB states that at this point they will not make use of the scan tool as they first have to consolidate the instrument
  • FXE is involved in the CAS tests

ITDM

  • As mentioned before the DAQ version is still old and needs an update. This contains many improvements
  • IBM discovered a bug in GPFS and published a patch. The bug could lead to data corruption
  • Decision: DAQ update and GPFS patch will be installed today in the morning.

 

  415   06 Nov 2017, 13:58 Gerd WellenreutherGeneralInformationTunnel access schedule updated

Dear all,

I updated the TAS to include the upcoming user runs up to and including the winter shutdown. As you can see, there will only be access to the XTDs in cases of urgency. In these cases, please contact Martin Dommach, Harald Sinn or Thomas Tschentscher.

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules&listViewLinkBack=true

Cheers, Gerd

  414   04 Nov 2017, 03:28 Jan GrünertGeneralInformationWP74 OCD plan until next shutdown
Attachment 1: WP74_OCD_Plan.xlsx
  413   04 Nov 2017, 03:26 Jan GrünertGeneralInformationWP74 OCD contacts for this weekend

4.11. from 08:00 until 5.11. 08:00 - Jan Gruenert by email (work mobile defect)

5.11. from 08:00 until 6.11. 08:00 - Naresh Kujala 9-6840 

  412   03 Nov 2017, 15:06 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

General

  • The rack layouts and rack placement were finalized during the PRR in August. The result is the basis of the network planning. The network is currently being installed and will most likely be finished by the end of the year. Changes to the layout that affect the network planning are no longer possible. 
  • The TC/XHEXP1 meetings shall be combined into one bi-weekly meeting starting on 9 Nov. A separater new invitation shall be sent

Instruments

  • SPB
    • Currently preparing for the next user period
    • A lot of work still needs to be done for bringing the laser to the interaction point. This mostly concerns the beam pipe
    • Apparently there was an interlock problem related to SPB components in XTD9. SPB components were taken out of the EPS loop without SPB being informed
      • Comment from T. Freyermuth: The condition of SPB devices was preventing beam operation and noone of SPB could be reached
  • FXE
    • A lot of progess was made together with CAS
    • This afternoon a DAQ stress test with several detectors is planned
    • The encoders on the CRL lens stack were disabled as this complicates the operation at the moment

Tunnels

  • XTD9 was closed yesterday. Beam could arrive this weekend
  • About one shift worth of work remains in XTD10 before readiness for lasing
  • KMONO measurements took place on Wed/Thurs from 3PM - 3AM
  • Some DAQ and KARABO issues surfaced
  • But it was possible to record donut radiation patterns
  • Now commissioning of HIREX is being prepared
  • XGM requires some minor re-commissioning

Laser

  • Beam is in the ILHs. 
  • Still problems dealing with the capricious AC system
  • The promised specs of the laser beam are met
  • Synchronization is still to local timing. Synchronization to the machine timing only comes next week and no time remains for testing before users

Detectors

  • LPD:
    • 15/16 super modules are operational
    • This week calibration data from charge injection were taken
    • The detector is ready for users
  • AGIPD:
    • Detector is fully working again after a big rearrangement of cables
    • It was found the the clocking-control cable is too long and needs to be re-routed during the winter shutdown. The detector will be fine for the next users
    • Work is ongoing on the firmware in order to allow 30 frames/train rather than only 15

Electronics

  • SPB:
    • Updated the tunnel loop for SPB
    • Loop 12 is up-to-date
    • Loop4: The mirror mounts were tested locally
    • The is no feed back on interlock behaviour in loop 5. If there should be problems individual conditions can be disabled
    • For loop 7 zoom and focus and mirror mounts to be tested
    • Same applies to loop 2
  • XTD9:
    • want to update EPS loop to include removed SPB components today
    • Vacuum loop to be updated next week
  • XTD10
    • Some ion pumps show an error condition in KARABO. J. Tolkiehn is investigating

ITDM

  • User handling for log books has changed. Please consult the help log book for details or consult J. Szuba
  • The meta data catalogue was updated so that instruments can now annotate with their own credentials
  • Folders for the next two user proposals are created, also in the online environment
  • A new version of the DAQ software needs to be deployed in order to fix a bug in slow data daq

CAS

  • A number of issues need to be resolved with Diagnostics as already mentioned earlier
  • The Gotthardt Detector is now working in FXE
  • The promised reduced scope for the scantool has been achieved. It will be made available to the users asap, most likely today
  411   02 Nov 2017, 10:09 Janusz SzubaGeneralInformationAutomatic logbooks for beamtimes

Hi

Logbooks for the first two upcoming proposals in FXE and SPB instruments have been set up and all missing users in the past 6 proposal have been corrected. From now on, user registration is disabled, logbooks for next proposal will be set up automatically once the proposal info is migrated from UPEX to MdCnix. Any subsequent changes (like a new user added in MdCnix to a given proposal) has to be done on request either by admin of particular logbook or ITDM.

Standard setup of beamtime logbooks is the following

  • Admin list contains ITDM staff, instrument support group (as given by relevant instruments), PI and Main proposer as given in UPEX
  • User access is given ITDM staff, instrument support group and all participants of a given beamtime.

All subsequent customisation, including adding extra users, should be made by logbook admins.

Regarding logbooks for the past proposals in September, not all was corrected according to the above defaults, especially there may be wrong email addresses put at that time by users during registration -this can be fixed as well, taking info from UPEX.

Also HELP logbook was updated.

This change affects also XFEL internal usage of ELOG, as from now on, a new user has to be added by a relevant admin.

Cheers, 

Janusz
 

  410   01 Nov 2017, 11:24 Adriano ViolanteXHEXPInformationStatus after 9.30 dispatch meeting

Instruments

  • SPB:
    • When PLC loops are restarted, still many device parameters are not properly recovered. This generates a lot of work for the team. Recently an entire day was lost.
      • The procedure now foresees a notice to the relevant group before each loop update. In particular, CAS will be asked for the last version of the parameters export to be re-implemented. The relative script was also updated and some bugs were fixed.
    • Loop 4 (maybe also 3) - MKB system - needs to be updated. This will be done as soon as the last eplan export is available.  
    • XTD9 loop needs to be updated with proper encoder names. CAS takes care of the missing names.
    • AGIPD Interlock system (loop 5) shows some issues concerning valves. It is not clear whether this is due to real problems with the interlock definitions or just to the fact that the system is not yet in vacuum. Chember will be pumped today and then the final test will be done.
  • FXE:
    • Preparation for the users run ongoing. PP laser treatment test ongoing.

Tunnels

  • XTD10/SASE3
  • XTD9/SASE1
    • Si-Mono: work has been done during the week end. The idea is to have it running without tilt stages.

Laser

  • Delay control is available through KARABO but requires ca 2 days testing: ongoing.

Electronics

  • Currently the work that is still needed for SASE3 is being evaluated
  • Planning for the next shut down is going on

CAS

  • Working on DOOCS/KARABO integration together with DESY. Ongoing. General frame being built. Then priority on the FEL imager needed for first lasing.
  • A. Silenzi and S. Hauff worked on the AGIPD procedures: Cooling down procedure successfully tested.
  • Work continues on reading and writing data for the GreatEyes detector: The idea is now to test the parallel operation of LPD, Gotthard and Greateyes detector.

ITDM

  • The folders for the next user run are being set up next week. GPFS is now fixed.
  • SPB requested 18 new cameras. This effort needs to be carefully coordinated. CAS and ITDM are working on that.
  409   27 Oct 2017, 11:38 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

Instruments

  • SPB:
    • When PLC loops are restarted, still many device parameters are not properly recovered. This generates a lot of work for the team. Recently an entire day was lost.
      • Answer from AE: The parameters are extracted by CAS using KARABO. It is possible that not all necessary values are returned. AE is also investigating whether there are some bugs in the firmware procedure. It is pointed out that the change from 24V to 48V led to a huge amount of changes. 
    • Injector motion is now working along all three axes
    • AGIPD automatic power up procedure is now working. Tests continue
  • FXE:
    • The spurious data losses with the Gotthard Detector were further investigated. They are not resolved but they do not limit operation with 30 bunches

Tunnels

  • XTD10/SASE3
    • All vacuum sections are connected, locally tested and ready
    • Part of the interlock system is also ok. A global test of the entire integrated system still needs to be performed
    • The biggest challenge was the gas attenuator section with 7 turbo pumps. All pumps were locally test and integrated into the control system. The vacuum is very good (10-9) Now they are put into a  safe state.
    • R. Villanueva would like to be informed in case whenever the interlock in XTD10 is broken so he can inspect the system
    • XGM is connected to the overall vacuum and tested. The interlock is working and it is also connected to DOOCS
    • The two imagers requested for first lasing (SR/FEL) are available
    • Approximately one shift would be needed to take care of a few remaining tasks before first lasing
  • XTD9/SASE1
    • The HiRex is fully operational
    • The Si-Mono needs work but this was de-prioritized as it is not needed for the next users

Laser

  • Full specs were reached for both the SPB and FXE beams. The laser is in the respective instrument laser hutches
  • The laser was covered in order to reduce effects of any remaining air drafts
  • Differential mode compensation is available. Common mode compensation is not
  • Work still continues on the 4 agree-upon bunch patterns. Another meeting is set up for today
  • Delay control is available through KARABO but requires ca 2 days testing

Detectors

  • For AGIPD both  for turning on power and cooling automatic procedures were implemented and tested
  • LPD was moved to Karabo 2.18. 
  • One charge injection run was done and the system operated for 2 days without interruts

Electronics

  • Currently the work that is still needed for SASE3 is being evaluated
  • Planning for the next shut down is going on

CAS

  • Currently planning next OCD period
  • Very good progress was made in SASE3
  • Working on DOOCS/KARABO integration together with DESY
  • Work on the laser is progressing
  • A. Silenzi and S. Hauff worked on the AGIPD procedures
  • Work continues on reading and writing data for the GreatEyes detector
  • As reported by the detector group, the stability of the DAQ is much improved

ITDM

  • A major problem caused the Maxwell cluster to be down for 3 days. IBM Technicians had to be called
  • In this context it became clear that the communication with the users must be optimized
  • Now the system is ok
  • The folders for the next user run are being set up next week. This was delayed due to the technical problems
  • For the experiment log books the users will now be automatically set up
  • The update of the DAQ for AGIPD is somewhat delayed due to the long time taken for the recabling
  • SPB requested 18 new cameras. This effort needs to be carefully coordinated.
  408   26 Oct 2017, 17:34 Tobias HaasXTD10InformationXTD10 closed and interlocked

The SASE3 tunnel XTD10 was searched by Janni and her team and interlocked about one hour ago.

It is now almost ready for first lasing in SASE3:

  • The vacuum is completely connected and closed from V0 all the way to the end
  • All valves, pumps and gauges are included in the PLC loop and can be controlled from Karabo
  • The gas attenuator section was completely tested and is also under PLC control. The turbos are isolated from the rest of the vacuum and put into a safe state
  • All three DPS's are operational, tested and included
  • All imagers are operational
  • The XGM vacuum is part of the general vacuum and controlled and monitored from KARABO
  • The interlocks of the DPSs were succesfully tested

In order to allow beam, it is estimated that one access with approximately 8h would be needed to sort out some remaining smaller issues

I would like to thank the entire team from Vacuum, Optics, Diagnostics, AE and CAS for the big effort and in particular C. Youngman for coordinating.

  407   25 Oct 2017, 15:20 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

General

  • After discussion between XFEL Vacuum (J. Eidam) and DESY XFEL Run Coor (Matthias Scholz) it is agreed that XTD10 will only be closed on Thursday and that XFEL will search the tunnel between 15:00 and 17:00. (This information became available only after the meeting)
  • This gives some extra time to finish activities in XTD10

Instruments

  • FXE
    • Currently A. Parenti and FXE are hunting a spurious data loss in Gotthards
    • Otherwise the software is surprisingly stable
  • SPB
    • Slow progress towards getting the Detector going again
    • The task to ease the acquistion needs help from the DESY developers

Tunnels

  • XTD10:
    • A lot of progress was made by the team working in XTD10: Thank you very much!
    • The complete tunnel is under vacuum
    • Also the missing pieces between XGM/MCP and the rest were inserted 
    • Cleanup is done
    • Denis and Raul are still working on local tests of the gas attenuator system
    • Testing is somewhat slowed down because the GUI client no longer allows active manipulation of components at the moment. So the ikarabo interface is used
    • Work also continues on getting the mov1 loop going
    • Local tests are done on all imgers except on the exit slit imager
  • XTD9
    • The FXE Mono is not finished. It needs a restart of the motion loop
    • AE will check when this is possible, but there are a lot of constraints from users and from resources
    • Since there has been a change in the interlock system, SPB needs to be in place when the shutter to XTD9 is opened. There is a risk of triggering the interlock and shutting down the machine.

AE

  • Almost everything that was planned could be achieved in XTD10
  • Some of the remaining work can be finished when the tunnel is closed.
    • Work is going on to finish the controls
  • Even the interface to MPS that was originally not foreseen is now available

CAS

  • The workaround that allows to modify parameters even though the protection does not allow this, is now in place. This should help with the trouble observed by the Vacuum Group.
  • A new bug in the PLC was observed in this context. Changing a parameter whose name has changed leads to a crash. 

ITDM

  • Currently the MAXWELL cluster is unavailable.  This is a serious software/hardware issued. The manufacturer is actually on site to fix the issue today
  406   23 Oct 2017, 16:54 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

FXE:

  • Pump-probe beam is now in the instrument hutch
  • Currently working on setting up the remainder of beam steering and compression
  • Gotthardt is working. Tests are going on a and some issues with the DAQ still need to be solved
  • The sample stack needs to be moved up stream for the next user period

SPB

  • Still working on AGIPD. Many cables needed to be disconnected and are being re-connected now
  • AE finished all the scheduled modifications on the crates and are doing some more that were originally not scheduled
  • A few details with the interlock need fixing
  • The newest version of Beckhoff_Devices needs to be deployed
  • The previously observed slowness of the GUI is probably fixed but needs more real life exposure. Final test are being done
  • The full infrastructure should be back and operational Tuesday/Wednesday

Tunnels

  • Work is going on on the gas attenuator
  • Interlocks are ready for the WP74 DPS and the XGM
  • Detailed steps to completion in XTD10 were re-iterated by C. Youngman. Additional coordination off line after the meeting
  • L. Samoylova mentioned that the mirror encoders in XTD2 are currently in error. B. Baranasic will have a look

AE

  • In the new version of KARABO, the rights for reading and writing parameters from and to the PLC are now being observed, differently from before.
  • This causes problems. P. Gessler mentions that he is very concerned that such a major change is introduced so briefly before the next user period. This puts AE under great pressure and leaves almost no time for testing
  • All the SASE1 loops were updated because of the new "feature" in KARABO
  • A. Silenzi mentions that he can implement a workaround for this "feature" so that loops need no updating. However, this requires a new KARABO release

CAS

  • KARABO2DOOCS: Meeting today with the DESY control people at 1430 in BKR
  • ScanTool: Meeting foreseen Tuesday/Wednesday with the instruments
  • The GUI slowness necessiated a hotfix which was verified within CAS. Now waiting for feedback from instruments
  405   20 Oct 2017, 15:42 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

General

  • SASE1 Instruments are asked to comment on progress with KARABO 2.1.18
  • Tunnels XTD9/10 will be closed on 25 Oct at 1700

Instruments

  • FXE (A. Galler):
    • KARABO 2.1.18 appears to be working ok. The device instantiation problem reported earlier in the week is no longer there. However, not all equipment could be tested yet
    • Work currently focusses on preparing for the next User Run and in particular on preparing the sample environment
  • SPB (A. Mancuso)
    • A situation similar to the on in FXE is observed. Full proof of functionality of KARABO will only be possible once AGIPD is fully assembled again which is next week at the earliest
    • Currently, work is going on inserting the AGIPD cables into drag chains so that the detector can be moved on the CSS. This work should finish today

Tunnels (J. Eidam, R. Villanueva, T. Maltezopoulos)

  • Leaky vacuum pipes were successfully welded
  • Absorber section is closed and under vacuum
  • MCP detector is connected
  • DESY finished work on the upstream section in the dump
  • The gas attenuator has very good vacuum (10^-9)
  • Differential pumping is ready from the HW side
  • Cabling for the gas attenuator section is still going on
  • Alignment was done in view of the small apertures in the gas section
  • Local tests can start now (This is 3 days behind the schedule made at the beginning of the week)
  • PES is installed but will only act as a pipe.

Laser (M Lederer)

  • NOPA Beam is now going to FXE
  • Next week it will be brought to SPB
  • The KARABO version used is several versions back with respect to the one in the instruments and there are still a few minor problems
  • On Monday work on programming the laser burst patterns together with the respective people at DESY shall start
  • Continue testing
  • There appears to be a shortage of conversion crystals for the instruments. New ones have been ordered but are not there yet.

Detectors (P. Lang)

  • AGIPD HW work is on track and finishes thie week. Software work starts next week
  • LPD control system works ok with KARABO 2.1.18. 
  • Some small issues with LPD calibrations

ITDM (K. Wrona)

  • Two updates were done for the DAQ of LPD
  • Some issues were found with the RC
  • It was tried to install a new version. This was already done for AGIPD

AE (T. Freyermuth, B. Baranasic, J. Tolkiehn)

  • SASE1 Tunnel:
    • Vacuum loop was updated without problems
    • The motion loop was updated a few times because there were problems with some motors. The work continues today
    • EPS loop was updated
    • On Monday/Tuesday the SPB loop will be updated to account for the terminal changes from 24V -> 48V
  • SASE1 Instruments:
    • Loop 5 with the redundancy module is now ok
    • Loops 2, 4, 7 were updated
    • Next come loops 1 and 12
    • All modified modules (24V -> 48V) are installed but a few more might need to be changed
  • SASE3 Tunnel:
    • Tried to get the vacuum loop up
    • But there are still many mistakes and inconsistencies with names
    • A big amount of work is still needed for the gas attenuator

CAS (S. Brockhauser)

  • KARABO 2.1.18 was deployed
  • Good progress was made with the Gotthard and GreatEyes Detectors
  • For GreatEyes a specific driver on a specific computer will be used
  • The problem with extra delays in the GUI client was resolved
  • Scantool:
    • A plan of what to implement exists. This needs to be checked with the instruments
    • A first part of this tool is actually already implemented
    • A. Galler points out that he sees the problems predominantly in the integration with the rest of the environment
  • DOOCS2KARABO
    • An example for pipelined data was sent to DESY
    • There is no showstopper seen, but the work is progressing according to priorities which are 1 Laser, 2 Undulators, 3 DOOCS2KARABO
    • Diagnostics points out that the will need both the scantool and DOOCS2KARABO for their commissioning period from 1 - 7 Nov. CAS should please provide information so that the tools can be tried
  404   16 Oct 2017, 17:11 Gerd WellenreutherGeneralInformationTunnel access schedule updated

Dear all,

I updated the schedule for the tunnel access in the upcoming weeks based on the discussions in the PSO- and the Operation Board-meetings:

https://docs.xfel.eu/share/page/site/PSC/wiki-page?title=Tunnel_access_schedules

Cheers, Gerd

  403   16 Oct 2017, 15:54 Tobias HaasGeneralInformationStatus and Plans after 01:30 meeting

Dear Tobias,

Just to clarify that the last information that we got from SPB was the message below. From this information and the technical evaluation that we did the issue is not strong enough to classify Karabo 2.1.18 as "Currently it is not operational. An operational control system is needed soon". This statement is too strong and does not contain technical fundamental. 

Anyway, we already mentioned in the meeting that since Friday we have been working to solve the problem and we already have alternatives. We will publish a patch (Karabo 2.1.18.x) today or tomorrow.

Finally, I talked offline with Adrian and we agreed about some topics. Especially to review the "mandatory" word in the release process.

Hugo


From: "Marc Messerschmidt" <marc.messerschmidt@desy.de>
To: "Hugo Santos" <hugo.santos@xfel.eu>, "chen xu" <chen.xu@xfel.eu>, "cyril danilevski" <cyril.danilevski@xfel.eu>, "Marc Messerschmidt" <marc.messerschmidt@xfel.eu>, "Jolanta Sztuk-Dambietz" <jolanta.sztuk@xfel.eu>, "Britta Weinhausen" <britta.weinhausen@xfel.eu>, "Alessandro Silenzi" <alessandro.silenzi@xfel.eu>
Cc: "John Wiggins" <john.wiggins@xfel.eu>
Sent: Friday, 13 October, 2017 20:47:41
Subject: AW: GUI Responsiveness in SPB

I saw some issues today as well when instantiating devices, but nothing as bad as yesterday. The gui delay also seemed functional and went only up top around 30s.

Marc 

 

 

Tobias Haas wrote:

General

  • Closing of XTD2/4/9/10 foreseen for Oct 23. Possibly XTD9/10 can be re-opened afterwards and stay open until 28 October

Instruments

  • SPB:
    • A lot of cabling work was done. Cable drag chains were installed for AGIPD to move it to another posiiton
    • Roughing pipes were installed
    • Update of KARABO to 2.1.18 was not as smooth as expected. Currently it is not operational. An operational control system is needed soon
    • M. Messerschmidt sent another message with problems early today
  • FXE
    • Final testing ongoing of Gotthardt with KARABO
    • Cable management and cable re-routing was done last week

Tunnels

  • V0 valve was replaced
  • The first vacuum section down stream from V0 was successfully closed and leak tested
  • DESY Vacuum group is currently working on the respective section upstream of V0
  • Welders are here today to work on the two pipes with leaks. They spent all morning on safety trainings
  • The gas attenuator was leak tested on the "good" side (where vacuum section is currently ok/no leaks)
  • Pumping has started on the diff pumping section by Vac
  • Many small things (mostly cabling and electronics)_ remain to be done

Diagnostics

  • Group was not represented by some infos from J. Tolkiehn
  • More tests are needed for the Diff Pumping of the Diagnostics group.
  • Diff Pumping and PES require more crates to be connected

AE

  • SPB loop 5 was cycled.
  • The issues with the turbo pumps is being worked on today (Pumps do not turn on after loop is cycled)
  • Work is ongoing on the MKS gauge controller
  • In the SASE1 tunnel the motion loop will be updated tomorrow and the vacuum loop on Wednesday 
  • At this time the SPB loop in XTD9 will also be cycled
  • FXE is asked to provide information on what works need to be done
  • SASE3 Tunnel:
    • Interlock definitions arre prepared but need to be implemented
    • Many local tests still ongoing
    • There are still problems with inconsistent/wrong device names
    • Two motions loops (mov1/mov2) mov1 contains the most important components upstream
    • EPS not expected for some time

ITDM

  • GPFS upgrade today
  • redundancy data link might be done today
  • Tomorrow: DAQ SW upgrade for LPD (Is still at KARABO 2.1.16.4)

CAS

  • Long discussion happened on what the word "mandatory" means in conjunction with the most recent KARABO release
  • CAS will look into the specific problems in SPB and FXE that have been brought up
  • H. Santos discussed the scan tool with S. Brockhauser on Saturday

 

  402   16 Oct 2017, 15:22 Tobias HaasGeneralInformationStatus and Plans after 01:30 meeting

General

  • Closing of XTD2/4/9/10 foreseen for Oct 23. Possibly XTD9/10 can be re-opened afterwards and stay open until 28 October

Instruments

  • SPB:
    • A lot of cabling work was done. Cable drag chains were installed for AGIPD to move it to another posiiton
    • Roughing pipes were installed
    • Update of KARABO to 2.1.18 was not as smooth as expected. Currently it is not operational. An operational control system is needed soon
    • M. Messerschmidt sent another message with problems early today
  • FXE
    • Final testing ongoing of Gotthardt with KARABO
    • Cable management and cable re-routing was done last week

Tunnels

  • V0 valve was replaced
  • The first vacuum section down stream from V0 was successfully closed and leak tested
  • DESY Vacuum group is currently working on the respective section upstream of V0
  • Welders are here today to work on the two pipes with leaks. They spent all morning on safety trainings
  • The gas attenuator was leak tested on the "good" side (where vacuum section is currently ok/no leaks)
  • Pumping has started on the diff pumping section by Vac
  • Many small things (mostly cabling and electronics)_ remain to be done

Diagnostics

  • Group was not represented by some infos from J. Tolkiehn
  • More tests are needed for the Diff Pumping of the Diagnostics group.
  • Diff Pumping and PES require more crates to be connected

AE

  • SPB loop 5 was cycled.
  • The issues with the turbo pumps is being worked on today (Pumps do not turn on after loop is cycled)
  • Work is ongoing on the MKS gauge controller
  • In the SASE1 tunnel the motion loop will be updated tomorrow and the vacuum loop on Wednesday 
  • At this time the SPB loop in XTD9 will also be cycled
  • FXE is asked to provide information on what works need to be done
  • SASE3 Tunnel:
    • Interlock definitions arre prepared but need to be implemented
    • Many local tests still ongoing
    • There are still problems with inconsistent/wrong device names
    • Two motions loops (mov1/mov2) mov1 contains the most important components upstream
    • EPS not expected for some time

ITDM

  • GPFS upgrade today
  • redundancy data link might be done today
  • Tomorrow: DAQ SW upgrade for LPD (Is still at KARABO 2.1.16.4)

CAS

  • Long discussion happened on what the word "mandatory" means in conjunction with the most recent KARABO release
  • CAS will look into the specific problems in SPB and FXE that have been brought up
  • H. Santos discussed the scan tool with S. Brockhauser on Saturday
  401   13 Oct 2017, 13:48 Tobias HaasGeneralInformationStatus and plans after SASE1 Status/SASE3 Readiness Meeting

General

  • A recent survey has shown that the experiment hall and the end of the tunnels XTD9 and XTDD10 have settled by as much as 6.5mm. If components need to be re-aligned a completely new network of points would need to be established. This is not foreseen before the end of the year. A plot is attached to these minutes

Instruments

  • SPB: 
    • Recabling of many components is currently going on
    • AGIPD has been opened and some electronics are being exchanged
    • Preparations are being made to move AGIPD also to a more downstream position
    • Laser table for bringing the PP laser to the sample has been set up
  • FXE:
    • Cables in the hutch have been re-routed and cleaned up
    • Preparations for the PP laser are well advanced as the Tangerine laser was already brought to the sample

Laser:

  • Work is going on under pressure as the time available until first laser beam is short
  • 3 Days were lost due to unannounced shut downs of the AC systems. 
  • Both the SPB and the FXE NOPAs are running
  • There are problems with triggering the ADC and hence the timing drift cannot be locked. The problem was reproduced in the lab and fixed. But the fix needs to be transferred to the laser hutch
  • Tests with the machine timing are still outstanding and it seems that there are some resource issues on the DESY side
  • Currently the optical sync system from DESY provides too little power and hence the laser cannot be synchronized by optical X-correlation. For the upcoming run period sync to the RF system will be used. This means that up to ps drifts are to be expected
  • The laser is very sensitive to changes in the humidity. This has been traced to the construction of the laser tables which involves wooden side panels. The problem has been discussed with Newport and the laser tables for SASE2/3 will be modified to avoid this

Tunnels:

  • Vacuum XTD10/SASE3:
    • There are currently two leaks in welded beam pipes. The welders are coming on Monday to fix this
    • After the leaks are fixed the MCP can be connected to the vacuum
    • And then finally this entire vacuum section can be closed
    • There are several leaks in the dump area. In particular the V0 valve is leaking and one flange. DESY will exchange V0 next week.
    • The pipe that hit in the vacuum incident in September appears not to be leaking
    • The absorber appears to have been damaged in the September incident and needs to be replaced. The SASE2 absorber could be used
    • Vacuum tests were done for the differential pumping downstream of the gas attenuator
    • Work is still being done one the differential pumping of the Diagnostics group
    • Interlock needs to be set up and tested
    • M. Petrich defined the interlock conditions and sent them to AE. This still needs  to be implemented
    • Cabling was done for the gas attenuator
  • Diagnostics
    • Inhibits for XGMs in in XTD2/9/10 was implemented
    • Loops were successfully rebooted in XTD9
    • A lot of work done on HIREX (camera exchanged, Gotthard connected to Beckhoff and tested)
    • Today repairs will be done on the MCP in XTD2
    • For both MCPs in SASE1/3 additional Beckhoff terminals will be installed
    • Next week focus will be to finish in XTD10 and get plc control of installed devices
    • All 10 imagers in SASE3 were tested. 3 needs fixes/repairs

AE

  • In order to close the loops in SASE3, focus should be given to the most relevant components
  • Component lists were circulated. Please check!
  • Work on KMONO is complete
  • What should happen to the CRL of FXE in XTD2? Statement from A. Galler: Nothing for now as it can be operated for the upcoming user run
  • Work on the instruments is ca 50% done
  • In SASE3 most local test are done
  • It will be tried to get the Vacuum and Motion 1 loop going next week
  • The EPS loop is not so critical

ITDM

  • The announced GPFS update was postponed several times but will now happen next Monday
  • The DAQ for AGIPD was upgraded (The SW used so far was dating back to May)
  • The time allocated was 3h. The upgrade was done after 1h, so the remaining 2h were used for tests.
  • Longer runs could be taken without major problems
  • People should be aware that during the first three days next week, the detectors cannot be operated due to the GPFS upgrade

Software

  • Planning is being aligned between CAS (H. Sanchez) and PSPO (M. Areste)
  • Progress was made for the laser. The software however will remain at version 2.13 since they do not need DAQ and the scan tool
  • Yesterday version 2.18 was released. It is already deployed for SASE1
  • Note that this version is not complatible with previous GUI clients
  • Next week: Work with AE for SASE3
  • There is no news on the scantool as S. Brockhauser was not present
  • FXE should be updated to 2.18 today
  • The DOOCS/KARABO interface is not considered a priority on the DESY side. They are working on the undulator controls

 

Attachment 1: Zusammenstellung_Nivellement_Oktober_2017.pdf
Zusammenstellung_Nivellement_Oktober_2017.pdf
  400   09 Oct 2017, 14:00 Adriano ViolanteGeneralInformationStatus report after dispatch meeting 13:30

Tunnels (J. Eidam, J. Gruenert)

  • SASE3:
    • ​1st section is not recovered yet, ongoing.
    • Diff. pumping XGM is being installed
    • Leak tests ongoing
    • MCP manipulator XTD2: company got the training - repair ongoing 
    • Absorber was moved ==> has to be realigned by MEA
    • A. Koch is commissioning the imagers. Some bugs were found and will be solved with the new release of Karabo.
    • MCP, FEL-Imager: Crates are there, AE still has to understand and define the priorities of their work
    • No access to the tunnels with visitor cards! (Access to tunnels requires rebreather training, knowledge about procedures, warnings, dosimetry etc.)
    • Exchange of gotthard detector in XTD 9 ongoing.
  • SASE1:
    • ​XGM Inihbit system will be brought up tomorrow. VAC loop needs to be brought down . Will be notified with a message in the logbook.

 

Electronics (P. Gessler)

  • Responsibles
    • SASE1: Bernard Baranasic
    • SASE3: Jan Tolkiehn
    • Instruments SASE1: Tobias Freyermuth
  • Fast valve terminals exchanged today. 
  • Documentation by WP74 + Vacuum is not consistent ==> AE will provide an Excel-sheet to be used, which should give a consistent picture of all terminals, connections etc. Ongoing

ITDM (K. Wrona)

  • Both online & offline-filesystems have to be updated. Postponed to TOMORROW (16.10.): Online-FS will be down, offline-FS will be worked on continuously, should not be down, but at reduced performance, since update will be server by server.
  • DAQ-Upgrade: still running Karabo 2.1.12 on detectors, which requires lots of restarts. UPGRADE: TOMORROW 13==>16 to Karabo v.2.1.16

Software (H. Santos, A. SIlenzi):

  • Karabo 2.1.18 being tested. Test to continue for a couple of days before deployment.
  • S. Brockhauser will report on the status of the scan tool this Friday.

 

  399   06 Oct 2017, 14:42 Janusz SzubaGeneralInformationOnline storage update on Monday 9th Oct

After today's discussion with WP75 group, we decided to postpone the update of the online storage to the next Monday 16th October.

Janusz Szuba wrote:

We would like to take the oportunity of the shutdown period and update the online storage next week on Monday 9th Oct. This will take approx. half a day. During that time the online storage will not be available on the online computing cluster as well as on pclayer servers used for DAQ. In the same way, the nfs export for data logger is also affected.

Regarding data collected on the online cluster, although it should not have any influence, but as a precautionary measures, please make sure that all relevant data from all past proposals  are migrated to offline storage, which in practice means requesting migration via metadata catalog web interface. If there are still issues with particular runs not being able to be migrated, please contact ITDM.

regards,

Janusz 

 

  398   06 Oct 2017, 11:15 Gerd WellenreutherGeneralInformationStatus and plans after Readiness Meeting

SPB (B. Weinhausen): 

  • Users took good data
  • Problems occured with the DAQ / control system
  • Working on diamond @ beamstop
  • Preparation for several updates / activities regarding AGIPD

FXE (A. Galler):

  • User took good data, analysis is ongoing
  • Will rework cable routing/connections
  • Chiller was moved to the rackroom, redo the connections etc.

Tunnels (M. Dommach, H. Sinn, J. Gruenert)

  • SASE3:
    • ​1st section is not recovered yet, DESY Vacuum has to replace the last valve here, works have not started yet
    • 2nd section with K-Mono, which was vented, is now pumped again.
    • Diff. pumping in gas section has been tested, upcoming are interlock tests
    • Gas attenuator still needs a lot of cabling, cooling water and compressed air
    • Leak test starts
    • MCP has to be connected next week
    • Leak @ absorber: has to be reweld by Trinos, or a blind flange has to put right after the absorber ==> should not hinder the 1st lasing attempt.
    • Absorber was moved ==> has to be realigned by MEA
    • A. Koch is ready to commission several imagers as soon as the motion loop is up&running
    • Next Monday AE, WP74, Vacuum etc. will finalize which components will be implemented in the motion loop during the October shutdown
    • MCP, FEL-Imager: Crates are there, AE still has to understand and define the priorities of their work
    • Repair of MCP: Access to XTD required, WP74 to clarify with SRP.
    • No access to the tunnels with visitor cards! (Access to tunnels requires rebreather training, knowledge about procedures, warnings, dosimetry etc.)
    • Hirex was calibrated @ 9keV, is ready to measure the bandwidth, was determined to be 20eV FWHM

Laser (M. Lederer)

  • NOPA-Alignment progressing
  • Some setbacks due to shutdown activities by MKK, which affected the humidity stability ==> tables bend
  • Laser synchronization has to be implemented by WP18, is questionable, the responsible expert is going on vacation next Friday.
  • Motors seem to move reliably now
  • No process yet to let the users easily change the laser burst pattern via software. Currently, BKR is being informed about our needs, and they "drive" the laser through timing signals from BKR.
  • Laser group will operate in a shift pattern:
    • 8-16h morning shift
    • 15-23h late shift

Detectors (M. Kuster)

  • AGIPD:
    • Decided not to modify the vacuum
    • Working on internal calibration
    • Working on increase of the bias voltage to improve the performance
    • Electronics are being replaced partly
  • LPD:
    • Upgraded the chiller
    • A leak occured due to the increased pressure, since the chiller was put into the rack room ==> pressure reducer is used now
    • Will exchange faulty super-modules
    • Working on the internal calibration
  • Planning next beamtime & shift schedule as laser group has setup
  • There still exists a risk to collide AGIPD-modules, meeting with the AGIPD-consortium to agree on changes for the next detectors
  • Problem with additional frames which had to be taken during data acquisition: pattern upgrade / firmware upgrade required, will not come in October.

Electronics (P. Gessler)

  • Responsibles
    • SASE1: Bernard Baranasic
    • SASE3: Jan Tolkiehn
    • Instruments SASE1: Tobias Freyermuth
  • Lots of updates are required, could all be done in short succession ==> can the beamline be put in an insensitive state, to allow quick series of updates?
  • Documentation by WP74 + Vacuum is not consistent ==> AE will provide an Excel-sheet to be used, which should give a consistent picture of all terminals, connections etc.

ITDM (K. Wrona)

  • Both online & offline-filesystems have to be updated. Monday: Online-FS will be down, offline-FS will be worked on continuously, should not be down, but at reduced performance, since update will be server by server.
  • How should ITDM communicate this to the user? Suggestion: Setup an email-list for users to subscribe to, so those individuals who actually work on the data get the updates, and not every user.
  • Capacity of the offline FS has to be increased
  • DAQ-Upgrade: still running Karabo 2.1.12 on detectors, which requires lots of restarts. Will upgrade to .13 first.
  • Still there is no proper commissioning of the Gotthard. Testing would be appreciated to improve reliability ==> time required from Detector and Instrument

Software (S. Brockhauser):

  • Karabo 2.1.18 to be released today, but not immediately to be installed. To be deployed after PLC upgrades.
  • Karabo + DOOCS: A meeting took place with T. Wilksen (responsible for DOOCS). ==> interface for slow data / digits exists, but not for pipelined data / cameras / images
  • Highest priority of the DOOCS-Team are the SASE3 undulators, which is not in line with the XFEL priorities (improving the stability / reliability of the SASE1 operation)
  • C. Youngman to work on the master equipment list for SASE3
  • A. Galler: Is the current situation really bearable for the instruments, can we actually call this upcoming beamtime "user time"? ==> S.Brockhauser: Lots of problems have been solved, new will arise...
  • A. Galler: Will we be able to do a simple blade scan through a laser beam and get a plot?
    ==> S. Brockhauser: We are working on this. You can have plots showing motor positions, intensity, and then correlate this.
  • H. Sinn: Basically, we still to not have a working scan tool, which is a big drawback. This is one of the very basic functionality every user will expect!

Priorities & Schedule (G. Wellenreuther, H.Sinn):

  • Operation board should decide about updated schedule next week
  • Users & instruments request time to with beam to set up before user time!
  • OCD-contacts for TS were provided (again?)
  • In general, interface to OCD should be harmonized for all groups (e.g. always have a fixed cellphone number)
  • Activities for the shutdown have been gathered by PSPO, and are being provided to the support groups for crosschecking.
  • No major collisions/ bottlenecks appeared apart from the high workload / demand for support by AE and CAS.
  • AE and CAS will crosscheck and provide feedback to PSPO regarding resources & priorities if required.
ELOG V3.1.4-7c3fd00