General

• Intensity and availability of beam was good over the weekend
• Large drifts and jitter were observed that are significantly higher than previously
• Beam has two horizontal lobes as if it is too large for the mirrors

Instruments

• SPB:
  • Many tasks were achieved during the weekend, e. g. instrument alignment, detector calibration
  • Issue with encoders whose cables are close to motor cables persists. These encoders are being disabled
  • Rotational SmarACT stages still do not work and are needed for this week's user experiment
• FXE
  • At shift start on Saturday after reloading beam parameters there was no beam in the hutch.
  • Beam had to be brought back by hand. This was a large effort
  • This situation did not repeat on Sunday
  • LPD detector had problems: Only one super module correctly recorded data. Comment from S. Hauf: FXE should have called the detector expert
  • PP laser synchronisation does not work properly yet: Increments when adjusting timing are too coarse. Scan range is limited to 18ns

Detector

• Detector group was not informed about the LPD issues in FXE. 
• AGIPD calibrations were successfully taken on Sunday

ITDM

• Still work is needed for slow data DAQ
• DAQ behaves differently in SPB and FXE. In FXE it appears overloaded and in SPB it works fine
• Comment from CAS: 
  • Many changes were made in the KARABO framework to allow asynchronous operation
  • These changes have to be also pulled through to DAQ. This will only be done during the winter shutdown
  • Until then, DAQ is very vulnerable to any delay in the system

CAS

• No further comments

AE

• Currently working on problems with rotational stages in SPB

General

• User experiments resume tomorrow morning
• Beam should be coming back late this morning or early this afternoon
• After the Operation Coordinaton Meeting H. Sinn, L. Samoylova and the instruments will align the mirrors
• Beam to the instruments should be available directly after this

Instruments

• SPB:
  • The last night with beam on Monday was used to take flat field calibrations with AGIPD
  • The timing for overlap with the optical laser was adjusted
  • 5 out of 6 SmarACT rotational stages are now operations. Some problems remain and AE is talking to the manufacturer
• FXE
  • Request for use of the beam after 9 PM
  • Can be given to SPB starting at 5 AM tomorrow for final setup 

Detectors

• Connection to the ONDA system was set up for the detectors
• Significant latency develops quickly an saturates at approximately 2 mins

ITDM

• Simultaneous use of the DAQ system by both instruments should be done carefullly

AE

• no further comment

CAS

• no further comment

General

• User operation ongoing in SPB/SFX and FXE

Instruments

• FXE:
  • User operation started during the night
  • Some concerns over running LPD as the sample chamber is from diamond and direct reflections could cause damage
  • Generally instrument worked very well
  • Synchronization with PP laser is now in a state where it can be tested
• SPB
  • Many data taken for detector calibration
  • Severe issues with KARABO
    • Response is very slow
    • Major crash this morning
    • Detector controls complex and instable
  • A failure of AGIPD related to FPGA not understood
  • Strange interlock issue: EPS interlocks appeared disabled

Tunnels

• L. Samoylova spent many hours on Wednesday to set up the DAQ in the tunnel
• With the help of many support people she managed to get the system to work
• The system is instable and also seems to record many more parameters than are needed
• Use of the system is very clumsy
• Comment from ITDM (L. Maia): The data sources were specified by the users. It is acknowledged that the system needs a lot more work
• Diagnostics is working with CAS in a squad to work down a list of open redmine tickets
• The DOOCS - KARABO bridge is now working. The FEL imager is routinely available in DOOCS to the BKR crew
• DESY set up a statistics server to help collect data about beam pointing instabilities

Laser

• Local pattern generation synchronized to machine timing is working
• Overlap finding together with FXE was done but has to be tested under real condition next week
• A few suddent shutoffs of the front end occured. The origin might be understood
• There are large timing drifts in the ADCs that are currently not understood (up to ms)

ITDM

• Next Wednesday there will be a 1h shutdown of the load balancer in the DESY RZ. This should affect mostly the web services
• It should be noted that the DAQ ran quite stably in SPB and FXE. More than 20 runs were collected in each case without a restart

CAS

• Two hot fixes are out (2.1.18.4/5). These address  very specific issues
• A major new release (2.2) is planned for the winter shutdown. This will be mandatory
• Comment from instrument: It must happen early so that wrinkles can be ironed out!)

General:

• User Operation ongoing until Tuesday 5 Dec 8 AM
• CAS shall present a release plan for KARABO 2.2 next week
• Dispatch meetings shall move to a Mo/Wed 13:00 schedule starting next Wedneday 6 Dec

Instruments

• FXE:
  • Scantool was successfully used to establish overlap with laser beam (Tangerine) 
  • Some problems with slow DAQ in particular also for tunnel components
  • Comment from ITDM: There are known issues with the slow data acquisiton. A new version exists that was intentionally not deployed before the end of UOP
• SPB
  • Just finished first shift of laser user experiment
  • No big problems
  • All goals in the optimistic schedule were achieved

Tunnels

• Currently preparing for the shutdown
• Sticking condition of mirror bender could be reproduced in the lab
• WP74 just installed the last missing pieces in XTD1

Laser

• Running
• temporal and spatial overlap with beam is achieved
• But now instruments do not intend to use the PP laser for the last experiment

Detectors

• Both detectors are working
• The DAQ for slow data is crashing occasionally
• Gotthard is fully operational but there problems getting the data out. Help from PSI is needed

AE

• No problems

ITDM

• No problems

CAS

• CAS-XPD Running:
  • We will finish next Monday
  • We have been addressing issues related to MCP, Karabo-DAQ, Cameras, and K-Mono
  • Deploy K-mono today
  • Review DAQ Groups in Karabo: Cyril (interface to ITDM), Liubov, Koch
  • Gotthard we have to address in a separate and more systematic discussion: trigger, operation, calibration...
• XGM memleak - we already have a fix to get it stable but have to check when we will be able to upgrade (only device).
• Internal discussion about CAS/Karabo priorities for the winter shutdown

General:

• User operation with good beam conditions over the weekend
• One interlock trip in the FXE branch in XTD9. Restored by CAS
• Shutdown starts tomorrow at 08:00.
• Next dispatch meeting takes place Wednesday at 13:00. A reminder will be sent.

Instruments

• SPB:
  • Started with the actual sample during the night shift
  • Some DAQ issues
  • 108TB data was recorded
  • Problems appear to occur between 01:00 and 02:00  and between 13:00 and 14:00 
• FXE
  • Running is ok. 
  • Temporal and spatial overlap with optical laser established
  • Currently taking pump-probe data
  • People on shift were uncertain about how to recover from interlock trip
    • Statement from Vacuum: Pumps can restarted/valves open if the respective button in KARABO can be clicked

CAS:

• Delay between data taking and "online" analysis in the "Hummingbird" software is about 0.3s

ITDM

• On Friday, there was a global network outage affecting the Office Network on the DESY site.
• The transponder system in the experiment hall was down
• SRP organised a guard at the hall entrance so that the hall had not to be evacuated
• The control network (XDAC) was not affected

Please note following info from DESY:

Author: F. Saretzki



Ab sofort werden die Rack-Löschanlagen in den südlichen Tunneln in Betrieb genommen. Die Anlagen bleiben danach scharf.



 

General

• Starting from next week shutdown activities will be tracked using M. Aresté's list
• Dispatch meeting on Monday and Wednesday at 13:00
• Next Friday Readiness meeting also at 13:00 because of conflict with XDAC

Instruments

• FXE:
  • Currently focussing of implementing all default parameters into the next firmware release
• SPB:
  • Major detector work with AGIPD disassembly started

Tunnels

• Currently testing firmware and software in SASE3
• Installation of first mirror starts next week. Mirror installation takes about 1 wk/mirror
• CRL in XTD2 will be vented to align internal mask and fix stuck arm
• Drilling takes place at the end of XTD9 for the installation of the FXE diamond grating
• Mirror cooling will not be installed this shutdown. It is not needed for the planned bunch rates and might introduce unwanted effects/vibrations
• Activities of Diagnostics:
  • Photonics Science camera in SASE1 exchange.
  • Tests are being carried out in SASE3 together with J. Tolkiehn
  • Cabling is going on in XTD1 on the XGM
  • There is a noise problem on the Gotthardt detector in the HIREX. Need support from AE to find the origin

AE

• Motion loop in SASE3 will be split into mov1 and mov2. This will allow to sort components into more and less stable sets and keep one of the loops up most of the time

Laser

• Laser activities had to start late due to problems with AC and laser interlock (delays of 6 and 5 months respectively)
• Availability of trigger pattern from the machine was also a problem
• Specs could just be reached for the Nov user period
• Eventually the PP laser was not used for any of the experiments
• Work on the laser is far from finished and will require 3 more months to come to full funtionality
• Currently, the Karabo version is being upgraded and the most up-to-date PLC firmware is being installed
• These updates had intentionally been delayed until the end of the user period.

Detector

• Detector group not present at the meeting
• Information from SPB: major work is ongoing on AGIPD

AE

• Train builder is being tested as it was not used so far. This required re-cabling
• PLC motor loops are also being split in SASE1
• Many activities in SPB
  • Remaining modifications of modules have started
  • Export of input for firmware generation for loop 8 has been provided by EETF
• Waiting for go in FXE
  • Need to work on the interface for the EPS system
• No tasks currently from laser group
• New information available on SmarACT stages: The observed problems arise from a cable problem and can be solve by configuration
• The problems with movers in the sample chamber in SPB (non-functioning encoders) arise from cable cross talk at the vacuum feed through. This require hardware re-design
• In SASE2 there are tests coming up together with the diagnostics group

ITDM

• Train builder tests are scheduled next week
• Recabling of Infiniband network happens on 18/19/20 Dec. No DAQ and controls available on these days
• A summary of DAQ statistics was shown (see slides on INDICO)

CAS

• Release of KARABO 2.2 on 18 Dec
• Freeze of functionality on 21 Jan
• Discussion reveals that a discussion on specific deployment dates and needs of different groups is urgently needed (Instruments, Tunnels, DAQ)

FYI

Author: F. Saretzki

English version below

Ankündigung:

Aufgrund von Sachverständigenprüfungen wird es am folgenden Termin zu Auslösungen des Evakuierungsalarms kommen:

Gebäude: XHE2/XS2, XHE4/XS4, XSDU1, XTD1, 3, 5, 6, 7 und 8
Datum: 13.12.2017 ca. 09:00 Uhr – 11:00 Uhr


Sie brauchen bei ertönen der Sirene in diesen Zeiträumen das Gebäude nicht verlassen.

============================================================================
Announcement:

Due to expert testing there will be an evacuation alarm on the following date:

Building: XHE2/XS2, XHE4/XS4, XSDU1, XTD1, 3, 5, 6, 7 and 8
Date: 13th of December 2017 between 09:00 a.m. and 11:00 a.m.


You don’t need to exit the building while the alarm occurs.

 

General

• Meeting on Friday will take place at 1300 due to overlap with XDAC
• M. Areste shall ask for updates on task list on Thursday

Instruments

• SPB
  • Work on AGIPD is ahead of schedule. Closing of detector should happen at the beginning of next week
  • Additional installations for roughing pump system is going on and will probably finish this year

Tunnels

• Diagnostics group currently working in the Southern tunnels, cabling components and aligning imagers
• Colleagues from Russia are coming 3 - 20 Jan to work on MCP in SASE3
• KMono local tests in SASE3 should also happen in early Jan
• Diagnostics requests scenes for XGM in XTD1

AE

• Vacuum loop in SASE3 will be modified. Some components shall be moved to a separate loop, e. g. Soft Mono
• SPB tunnel loop will be re-deployed to include the attenuator
• The motion loop in SASE1 will also be split to allow more rapid updates for newly integrated components
• Z-motion crate for detector in SPB will be done until next week or early next year

CAS

• Currently working with AE and ITDM on refining release schedule for KARABO
• Testing the version for release next week. This will then first be used by ITDM and is needed for the firmware upgrade of AGIPD
• Work is going on on the XGM train ID problem
• FXE will be moved to a separate topic (broker) to disentangle some dependencies at least temporarily
• FEL imager performance problem was investigated. This runs on a server that is very busy and should be moved to a different server

The PLC program was modified and downloaded to the PLC, due to allow work on the XGM crate in rack 1.3.

The PLC vacuum loop includes now all vacuum crates in the racks from rack 1.1 to rack 1.7 except rack 1.3 (XGM)

Dear collogues,

we have added a hotfix to the framework, to make sure, that no motor can be moved if the feedback signals are missing. This update will be done due to the mentioned issue in the readiness meeting this morning.

We will update the loops again after the weekend, with a proper solution.

The hotfix will send all devices of a loop into error if the feedback of a terminal in the loop is missing.

I will write a new entire in the Logbook, if we are done with the updates.

Cheers,

    Tobias

Dear collogues,

all loops are up and running after deploying the hotfix.

Have a nice weekend,

    Tobias

 Confirmed from Karabo side.

Dear cllougues,

we will run an update of the EPS-loop, in order to have a proper solution for the latest hotfix, as it was announced this morning in the standup meeting.

Cheers,

    Tobias

The first EtherCAT coupler in 1.7 VAC was broken and is replaced now. The loop is up and running again.

Cheers,

    Tobias

We have to disconnect the EPS crate in RACK 1.14 in order to check the interface to the MPS.

I will keep you updated

The EPS-loop is now complete again.

The interface from EPS to MPS is working now. One broken wire in the cable between EPS and MPS was found. We are using now other pairs for the signals "SA1_SYS_EPS2MPS/DCTRL/MODE1", "SA1_SYS_EPS2MPS/DCTRL/MODES" and "SA1_SYS_EPS2MPS/DCTRL/MODEM".

Cheers,

    Tobias

Hi all,

in order to integrate the CRL in XTD2 I am going to update the motion loop.

I will let you know as soon as this is done.

Cheers,

    Tobias

AE on call service:

    Contact person:

        Tobias Freyermuth

    Phone:

        9 - 6822

Hi all,

due to ongoing tests with the PBLM motor drivers I have to set the Elmo drivers in PreOp mode.

That should not affect any other terminal in the loop.

Cheers,

    Tobias