General Announcement:

CTRL:

• nothing to report

DA:

• Nothing to report

EEE:

• XTD 10 ZZ uTCA excahange

VAC:

• nothing

ME:

• nothing

XPD:

• within next three days XPD equipment to be checked with x-rays
• DAQ test next Monday
•  

DET:

• LPD
  • communication with RAL and STFC, no sign of corrosion --> single module to be taken out and test in the lab

ITDM:

• nothing

XRO:

• nothing
•  

LAS:

• redmine tickets

UNSYS: 

• nothing to report

SPB/SFX:

• ITDM DAQ test today

FXE:

• nothing --> hutch being prepared
• PLC loops 4 and 7 are down due to hardware issues on the PLC --> EEE on it.
  • sample stack --> loop4 preparation delayed

MID:

• pre is going on, everything

HED:

• nothing

SCS:

• nothing

SQS:

• nothing to report

AOB:

• Zunaira --> beam shutter keys

General Announcement:

CTRL:

• InFlux, still experiencing delay with querying the database (20min, karaba; 5min Grafana)
  • Jan reports that in the XGM trendlines a similar effect is seen

DA:

• nothing to report

EEE:

• Friday call from BKR, because of MPS blocking
  • FXE SHUT MPS signal was not reset
  • ZA comments, that the reason is not yet clear --> ZA is contact with Rathjen and MPS experts

VAC:

ME:

XPD:

• For the FXE SHUT it seems MPS and karabo data are differeing
•  

DET:

• LPD
  • Hazem is back on Friday --> Supermodules will be tested then in the Lab
• AGIPD
  • special dark procedur currently not working well

ITDM:

• nothing to report

XRO:

• This week, component check in all SASE tunnels
•  

LAS:

• nothing to report

UNSYS: 

• Friday
  • SA1 cell 10, Suren exchanged the coupler

SPB/SFX:

• nothing to report

FXE:

• nothing to report
  • PLC issues loop4 and loop7 solved

MID:

• nothing to report

HED:

• nothing to report

SCS:

• nothing to report

SQS:

• nothing to report

AOB:

• Zunaira --> beam shutter keys

General Announcement:

CTRL:

• InlFlux logging, some statisitics had been gathered --> but no root cause had been found.
  • Load Balancer would needed to be restarted --> data inside buffer would be

DA:

• all good

EEE:

• PLC team is still busy with check up of the "fuse" issue
• MID cabling going

VAC:

ME:

XPD:

• one XGM didn't ecevice a signal
  • quickly resolved
• downstream XGM's are running as well

DET:

• AGIPD
  • dark procedure fixed and seems to be working now at SPB/SFX, has to be installed at HED and MID
• LPD
  • 3 supermodules had been moved to the DET LAB --> so far visual inspection does not show major issues
  • no module powered yet
     

ITDM:

• nothing to report

XRO:

• SA1 was aligned
  • SPB/SFX done

LAS:

• humidty in PPLAS
  • TS working on it
    • PKG after maintenance did not come back to specs

UNSYS: 

• nothing to report

SPB/SFX:

• nothing to report

FXE:

• nothing to report --> shutter porblem solved

MID:

• nothing to report

HED:

• nothing to report

SCS:

• triggers

SQS:

• nothing to report

AOB:

General Announcements
none

CTRL
- late train IDs at XGM, being sorted

DA
- nothing to report

EEE

DET

ITDM
- nothing to report

XRO
-

LAS
- Failure of Power SUpply on AMPHOS in SA3, had been replaced over the weekend,
- observed jump on ODL in SA2 around midnight --> being investigated

UNSYS
- nothing to report
- PKG status to be discussed

SPB/SFX
- nothing major to report

FXE

MID
- Feedback for M2 stopped working, started calculating wrong values

HED
- nothing to report

SCS
- problems with DAQ: DA1 stuck --> DOC and ITDM are aware

SQS
- nothing to report

AOB

General Announcement:

CTRL:

• nothing to report

DA:

• nothing to report

EEE:

• nothing to report

VAC:

• nothing to report

XPD:

• nothing to report

DET:

• nothing to report (SPB reports JF1M issue --> suspected readout board issue)

ITDM:

• issues with Nagios
  • misconfig by puppet mangement (DESY)

XRO:

• nothing to report

LAS:

• nothing to report
  • SA3 PPL up and running

UNSYS: 

• Phaseshifter tables for higher energies in blue week had been uploaded
• two quad movers from SA2 moved to 0
  • triggered by doocs --> quad movers moved back but it is unclear if they actually returned the BBA optimized position
  • clarification with M. Scholz confirms that quad movers did not move.

SPB/SFX:

• JF
• nothing to report

FXE:

• User Experiment starts
• C&C update requested until next week for LPD (uTCA)

MID:

• all fine at MID

HED:

• nothing to report

SCS:

• not present

SQS:

• nothing to report

AOB:

General Announcement:

CTRL:

• nothing report

DA:

• nothing to report

EEE:

• several motors in SPB/SFX had trouble, all originating from the same crate

VAC:

• VAC problems at HED last Wed
  • monochromator section in the tunnel, two unidentified vacuum bursts

XPD:

• nothing to be reported

DET:

• nothing to report

ITDM:

• nothing to report

XRO:

• FXE (PZ) reported a large jump in the bender setting, sebsequently Ry and pitch drifted for days and stabilized later
  • M1 M2 alignment changed as well
  • currently stable and being investigated

LAS:

• nothing to report

UNSYS: 

• nothing to report

SPB/SFX:

• unstable pulse energy jittering ~ 35% over days.

FXE:

• Experiment went well, apart from minor issue with calibration pipeline with JF

MID:

• all fine

HED:

• nothing to report

SCS:

• not present

SQS:

• not present

AOB:

General Announcement:

• This meeting is a tool to share information/experiences across scientific and technical groups

CTRL:

• system are working

DA:

EEE:

• next week
  • update on GATT and EPS loop in SA3 in roder to allow for new protection of GATT for the whole beaemtime
  • SA2 on Thursday (maybe Friday), try to correct backlash issue --> loop update

VAC:

• all is running
• HED VAC event
  • miscommunication between operators led to this issue

XPD:

• no issue to report
• next week, during the whole week there'll be a high photon energy test at SA1 and SA2 see a big involvement of XPD
  • 24 keV @SA1 (Tue)
  • ~30 keVC @ SA2 (Wed/Thu)
  • PI/Planner is Frank Brinker

DET:

• investigating JF1M issue of 4-6Hz instead of 10 Hz operation

ITDM:

• nothing

XRO:

• SA1 M1/M2 drifts under investigation
• XRO is involved as well with next weeks high photon energy effort as well as EEE loop updates

LAS:

• sub sync

UNSYS: 

• nothing

SPB/SFX:

• user exp this week, 24 keV next week

FXE:

MID:

• all fine at MID

HED:

• next week high photon energy

SCS:

• nothing to report

SQS:

• next week material tests, maybe also detector group.

AOB:

General Announcement:

CTRL:

• no update

DA:

• nothing

EEE:

• nothing

VAC:

• nothing to report

XPD:

• nothing special, shtudown tasks are in schedule, HXPES / XTED2 cable work ongoing

DET:

• nothing special

ITDM:

XRO:

LAS:

• lasers ramped down

UNSYS: 

• APPLEXv inv place, VAC to be closed next year

SPB/SFX:

• working on mirrors
  • checking vacuum today
• not clear what is the status with the movement of racks wrt SXP (SPB/SFX optics hutch racks currently powered down)
  • electrical installation?
  • PG comments: Brad won't allow operation until 230 V power distribution is calrified, once Brad gives the OK, the racks can be switched on
    racks are being finalized by EEE, but some checks need electrical power
  • RB: SPB/SFX were promised that the move of the racks were finished before X-Mas, but it seems the electrical work is not up to satandards

FXE:

• shutdown task in schedule

MID:

• all fine
  • some loop updates planned for Today

HED:

SCS:

SQS:

• nothing

AOB:

• light OCD over X-Mas for XPD in case of floowings, power outages.lists will be

General Announcement:

CTRL:

• no major updates

DA:

• no major updates

EEE:

• 1 MID plC dead
  • replaced on Monday
• cable work planned for next week
  • HED, SXP

VAC:

• In general excellent progress
• closing of M2 XTD2 (SA1) in progress, sealing issues mitigated

XPD:

• XTD10, XTD6
  • improvements, commissioning ongoing
• clean room tent water leak reported on 23.12.
  • fixed by TS befroe X-Mas

DET:

• no major issues

ITDM:

• Jan 19: dCachce maintenance

XRO:

• progress as planned
• changes to existing components will be posted to respective elogs

LAS:

UNSYS: 

• progress as planned

SPB/SFX:

• Vac monitoring recoverd

FXE:

MID:

• PLC outage on Jan 1 st caused VAC issues (reported by EEE)
  • fixed
• no access to MID optics hutch due to interlock test

HED:

• everything on schedule
• leak on a hose to humidifier device in A12. This leaked a few litres on the floor.
  • to be fixed by Caverion

SCS:

• progress as planned

SQS:

• progress as planned

General Announcement:

CTRL:

• observation from DOC
  • macros some times after the upgrade to Python 3.8 --> please contact CTRL?

DA:

• significant upgrade to Offline Analysis -->
  • proc data won't inlcude a copy of raw data any longer

EEE:

• fast electronics

VAC:

• no issues

XPD:

• no incidents
  • 2 tunnel accesses on Monday --> succesful

DET:

• DET commissioning at SPB (AGIPD JUNGFRAU 1M), MID (AGIPD), HED (ePIX, JUNGFRAU)

ITDM:

• E0.604 rack room, Wed next week
  • fixing power distribution units in every rack --> TS and external company will need a full day
  • affected services are: Lab DAQ, LAB control servers (EEE; CTRL, DET), GitLab runners, one machine for data analysis, PLC database, karabo project database (?)
• Yesterday short interruption with influx reading note --> test requested for next Monday causing a 1 hour interruption

XRO:

• SA1 / SA2 M3 PBLMs have been equipped with diamond plates and can be used for beam stability measurements
• Backlash Controls implemented last week --> values
• report of lost positions of vertical slit in SA3
  • in case of similar issues please contact DOC prior to self fixing.

LAS:

• SA2 PPL is good
• SA3 PPL planned to be up today
• SA1 PPL being investigated as agreed with SA1 instruments

UNSYS: 

• transparency test  for APPLE X went successful

SPB/SFX:

• SPB/SFX aligned since yesterday, no issues

FXE:

• blown fuses in crates --> solved with EEE

MID:

• all fine

HED:

• Com yesterday
  • outstanding issue: fastADC not working EEE aware of that

SCS:

SQS:

• GATT always worked but very slowly responding
  • OCD contacted
  • Raul comments that this is a known issue and should be communicated to CTRL
  • SH nesures that Raul creates the ticket
• XTD10_PES
  • digitizer display frozen almost all time (connected to 2 DAQs) --> reported to DOC
  • SH comments that when connecting to two DAQs (tunnel and SQS), should be connected to only one DAQ.

General Announcement:

CTRL:

• PP LAS restarting device servers for upgrading the device tag --> autoinstantiate some servers all topics

DA:

• Planned maintenance today for Cal and gpfs delayed at least until next Monday

EEE:

• nothing major to report

VAC:

• 1 ZZ ongoing to XTD4 to shiled an ion pump controller, suspicion of radiation damage.
  • AK Dump XTD10? --> yes

XPD:

• similar Diamond cracks as in solid attenuator had been observed in the transmissive imagers
• Last weeek FEL imager issue had been clarified (image was rotated by 180 °, occured due to faultly config load)

DET:

• AGIPD1M (SPB/SFX), Friday morning some DAQ issues induced train losses. Didn't happen again after careful checking

ITDM:

• Issue at HED with data migration --> discussion will take place

XRO:

• Tomorrow and after tomorrow late shift some CTRL support might be required

LAS:

• PLL LAS SA1 sync issues (LbSYNC informed and arriving 10:00)
• PPL LAS3 NOPA setup

UNSYS: 

• ZZ ongoing

SPB/SFX:

• Experiment went well, no major issue
• Run Controller crashd a few times over the weekend. Could be restarted but crashes some time later and has to be restarted

FXE:

• affecetd by LAS Sync issue
• issue with doocs panel for shutters (yifeng)

MID:

• all fine
• SSRY motor issue (ELMO)
  • Nerea updated the ticket --> motor is working

HED:

• DA1 got stuck for a while (fixed by restarting)
• lot of short runs (1min) and only one train isneed from that -> requires fast data migration and analysis -> doesn't always work
  • discussion with ITDM commencing
• PAM was missing from DAQ

SCS:

• no major issues
• RIXS on liquids went well

SQS:

• mono shift becuase

DOC /DRC

issues with call forwarding

General Announcement:

CTRL:

• influxDB outage (telegraf issue)
  • SA1 and SA3 affected
  • currently fixed but linked to a device in SQS

DA:

• nothing major

EEE:

• PLC colleagues are working on CR and support requests
• pulling cables for SQS and SCS
• reported to XO uTCA in XTD1
  • remote reboot planned for Friday, reserve ZZ slot for Friday

VAC:

• repair of XTD2 solid attenuator ongoing

XPD:

• FD @ SA2

DET:

• nothing to report

ITDM:

• nothing to report
• next Monday GPFS update will take place

XRO:

• mirror calibration started in SA2
  • beam path through offset mirros not visible -> no beam on M3 pop in, but visible on M2 pop in MCP in beam?
  • JG: no MCP in beam, how about PBLM?
  • LS: no PBLM in beam
  •  

LAS:

• Sync issue from Monday cannot be replicated by Sebastian -->
• tests for SPB happening tomorrow

UNSYS: 

• circular polratizaton lasing achievd in APPLE-X in left and right hand polarization, also linear horizontal and vertical and 45° measured at 900 and 700 eV
  • thanks to UNSYS and DESY APPLE-X comm team.

SPB/SFX:

• no major

FXE:

• nothing to add

MID:

• all fine

HED:

• nothing to report

SCS:

• reservation of nodes for jupyter hub.
  • requested during drc hour fix to reserve 3 nodes
    • didn't work out
  • is it possible to reserve 3 nodes by default?
• Olivier reports there was a misunderstanding and it got fixed very quickly
• Janusz reports that by standard 4 nodes are reserved for each experiment.

SQS:

• nothing to report

General Announcement:

CTRL:

• small intervention --> udpate of karabo-gui client exflagteway

DA:

• maxwell gpfs software maintenance --> offline calibration pipeline affected see ITDM

EEE:

• OCD call on rack 2.01 which was powered down. After fuse repaired

VAC:

• see above ...

XPD:

• Rack 2.01 --> Camera of SR imagare is not working
  • situation not clear, being investigated --> IP conflict?
  • Janusz suggests to powercycle the camera

DET:

• many to report

ITDM:

• 10:00 maxfell gpfs update

XRO:

• SA2 mirror calibration

LAS:

• remainder change of set point for SCS to 112 kHz

UNSYS: 

• nothing to report

SPB/SFX:

• nothing to report

FXE:

• preparing for user experiments
  • concerned about optical sync of PP Laser, probelm of last week was fixed but no reason found
  • test last thursday optical sync lost again while moving optical delay stage
  • FXE

MID:

• alignment of optical laser alignment in joined hutch.
  • please

HED:

• nothing to report

SCS:

• nothing to report
  •  

SQS:

• last week Thursday night
  • mono drift investigated --> preliminary analyisis points to the grating as the drifting component

General Announcement:

• No User operation meeting this Friday, instead cw 16 / cw 17 check will happen in Daily Operations Meeting at 15:30 on Thu Apr 14. Link available on ttfinfo in XFELelog

CTRL:

• nothing major to report

DA:

• nothing major to report

EEE:

• nothing major to report

VAC:

• nothing to report

XPD:

• nothing to report

DET:

• offline calib with JUNGFRAU --> tickets closed already

ITDM:

XRO:

• request to SA1 instruments
  • please check the repaired XTD2 solid attenuators
  • attenuators cannot sustain too much power, offline discussion follows
• SA2 mirror recalib
  • roll of M1 and M2 was completely misaligned, recalibration successful
  • geometrical center of the kinematic system roughly identified
    • required for Euler MDL,

LAS:

• nothing to report

UNSYS: 

• nothing to report

SPB/SFX:

• nothing to report

FXE:

• not present

MID:

• pump probe looks stable and getting ready for the user
  • BAM doocsbridge stopped --> manual restart necessary

HED:

• nothing to report

SCS:

• nothing to report
  •  

SQS:

• nothing to report

DRC

• one hour data lost in influxDB in SA2

PRC

General Announcement:

CTRL:

• bug noticed in BEckhoof karabo device --> already fixed from some topics, see xo-info comm from yesterday

DA:

• all fine

EEE:

• CRL-HED-OPT isue solved

VAC:

• two devices went into error --> likely because of Beckhoff Karabo bug

XPD:

• nothing to report

DET:

• AGIPD 1M at SPB partially vented yesterday --> detector went into interlock, no damage
  • who should be called at SPB (or any instrument for that matter)
  • Turbo pumps went into error state which triggered the partial vent --> karabo Beckhoff update?
  • Wajid comments: karabo beckhoff update does not correlate with this incident, judging from time stamps

ITDM:

• no update

XRO:

• nothing to report

LAS:

• nothing to report

UNSYS: 

• not present

SPB/SFX:

FXE:

• software limits of phase shifter of PP Laser
  • DK solved by PP Laser

MID:

• all fine at MID
  • HXRSS very much appreciated

HED:

• HIREX offset

SCS:

SQS:

DRC

PRC

• SCS incident on Monday, photon energy scanning script was left running --> had to be stopped manually

General Announcement:

• M. Döring:
  • almost all of the activities that should have been done until last Friday had been done

CTRL:

• last week karabo topic updates had finishedexcept for LAS
•  

DA:

• nothing

EEE:

• SA3 exit slit

VAC:

• leak search on M6 (SA3) chamber
• leak search in XTD9
• all in all well

XPD:

• almost all tasks have been completed
• XTD1 new crate connected on Friday,

DET:

• nothing

ITDM:

• all updates are finsihed
• some activities continuing on the offsite --> should be comüpletely transparent
• after the prox change there are some PCs which are using the old host names --> relavant people will be contacted by Janusz Szuba

XRO:

• M6 mirror installation beign finished
  • end switch checks shoudl be finished by today
  • ramp up procedure being written down
    • instruments are asked to submit any additional requests for ramp up

LAS:

• fastADCs should be finished by Wed this wek

UNSYS: 

• control system has to be adjusted after removal of APPLE X, i.e. the control system looks for motors and

SPB/SFX:

• checks

FXE:

• motor commissioning ongoing

MID:

• nothing
  •  

HED:

• when modifying cables downstairs, make sure modules upstairs are switched off

SCS:

• PLC updates

SQS:

• not present

DRC

PRC

• SCS incident on Monday, photon energy scanning script was left running --> had to be stopped manually

General Announcement:

• none

CTRL:

• fastADC karabo Version updated
  • everything looks fine
•  

DA:

• nothing

EEE:

• PLC and cabling as scheduled
• fastADC update
  • MID and SPB issues being solved by lunch time

VAC:

• HED Mono
• SXP tunnel interface

XPD:

• ZZ on Tuesday for XTD1 SR imager camera reset

DET:

• DSSC update
  • test not yet conclusive on whether operation is safe or not.
  • current suggestion: postpone experiments until repair is done
• AGIPD firmware update went well

ITDM:

• nothing

XRO:

• not present

LAS:

• Timing issues solved, solved by Lars Froehlich
• LAS may suffer from limited available resources

UNSYS: 

• all fine

SPB/SFX:

• issue with Elmo motors

FXE:

• nothing

MID:

• nothing
  •  

HED:

• nothing

SCS:

• mobile compressor will be used this week

SQS:

• nothing

DRC

PRC

General Announcement:

• none

CTRL:

• everzthing as planned
•  

DA:

• currently there is a gpfs issue affecting the online cluster
  • delays in online analysis have to be expected
  • delays in obtaining new constants, can take much longer than normally -->

EEE:

• nothing to report
• a couple of boards are not reachable, maybe connected to the power glitch we've had

VAC:

• ongoing activities
  • XTD 6&9
    • HED Mono works closing
    • SXP beamline ready for beam
  • operation at very low photon energies at SA3:
    • GATT will use Ar --> System had been tested
• power glitch yesterday was seen in water distribution

XPD:

• ZZ on Tuesday for XTD1 SR imager camera
  • successful

DET:

• nothing major to report

ITDM:

• nothing

XRO:

• nothing major to report
  •  

LAS:

• nothing major to report

UNSYS: 

• nothing to report

SPB/SFX:

• issue with Elmo motors had been mitigated
• DA and Detector recovery

FXE:

• acces to air comperssor and support for moving LPD required for July 22nd for BTM tests.

MID:

• after loop update wrong parameters had been put to LDM motors
  •  

HED:

• not present

SCS:

• nothing to report

SQS:

• nothing to report

DRC

PRC

General Announcement:

CTRL:

• Today "bring up" works at FXE

DA:

• Nothing special to report

EEE:

• PLC team is trying to find solutions for slowness in big loops

VAC:

• nothing to report

XPD:

• nothing special

DET:

• nothing major to report

ITDM:

• SA2 GPFS updates today

XRO:

• not present

LAS:

• SA1 AMPHOS back as

UNSYS: 

• K- Mono Scan last week
  • S. Arbeghyan assisted but new undultaor offsets will only be implemented after Aug 5th

SPB/SFX:

• nothing special to report
• bring up days are very successful

FXE:

MID:

• ITDM please keep MID in the loop with SA2 GPFS updates, as MID plans to save data today
• opening of the beam shutter is an issue
  • over the weekend MID was forced to open

HED:

• shutter issue as MID

SCS:

• nothing to report

SQS:

• nothing to add

DRC

PRC

• s

SH

• VAC loops SA1 SA2, don't panic because of slowness

General Announcement:

CTRL:

• nothing special

DA:

• nothing

EEE:

• actions from Monday (SA3 PLC master update,  PNACT testing) successful

VAC:

• vacuum ok

XPD:

• nothing

DET:

• nothing major to report

ITDM:

• nothing to report

XRO:

• nothing to discuss

LAS:

• nothing

UNSYS: 

• nothing

SPB/SFX:

• nothing

FXE:

• all fine

MID:

• all fine

HED:

• undulator setting off by 100 eV --> HED had means to accurately measure the delivered photon energy and compensate for that.

SCS:

• nothing

SQS:

• nothing

DRC

• nothing

PRC

• nothing