SPB:

• nothing to report

FXE:

• MC2 library motor commissioning on-going

MID:

• SDL motor parameters defined, however one crate is blocked by Wille and 6 motor connectors (Harting material) are still missing
  once fixed, motor paramaters can be uploaded
• WCB (wall connection boxes) in otics hutch are partially finished

HED:

• Wille is working on some crates and the respective Beckhoff loops are down for the moment
• loop updates from Tuesday on

 EEE:

• additional limit switched for XTD6 CRLs (MID) are being installed
• HED is asking for limit switches on their XTD6 CRLs as well --> EEE and HED will investigate when and how
• cable change in FXE starts this week with exchaging wrongly poled SHV cables

CTRL:

• nothing to report

VAC:

• nothing to report

XRO:

• nothing to report

SPB:

• nothing to report

FXE:

• not present due to group retreat

MID:

• Mono done, no severe issues from commissioning weekend

SCS:

• not present

 EEE:

• SQS pnCCD cables have been installed, waiting for crate to connect

CTRL:

• two minor OCD call and one regarding AGIPD
  • SPB not aware that no support was scheduled for AGIPD

 XRO:

• piezo karabo device and firmware in HED mono need to be checked for compatibility

ITDM:

• 2 calls
  • derver down and restarted
  • data aggregator down and restarted
• new RunConfig delpoyed at FXE

DA:

• jupyterhub @SCS showing performance issues --> investigated by DA

SPB:

• AGIPD improvement campaign great success
• EEE --> 2nd C&C worked just fine

FXE:

• BTM tests successful
• Johann MDL motors should be in closed loop
• JF CTRL device crashed

MID:

• AGIPD Flat Fields done
• Data Correction pipeline was down --> 1 day lost
• PPLaser comm this week
• extrafoam issues

SCS:

• nothing to report

 EEE:

• updates for SDL (MID)
• fastADC issue --> don't take more than 40k samples; bugfix in progress

CTRL:

• 8 OCD events
  • mostly deployment isues
  • FXE DA3 overload
  • old version of scantool @ SCS

 XPD:

• unusual behaviour of beam pos measured with XGM
  • cross check done yesterday

ITDM:

• DA crashes --> being investigated

HED:

• tecnosoft terminal issue
  • motor ran into Ty limit (CCW) and got stuck due to tecnosoft logic

SPB:

• interument parked
• power cut --> someon needs to come in
• NJ: what'S that status of the VAC
• KC & RB: safe state, roughing system still running, in the event of a power cut this needs to be looked at

FXE:

• nothing to report
• NJ: VAC in safe state? --> MK yes everything is switched off

MID:

• instrument parked, VAC is running but pumps will be stopped today and partially vented
  • gate valves are closed hopefully the VAC stays

SCS:

• SCS in safe state, all pumps running
• valves around KB are permenantly closed
• in the event of a power cut all other gate valves will close
• loop update done yesterday

 EEE:

• preparation for safe state ongoing
  • discussed

CTRL:

• is master timing going away or not?
• if master timing goes down, some devices
• ICI contacts identify what can be remotely done and what not

• u

ITDM:

• nothing special

HED:

• HED is going to sleep within the next two days

DA:

• focus on long standing tasks

XRO:

• beam transport is going to be parked today

Currently EuXFEL enforces a hand-off policy for instrument/tunnel hardware. That includes remote control of motors, pumps, etc.. as well as changing config Files. If and when this policy is lifted will be communicated during Dispatch and Technical Integration Meetings. Please also don't hesitate and contact XO in case of any questions.

SPB:

• reached safe state
• rougning pumps still runnning and valves open
• PLCs need to run for monitoring

FXE:

• n

MID:

• absolutely safe
• cooling water connection
• PLCs switched off

SCS:

• S

 EEE:

• MID switched off completely, all other PLCs are running
• MID utca also shut down
• Jörn, plenty of time for additional planning

CTRL:

• broker logs ok from last weekend
• ICI contacts are in discussion instruments

ITDM:

• while instrument schutdown, network and servers are still running

HED:

• safe state, all motors switched off ( parameters saved) all valves close, ion pumps are stil running.

DA:

• work to do

XRO:

• M1s parked on Friday, tunnel seems ok, lenses seem to be fine
• small problems in the GUI,

XPD:

• x-ray diagnostics had been secured last week, i.e. power ramped down. XGM vaccum needed.

FXE:

• Jungfrau chiller switched on, AMPHOS shutter chiller
• OGT3 no pump cart, ion pumps still running, gate valves closed

VAC:

• V0 closed, OCD looking at vacuum twice a day, PLCs needed and running

Currently EuXFEL enforces a hand-off policy for instrument/tunnel hardware. That includes remote control of motors, pumps, etc.. as well as changing config files. If and when this policy is lifted will be communicated during Dispatch and Technical Integration Meetings. Please also don't hesitate and contact XO in case of any questions.

SPB:

• instrument is fine in safe state.
• anyone having a VAC monitoring script connected to an SMS service?
  • SH: comments that CTRL is working on that on a more general scope, but that needs a bit more time

FXE:

• JungFrau checked and powered down.

MID:

• everything is in a safe state.
  • Martin D. reported a VAC issue --> gonna be clarified
    • MD reports a spike in the VAC which might indicate a leak in a bellow

SCS:

• VAC interlock closed
• eplans for hRIXs electronics
• MT wrote a rudimentary webservice for monitoring karabo stuff via a webbrowser --> not to be used until sanctioned by CTRL

 EEE:

• general remark: EEE provides two key systems: utca, and PLCs
  • currently only PLCs for critical systems are covered by OCDs
  • however all other PLCs are kept running, but not necessarily fixed on-site
  • utca not critical --> thus no OCD for that

CTRL:

• all systems running
  • limitations on karabo in the control network as stated in MB message --> finding out what that means for work on CTRL topics

ITDM:

• everything working, dropbox for urgently needed IT equipment available

HED:

• working on scenes / projects, recombining and cleaning up.

DA:

• not present

XRO:

• tunnel components secured, something on CRLs to be clarified with VAC

XPD:

• everything is fine

VAC:

• VAC in tunnel is stable
• new software for vacuum pump carts developed and being rolled out

CTRL:

• Broker Pile Up in 4 topics over the weekend
  • LAS affected in all topics, LA2 even twice
  • SPB affected once
• mitigation startegy worked

EEE:

• nothing to report

VAC:

nothing to report

XPD

• vac systems in the lab had been secured --> secuvac valves installed plus webcams

DET:

• nothing to report

ITDM:

• exflgateway hardware problem on Saturday --> fixed

XRO:

• nothing to report

SPB:

• rough pump turned off, possibly lack of cooling water
  • interlock tripped --> pressures raised
• were able to set remotely

FXE:

• nothing to report

MID:

• everything is fine

HED:

• VAC is fine, ion pumps are on, no roughing pumps running

DA:

CTRL:

• productive meeting with HED
  • CTRL will ask the other instruments if a similar concept could be agreed upon
• discussion with LAS this afternoon with respect to the current hardware policy

EEE:

• HED provided tasks that EEE shall do
• uTCA development ongoing
  • review how digitizers are being used (fastADC, SPdevices, clock & Control)

VAC:

• power glitch

XPD:

• nothing to report

DET:

ITDM:

• nothing to report, no indication of power glitch

XRO:

• no indication of power glitch

SPB:

• nothing to report

FXE:

• nothing to report, problem to read out pressure values
  • Bernd Baranasic: no ion pressure readings on OGT1
    • in PLC it looks ok, but karabo shows 0
  •  

MID:

• meeting with DESY safety colleagues wrt PUMA, going to be bundled with LDM safety
• discussing with Bruno about digitizers
•  

HED:

• see CTRL

SCS

• only valves in loop1 were closed
• every few days a pump is failing, new behaviour
  • JG: comments it is a known behaviour that one of the gauges showed an erratic behaviour (wrongly set epsilon in karabo)
  • GM: answers, what happened on Saturday was slighty different

DA:

CTRL:

• no bigger issues
• announcement: inital simulation environment ready which can simulate BECKHOFF devices
  • shall be opened to all groups for karabo development

EEE:

• continue work
  • shutdown task deadline may be too early
• if people have CRs CRDs, cable issues please submit

VAC:

• water pumps

XPD:

• nothing to report

DET:

ITDM:

XRO:

• nothing

SPB:

• nothing to report

LAS

• waiting for karabo policy

FXE:

• nothing to report

MID:

• nothing to report
• two weeks deadline extensions like EEE, DET

HED:

• nothing to report

SCS

• shutdown list week extension

DA:

CTRL:

• notificagion service device going to be tested this week
  • ITDM: SMS part is operational but needs to be implemented

EEE:

• shutdown task list to be submitted today

VAC:

• no one present

XPD:

• shutdown task list submitted

DET:

• nothing to report

ITDM:

• nothing to report
•  

XRO:

• nothin to report

SPB:

• VAC i

LAS

• w

FXE:

• shutdown list

MID:

• shutdown list
• Jörg and Jörn (EEE) align for shutdown task list

HED:

• aligning with Jörn (EEE) on shutdown taks

SCS

• working on shutdown list

DA:

General Announcement:

• Default beam positions are being implemented for each SASE at the FEL imager to reduce re-alignment offers for the instrumetns after shutdowns and facility development weeks

CTRL:

• minor OCD calls
  • 6541, 6542
• few MC2 motors not working -> target velocity was set to 0
• in SA1 the bunch pattern changes @10 Hz --> data logging system (with timers) gets overloaded ...
  • if this was to become a normal mode of operation, the system would need to be adpated --> as for now bunch pattterns are not logged anymore
• remote user access to karabo
  • soo to be tested with SQS
• inlfuxDB
  • FXE 95% migrated, probably 100% at the end of the day
  • LAS SA1 100% migrated
  • MID, SCS shall reply when migration is possible (causes a short down time of datalogging)
  • SPB, HED shall reply s.a.

DA:

• nothing to report

EEE:

• issue with encoders in SPB --> some errors had been disabled --> checks for cable lengths (OCD call)
• SQS interloop comunication working? EEE, yes, need to investigate
• N. Jardon reminds about the default parameter procedure
• pre shutter in SA3 works commence when Zunaira, Martin and Maurizio

VAC:

• nothing to report

XPD:

• intervention Monday in SA3, PES is operational

DET:

• no issues

ITDM: 

• last week new DAQ deplyoed in HED, FXE and SPB and tunnels, the three other instruments had been delayed
  • MID today, SQS yesterday
• scratch space in gpfs --> running out of space --> data needs to be removed
  • procedure not yet agreed upon
  • all involved parties should identify data that can be removed from scratch
  • 100 TB are left
• internal proposals for commissioning time needed
  • in the future please prepare those in advance

XRO:

• during the weekend, XRO was recommissioning of the beam transport
  • some qctivities were completed on Tuesday, M3 repositioned from damaged spot
  • recommissioning is completed --> however small problems are to be expected over the frst few days, please contact XRO

LAS:

• no issues

UNSYS: 

• cell 33 in SA2 communication crippled, ZZ planned for next monday

SPB:

• reasonably succesful first night with beam, instrument aligned with alignment laser
• tunnel EPS system --> the FXE_XTD9_popin blocked SPB to 1 pulse
  • MPS signals from SPB might be masked by the machine -> needs to be checked
• 1 OCD call to EEE
• dMigration to influxDB can be done

FXE:

• re align beam to FXE pop in
• mono re alignment
• issue with FastValva --> naming convention on the PLC level had been changed, now only capital letters are allowed --> Bernard is looking into this
• tunnel project using step function with motors throws an error on the slot
  • CTRL is workig on a fix already, it happens for very small steps
  • J. Nardon: was also reported by SQS
• all TView devices had to be restarted as they were out of sync

MID:

• alignment went smooth, until diagnostic endstation
  • data loggers are fine today to migrate to influxDB
    • Steffen aims at afternoon
• pre shutter
  • for CRL2 not possible to laign as the pre shutter always moved in
    • XRO looks into it

HED:

• beamline aligned
  • beam profile excellent in fornt of hutch
    • spatial jitter present, and horizontal movement
  • mirror feedback macro not working
• XGM investigation together twith Theo
• IPM didodes are not read out --> comment by Peter, check the timers
•  

SCS:

• no issues

SQS:

• beam from around noon, beamline aligned, set points on imagers need to be re found

General Announcement:

• XO discusses the need for sharing information.

CTRL:

• support calls
  • small movements cause errors on MC2 motors, fis is going to be tested today and it will be communicated when ti is going to beployed
• swithcing to influxDB only logging, if not heard otherwise then this should happen this afternoon (Monday)
  • SCS, SPB, HED, MID
• switching to influxDB in SA1, SA2 and SA3 to follow as well
• notification service updated such, that only more serious messafe are being send out

DA:

• no issues

EEE:

• cable installations in this week at HED, should start from tomorrow, 1.5 days (Jörn)
• few OCD calls,
  • soft mono in SASE3, motor not working
  • mirror
  • FXE motor runs into emergency stop --> fixed in the latest PC framework, but FXE is running an earlier version
  •  

VAC:

• no issues

XPD:

• i

DET:

• Thursday, OCD was not scheduled, despite some calls

ITDM: 

• bug for big detectors in DAQ had been addressed and should be deployed soon
  • this bug seems to also affect some slow detectors
    • only happens when there is no slow data added to the aggregator
• data reduction effort
  • one outcome: record bunch pattern in every run
    • instruments are asked to include the datasource XXX_timing
    • SH comments: in the future the bunch pattern information will be made available in the pipeline

XRO:

• some recommissioning last week
• sometimes mirrors are not moving

LAS:

• no issues

UNSYS: 

• c

SPB:

• good run for the first wek
• trouble with detector, not being able to record darks -> fixed next day
  • issue with a device that checks correct datasources
• FXE imager EPS interlock issue

FXE:

• FXE imager EPS interlock issue

MID:

• successful week
• a couple of OCD calls
  • CTRL, EEE, DET, everything solved
• AGIPD trainID offset by 1 wrt to the XGM
• karabo update --> changes are incompatible with existing scenes
  • e.g. number of digits
  • SH comments that it would be appreciated if those issue were reported to CTRL

HED:

• no issues
• MC2 motor update, is that going to be done today?
  • SH: yes and if possible for HED, then 11:30 - 13:30 would be the preferred time window.

SCS:

• exit slits did not move --> EEE OCD could help
  • SCS requests tthat components are tested before tunnels are closed
• Navitar didn't work
  • focussing lens lost position
• when component software is updated/upgraded, SCS would liek to see a test
• NG comments: motor parameters are sometimes wrong
• If things need to be modified, test beforehand

SQS:

• no issues

General Announcement:

• .

CTRL:

• INfluxDB migration is running smoothly, tomorrow the last topics will be switched to influx only
• some Basler / Andor cameras not working at 10 Hz is related to the data network --> depends

DA:

• n

EEE:

• Company Wille is on-site for hardware installations
  • hRIXS,
  • HED cable installations
  •  

VAC:

• no issues

XPD:

• i

DET:

• commissioning shifts with AGIPD
• pn CCD at SQS also on commissioning

ITDM: 

• issue from Monday wrt crippled DAQ when having no sow data in a datasource
  • the fix reported on Monday only works partially --> earliest estimate for fixing is now next week
•  

XRO:

• s

LAS:

• n

UNSYS: 

• no issues

SPB:

• OCD for Basler cameras
• OCD for MC2 stages (problem with encoders, EEE is informed)

FXE:

• Major issue with the machine wrt to big brother
• JUngFRAU seems to be more affected by the DAQ glitch
• GOTTHARD detector should be re-integrated into the DAQ
  • K. Wrona clarifies that only a reconfig of datasources is needed

MID:

• beam looks unstable
  • horizontal jitter is higher than usual

HED:

• no issues

SCS:

• no issues
  • Daniele: is the soft-mono solved?
  • TM comments: the pre mirror is in place piezos are working

SQS:

• no issues
• pnCCD commissioning

General Announcement:

• .

CTRL:

• few issues on SA3 vacuum server, over the weekend.
  • twice some devices were not running,
  • when restarting Beckhoff loops from the GUI please remember that it takes time until they are back
• device servers with - in their name. restarting with the service manager cannot cope with this --> fix is underway
• scheduling:
  • Beckhoff: small movement error fix is underway and it is suggested to update karabo Beckhoff servers later this week.
  • influxDB data logging
    • current version does not allow easy correlation data from two devices in Grafan (web front-end) an update to the next higher version would cure that
      • suggest to update Wednesday morning early
•  

DA:

• no issues

EEE:

• utca OCD calls
  • nothing serious
• PLC no OCD calls
  • some updates are planned for FXE, MID SCS on Beckhoff

VAC:

• no issues

XPD:

• no issues

DET:

• comm AGIPD detectors
  • detectors are stable, but issues with online processing

ITDM:

• new version of DAQ had been tested
  • should be deployed this week
• 1 OCD call for recording data with ADQdigitizer
  • requires coord between CTRL and ITDM and EEE
  • this operation mode had been tested
• 1 OCD call JUNGFRAU SPB in burst mode --> did not work
  • SH: was the receiver crashing?
  • KW: not clear, nothing was working in that config
  • SH: the problem had been identified at FXE --> fix should be deployed at SPB as well

XRO:

• update:
  • follwoing up the issue with the attenuator damage in SAx
•  

LAS:

• all three laser are running
  • some problems with shutters
    • SQS: shutter in ILH, cooling water refilled --> but interlocks could not be set
      • XO to coordinate discussion with Pouneh

UNSYS: 

• no issues

SPB:

• karabo sessions lost connections to GUI server on exflcon026
• JUNGFRAU receiver crashing
• DAQ took long time to change state
  • even switching to ignore after run take a long time
  • this relates to JUNGFRAU
• GOTTHARD specref device crasing multiple times
  • restarting had to be done multiple times
• optics interlock hutch was observed to be broken on Thursady night

FXE:

• digitizers at some point stopped
  • connection to boards and triggers
  • BF: this has to be investigated!
• beam stability
  • mirror cooling
• EXAFS within 700 eV had been measured
• PP Laser was very stable

MID:

• VAC incident in CRL2 --> which closed V0 --> VAC solved the issues
• due to high pulse energies small damage to attenuator and instrument component had been
• DAQ update scheduling
•  

HED:

• beam is jittering (when collimated 100 um are observed)
• CRL scene optics hutch is mixed up for one or two motors --> Thomas Preston investigate

SCS:

• no issues

SQS:

• interlock / shutter issue

General Announcement:

• .

CTRL:

• SA3 vac loop
  • devices lose connection to the PLCs every now and then (every few days)
• karabo fix for small movement error on MC2 motors
  • for the tunnel topics CTRL suggests to do that Thursday morning
• InfluxDB

DA:

• no issues
• upcoming commissioning
  • da infrastructure up and ready

EEE:

• cw36, cable installations require

VAC:

• no issues, normal activities

XPD:

DET:

• no issues

ITDM:

• continuing with Monday tasks

XRO:

LAS:

• increas priotity on karabo integration for lasermet interlock
• SH: for SA1 and SA2 much of the interlock data is

UNSYS: 

• today ZZ --> cell 33 should be fixed today as well as cell1

SPB:

• nothing outstanding
  • DAQ testing
  • VAC intervention

FXE:

• FXE and XRO is switching off M3 cooling
• no update on the digitizer issue

MID:

• no issue

HED:

• MC2 motor timeout problem, karabo fix seems to work
  • there are other time out errors that ar not addressed by this
  • SH: suggests a zoom session for demonstration of these
  • TP: e.g. stop button gives time

SCS:

• no issues
• installation of girder
• bestec is working on hRIXS
• recommissioning of Mono

SQS:

• laser interlock issue
  • Lasermet is visiting today

General Announcement:
- An informal Run coordinators' BKR chat to exchange ideas happens every day at 15:00. 
It is a forum in case technical and scientific people wish to make formal requests. XO will send an e-mail with Zoom link.
- XO would like to ask instruments to pay attention to the parameters of the Delivery and Operators table, especially regarding the bunch limits for the North branch.
Ideally we shall discuss one week before beam delivery.
- H. Sinn notices that if SPB move some motors, this is temporarily cutting SA3 beam due to MPS. SQS has not observed this effect. Harald will further investigate. 
Adrian M. says SCS impacts SPB in different ways.
- A. Galler mentions that SRP/task force will distribute updates related to the currently raising Covid issues.

CTRL:
- One group of motors at the same SCS/PLC was not accepting changes. A re-start with servers in small loops must help.
- Timing servers (switching off archive): CTRL wants to perform today or tomorrow morning the installation of the new version. Nobody seems against it.

DA:
- Nothing to report.

EEE:
- EEE understand the team request regarding the 5 Hz trigger will be done tomorrow.
- EEE waits for confirmation of SEPS update, possibly happening today or tomorrow. XO mentions Z. Ansari will be able to confirm today after lunch time.

VAC:
- Nothing to report.

XPD:
- Not present.

DET:
- XO inquires on the DSSC activities in SCS: DET says it may be delayed a bit due to other activities; DET got a ticket regarding operation; DA believes the DSSC was 
running and used.

ITDM: 
- Comment on DSSC: it was used but we shall not forget that on the first day there was not a preparation and we could expect hiccups.
- A faulty memory module must be replaced and the exflgateway will be shut down shortly for this activity at around 12:00. An e-mail will be sent but nobody seems against 
to it. 

XRO:
- Nothing to report.

LAS:
- SA3 AMPHOS photodiodes still has problems. System is partially working and can be operated. LAS knows beam size and pointing do not change but not if loses power. 
AMPHOS is contacted. 
The consequence is that switching ON and OFF takes more time because it is done manually. Further measurements will be done and SA 3 instruments will be updated 
accordingly.
- LAS informs its OCD team is not responsible for synchronization issues. 
- SA1 and SA2 lasers are fine.

UNSYS:
- Small issues with SA3 PLC crashes: SA3 is operated without first cells. A re-start of the servers is needed, probably today or tomorrow. UNSYS will inform SCS and SQS 
when this is fixed. 

SPB:
- Nothing to report.

FXE:
- FXE says that SQS will need to perform some tests: this will be discussed in separate round.

MID:
- All fine.

HED:
- Data recording on Saturday night was not saved. Shift crew realized at the end of the shift that it was not saved - main data for the shift. 
- ITDM investigated the issue after ticket arrived yesterday. It confirms data was not recorded and can't see why this happens as ticket arrived late. Even the folder of 
the data was not recorded. 
For the future: there is an assumption to look at the data every few minutes. ITDM will share procedures. Further discussions on flexibility for different systems/setups 
have to be done.
- HED says the 5Hz test must be tomorrow by the morning. XO states that currently this is only a test then we will see what happens. HED has this understanding.
- HED has recorded several hundreds of microns in horizontal jitter. XRO says that it is important to work together for investigating the jitter. 
HED will send the data to XRO and/or the coordinator of stability task force.

SCS:
- Not present.

SQS:
- Preparing next week user operation; still some problems with Zoom connection: SQS and ITDM will discuss the issue.

General Announcement:

• .

CTRL:

• issues SA3 vacuum loop
  • gets more and more diffcult to recover
    • problem not clear, PLC comminucation is lost without obvious reasons
    • no software changes took place from either CTRL nor EEE ... component issue?
• fastadc issues --> EEE
• mirror in SA3 not always following a move ommand XTD10 M1

DA:

EEE:

• OCD at HED --> hexapod
• fineDelay

VAC:

• preventive action in XTD10 for GATT scroll pump

XPD:

• SA2 imager XTD6 pop in 45 after M3 small mark on the imager --> imager still usable

DET:

• nothing to report

ITDM:

• OCD call data aggregator at MID
  • need to understand better how to avoid the situation
• users complained to maxwell service (DESY directly) about limited resources becuase of high load by other users
  • there are reserved nodes --> use them!
• issues with scratch
  • more than 300 TB had been accumulated within a relatively short time
    • it seems that data was calibrated to scratch not to proc
    • MID comments --> a special calib mode was used that could not save to proc which is why it was saved to scratch
• problem with one of the proxy servers --> degraded performance of data migration

XRO:

LAS:

• issue at SA1 with Galvo
  • after shift change to FXE, laser was not working --> Galvo was out of phase
  • FXE resetted something (not clear) and the laser came back
  • LAS investigating the issue

UNSYS: 

• nothing special
  • a couple of times the undulators opened suddenly without any command issued.
  • O. Henssler from DESY proveds the logs

SPB/SFX:

• coupling with SA1 and SA2 (SA2 using train on demand) is an issue
  • beam walks off the XTD9 XGM aperture
• offset mirrors M2
  • pitch movement got stiff
• smaract cotnrollers
  • most fixed by increasing timeout windows

FXE:

• shutter can be opened despite closed vavales

MID:

• used pulse on demand
  • users were happy

HED:

• damage on M3 mirror
  • due to focusing scheme --> 9keV and intermediate focus --> during alignment unfortunately the focusing point came too close to M3
• meeting with XPD and XRO at 11:30
• hexapod issue reported to EEE is not quite fixed
  • ´status is not reporting the correct status of the hexapod
  • hexapod started working at some point and fell back to weird operation later again
• CRL arm7 seems to be at the limit of the x-axis
  • if FEL stays where it is, XTD1 chamber might need to be moved
• JUNGFRAU issue
  • up to a point the data is fine in the preview
    • after some runs data seen in the preview is not written to file

SCS:

• nothing to report

SQS:

• l

General Announcement:

• .

CTRL:

• problems in SA3
  • VAC loop unstable, devices cannot be brought into a stable state --> what used to be a restart every  two weeks, cannot be recovered these days
  • issues with motors developing a short time out
    • somewhere there is a beavy load on Beckhoff loops
    • TM comments: at times the movements happen very late after issuing the command
  • influxDB sees high load
    • any modification of or running a new macro?
    • Please let CTRL know
  • the issue started with VAC --> devugging shoud start there

DA:

• extrafoam OCD call

EEE:

• working together with CTRL on SA3 issue
• OCD call from SCS but call was not correctly frowarded

VAC:

• not present

XPD:

• no issues

DET:

• OCD call from SCS --> was fixed

ITDM:

• nothing to report

XRO:

• nothing to report

LAS:

• no issues

UNSYS: 

• nthing to report

SPB/SFX:

• all fine

FXE:

• people on shift could not initiate a hutch search
• GABS access right missing (Gebietsabsuche)
  • introduced without informing FXE

MID:

• waiting for invitation AGIPD trainID solution

HED:

• JUNGFRAU issue presists
• HexaPOD is working again
  • work around

SCS:

SQS:

• DAQ issue
  • no data was recorded without
• vac is an issue but not endangering
• motor issue is vital to the instrument

General Announcement:

• .

CTRL:

• recurrent error reprted by LAS  --> drift compensation in SA2 --> need feedback
• minor issues reported instruments during startup

DA:

EEE:

• Issue on April 4th --> 4 PLCs shut down without a cause, (looks like power cut)
• MID crate went down at 9:00
• SA3 utca shut down (could be voltage drop, could be ehternet) (9:35)

VAC:

• VAC-OCD recovered SA2 vacuum state (valves, and ystem) (see EEE SA2 incident above)
• VAC obseved that the MID Mono warmed up, the cryo warmed up presumably after a PLCs update which interrupted an interloop communication

XPD:

• nothing report

DET:

• everything running fine

ITDM:

• SA1 baclony room checked
  • some racks lost power --> investifation going on (possibly happened on Sat. Apr 3rd 8:00 am)

XRO:

• incorrect motor velocity in SA3 --> fixed
•  

LAS:

• small issue in SA2 with slow feedback --> see CTRL report

UNSYS: 

• problem on SA1 central control node, after a power cycle aircoils and phase shifters were unresponsive

SPB/SFX:

• Laser Interlock issue was solved, power spike involved that reported by MEA3 ( klaus Witt).

FXE:

• no issues

MID:

• M1 mirror was moved completely out on 1st of Apr.
  • --> Naresh Kujala moved M1 out.

HED:

• Laser interlock issue on Tuesday
• with regards to the MID Mono cryo, there had been HED PLC updates
• calibration pipeline issue on AGIPD
• HED utca in optics hutch went off

SCS:

• no issues

SQS:

General Announcement:

- User Operations Meeting will resume on Friday

CTRL:

- Last week found small bug for the fastADC – short down time requested

DA:

- Nothing to report.

EEE:

- Fast electronics – are planning a ZZ

- Fuses issue – MID & HED medium priority crates fixed

- SASE 3 vacuum loop has to be updated – today afternoon or tomorrow around lunch timewill be coordinated by XO with EEE and MXL

VAC:

- Nothing to report

XPD:

- Start up of equipment

DET:

- Nothing to report.

ITDM:

- Nothing to report.

XRO:

- Nothing to report.

LAS:

- Alignment Laser interlock issue at SA3, being investigated

UNSYS:

- Nothing to report.

SPB:

- some motor issues after crate restarts in connection with but not caused by the works adressing the fuse issue

FXE:

- Nothing to report.

MID:

- all fine

HED:

- nothing.

SCS:

- Nothing to report.

SQS:

- Nothing to report.