SASE2 :
The unavailability of karabo and possibly missing cooling water is a problem for the tunnel components as well.
XGM in XTD6 : server error since about 11am. We don't know about the status of this device anymore, neither from DOOCS or karabo.
VAC-OCD is informed and will work with EEE-PLC-OCD to check and secure SA2 tunnel systems possibly via PLC.
Main concerns: XGM and cryo-cooler systems. Overheating of racks which contain XGM and monochromator electronics.

16h00 SA2 karabo is back online (partially) !

Another good news: 
together with VAC-OCD we see that the vacuum system of SA2 tunnel is fine, unaffected by the power cut and the karabo outage. Pressures seem fine.

Janusz Malka found for the SA3 GPFS servers that the balcony room redundancy cooling water didn't start, and no failure was reported.

16h25 DOC info
Cooling in rack rooms ok. GPFS servers are up, ITDM still checking. DAQ servers up.

16h30 XTD6-XGM is down.
Vacuum ok but MTCA crate cannot be reached / all RS232 connections not responding. XPD and Fini checking
but looks like an access is required. Raimund Kammering also checking crate.

16h30 FXE:
Optical laser shutter in error state. LAS-OCD informed. Therefore FXE is not taking beam now.
Actually the optical laser safety shutter between FXE optical laser hutch and experimental hutch has a problem and LAS_OCD cannot help. SRP to be contacted.

16h40 info ITDM:
recovery of SA2 GPFS hardware after cooling failure is completed. Everything seems to be working. Only few power supplies are damaged but there is redundancy.

16h40 info RC
SA2 beam ins now blocked by Big Brother since it receives an EPS remote power limit of 0 W. This info is now masked until all SA2 karabo devices will be back up. (e.g. bunchpattern-MDL is still down)

17h30 info by VAC-OCD
Vacuum systems SA1+SA2+SA3 are checked OK. Only some MDLs needed to be retstarted.
Some more problems with the cryo compressor on monochromators. 2 on HED and 1 on MID. Vacuum pressure is ok but if
users need the mono it will not be possible. VAC-OCD checks back with HED when they are back in the hutch.