After going in and reconnecting the camera, we observe att. 1.
According to logs the camera device server was restarted around 11pm, however the camera instance was never reinstantiated after the restart. The Karabo GUI did not indicate that properly, instead showed a frozen image that looked as if the camera works properly.
(see history of camera state in att. 2)
Device Server log shows that after camera connection was lost, the device server restarted?! Was this restart done on purpose?
2023-11-23T23:02:22.499 ERROR karabo.AravisBasler2Camera : SQS_AQS_UPP/CAM/A2A1920_2: Could not synchronize timestamp: GigEVision write_memory timeout
2023-11-23T23:02:27.504 ERROR karabo.AravisCamera : SQS_AQS_UPP/CAM/A2A1920_2: arv_device_get_float_feature_value failed: GigEVision read_memory timeout
2023-11-23T23:02:30.007 WARN karabo.AravisCamera : SQS_AQS_UPP/CAM/A2A1920_2: Control of the camera exflqr50765 is lost
INFO Successfully loaded plugin: "libslsDetectors.so"
INFO Successfully loaded plugin: "libaravisCameras.so"
2023-11-23T23:02:33.773 INFO openMq : Opened TCP connection to broker %s:%d.
2023-11-23T23:02:33.775 INFO openMq : Connection ping enabled (ping interval = %d second).
2023-11-23T23:02:33.778 INFO openMq : Connection connected to broker
2023-11-23T23:02:33.778 INFO karabo.net.JmsConnection : Opened TCP connection to broker tcp://sa3-br-sys-broker-2:7777
2023-11-23T23:02:33.779 INFO karabo.core.DeviceServer : Logfiles are written to: "/scratch/xctrl/karabo/var/log/cppServer/sqs_nqs_vmi_cam/device-server.log"
2023-11-23T23:02:34.785 INFO karabo.xms.SignalSlotable : Instance starts up in topic 'SQS' as 'cppServer/sqs_nqs_vmi_cam' - Karabo 2.16.6
2023-11-23T23:02:34.786 INFO cppServer/sqs_nqs_vmi_cam : DeviceServer starts up with id: cppServer/sqs_nqs_vmi_cam
2023-11-23T23:02:34.786 INFO cppServer/sqs_nqs_vmi_cam : Updated list of devices available: AlarmService,AravisBasler2Camera,AravisBaslerCamera,AravisCamera,AravisPhotonicScienceCamera,CameraImageSource,DataLoggerManager,FileDataLogger,FileLogReader,Gotthard2Control,Gotthard2Receiver,GotthardControl,GotthardReceiver,GuiServerDevice,ImageSource,InfluxDataLogger,InfluxLogReader,JungfrauControl,JungfrauReceiver,PropertyTest
2023-11-23T23:02:34.934 INFO karabo.core.DeviceServer : Sending instance update as new device plugins are available: AravisBasler2Camera,AravisBaslerCamera
2023-11-23T23:02:34.934 INFO cppServer/sqs_nqs_vmi_cam : Keep watching directory: "/scratch/xctrl/karabo/plugins" for Device plugins
2023-11-23T23:02:34.934 INFO cppServer/sqs_nqs_vmi_cam : Starting Karabo DeviceServer (pid: 235604) on host: sqs-rr-sys-con-cams-1, serverId: cppServer/sqs_nqs_vmi_cam, Broker: tcp://sa3-br-sys-broker-2:7777
2023-11-23T23:02:35.073 INFO karabo.core.DeviceServer : Successfully connected to time server 'SQS_RR_UTC/TSYS/TIMESERVER'
==> Status of Taylor Cone unknown for runs 334 - 401
We had to physically unplug and replug the ethernet cable to the camera to get it to work again. |