Smart camera reduces unexpected complications in hospitals

Suppose you are a heart or cancer patient and you have had surgery. Once the surgery is successful, and you no longer need to stay in the Intensive Care or Medium Care Unit, you are usually transferred to a regular nursing ward to continue your recovery.

There you will not be constantly monitored. However, even then it is vital that doctors and nurses keep an eye out for complications. In most hospitals someone will come to your bedside every six to ten hours to take your pulse and check your breathing.

That might sound like a sensible arrangement. However, the time between those checks is quite long, with the risk that something might happen in the meantime. Moreover, the check-ups can be quite annoying, especially when you are sleeping. Wouldn't it be nice to have a system that could remotely monitor you permanently, without bothering you?

Intelligent camera

Researchers at TU/e and the Catharina Hospital in Eindhoven believe they have found the solution: an intelligent camera that constantly checks your heartbeat and breathing, and sends a signal to doctors and nurses when something is wrong.

The researchers believe this will reduce complications and unexpected deaths, about 40 percent of which occur in a regular nursing ward.

"The camera uses artificial intelligence," says Sveta Zinger, associate professor at the Departement of Electrical Engineering who specializes in medical image analysis. "The camera is focused on the patient's chest and face. This allows it to see exactly if there are any subtle changes in the color of your face or the movement of your breast, which could indicate possible complications."

From lab to hospital

The camera has been extensively tested in the laboratory over the past few years. "That showed that a smart camera can remotely measure vital data, such as heart rate, breathing, temperature and the amount of oxygen in the blood. Using machine learning, we can then interpret that data as well," Zinger explains.

"To see if the camera also works in a clinical setting, we are going to conduct an experiment in the ICU department at Catharina Hospital. Ten patients and five healthy volunteers will be monitored non-stop for eight hours, during which time we will combine the data from the camera with the measurement data from the ICU equipment, the 'gold standard' when it comes to monitoring. This will allow us to train the camera in situations it is not yet familiar with, for example when there is less light, or when a patient turns their face away."

Anonymous pixels

For those who might have reservations about having a camera next to their bed, Zinger emphasizes that the privacy of the patient is paramount. All participants in the study have to give their express permission, and there is a button on each camera that allows the patient to turn the device off, if he or she so desires. 

"What’s more, the patients are not personally identifiable in the video recordings. For the algorithms to do their work, they only need anonymous pixels," says the researcher. Zinger also knows from experience that many patients have no problem at all with camera monitoring, because they also see the benefits.

Benefits for patients

The Eindhoven-based Catharina Hospital sees great benefits for the smart camera. "Our hospital specializes in cardiovascular diseases and also treats many cancer patients. It is particularly important for this group that they are closely monitored. Complications can adversely affect recovery and in some cases can even be fatal. Timely intervention is therefore of great importance," says anesthesiologist Arthur Bouwman.

Moreover, a camera is less stressful than regular checks with an electrocardiogram, in which the patient has electrodes taped to his body, or a saturation meter, in which a clip is slipped over the finger to measure the amount of oxygen in your blood. "The patient can now freely move around and is no longer restricted by cables with electrodes. This is great news, because nothing is more damaging to a patient than lying still in bed," says Bouwman.

The doctor expects that the camera can be used mainly on nursing wards. "I don't think the camera can ever replace the classic monitoring in the ICU and medium-care. There is too much 'noise' in those places for that, which distorts the image. But we do want to start testing the camera for home monitoring".

False alarm

The first results of the study are expected in the summer of 2021. Ultimately, the FORSEE project, an initiative of the Eindhoven MedTech Innovation Center (e/MTIC ), and subsidized by ZonMw, the Dutch Research Council, the Heart Foundation and the Dutch CardioVascular Alliance (DCVA), will last five years.

The researchers will also be looking at the effect of the cameras on the patients (do they like the cameras or not?) and on the nurses (are the cameras easy to use and don't they set off to many false alarms?).

The project will also examine how machine learning can help doctors and nurses interpret the vast amounts of data collected by the cameras, so they can make better clinical decisions.