Smart camera eases pressure on nursing staff

On Monday 2 March, IO+ was invited to the Catharina Hospital for a presentation on the Advance ForSee project. The project is a collaboration between Catharina Hospital, the Eindhoven University of Technology (TU/e) and Philips. They have developed AI-enabled cameras to detect changes in a patient’s vital signs after surgery. Less administrative work means nursing staff can spend more time with patients. “This saves an average of 10 minutes per patient,” says cardiologist and professor Lukas Dekker.

For the time being, the camera is intended solely to monitor cardiac patients. Following open-heart surgery, this group is particularly vulnerable to heart rhythm disorders and internal bleeding. Ultimately, according to the researchers, the aim is to deploy the camera for patients who have undergone procedures such as vascular surgery. “The model has to be adapted for each patient group,” says anaesthesiologist and professor Arthur Bouwman.

Detecting complications at an early stage

During recovery after heart surgery, around 10% of patients on the ward experience complications. “In addition, 40% of unexpected deaths in hospital occur on a general ward,” Dekker explains.

In intensive care, patients’ vital signs are continuously monitored using advanced equipment — the ‘gold standard’, according to Dekker. On a general ward, checks are carried out only every six to ten hours. What happens between these checks is crucial for detecting complications early, he says.

Due to budget cuts and a shortage of beds, doctors, nurses and other hospital staff are under considerable pressure. “We are expected to do more and more with less and less,” says Dekker. As a result, continuous monitoring is difficult, meaning complications are sometimes detected too late. Video monitoring offers an innovative and cost-effective solution.

After open-heart surgery, patients remain in hospital for at least five days, says researcher and cardiologist in training Dr Gijs van Steenbergen. Using signals from the cameras and the software, doctors can determine whether there are complications, whether care needs to be adjusted, or whether patients can be discharged earlier. “In this way, we can provide targeted and personalised care,” Van Steenbergen says.

How does the camera work?

Three cameras are aimed at the patient’s head and chest. One camera is used during the day and two (infrared and black-and-white) at night, explains Rik van Esch, a PhD candidate in electrical engineering at TU/e.

Images are recorded and analysed by the software 24/7, without any human involvement, Van Esch says. Using machine learning, the software can alert doctors and nursing staff to notable changes in a patient’s condition.

With the addition of AI, the system can also make predictions about possible complications based on the detected vital signs.

Requirements of patients and nursing staff

“Strangely enough, there was more reluctance among nursing staff than among patients regarding its use,” says Dekker. For nurses, it is essential that the system reduces the administrative burden rather than adding to it. Because they no longer have to take measurements themselves and enter the data manually, they can devote that time to patient care.

According to Dekker, the most important condition for patients is that they can ensure privacy whenever they wish — for example during personal care or family visits. To enable this, a mechanism has been devised that allows a cover to be placed over the cameras by pulling a cord. “Switching the cameras off completely was also an option, but shutting them down and restarting both the cameras and the system took too long,” Bouwman explains. With the cover in place, the cameras can continue running.

Once the equipment is fully operational, measurements may be stored for a maximum of six months after the patient has been discharged from hospital, Dekker says.

How far has the project progressed?

The project began in 2021 and is currently still in the development phase, Bouwman explains. So far, the doctors and researchers have tested Advance ForSee on “hundreds of patients”, according to Dekker. Scientific research and publication in academic journals are also required before various parties can grant approval, he adds. These include bodies that oversee privacy, ethics and accountability.

Like other AI and machine learning models, the software must learn to deal with a wide range of situations — for example, skin measurements in someone with facial hair or scars, or in different sleeping positions. It must also learn to account for different skin tones.

Your facial colour changes with your heartbeat — a phenomenon Bouwman refers to as micro-blushing. The cameras use this to measure the patient’s heart rate. The system works most effectively when the patient has a lighter complexion and when there is sufficient light in the room. “With patients who have darker skin, it is still difficult for the cameras to detect changes in facial colour,” Dekker notes.