Extracting blood-induced color changes on the face for non-contact heart rate estimation

Heart rate (HR) estimation is an essential component of health monitoring, and provides useful insights regarding the physiological and emotional state of humans. In the past decade or so, researchers have explored newer approaches for contactless HR estimation, primarily to overcome the discomfort or dermatitis associated with conventional methods that require physical contact. Non-contact HR estimation using cameras is an example of one such method.

The method focuses on the blood volume pulse (BVP), that causes slight temporal changes in facial skin color captured in videos. By examining these color variations, it becomes possible to estimate HR. However, due to the small magnitude of these color changes, the accuracy of HR estimation is adversely affected by facial movements, ambient lighting variations, and noise.

To address these challenges, a team of researchers from Japan have now developed a novel method that leverages the temporal characteristics of the blood pulse. Importantly, it builds on the ability of the pulse to exhibit quasi-periodic behavior, which distinguishes it from noise artifacts.

The study was led by Dr. Yoshihiro Maeda, Junior Associate Professor, from the Department of Electrical Engineering at the Tokyo University of Science and is published in IEEE Access. Professor Takayuki Hamamoto and Kosuke Kurihara from the Tokyo University of Science and Associate Professor Daisuke Sugimura from the Department of Computer Science, Tsuda University, were also a part of this study.

The proposed method utilizes dynamic mode decomposition (DMD), a technique that analyzes spatio-temporal structures in multi-dimensional time-series signals. It also employs adaptive selection of the optimal spatio-temporal structure based on medical knowledge of HR frequencies.

"Our method, unlike previous applications of DMD, effectively models and extracts the BVP signal by incorporating physics-informed DMD in a time-delay coordinate system, taking into account the nonlinearity and quasi-periodicity of the BVP dynamics," explains Kosuke Kurihara, a Ph.D. student.

The proposed method relies solely on tracking time-series data from videos of a person's face, eliminating the need for any attached detectors on the person's body. In this method, the video time-series of the face, monitoring continuous changes, are converted into RGB time-series signals, which helps in extracting information of blood volume changes occurring beneath the skin. After effectively dealing with noise or misinformation that might creep into the data, the observed RGB signals are then converted to pulse wave information data.

Using the DMD method in a time-delayed coordinate system with conservative dynamics modeling, pulse waves containing major and accurate information can be extracted to estimate HR.

To demonstrate the efficacy of this method, the researchers used 67 facial videos from three publicly available datasets—namely TokyoTech Remote PPG dataset, MR-NIRP dataset, and UBFC-RPPG dataset. The results of this method were then compared with other non-contact HR estimation methods, including DistancePPG, SparsePPG, SAMC, Hierarchical, and MTTS-CAN.

Interestingly, the proposed method adaptively selects the dynamic mode that contains the most pulse wave components, based on the knowledge of the typical range of pulse wave components. As a result, the method showed a 36.5% improvement in estimation accuracy compared to conventional methods, especially in scenes with ambient light fluctuations.

"This achievement is expected to play a significant role as a fundamental technology for vital monitoring systems in the medical and fitness fields. The breakthrough contactless method holds great potential for non-contact heart rate estimation in various applications, such as remote health monitoring and physiological assessments," concludes Dr. Maeda.

The research findings provide new possibilities for enhancing health care technologies and improving overall patient comfort and well-being. Going ahead, further research will be needed to explore techniques that incorporate multispectral information, which can contribute to reducing noise and improving the accuracy of the method.