Model uses movie-watching fMRI and eye-tracking to predict cognitive scores

In a recent article published in the journal Psychoradiology, researchers from Northwestern Polytechnical University have unveiled "Attention-CensNet" (A-CensNet), a fusion model that predicts cognitive scores by amalgamating movie-watching functional magnetic resonance imaging (mfMRI) and eye-tracking data.

In this model, participants serve as nodes, mfMRI data are translated into node features, and eye-tracking details forge the connections between participants, creating graph edges. By employing multiple graphs as channels, A-CensNet merges graph embeddings to offer a consolidated interpretation.

Remarkably, A-CensNet outperforms models based solely on individual modalities and other leading techniques, underscoring the synergy between brain functionality and behavior in gauging cognitive traits.

This synergy is not only pivotal for understanding cognitive aberrations and potential psychiatric markers but also suggests potential in diagnosing disorders like autism, characterized by attention disparities. Although initial tests on emotion scores were less definitive, the incorporation of advanced algorithms, such as LSTM and BERT, in future iterations may enhance predictions.

While the model is primarily tailored for movie-watching, its application isn't limited. Combining other modalities like resting-state fMRI could predict other personal traits. Most importantly, integrating varied data sources as shown in this study can amplify cognitive prediction accuracy, crucial for distinguishing between patients and controls in disease research.