Digital pens provide new insight into cognitive testing results

During neuropsychological assessments, participants complete tasks designed to study memory and thinking. Based on their performance, the participants receive a score that researchers use to evaluate how well specific domains of their cognition are functioning.

Consider, though, two participants who achieve the same score on one of these paper-and-pencil neuropsychological tests. One took 60 seconds to complete the task and was writing the entire time; the other spent three minutes, and alternated between writing answers and staring off into space. If researchers analyzed only the overall score of these two participants, would they be missing something important?

"By looking only at the outcome, meaning what score someone gets, we lose a lot of important information about how the person performed the task that may help us to better understand the underlying problem," explains lead author Stacy Andersen, Ph.D., assistant professor of medicine at Boston University School of Medicine (BUSM).

Researchers with the Long Life Family Study (LLFS) used digital pens and digital voice recorders to capture differences in study participants' performance while completing a cognitive test and found that differences in 'thinking' versus 'writing' time on a symbol coding test might act as clinically relevant, early biomarkers for cognitive/motor decline.

Participants in the LLFS were chosen for having multiple siblings living to very old ages. Longevity has long been associated with an increased health span and thus these families are studied to better understand contributors to healthy aging. The participants were assessed on a number of physical and cognitive measures, including a symbol coding test called the Digit Symbol Substitution Test.

This timed test requires participants to fill in numbered boxes with corresponding symbols from a given key and assesses both cognitive (attention and processing speed) and non-cognitive factors (motor speed and visual scanning). To allow researchers to collect data about how a participant went about completing the task, the participants used a digital pen while completing the test. On the tip of this pen was a small camera that tracked what and when a participant wrote. The LLFS researchers divided the output from this digital pen into 'writing time' (the time the participant spent writing) and 'thinking time' (the time not spent writing) and looked at how these changed over the course of the 90-second test.

The researchers then identified groups of participants that had similar patterns of writing time and thinking time across the course of the test. They found that although most participants had consistent writing and thinking times, there were groups of participants who got faster or slowed down. "This method of clustering allowed us to look at other similarities among the participants in each group in terms of their health and function that may be related to differences in writing and thinking time patterns," said coauthor and lead biostatistician Benjamin Sweigart, a biostatistics doctoral student at Boston University School of Public Health. The researchers found that those who got slower in writing the symbols during the test had poorer physical function on tests of grip strength and walking speed. In contrast, those who changed speed in thinking time had poorer scores on memory and executive function tests suggesting that writing time and thinking time capture different contributors to overall performance on the test.

According to the researchers, these findings show the importance of capturing additional facets of test performance beyond test scores. "Identifying whether poor test performance is related to impaired cognitive function as opposed to impaired motor function is important for choosing the correct treatment for an individual patient" adds Andersen. "The incorporation of digital technologies amplifies our ability to detect subtle differences in test behavior and functional abilities, even on brief tests of cognitive function. Moreover, these metrics have the potential to be very early markers of dysfunction."