Collaborative psychology research finds AI chatbot can infer someone's personality

Personality tests are a popular job candidate selection tool; however, they often rely on self-reported measures, which can be biased, inaccurate or fake. A collaborative research team that includes a Kansas State University psychologist found that artificial intelligence could help with this task.

Tianjun Sun, assistant professor of psychological sciences in K-State's College of Arts and Sciences, is part of the research team that conducted a study with more than 1,500 undergraduate students to determine whether a person's conversation with an AI chatbot could extract personality-related information.

The study, "How well can an AI chatbot infer personality?: Examining psychometric properties of machine-inferred personality scores," was recently published in the Journal of Applied Psychology.

Sun is a co-leading author with researchers from Auburn University, the University of Illinois Urbana-Champaign, the University of South Florida-St. Petersburg and the United States Air Force Academy. The American Psychological Association also designated the article as a high-impact article.

To examine the properties of the AI personality scores, the researchers had each participant engage in a conversation with an AI chatbot for 20–30 minutes and complete self-reported personality assessments for comparison. Grade point average and peer-rated college adjustment were used to measure personality outcomes. Researchers also used participants' chat responses to train the AI to predict the self-reported personality scores.

The researchers found that the AI personality scores satisfied many of the standards that help researchers evaluate the quality of a psychological test. The scores were consistent and similar to the self-reported personality measures. In some analyses, like when measuring the ability of extroversion to predict GPA and college adjustment, AI scores had predictive power that extended beyond self-reported personality scores.

"Our study provides promising support for the future of machine learning techniques in personality assessment," said Sun. "As these techniques become more common in employee selection, organizations should be aware that this might help them save time and still accurately measure personality. More research is needed to learn whether it is fair to minority groups or susceptible to faking and its legal and ethical implications."

Sun is an expert in industrial-organizational psychology and studies personnel selection, individual differences and quantitative methods. She is passionate about the responsible use of psychometric tools and advanced technology to improve psychological sciences and solve organizational problems.