Public release of AI to estimate biological sex from fundus images

The Japanese Ophthalmological Society and the National Institute of Informatics have developed and public-released an AI model to estimate an individual's sex from fundus images, using data collected by the Japan Ocular Imaging Registry (JOIR), a national ophthalmological database established with support from the Japanese Agency for Medical and Health Sciences (AMED).

We expect that researchers will apply this AI model in future research, and it helps elucidate the pathophysiology of diseases where the onset frequency differs depending on biological sex.

Deep learning has emerged as a significant breakthrough in machine learning and is widely used in artificial intelligence (AI), particularly in image recognition. Numerous reports suggest that the accuracy of image recognition has surpassed that of humans.

In 2017, NII established the Research Center for Medical Bigdata (RCMB) to develop medical assistance AI and created a database of medical image big data. NII also developed and operates an integrated cloud environment (cloud platform) equipped with high-capacity data servers and GPU servers for machine learning calculation.

Medical image analysis researchers across Japan, including Nagoya University, are connected to the cloud platform as part of NII's analysis team, and are collaborating to conduct research and development of various medical assistance AIs.

In 2019, JOS established the General Incorporated Association Japan Ocular Imaging Registry (JOI Registry or JOIR) to collect various ophthalmic data from ophthalmology-related facilities nationwide, aiming to promote and support research and development of medical assistance AI. The collected ophthalmic images in the JOIR database are anonymized and sent to NII's cloud infrastructure to develop medical support AI using available computing resources on the cloud infrastructure.

In recent years, the development of AI using medical images has progressed rapidly, and it has become clear that it is possible not only to determine the presence or absence of a disease but also to estimate the condition of the individual whose image was taken.

It has become clear that AI can be used to estimate age, sex, smoking status, blood sugar level, etc., from fundus images taken of the retina, which is the light-sensing part of the eye, and the information obtained from this is used by ophthalmologists. It has the potential to be used in medical research targeting not only regional diseases but also various systemic diseases. However, in the research reported so far, the developed AI has not been made public, so it has not been possible to use it for other research.

To address this issue, JOS collaborated with NII to develop an AI that estimates a person's age based on fundus images, and in January 2023, the AI model (estimation method) has been made available to a wide range of researchers without any charge. As the second step, we have developed an AI that estimates biological sex from fundus images and are making it available to the public free of charge.

The model we developed used 131,031 fundus images of people between the ages of 17 and 94 with sex labels as training data and performed deep learning with gender as the correct answer. During training, we used 16 popular deep learning models (DenseNet-121/169/201, Inception-V3, Inception-ResNet-V2, MobileNet, MobileNetV2, Xception, EfficientNet-B0/B1/B2/B3/B4/B5/B6/B7).

As a result, the accuracy with which the sex estimated from the fundus images of the validation data matched the actual gender was 92.0% (AUC 0.971) for the highest model, which is equivalent to the gender estimation models used for other races. Among these, we will publish two models: EfficientNet-B7, which was the most accurate, and MobileNet, which was lightweight and highly accurate.

By utilizing these AI models, researchers may elucidate the pathophysiology of diseases whose onset frequency differs depending on biological sex or supplement information when the sex information is missing in their research.