Analysis of large-scale neuronal imaging enables closed-loop investigation of neural dynamics

In a study published in Nature Neuroscience, Du Jiuli's group and Mu Yu's group at the Center for Excellence in Brain Science and Intelligence Technology of the Chinese Academy of Sciences (CAS), and Hao Jie at the Institute of Automation of CAS, utilized data processing techniques from astronomy and a field programmable gate array graphics processing unit (FPGA-GPU) hybrid architecture to perform real-time registration, signal extraction, and analysis on data streams of up to 500MB/s.

They achieved the real-time analysis of hundreds of thousands of neurons in the zebrafish brain for the first time, enabling decoding of the activities of arbitrarily selected neuron ensembles to control external devices.

Whole-brain neuron activity imaging is a powerful tool for deciphering the principles of the brain. However, its enormous data processing demands have become a bottleneck, making real-time analysis and closed-loop research of brain functions challenging.

Inspired by rapid radio burst detection technology in astronomy, the researchers employed the design of the FX system and utilized the flexibility of FPGA programming to establish an optical neural signal preprocessing system.

This system regularizes signals from optical sensors and sends them to a GPU-based real-time processing system for high-speed nonlinear registration, neural signals extraction and decoding, and obtaining feedback signals for controlling external devices.

The system generated feedback signals by continuously monitoring the activities of zebrafish whole-brain neurons with a feedback delay of less than 70.5 milliseconds.

The performance of the system was demonstrated in three closed-loop brain science research scenarios: real-time optogenetic stimulation locked to the activity of arbitrarily selected neuron ensembles, real-time visual stimulation locked to specific brain functional states, and virtual reality directly driven by neuronal activities in the brain.

By functionally clustering neurons in the whole brain, the spontaneous activity of the selected ensembles was used as a trigger signal to implement real-time optogenetic stimulation on target neuron ensembles. Compared to open-loop stimulation, closed-loop stimulation effectively activated downstream brain areas.

By real-time monitoring of the activity of the locus coeruleus (LC) norepinephrinergic system, visual stimulation was applied during the excitatory phase of LC neurons representing the animal's awake state, resulting in stronger responses of neurons across the brain. This indicated that brain states modulate the processing of visual information, and that closed-loop sensory stimulation facilitate the study of the interaction between internal brain states and the external environment.

Real-time dimensionality reduction of all brain neurons' activities to multiple neuron ensembles and closed-loop coupling with the visual environment enables the establishment of a virtual reality system directly driven by the activities of neurons in the brain. In this virtual reality system, the gain coupling between the neuronal activity and the environment can be adjusted arbitrarily, allowing the neuron ensemble controlling the environment to adaptively adjust its output based on gain change.

Leveraging the real-time analysis of big data streams and high-throughput whole-brain imaging technology, the researchers will screen out neuronal activity characteristics suitable for optical brain-machine interface (BMI), uncover the underlying mechanisms, and develop more efficient optical BMI technologies.

This study marks a crucial step in the application of techniques such as virtual reality based on whole-brain cellular-resolution optical imaging and optogenetic control in the field of closed-loop whole-brain-scale research.