Convolutional Neural Networks (CNNs), with its exceptional image recognition capa­bilities, have performed outstandingly in the field of AI and notably within platforms like ChatGPT. Recently, a team of Chinese researchers from University of Shanghai for Science and Techno­logy have success­fully introduced the concept of CNNs into the field of optics and realized convolu­tional all-optical neural network, bringing revolu­tionary progress to AI imaging technology.

Led by Min Gu and Qiming Zhang from School of Artificial Intelligence Science and Technology (SAIST) at the University of Shanghai for Science and Techno­logy (USST), the research team has developed an ultrafast convolu­tional optical neural network (ONN), which achieves efficient and clear imaging of objects behind scattering media without relying on the optical memory effect. The core of CNNs, convolutional operations, extracts local features from images and constructs more complex and abstract feature represen­tations layer by layer, greatly advancing the fields of image processing and pattern recognition. However, applying the concept of convolution network operations to the field of optics faces the challenge of converting electronic signals into optical signals. The research team ingeniously designed an all-optical solution, performing convolution network operations directly in the optical domain, eliminating the cumbers­ome signal conversion process, and achieving true optical computing speeds.

The key to this research is the construction of a multi-stage convolutional ONN, composed of multiple parallel cores capable of operating at the speed of light, directly extracting features from scattered light for rapid image recon­struction. This process not only greatly improves imaging speed but also significantly enhances image quality, making imaging in complex scattering environ­ments possible. Moreover, the convolutional ONN's compu­tational speed reaches 1.57 Peta operations per second (POPS), providing robust support for real-time dynamic imaging.

Another highlight of this research is its multi­tasking capability. By simply adjusting the network structure, the same convolutional ONN can perform a variety of different image processing tasks, such as classi­fication and reconstruction, concurrently – a first in the field of optical artificial intelli­gence. Qiming Zhang noted, “This combination of flexibility and efficiency not only underscores the importance of convo­lutional networks in artificial intelligence but also opens up new avenues for optical imaging techno­logy.” The advent of this research is not only a successful trans­plantation of convo­lutional neural networks into the optical field but also a significant boost to AI imaging techno­logy. Min Gu stated, “In the near future, convo­lutional optical neural networks will play an increasingly vital role in autonomous driving, robotic vision, and medical imaging.” (Source: USST)

Reference: Y. Zhang et al.: Memory-less scattering imaging with ultrafast convolutional optical neural networks, Sci. Adv. 10, adn2205 (2024); DOI: 10.1126/sciadv.adn2205

Link: Institute of Photonic Chips, University of Shanghai for Science and Technology, Shanghai, China.