Views & Comments

Optically Digitalized Holography: A Perspective for All-Optical Machine Learning

  • Min Gu a ,
  • Xinyuan Fang a,b ,
  • Haoran Ren c ,
  • Elena Goi a
Expand
  • a Laboratory of Artificial-Intelligence Nanophotonics, School of Science, RMIT University, Melbourne, VIC 3001, Australia
  • b National Laboratory of Solid State Microstructures, College of Engineering and Applied Sciences, Nanjing University, Nanjing 210093, China
  • c Chair in Hybrid Nanosystems, Nanoinstitute Munich, Faculty of Physics, Ludwig-Maximilians-University Munich, Munich 80539, Germany

Published date: 14 Jun 2019

Copyright

2019

Cite this article

Min Gu , Xinyuan Fang , Haoran Ren , Elena Goi . Optically Digitalized Holography: A Perspective for All-Optical Machine Learning[J]. Engineering, 2019 , 5(3) : 363 -365 . DOI: 10.1016/j.eng.2019.04.002

Holography, which was invented by Dennis Gabor in 1948, offers an approach to reconstructing both the amplitude and phase information of a three-dimensional (3D) object [1]. Since its invention, the concept of holography has been widely used in various fields, such as microscopy [2], interferometry [3], ultrasonography [4], and holographic display [5]. Optical holography can be divided into two steps: recording and reconstruction. A conventional hologram is recorded onto a photosensitive film as the interference between an object beam carrying the 3D object information and a reference beam. Thereafter, the original object wavefront is reconstructed in the 3D image space by illuminating the reference beam on the recorded hologram.
Digital holography was invented by Brown and Lohmann in 1966, marking a milestone breakthrough in optical holography based on computer-generated holograms (CGHs) [6]. Instead of performing complex two-step optical holography, CGHs provide a simple way to obtain the amplitude and phase information of a digital hologram based on various computational algorithms. CGH-based digital holography has recently been realized through both passive [7] and active photonic devices [8].
The advent of the computer-addressed spatial light modulator (SLM) opens up the possibility of dynamic digital holography that is capable of rapidly switching holograms within only a few microseconds [9]. SLM-assisted digital holography has been applied in 3D displays [10], holographic encryption [11], digital holographic microscopy [12], optical data storage [13], optical trapping [14], and so forth. However, several compelling challenges still remain for digital holography, including a small field of view, low resolution, narrow bandwidth, optically thick holograms, and multiple diffraction orders.
To overcome these challenging issues, high-resolution and optically thin metasurfaces have been put forward in order to digitalize CGHs [15]. Unfortunately, the formidable complexity and high cost of the fabrication methods—namely, electron-beam lithography and focused ion-beam lithography—limit the practical applications of small metasurface holograms. Optically digitalized holography (ODH) has recently been proposed and demonstrated [1618], opening up the possibility of using optical methods to generate high-resolution, large-scale, and cost-effective holograms [1921]. The new method is based on the vectorial Debye diffraction theory [22] in conjunction with inversed Fourier transform [2325].
The 3D direct laser writing technique has been experimentally used in ODH to optically digitalize CGHs in different photosensitive materials. A tightly-focused femtosecond laser beam is scanned on a photosensitive material to print 3D nanostructures, where different-sized nanostructures correspond to multilevel amplitude and/or phase modulation in the CGHs. It is notable that the recent development of super-resolution direct laser writing techniques holds great promise for digitalizing ultrahigh-definition CGHs with extremely small pixels [26]. On the other hand, galvo scan mirrors and diffraction-limited two-dimensional (2D) [23,24] and 3D [25] multifocal arrays have enabled fast and parallel direct laser writing with a throughput that is increased by orders of magnitude. As a result, ODH-based holograms with high resolution and a large size enable floating displays of holographic images with an ultra-wide viewing angle and a high spatial bandwidth product. In this context, an ODH hologram with a resolution of 550 nm was fabricated in graphene oxides [16,17] and photoresist [18], resulting in a 3D display with an ultra-wide viewing angle of 52° [17]. Moreover, an ultra-thin ODH hologram with an optically thin thickness of 20 nm was fabricated by exploiting multi-reflection phase accumulation in a topological insulator thin film [27].
Recently, artificial intelligence has attracted a surge of interest, due to its widespread application in medical image analysis [28], molecular and material science [29], speech recognition [30], and so forth. It is envisioned that optical holography can provide great advantages to artificial intelligence. Pioneering work extending optical holography to artificial neural networks dates back to the 1990s [31]; in that work, the activity of each neuron was coded in the amplitude or intensity of optical beams. Due to the angle selectivity in Bragg diffraction, a complex mapping relationship in neurons can be represented by a 3D volume hologram based on the multiplexing of holographic gratings. However, the lack of practical devices at that time that could implement a holographic device acting as complex neurons prevented the advancement of this idea. Recently, ODH has enabled the fabrication of high-resolution holographic devices performing the function of artificial neural networks. All-optical machine learning using diffractive deep neural networks has been successfully demonstrated to perform image classification in the terahertz (THz) band [32]. To achieve the learning function, multilayer holograms were computationally designed based on advanced deep-learning algorithms and were experimentally fabricated by 3D printing.
Extending 3D printing [32] to 3D high-resolution laser printing [26,33] can provide an all-optical machine learning chip ranging from the THz to visible regions (Fig. 1). The merging of ODH with artificial intelligence will lead to significant breakthroughs in both fundamental research and practical holographic applications in future. We envisage that extending the working wavelength from the THz to visible frequency range will open up new perspectives for applications such as a smarter imager [34], light fidelity (Li-Fi) [35], and security access. However, the implementation of a high-definition holographic display based on artificial intelligence presents a formidable task for computation that lies significantly beyond current capabilities; therefore, new computational algorithms must be developed to mitigate this challenge. We have thus embarked on an exciting journey to explore new artificial intelligence-based ODH. Alternatively, optical machine leaning can be implemented on on-chip nanophotonic circuits [36]. The combination of these two approaches may provide an entirely new platform for neural technology and engineering in brain-like exploration that can benefit the development of new medical procedures for curing mental disorders—which currently demand an approximate annual cost of $1 trillion around the world and $90 billion in China.
Fig. 1 (a) All-optical machine learning based on a multilayered ODH chip. (b) A monolithic design combines four different holographic layers that work collectively to perform image classification. In this example, the multilayered chip can classify the animal images, recognizing the butterfly as an insect. (c) Each layer of the chip consists of an ODH. (d) Schematic illustration of an ODH fabricated by high-resolution 3D direct laser writing, which enables the extension of the operation wavelength from the THz to visible region for a diverse range of applications.

Full size|PPT slide

Acknowledgements

Min Gu acknowledges support from the Australian Research Council (ARC) through the Discovery Project (DP180102402). Xinyan Fang acknowledges support from a scholarship from the China Scholarship Council (201706190189). Haoran Ren acknowledges financial support from the Humboldt Research Fellowship from the Alexander von Humboldt Foundation.
[1]
Gabor D.. A new microscopic principle. Nature. 1948; 161(4098): 777.

[2]
Gabor D.. Microscopy by reconstructed wave-fronts. Proc R Soc Lond A Math Phys Sci. 1949; 197(1051): 454-487.

[3]
Powell R.L., Stetson K.A.. Interferometric vibration analysis by wavefront reconstruction. J Opt Soc Am. 1965; 55(12): 1593-1598.

[4]
Baum G., Stroke G.W.. Optical holographic three-dimensional ultrasonography. Science. 1975; 189(4207): 994-995.

[5]
Leith E.N., Upatnieks J.. Wavefront reconstruction with diffused illumination and three-dimensional objects. J Opt Soc Am. 1964; 54(11): 1295-1301.

[6]
Brown B.R., Lohmann A.W.. Complex spatial filtering with binary masks. Appl Opt. 1966; 5(6): 967-969.

[7]
Verbeeck J., Tian H., Schattschneider P.. Production and application of electron vortex beams. Nature. 2010; 467(7313): 301-304.

[8]
Zhang Z., You Z., Chu D.. Fundamentals of phase-only liquid crystal on silicon (LCOS) devices. Light Sci Appl. 2014; 3: e213.

[9]
Javidi B., Kuo C.J.. Joint transform image correlation using a binary spatial light modulator at the Fourier plane. Appl Opt. 1988; 27(4): 663-665.

[10]
Downing E., Hesselink L., Ralston J., Macfarlane R.. A three-color, solid-state, three-dimensional display. Science. 1996; 273(5279): 1185-1189.

[11]
Li J., Kamin S., Zheng G., Neubrech F., Zhang S., Liu N.. Addressable metasurfaces for dynamic holography and optical information encryption. Sci Adv. 2018; 4(6): eaar6768.

[12]
Rosen J., Brooker G.. Non-scanning motionless fluorescence three-dimensional holographic microscopy. Nat Photonics. 2008; 2(3): 190-195.

[13]
Heanue J.F., Bashaw M.C., Hesselink L.. Volume holographic storage and retrieval of digital data. Science. 1994; 265(5173): 749-752.

[14]
Grier D.G.. A revolution in optical manipulation. Nature. 2003; 424(6950): 810-816.

[15]
Ni X., Kildishev A.V., Shalaev V.M.. Metasurface holograms for visible light. Nat Commun. 2013; 4: 2807.

[16]
Li X., Zhang Q., Chen X., Gu M.. Giant refractive-index modulation by two-photon reduction of fluorescent graphene oxides for multimode optical recording. Sci Rep. 2013; 3: 2819.

[17]
Li X., Ren H., Chen X., Liu J., Li Q., Li C., . Athermally photoreduced graphene oxides for three-dimensional holographic images. Nat Commun. 2015; 6: 6984.

[18]
Li X., Liu J., Cao L., Wang Y., Jin G., Gu M.. Light-control-light nanoplasmonic modulator for 3D micro-optical beam shaping. Adv Opt Mater. 2016; 4(1): 70-75.

[19]
Wang S., Ouyang X., Feng Z., Cao Y., Gu M., Li X.. Diffractive photonic applications mediated by laser reduced graphene oxides. Opto-Electron Adv. 2018; 1(2): 170002.

[20]
Zhang Q, Yu H, Barbiero M, Wang B, Gu M. Artificial neural networks enabled by nanophotonics. Light Sci Appl (In press).

[21]
Gu M., Zhang Q., Lamon S.. Nanomaterials for optical data storage. Nat Rev Mater. 2016; 1: 16070.

[22]
Gu M.. Advanced optical imaging theory.

[23]
Lin H., Jia B., Gu M.. Dynamic generation of Debye diffraction-limited multifocal arrays for direct laser printing nanofabrication. Opt Lett. 2011; 36(3): 406-408.

[24]
Gu M., Lin H., Li X.. Parallel multiphoton microscopy with cylindrically polarized multifocal arrays. Opt Lett. 2013; 38(18): 3627-3630.

[25]
Ren H., Lin H., Li X., Gu M.. Three-dimensional parallel recording with a Debye diffraction-limited and aberration-free volumetric multifocal array. Opt Lett. 2014; 39(6): 1621-1624.

[26]
Gan Z., Cao Y., Evans R.A., Gu M.. Three-dimensional deep sub-diffraction optical beam lithography with 9 nm feature size. Nat Commun. 2013; 4: 2061.

[27]
Yue Z., Xue G., Liu J., Wang Y., Gu M.. Nanometric holograms based on a topological insulator material. Nat Commun. 2017; 8: 15354.

[28]
Litjens G., Kooi T., Bejnordi B.E., Setio A.A.A., Ciompi F., Ghafoorian M., . A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42: 60-88.

[29]
Butler K.T., Davies D.W., Cartwright H., Isayev O., Walsh A.. Machine learning for molecular and materials science. Nature. 2018; 559(7715): 547-555.

[30]
Hinton G., Deng L., Yu D., Dahl G.E., Mohamed A., Jaitly N., . Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag. 2012; 29(6): 82-97.

[31]
Psaltis D., Brady D., Gu X.G., Lin S.. Holography in artificial neural networks. Nature. 1990; 343(6256): 325-330.

[32]
Lin X., Rivenson Y., Yardimci N.T., Veli M., Luo Y., Jarrahi M., . All-optical machine learning using diffractive deep neural networks. Science. 2018; 361(6406): 1004-1008.

[33]
Goi E, Gu M. Laser printing of a nano-imager to perform full optical machine learning [presentation]. In: Conference on Lasers and Electro-Optics/Europe; 2019 Jun 23–27; Munich, Germany; 2019.

[34]
Li L., Ruan H., Liu C., Li Y., Shuang Y., Alù A., . Machine-learning reprogrammable metasurface imager. Nat Commun. 2019; 10(1): 1082.

[35]
Haas H., Yin L., Wang Y., Chen C.. What is LiFi?. J Lightwave Technol. 2015; 34(6): 1533-1544.

[36]
Shen Y., Harris N.C., Skirlo S., Prabhu M., Baehr-Jones T., Hochberg M., . Deep learning with coherent nanophotonic circuits. Nat Photonics. 2017; 11: 441-446.

Outlines

/