Acta Aeronautica et Astronautica Sinica ›› 2024, Vol. 45 ›› Issue (20): 630439.doi: 10.7527/S1000-6893.2024.30439
• Aeronautics Computing and Simulation Technique • Previous Articles
Junshen LI1,2,3, Xiangyan MENG1,2,3, Nuannuan SHI1,2,3, Wei LI1,2,3, Ninghua ZHU1,2,3, Ming LI1,2,3()
Received:
2024-03-25
Revised:
2024-04-19
Accepted:
2024-05-23
Online:
2024-06-06
Published:
2024-05-29
Contact:
Ming LI
E-mail:ml@semi.ac.cn
Supported by:
CLC Number:
Junshen LI, Xiangyan MENG, Nuannuan SHI, Wei LI, Ninghua ZHU, Ming LI. Intelligent processing of optical neural networks: Technological evolution and future prospects[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(20): 630439.
1 | FUKUSHIMA K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4): 193-202. |
2 | LECUN Y, BOSER B, DENKER J, et al. Handwritten digit recognition with a backpropagation network[J]. Advances in Neural Information Processing Systems, 1989, 1 (4): 541-551. |
3 | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2): 1097-1105. |
4 | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[DB/OL]. arXiv preprint: 1409.1556, 2014. |
5 | SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]∥ 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2015: 1-9. |
6 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2016: 770-778. |
7 | HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[DB/OL]. arXiv preprint: 1704.04861, 2017. |
8 | CHANG J L, SITZMANN V, DUN X, et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification[J]. Scientific Reports, 2018, 8: 12324. |
9 | ARRATIA A, SEPÚLVEDA E. Convolutional neural networks, image recognition and financial time series forecasting[C]∥Workshop on Mining Data for Financial Applications. Cham: Springer, 2020: 60-69. |
10 | SZEGEDY C, TOSHEV A, ERHAN D. Deep neural networks for object detection[J]. Advances in Neural Information Processing Systems, 2013, 26. |
11 | HE S F, LAU R W H, LIU W X, et al. SuperCNN: a superpixelwise convolutional neural network for salient object detection[J]. International Journal of Computer Vision, 2015, 115(3): 330-344. |
12 | LI Q, CAI W D, WANG X G, et al. Medical image classification with convolutional neural network[C]∥ 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV). Piscataway: IEEE Press, 2014: 844-848. |
13 | LU L, ZHENG Y F, CARNEIRO G, et al. Deep learning and convolutional neural networks for medical image computing[J]. Advances in Computer Vision and Pattern Recognition, 2017, 10: 978-981. |
14 | KUMAR A, KIM J, LYNDON D, et al. An ensemble of fine-tuned convolutional neural networks for medical image classification[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(1): 31-40. |
15 | YAMASHITA R, NISHIO M, DO R K G, et al. Convolutional neural networks: an overview and application in radiology[J]. Insights into Imaging, 2018, 9(4): 611-629. |
16 | ZHOU C, XU D M, CHEN L, et al. Evaluation of fish feeding intensity in aquaculture using a convolutional neural network and machine vision[J]. Aquaculture, 2019, 507: 457-465. |
17 | WILLIAMS H A M, JONES M H, NEJATI M, et al. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms[J]. Biosystems Engineering, 2019, 181: 140-156. |
18 | LIN X, RIVENSON Y, YARDIMCI N T, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406): 1004-1008. |
19 | SHAHVERDY M, FATHY M, BERANGI R, et al. Driver behavior detection and classification using deep convolutional neural networks[J]. Expert Systems with Applications, 2020, 149: 113240. |
20 | CHEN L, LIN S B, LU X K, et al. Deep neural network based vehicle and pedestrian detection for autonomous driving: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(6): 3234-3246. |
21 | SIROHI D, KUMAR N, RANA P S. Convolutional neural networks for 5G-enabled intelligent transportation system: a systematic review[J]. Computer Communications, 2020, 153: 459-498. |
22 | KASTER J, PATRICK J, CLOUSE H S. Convolutional neural networks on small unmanned aerial systems[C]∥ 2017 IEEE National Aerospace and Electronics Conference (NAECON). Piscataway: IEEE Press, 2017: 149-154. |
23 | ZHOU L M, YAN H X, SHAN Y Z, et al. Aircraft detection for remote sensing images based on deep convolutional neural networks[J]. Journal of Electrical and Computer Engineering, 2021: 4685644. |
24 | DING P, ZHANG Y, DENG W J, et al. A light and faster regional convolutional neural network for object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 141: 208-218. |
25 | RADOVIC M, ADARKWA O, WANG Q S. Object recognition in aerial images using convolutional neural networks[J]. Journal of Imaging, 2017, 3(2): 21. |
26 | GALLEGO A J, PERTUSA A, GIL P. Automatic ship classification from optical aerial images with convolutional neural networks[J]. Remote Sensing, 2018, 10(4): 511. |
27 | XU X Y, TAN M X, CORCORAN B, et al. 11 TOPS photonic convolutional accelerator for optical neural networks[J]. Nature, 2021, 589: 44-51. |
28 | MENG X Y, SHI N N, LI G Y, et al. Optical convolutional neural networks: methodology and advances (invited)[J]. Applied Sciences, 2023, 13(13): 7523. |
29 | SLUDDS A, BANDYOPADHYAY S, CHEN Z J, et al. Delocalized photonic deep learning on the internet’s edge[J]. Science, 2022, 378(6617): 270-276. |
30 | COLBURN S, CHU Y, SHILZERMAN E, et al. Optical frontend for a convolutional neural network[J]. Applied Optics, 2019, 58(12): 3179-3186. |
31 | OKUMA N, KAWABATA K, SHIOZAKI K, et al. Topological origin of non-hermitian skin effects[J]. Physical Review Letters, 2020, 124(8): 086801. |
32 | TSAI F C F, O’BRIEN C J, PETROVIĆ N S, et al. Analysis of optical channel cross talk for free-space optical interconnects in the presence of higher-order transverse modes[J]. Applied Optics, 2005, 44(30): 6380-6387. |
33 | HU W H, LI X J, YANG J K, et al. Crosstalk analysis of aligned and misaligned free-space optical interconnect systems[J]. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 2010, 27(2): 200-205. |
34 | ASHTIANI F, GEERS A J, AFLATOUNI F. An on-chip photonic deep neural network for image classification[J]. Nature, 2022, 606: 501-506. |
35 | XU S F, WANG J, WANG R, et al. High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays[J]. Optics Express, 2019, 27(14): 19778-19787. |
36 | XU S F, WANG J, SHU H W, et al. Optical coherent dot-product chip for sophisticated deep learning regression[J]. Light: Science & Applications, 2021, 10: 221. |
37 | QU Y R, ZHU H Z, SHEN Y C, et al. Inverse design of an integrated-nanophotonics optical neural network[J]. Science Bulletin, 2020, 65(14): 1177-1183. |
38 | KHORAM E, CHEN A, LIU D J, et al. Nanophotonic media for artificial neural inference[J]. Photonics Research, 2019, 7(8): 823. |
39 | ATHALE R A, COLLINS W C. Optical matrix-matrix multiplier based on outer product decomposition[J]. Applied Optics, 1982, 21(12): 2089-2090. |
40 | RECK M, ZEILINGER A, BERNSTEIN H J, et al. Experimental realization of any discrete unitary operator[J]. Physical Review Letters, 1994, 73(1): 58-61. |
41 | ZHU W W, ZHANG L, LU Y Y, et al. Design and experimental verification for optical module of optical vector-matrix multiplier[J]. Applied Optics, 2013, 52(18): 4412-4418. |
42 | CLEMENTS W R, HUMPHREYS P C, METCALF B J, et al. Optimal design for universal multiport interferometers[J]. Optica, 2016, 3(12): 1460. |
43 | CHENG J W, ZHOU H L, DONG J J. Photonic matrix computing: from fundamentals to applications[J]. Nanomaterials, 2021, 11(7): 1683. |
44 | XU X Y, TAN M X, WU J Y, et al. Photonic perceptron based on a Kerr microcomb for high-speed, scalable, optical neural networks[J] Laser & Photonics Reviews, 2020, 14(10): 2000070. |
45 | ZANG Y B, CHEN M H, YANG S G, et al. Optoelectronic convolutional neural networks based on time-stretch method[J]. Science China Information Sciences, 2021, 64(2): 122401. |
46 | MENG X Y, SHI N N, SHI D F, et al. Photonics-enabled spiking timing-dependent convolutional neural network for real-time image classification[J]. Optics Express, 2022, 30(10): 16217-16228. |
47 | MENG X Y, SHI N N, LI G Y, et al. On-demand reconfigurable incoherent optical matrix operator for real-time video image display[J]. Journal of Lightwave Technology, 2023, 41(6): 1637-1648. |
48 | HUANG L, YAO J P. Optical processor for a binarized neural network[J]. Optics Letters, 2022, 47(15): 3892-3895. |
49 | XU Z Z, TANG K F, JI X, et al. Experimental demonstration of a photonic convolutional accelerator based on a monolithically integrated multi-wavelength distributed feedback laser[J]. Optics Letters, 2022, 47(22): 5977-5980. |
50 | HUANG Y Y, ZHANG W J, YANG F, et al. Programmable matrix operation with reconfigurable time-wavelength plane manipulation and dispersed time delay[J]. Optics Express, 2019, 27(15): 20456-20467. |
51 | LIU Z B, GAO S C, LAI Z Y, et al. Broadband, low-crosstalk, and massive-channels OAM modes de/multiplexing based on optical diffraction neural network[J]. Laser & Photonics Reviews, 2023, 17(4): 2200536. |
52 | ZHANG J Q, YE Z Y, YIN J H, et al. Polarized deep diffractive neural network for sorting, generation, multiplexing, and de-multiplexing of orbital angular momentum modes[J]. Optics Express, 2022, 30(15): 26728-26741. |
53 | MENGU D, LUO Y, RIVENSON Y, et al. Analysis of diffractive optical neural networks and their integration with electronic neural networks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2020, 26(1): 3700114. |
54 | YAN T, WU J M, ZHOU T K, et al. Fourier-space diffractive deep neural network[J]. Physical Review Letters, 2019, 123(2): 023901. |
55 | CHEN Y S, ZHU J F. An optical diffractive deep neural network with multiple frequency-channels[DB/OL]. arXiv preprint: 1912.10730, 2019. |
56 | LUO Y, MENGU D, YARDIMCI N T, et al. Design of task-specific optical systems using broadband diffractive neural networks[J]. Light: Science & Applications, 2019, 8: 112. |
57 | SHEN Y C, HARRIS N C, SKIRLO S, et al. Deep learning with coherent nanophotonic circuits[J]. Nature Photonics, 2017, 11: 441-446. |
58 | BAGHERIAN H, SKIRLO S, SHEN Y C, et al. On-chip optical convolutional neural networks[DB/OL]. arXiv preprint: 1808.03303, 2018. |
59 | XU X F, ZHU L Q, ZHUANG W, et al. A convolution neural network implemented by three 3 × 3 photonic integrated reconfigurable linear processors[J]. Photonics, 2022, 9(2): 80. |
60 | DE MARINIS L, COCOCCIONI M, LIBOIRON-LADOUCEUR O, et al. Photonic integrated reconfigurable linear processors as neural network accelerators[J]. Applied Sciences, 2021, 11(13): 6232. |
61 | XU X F, ZHU L Q, ZHUANG W, et al. Photoelectric hybrid convolution neural network with coherent nanophotonic circuits[J]. Optical Engineering, 2021, 60: 117106. |
62 | YANG Z, TAN W M, ZHANG T J, et al. MXene-based broadband ultrafast nonlinear activator for optical computing[J]. Advanced Optical Materials, 2022, 10(17): 2200714. |
63 | SOLDANO L B, PENNINGS E C M. Optical multi-mode interference devices based on self-imaging: principles and applications[J]. Journal of Lightwave Technology, 1995, 13(4): 615-627. |
64 | MENG X Y, ZHANG G J, SHI N N, et al. Compact optical convolution processing unit based on multimode interference[J]. Nature Communications, 2023, 14: 3000. |
65 | LIAO K, CHEN Y, YU Z C, et al. All-optical computing based on convolutional neural networks[J]. Opto-Electronic Advances, 2021, 4(11): 200060. |
66 | XU X F, ZHU L Q, ZHUANG W. Convolutional neural networks with coherent nanophotonic circuits[C]∥ 10th International Symposium on Advanced Optical Manufacturing and Testing Technologies: Intelligent Sensing Technologies and Applications. SPIE, 2021: 207-213. |
67 | JIANG Y, ZHANG W J, YANG F, et al. Photonic convolution neural network based on interleaved time-wavelength modulation[J]. Journal of Lightwave Technology, 2021, 39(14): 4592-4600. |
68 | BANGARI V, MARQUEZ B A, MILLER H, et al. Digital electronics and analog photonics for convolutional neural networks (DEAP-CNNs)[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2020, 26(1): 7701213. |
69 | XU S F, WANG J, ZOU W W. Optical convolutional neural network with WDM-based optical patching and microring weighting banks[J]. IEEE Photonics Technology Letters, 2021, 33(2): 89-92. |
70 | BAI B W, YANG Q P, SHU H W, et al. Microcomb-based integrated photonic processing unit[J]. Nature Communications, 2023, 14: 66. |
71 | XU S F, WANG J, ZOU W W. Optical patching scheme for optical convolutional neural networks based on wavelength-division multiplexing and optical delay lines[J]. Optics Letters, 2020, 45(13): 3689-3692. |
72 | XU S F, WANG J, ZOU W W. High-energy-efficiency integrated photonic convolutional neural networks[DB/OL]. arXiv preprint: 1910.12635, 2019. |
73 | OHNO S, TANG R, TOPRASERTPONG K, et al. Si microring resonator crossbar array for on-chip inference and training of the optical neural network[J]. ACS Photonics, 2022, 9(8): 2614-2622. |
74 | TAIT A N, FERREIRA DE LIMA T, NAHMIAS M A, et al. Silicon photonic modulator neuron[J]. Physical Review Applied, 2019, 11(6): 064043. |
75 | ONG J R, OOI C C, ANG T Y L, et al. Photonic convolutional neural networks using integrated diffractive optics[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2020, 26(5): 7702108. |
76 | FU T Z, ZANG Y B, HUANG H H, et al. On-chip photonic diffractive optical neural network based on a spatial domain electromagnetic propagation model[J]. Optics Express, 2021, 29(20): 31924-31940. |
77 | FU T Z, ZANG Y B, HUANG Y Y, et al. Photonic machine learning with on-chip diffractive optics[J]. Nature Communications, 2023, 14: 70. |
78 | CHEN Z J, SLUDDS A, DAVIS R, et al. Deep learning with coherent VCSEL neural networks[J]. Nature Photonics, 2023, 17: 723-730. |
79 | LI S R, YANG H B, WONG C W, et al. PhotoFourier: a photonic joint transform correlator-based neural network accelerator[C]∥ 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). Piscataway: IEEE Press, 2023: 15-28. |
80 | ZHU H H, ZOU J, ZHANG H, et al. Space-efficient optical computing with an integrated chip diffractive neural network[J]. Nature Communications, 2022, 13: 1044. |
81 | FELDMANN J, YOUNGBLOOD N, KARPOV M, et al. Parallel convolutional processing using an integrated photonic tensor core[J]. Nature, 2021, 589: 52-58. |
82 | XU S F, WANG J, YI S C, et al. High-order tensor flow processing using integrated photonic circuits[J]. Nature Communications, 2022, 13: 7970. |
83 | HUANG Y Y, FU T Z, HUANG H H, et al. Sophisticated deep learning with on-chip optical diffractive tensor processing[J]. Photonics Research, 2023, 11(6): 1125. |
84 | DONG B W, AGGARWAL S, ZHOU W, et al. Higher-dimensional processing using a photonic tensor core with continuous-time data[J]. Nature Photonics, 2023, 17: 1080-1088. |
85 | YIN R Y, XIAO H F, JIANG Y H, et al. Optical mode division multiplexing inspired photonic neural network accelerator[DB/OL]. Research Square preprint, 2023. |
86 | CHEN Y T, NAZHAMAITI M, XU H, et al. All-analog photoelectronic chip for high-speed vision tasks[J]. Nature, 2023, 623: 48-57. |
87 | GU Z Y, GAO Y S, LIU X Z. Optronic convolutional neural networks of multi-layers with different functions executed in optics for image classification[J]. Optics Express, 2021, 29(4): 5877-5889. |
88 | NAHMIAS M A, TAIT A N, TOLIAS L, et al. An integrated analog O/E/O link for multi-channel laser neurons[J]. Applied Physics Letters, 2016, 108(15): 151106. |
89 | MESARITAKIS C, KAPSALIS A, SYVRIDIS D. All-optical reservoir computing system based on InGaAsP ring resonators for high-speed identification and optical routing in optical networks[C]∥ Quantum Sensing and Nanophotonic Devices XII. SPIE, 2015, 9370: 608-614. |
90 | DUPORT F, SCHNEIDER B, SMERIERI A, et al. All-optical reservoir computing[J]. Optics Express, 2012, 20(20): 22783. |
91 | CHENG Z Z, TSANG H K, WANG X M, et al. In-plane optical absorption and free carrier absorption in graphene-on-silicon waveguides[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2014, 20(1): 4400106. |
92 | MISCUGLIO M, MEHRABIAN A, HU Z B, et al. All-optical nonlinear activation function for photonic neural networks[J]. Optical Materials Express, 2018, 8(12): 3851. |
93 | ZHANG H, THOMPSON J, GU M L, et al. Efficient on-chip training of optical neural networks using genetic algorithm[J]. ACS Photonics, 2021, 8(6): 1662-1672. |
[1] | Jiaqi LIU, Rongqian CHEN, Jinhua LOU, Xu HAN, Hao WU, Yancheng YOU. Aerodynamic shape optimization of high-speed helicopter rotor airfoil based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(9): 529828-529828. |
[2] | Xudong LUO, Yiquan WU, Jinlin CHEN. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 28822-028822. |
[3] | Shengzhe SHAN, Weiwei ZHANG. Air combat intelligent decision-making method based on self-play and deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328723-328723. |
[4] | Haiqiao LIU, Meng LIU, Zichao GONG, Jing DONG. A review of image matching methods based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(3): 28796-028796. |
[5] | Yunxiang FAN, Huanan AI, Mingzhen WANG, Kai CAO, Xuejun LIU, Hongqiang LYU. Unsteady hydrodynamic load reconstruction of seaplane based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(20): 129882-129882. |
[6] | Xiangxi KONG, Wenyuan QIN, Piaoyi SU, Yongzhao HUA, Xiwang DONG, Li WANG, Ying SU, Kun LYU. Damage assessment algorithm based on deep learning and fuzzy analytic hierarchy process [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(19): 329503-329503. |
[7] | Hui LI, Yinchao CHEN, Shaoshan SUN, Zhaoxin LIANG, Gang MAO, Bin QIAO, Yongbo LI. Fault diagnosis method of rotor system based on federated graph network [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(17): 530611-530611. |
[8] | Zibo ZHUANG, Chunhui ZHANG, Xing CHEN, Jingyuan SHAO, Pakwai CHAN. Clear⁃air turbulence recognition by Doppler⁃wind⁃lidar in terminal area based on DCGAN [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(16): 329748-329748. |
[9] | Yuanhao HU, Yibo SONG, Jiahui LIU, Xiaoli ZHAO, Wenxiang DENG, Jian HU, Jianyong YAO. Fault diagnosis of electro-hydraulic proportional servo valves based on attention convolutional capsule networks [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(15): 630407-630407. |
[10] | Jiawen LIU, Mingzhen WANG, Wenxuan OUYANG, Jian YU, Xuejun LIU, Hongqiang LYU. Intelligent denoising methods for Gibbs phenomenon of high⁃order discontinuous Galerkin numerical scheme [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 129323-129323. |
[11] | Zhaochen CHU, Tao SONG, Ren JIN, Defu LIN. Vision-based air-to-air multi-UAVs tracking [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 629379-629379. |
[12] | Yanfang LIU, Jiayu SHE, Qiufan YUAN, Rui ZHOU, Naiming QI. Real⁃time small target detection networks for UAV remote sensing [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 630119-630119. |
[13] | Junyu LI, Qiankun LIU, Ying FU. Infrared small object detection based on attention mechanism [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 628959-628959. |
[14] | Wentao LI, Yunqin HE, Wenbo LI, Yiyi ZHANG, Guozhu LIANG. 3D grain reverse design and shape optimization for solid rocket motor [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(11): 529089-529089. |
[15] | Xin SU, Runcheng GUAN, Qiao WANG, Weizheng YUAN, Xianglian LYU, Yang HE. Ice area and thickness detection method based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729283-729283. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||
Address: No.238, Baiyan Buiding, Beisihuan Zhonglu Road, Haidian District, Beijing, China
Postal code : 100083
E-mail:hkxb@buaa.edu.cn
Total visits: 6658907 Today visits: 1341All copyright © editorial office of Chinese Journal of Aeronautics
All copyright © editorial office of Chinese Journal of Aeronautics
Total visits: 6658907 Today visits: 1341