《1 Engineering research fronts》

1 Engineering research fronts

《1.1 Trends in Top 10 engineering research fronts》

1.1 Trends in Top 10 engineering research fronts

The Top 10 engineering research fronts in the field of information and electronic engineering are summarized in Table 1.1.1, encompassing the subfields of electronic science and technology, optical engineering and technology, instrument science and technology, information and communication engineering, computer science and technology, and control science. Among them, “reconfigurable intelligent surface assisted wireless communications” is among the popular topics published by Clarivate, and the nine other fronts are recommended by researchers.

The number of core papers published from 2015 to 2020 related to each front are shown in Table 1.1.2. Of these research fronts, “reconfigurable intelligent surface assisted wireless communications” is the most significant one regarding the number of core papers published in recent years, which is followed by “in-memory computing technology for intelligent computing”.

(1)    In-memory computing technology for intelligent computing

The field of in-memory computing technology, as its name implies, is a computing paradigm integrating memory and computing. It aims to transform the traditional computing- centric architecture into a data-centric architecture for direct data processing within the memory. Intelligent computing is a method encompassing the implementation of computational intelligence systems, setting extremely high requirements for the information interaction capacity between the computing unit and the memory. Computational intelligence systems based on the traditional Von Neumann architecture feature the inherent problems of short effective computation time and low energy efficiency ratio due to the separation of the computing unit from the memory. In-memory computing technology effectively reduces data interaction during computing and has the potential to provide the best solutions to data-driven high-performance intelligent computing needs. For the efficient implementation of this technology, dedicated high-performance chips with in-memory computing function should be used. In line with different types of memory media, the mainstream research and development (R&D) of in- memory computing chips currently focuses on traditional volatile memories (such as SRAM and DRAM) and non-volatile

《Table 1.1.1》

Table 1.1.1 Top 10 engineering research fronts in information and electronic engineering

《Table 1.1.2》

Table 1.1.2 Annual number of core papers published for the Top 10 engineering research fronts in information and electronic engineering

memories (such as NOR Flash). In recent years, non-volatile memory technologies, such as resistive random access memory (RRAM), phase change memory (PCM), and magnetic random access memory (MRAM), have brought new hope for the efficient implementation of in-memory computing chips. In-memory computing technology is a subversive technology with great potential. Scientific research teams in many countries and regions, including China, the USA, Europe, Japan, and South Korea, have conducted exploratory research on various levels such as material and process, chip circuitry, computing architecture, system integration, or supporting software. The main research directions in the industry include generic near-memory computing architecture, SRAM in- memory computing, DRAM in-memory computing, and RRAM/ PCM/Flash multi-value in-memory computing. In-memory computing technology for intelligent computing has a broad application prospect in the field of artificial intelligence & Internet of Things (AIoT).

(2)  Photonic-electronic integrated circuits

With the development of technologies like mobile Internet, cloud computing, and autonomous driving, the demand for data interconnection, high-performance computing, and multi-modal sensing shows exponential growth. As Moore’s Law approaches its limitation, the performance improvement brought on by the evolution of integrated circuit technology tends to diminish. The use of integrated photonic functions to obtain microelectronic chips with boosted speed, reduced energy consumption, and extended capacity has become one of the main technological paths in the post-Moore era. Photonic-electronic integrated circuit (hereinafter abbreviated as PEIC) refers to a mixed-signal chip that incorporates photonic devices, photonic circuits, or photonic on-chip systems with large-scale circuits via packaging, bonding, or single-chip preparation. This kind of chip integrates multiple functions, such as optical-electronical/electronical-optical conversion, information transmission, and optoelectronic mixed signal processing, showing broad prospects for optical communication, mobile communication, high-performance computing, data center, quantum information, sensing measurement, biomedicine, among other applications. The main research directions of PEIC include photonic-electronic integrated materials, photonic-electronic integrated devices, photonic-electronic integrated processes, photonic-electronic integrated circuit control methods, photonic-electronic integrated circuit architecture, optoelectronic collaborative design simulation, and research on the application of photonic-electronic integrated circuits. The research on PEIC has shown rapid progress in the past 20 years, especially concerning those based on silicon-based optoelectronics platforms using the industrial basis of integrated circuits and learning from the experience of integrated circuit ecosystem development. Thus far, silicon-based PEIC has established an industrial basis for commercial design simulation software, fabs, packaging plants, and system integration, and it has provided important applications in the fields of optical transmission, optical interconnection, optical computing, and optical sensing. PEIC is rapidly evolving towards photonic-electronic fusion networks-on-chip. It is expected that optoelectronics and microelectronics will eventually be monolithically integrated to open up a new direction of development. Nonetheless, challenges in realizing this vision are still emerging. Considering the development of integrated circuits, the aforementioned process could be accelerated.

(3)  Integrated microwave photonics

Microwave photonics (MWP) is an emerging topic in which radio frequency (RF) signals are generated, distributed, processed, measured, and analyzed using photonic techniques. With the rapid evolution of information systems such as perceptual detection and interconnection communication under the direction of integration and intelligence, conventional microwave and electronic technologies have encountered an issue of limited bandwidth. MWP combines the advantages of photonics technologies (such as large bandwidth, high multiplex, and low loss) and those of microwave technologies (including high precision, flexibility, and ease of modification), turning into a technology enabling various functions that cannot be implemented in the microwave field alone. Recent advances in photonic integration have propelled microwave photonic technologies to new heights. The possibility to interface hybrid material platforms in order to enhance light-matter interactions has led to the development of ultra-small and high-bandwidth electro-optic modulators, low-noise frequency synthesizers, and chip signal processors with spectral resolution enhanced by orders of magnitude, featured by a significant reduction in size, weight, cost, and power consumption of the microwave photonic link. Besides, the perfection of high-volume semiconductor processing has finally enabled the complete integration of light sources, amplifiers, modulators, isolators, and detectors in a single microwave photonic processor chip, and it has ushered the creation of a complex signal processor with multifunctionality and reconfigurability akin to electronic devices. Integrated microwave photonics technology is expected to unify the discrete form of single devices in the microwave photonics system, achieving maximized resource optimization through integration and supporting the multifunctional integration form of future equipment systems. Therefore, it is considered a disruptive technology for the next generation of information systems.

The main research directions of integrated microwave photonics can be divided into two categories. One is to apply optical methods to microwave systems, such as radar and electronic confrontation using the huge advantage of high bandwidth of the optical system to transmit and process microwave signals. The other is to employ various microwave techniques to optical systems to promote optical communication networks and systems. Intelligence is the main trend of information society development; like analog and digital signals, it is an important aspect of information systems of the future. Further research in integrated microwave photonics will focus on solving the scientific problems of cross-band, cross-scale, and cross- material integration faced by radar and information systems to give full play to the advantages of integrated microwave photonics with large bandwidth, multi-function, and high energy efficiency. The current frontier research focuses mainly on mathematical model construction, system framework innovation, functional device/chip innovation, material and key manufacturing process breakthroughs, comprehensive energy efficiency evaluation, etc. Furthermore, integrated microwave photonics technology is predicted to be the core technology of electromagnetic space integration and future sixth-generation mobile communication (6G), and at the same time, a disruptive technology that supports the cross- generational transformation of radar, communications, and electromagnetic warfare information systems in the military field.

(4)  General-purpose brain-inspired computing systems

Brain-inspired computing refers to computational theories, system architectures, chips, algorithms, and applications that are motivated mainly by the information processing mode or structure of the brain. Various brain-inspired computing systems centered on neuromorphic chips are rapidly advancing and showing their benefits in dealing with certain intelligent problems with low power consumption. The current research is drawing inspiration from the design methodology and development history of existing general- purpose processors, in order to realize complete hardware in combination with application requirements based on the computational completeness theory. At the same time, research on the basic software of brain-inspired computing is proposing high-level programming abstraction and certain unified development frameworks that are independent of specific chips. All of these works promote the evolution of brain-inspired computing systems from “customized” to “general-purpose;” that is, they realize general-purpose brain-inspired computing systems.

From the perspective of design methods, most of the existing neuromorphic chips are customized; i.e., the hardware functions and interfaces are usually determined by induction based on the requirements of target applications, as well as the toolchain software. This design methodology features the tight coupling of system hardware and software, and increases the difficulty of application development and migration, which is particularly disadvantageous for interdisciplinary research such as brain-inspired computing. Moreover, the fields of application of brain-inspired computing are expanding rapidly. Thus, it is difficult to determine whether the hardware functions and interfaces summarized based on existing applications can support the emerging applications, and it is also challenging to compare and evaluate different systems.

Researchers have gradually realized this problem and embarked on researches on the brain-inspired computing completeness theory, the corresponding hardware function and chip design, and unified development frameworks. Specifically, drawing inspiration from the computational complete theory and the design methodology of system hierarchy of general-purpose computers, the related theories and system architectures for brain-inspired computing are under investigation, which is the theoretical basis for decoupling the software and hardware of brain-inspired computing systems. From the aspect of chip design, it is necessary to explore the balance of function completeness and application efficiency while maintaining the high performance of brain-inspired computing, and especially, giving full play to the processing capabilities of neuromorphic circuits/devices. As for the research related to basic software, it needs to implement some unified application development frameworks through the appropriate abstraction and layering between applications and chips, such that hardware specifications and constraints can become “transparent” to application development.

The transformation of brain-inspired computing systems from “customized” to “general-purpose” will enable all types of personnel involved in this interdisciplinary research to focus on their respective professional fields and bring significant improvement to R&D efficiency. This is one of the key steps to rapid development and the formation of future large-scale industries.

(5)  Intelligent perception and safety control of autonomous unmanned systems

This research front comprises the fields of intelligent recognition, understanding, and environmental cognition of autonomous unmanned systems, as well as the reliable and safe control of their self-motions and behaviors. In addition to the need for conventional performance indicators like stability and tracking, the intelligent perception and safety control of autonomous unmanned systems faces new challenges, such as complex open environments, random dynamics, and game confrontation, thus creating new scientific problems and technical solutions. ① The complex open environment requires the autonomous unmanned system to have high- precision situational awareness technology that can cope with the reconstruction of real-time noise reduction and dynamic prediction of urban scenes under open, highly dynamic, high density, and large noise conditions. Currently, this field has many technologies such as the multi-level instance segmentation of images and videos, interactive prediction of moving obstacles, or the real-time reconstruction of static environments. ② Random dynamics require the autonomous unmanned system to possess safe and reliable dynamic motion programming and scheduling technology that can deal with the high-dimensional and strong real- time problems of navigation and controlled decision-making under the condition of a complex and harsh environment with variable loads. The recently emerging relevant algorithms include multi-sensor tight coupling fusion, visual positioning based on feature learning, route conflict management, and automatic planning control. ③ Game confrontation requires the autonomous unmanned system to have intelligent collaborative decision-making technology that can cope with the interactive learning problems of multiple agents under the conditions of an uncertain environment, incomplete decision-making information, and restricted communication. The current associated technologies comprise multi-agent reinforcement learning, generative adversarial networks, and distributed robust optimization. For the intelligent perception and safety control of autonomous unmanned systems, the research directions include building a theoretical system, building simulation platforms, generating test cases, and establishing demonstration projects.

(6)  Artificial intelligence enabled system engineering

Artificial intelligence (AI) enabled system engineering, also known as intelligent system engineering (ISE), refers to the use of AI technology to integrate and innovate the organization and management methods of system planning, research, design, manufacturing, testing, and application, so as to make the realization and application of the system reach a new mode and higher efficiency. It is a novel concept and discipline emerging along the intellectualization of system engineering and AI engineering. According to the application of AI technology in different stages of system engineering, it is divided mainly into ISE modeling, ISE analysis, ISE synthesis, and ISE simulation.

Thus far, research on this front is still in its infancy, and the development of each branch direction is relatively independent. The existing achievements are aimed mainly at one aspect of a specific field, such as the modeling and analysis of complex systems in a particular subject; few studies have comprehensively integrated technologies at all levels to systematically solve the whole process problems of practical complex systems. With the continuous development of AI technology, the implementation methods of ISE are increasingly more diversified. The combination of data-driven deep learning methods and mechanism-driven traditional modeling optimization procedures will be further expanded both in depth and in breadth, and the technologies at different levels and along different directions will also be deeply integrated. At the same time, the application fields of ISE technology are expanding, which is conducive to finding more efficient ways to research, design, manufacture, test, and perform the operation management of various complex systems. As a trend, ISE technology will play an important role in promoting the development of cutting-edge technology fields (e.g., the manufacturing of complex industrial products and the R&D of aerospace equipment) and the efficient governance of social systems. Meanwhile, ISE is gradually showing the development trend of multi-disciplinary intersection and integration. The combination of big data technology, cloud computing technology, and knowledge automation in specific fields is going to become an important development layer of ISE.

(7)  Quantum intelligent algorithms

Quantum intelligent algorithms are a new type of algorithms combining quantum computing and classical intelligent algorithms, which can break through the limits of classical intelligent algorithms. In recent years, quantum intelligent algorithms have attracted extensive attention in both academia and industry. Compared with classical algorithms, several quantum intelligent algorithms have shown their superiority. For example, quantum support vector machines can provide a significant speedup in finding support vectors and calculating the kernel function matrices; combining the advantages of quantum computing and neural networks, quantum neural networks are expected to improve the computing efficiency of neural networks and solve the difficulties of big data and the slow training process in learning models. In addition, the quantum principal component analysis algorithm, the quantum reinforcement learning algorithm, and others, show incomparable superiority to classical algorithms. Quantum intelligent algorithms rely on the development of quantum computer hardware. In the new era of noisy intermediate-scale quantum (NISQ), the theoretical research directions of quantum intelligent algorithms include the platform development of quantum intelligent algorithms, quantum convolutional neural network with parameterized quantum circuits, quantum adversarial generative neural network, the robustness and attack ability of quantum intelligence algorithms, etc. Currently, technology companies in the USA, such as IBM, Google, Microsoft, and Rigetti, are leading the development of quantum computer physical systems, architectures, software, and intelligent algorithms. Chinese technology companies, such as Alibaba, Tencent, Huawei, Baidu, JD.com, ByteDance, and Origin Quantum, have also deployed quantum intelligent computing industries. It can be predicted that killer applications will appear in the next few years on the NISQ processor, and quantum intelligence will play major roles in biomedicine, financial technology, material chemistry, military industry, and other fields.

(8)  Ultrafast submicron microscopic imaging

Ultrafast submicron microscopic imaging is a high-precision optoelectronic imaging technique with temporal and spatial resolution of <1 ns and <1 μm, respectively. It is of great importance to explore the microcosm of the world around us. Ultrafast optical spectroscopy could study the photo- dynamic processes and mechanism in matter on femto/ picosecond time scales. Moreover, commercial imaging microscopy is highly sensitive in the spatial domain, leaving the temporal domain largely unexplored. Ultrafast submicron imaging microscopy is a combination of ultrafast optical spectroscopy and high-precision imaging microscopy, which can be used to study various ultrafast dynamic processes on a submicron scale, which is crucial in physics, chemistry, material science, biomedicine, etc. According to the working principle, ultrafast submicron imaging microscopy can be divided into three major branches: ultrafast optical microscopy, ultrafast scanning probe microscopy, and ultrafast electron microscopy. Ultrafast optical microscopy relies on the combination of ultrafast laser pulses and optical microscopy, and these techniques are mature, low-cost and have high temporal and spatial resolution. However, the major disadvantage of this kind of microscopy is that its spatial resolution is limited by the optical diffraction limit. Ultrafast scanning probe microscopy is based on the combination of ultrafast laser pulses and scanning probe microscopy (such as SEM, AFM, and NSOM); therefore, its spatial resolution is considerably better than that of ultrafast optical microscopy. Notwithstanding, ultrafast scanning probe microscopy is still under development and has high cost. Ultrafast electron microscopy is based on the coherent imaging of the photoemission of electrons after ultrafast pulse excitation; it uniquely combines both nanometer-scale spatial resolution and sub-picosecond temporal resolution. The main disadvantages of this technique are its high cost and damage to the sample.

In material science, ultrafast submicron imaging microscopy can be used to visualize the correlation between the spatial distribution of carrier dynamics and the micromorphology of nanomaterials, which is crucial for a deeper understanding of the physical properties of materials. In biomedical research, ultrafast submicron imaging microscopy can resolve the distribution and evolution properties of tumor labels and the energy transfer processes of nanomaterials in vivo. This information is of great importance in the development of novel drugs and cancer therapies.

With the rapid development of laser and detector technology, novel ultrafast imaging microscopy with shorter wavelengths (UVC to X-ray), better spatial resolution (<100 nm), higher sensitivity (single molecule detection), and higher frame rate (>1 000 fps) will inevitably play a central role in modern optoelectronic technologies.

(9)  Multimodal automatic machine learning

Multimodal machine learning aims to process and understand multimodal information by means of machine learning. Multimodal machine learning emerged in the 1970s. It experienced several stages of development, and then entered the era of deep learning after 2010. Generally speaking, modality refers to “the way something happens or is perceived”, such as vision or touch. When a machine learning task contains multiple such modalities, it is called a multimodal problem. Multimodal machine learning focuses mainly on three modalities: natural language that can be written or spoken, visual signals usually represented by images or videos, and sound signals that encode sound and paralinguistic information, such as rhythm and sound expression.

The main challenge of multimodal machine learning is the heterogeneity of data. At present, there are five main research directions of multimodal machine learning. ① Multimodal representation learning, that is, to mine the complementarity among multiple modes and eliminate the redundancy in order to learn better feature representations. The heterogeneity of multimodal data makes it challenging to construct such a representation. For example, language is a symbol with highly abstract semantics, while audio and video are signals. ② Modal transformation, that is, to transform the information of one mode into the information of another mode, such as machine translation, speech translation, and picture description. ③ Modal alignment, that is, to find the corresponding relationship between the internal components of different modalities from the same object that is subject to analysis, for example, to associate and align a movie with its subtitles. ④ Multimodal fusion, that is, to perform target prediction by fusing multimodal information, including pattern recognition, semantic analysis, and other tasks. For instance, in speech recognition tasks, the accuracy of recognition can be improved by using the fusion data of speech signals, semantic features, and even lip movements. ⑤ Collaborative learning, that is, to carry out machine learning with the aid of information from another mode. When labeled samples are scarce, collaborative learning can effectively solve this problem.

In pace with the advance of deep learning, engineers need to decide the best neural network architecture, training process, regularization method, super-parameters, etc., all of which have a great impact on the algorithm performance. A multimodal task usually needs more complex neural networks and larger parameter optimization space than a unimodal task. Hence, it is difficult to make decisions manually. Multimodal automatic machine learning aims at designing automatic algorithms using data-driven methods. Its main research directions include: ① automatic feature engineering, that is, to automatically extract typical features from the original data; ② automatic model selection and hyperparameter optimization, that is, to find the most suitable algorithm model for solving task problems, and to automatically determine the hyperparameter of the model; and ③ neural architecture search, that is, to automatically find the best neural network architecture from a series of candidate architectures based on specific evaluation functions and search strategies.

The main development directions of multimodal automatic machine learning comprise: ① studies on automatic machine learning algorithms for multimodal tasks and multi-objective problems; ② research on more flexible variable representations for parameter space searching, and finding more convenient representation methods for modal migration; ③ designing more effective evaluation functions; and ④ research on transfer learning among multimodalities to further improve the efficiency of automatic machine learning.

(10)    Reconfigurable intelligent surface assisted wireless communications

Reconfigurable intelligent surface (RIS) assisted wireless communications is a type of communication system based on RISs to freely regulate the wireless environment and construct a new system architecture. Developed from information metamaterial technology, RISs are artificial surface structures with programmable electromagnetic properties, and are usually composed of a large number of carefully designed electromagnetic unit cells. By applying control signals to the adjustable elements on the unit cells, the electromagnetic properties of the unit cells can be dynamically controlled. Hence, the space electromagnetic waves can be actively and intelligently controlled in a programmable fashion, and electromagnetic fields can be generated whose parameters, such as amplitude, phase, polarization, and frequency, are controllable. This technology provides a broad interface between the physical world of RISs and the digital world of information science, and is extremely attractive for the development of future wireless networks.

RISs have the characteristics of low cost, low energy consumption, field programmability, and ease of deployment, and thus are becoming the candidate technologies of 6G. At present, the main research directions of RISs are focused on: ① deployment on the surfaces of various objects in the wireless propagation environment to construct an intelligently programmable wireless environment, including applications such as coverage improvement, capacity enhancement, secure communication, interference suppression, wireless energy transmission, and RIS-aided positioning and sensing; and ② the direct modulation of baseband information to the radio frequency carrier by using RISs, which can be used to construct wireless transmitter systems with new architecture and reduce the hardware complexity and cost. The main future development trends of RIS assisted wireless communications include metasurface hardware architecture and control algorithms, new theories of intelligent environment communications and RIS baseband algorithms, new wireless network architectures, and the measurement verification of prototype systems.

《1.2 Interpretations for three key engineering research fronts》

1.2 Interpretations for three key engineering research fronts

1.2.1 In-memory computing technology for intelligent computing

The basic concept of in-memory computing technology can be traced back to the concept of “logic-in-memory computer”, which was first put forward by Kautz et al. of Stanford Research Institute in 1969. Due to the constraints of chip design complexity and manufacturing cost, as well as the lack of killer applications of big data, the research on in- memory computing technology in the early stage stayed in the laboratory. Since 2012, the deep neural network (DNN) algorithm has developed rapidly in the field of intelligent computing, breaking the computing-centric rule of traditional algorithms and generating a data-centric computing requirement instead. The layout with the memory separated from the computing unit on the basis of the classical Von Neumann architecture has encountered serious bottlenecks in terms of performance and power consumption, and the crisis of Moore’s Law that came about in the same period has accelerated the deterioration of this situation. In this context, in-memory computing technology has become a research hotspot in the academic community and has entered a fast track to industrialization. Since intelligent computing applications such as DNN are not only computing- intensive but also data-intensive, there is a pressing need to improve the hardware computing power and memory access bandwidth. The concept of “in-memory computing”, which took a back seat in the 1990s, was resurrected by the designers of AI computing architecture, thanks to alleviating or eliminating problems such as performance bottleneck and low energy efficiency caused by the traditional Von Neumann architecture. In the scenario of intelligent computing, more than 95% of operations in the most widely used DNN algorithm are vector matrix multiplication; in- memory computing is used mainly to accelerate this part of operations. According to the literature, the paradigm of in- memory computing can increase the speed by more than 50 times with about 5% of the power consumption of the Von Neumann architecture. At the International Symposium on Microarchitecture 2017 (Micro 2017), NVIDIA Corporation, Intel Corporation, Microsoft Corporation, Samsung Electronics, ETH, University of California, Santa Barbara, and others, launched their prototypes of in-memory computing systems.

According to the data representation method, in-memory computing technology may belong to digital architecture, analog architecture, and hybrid digital-analog architecture; meanwhile, based on the implemented basic device structure, it may fall into general near-memory computing architecture, SRAM in-memory computing, DRAM in-memory computing, RRAM/PCM/Flash multi-value in-memory computing, and RRAM/PCM/MRAM two-value in-memory computing. In recent years, academic research has focused on the in-memory computing implemented by RRAM, and Duke University, Purdue University, Stanford University, University of Massachusetts, Nanyang Technological University of Singapore, Hewlett-Packard Development Company, L.P., Intel Corporation, and Micron Technology, Inc. have all released their related test chip prototypes. The entrepreneurial hotspot of the industry is the in-memory computing chip of NOR Flash. Mythic and Syntiant of the USA, and Witintech Co., Ltd. and Zbit Semiconductor, Inc. of China have all launched products that can be mass-produced for commercial purpose. In addition, in recent years, researchers have begun to explore ways to implement intelligent computing by the close integration of perception, memory, and computing. For example, a perceptive in-memory computing unit may be formed by adding binary metallic oxides, perovskite, polymers, and organic materials in the back end of line, so as to further reduce system delay, improve performance, and reduce power consumption, and it may find future applications in various fields such as computer vision, tactile sensory neuron system, and speech recognition.

Table 1.2.1 lists the countries with the greatest output of core papers on “in-memory computing technology for intelligent computing”. Both with a solid research foundation, China and the USA are ranked first and second, respectively, regarding their output of core papers. The main international cooperation partner of China is the USA (Figure 1.2.1). Among the Top 10 institutions with the greatest output of core papers (Table 1.2.2), four are from China. In terms of institutional cooperation (Figure 1.2.2), three Chinese research institutions collaborate closely with each other, and all of them are linked

《Table 1.2.1》

Table 1.2.1 Countries with the greatest output of core papers on “in-memory computing technology for intelligent computing”

with RWTH Aachen University. As for the number of cited core papers (Table 1.2.3), China accounts for 34.99%, 10.25% higher than the second place holder (the USA). Among the Top 10 institutions with the greatest output of citing papers, the top six are all from China (Table 1.2.4), which shows that this country pays great attention to this research front.

1.2.2 Photonic-electronic integrated circuits

Currently, no recognized technical solution or research roadmap exists for photonic-electronic integrated circuits. Researchers around the world are in pursuit of technological breakthroughs or solutions at various levels, such as process devices, design software, and system applications. The concept of photonic-electronic integrated circuits was put forward more than 20 years ago, and early on, photonic devices were dedicated to be integrated with passive circuits on a single chip. Due to the lack of high-speed transistors and microwave devices in the circuit, the first photoelectronic integrated chips could achieve only simple photoelectronic signal conversion or optical multiplexing functions. The integration of large-scale photonic and electronic integrated circuits is the feature of the present photonic-electronic integrated circuits. The package integration or monolithic integration of a large number of optoelectronic unit devices allows for the multi-function and system-level integration

《Table 1.2.2》

Table 1.2.2 Institutions with the greatest output of core papers on “in-memory computing technology for intelligent computing”

《Figure 1.2.1》

Figure 1.2.1 Collaboration network among major countries in the engineering research front of “in-memory computing technology for intelligent computing”

《Figure 1.2.2》

Figure 1.2.2 Collaboration network among major institutions in the engineering research front of “in-memory computing technology for intelligent computing”

《Table 1.2.3》

Table 1.2.3 Countries with the greatest output of citing papers on “in-memory computing technology for intelligent computing”

《Table 1.2.4》

Table 1.2.4 Institutions with the greatest output of citing papers on “in-memory computing technology for intelligent computing”

that is similar to microelectronic chips. To realize large-scale photonic-electronic integrated circuits, it is necessary to make breakthroughs in the four aspects of process, device, design, and application. Therefore, the interpretation of technological frontiers in this field is also carried out from the four aspects discussed below:

First, the processes of photonic-electronic integrated circuits are divided mainly into monolithic integration and packaging integration. Monolithic integration relies mainly on silicon- based CMOS and SiGe BiCMOS processes, which makes photonic and electronic circuits monolithically prepared on the same substrate in the wafer preparation process. On the other hand, packaging integration adopts different processes to integrate photonic and electronic circuits on different chips, and then applies hybrid integration into core particles through 2D/2.5D/3D packaging processes. The main R&D institutions pursuing this direction include Intel, Global Foundries, Taiwan Semiconductor Manufacturing Company (TSMC), Tower, STMicroelectronics, Innovations for High Performance microelectronics (IHP), and the National Institute of Advanced Industrial Science and Technology (AIST) of Japan, etc. The capability of chip processing technology is basic support for the development of the entire photonic-electronic integrated circuits, and it also determines the degree of autonomy and controllability. However, the selection of process nodes and integration routes of photonic-electronic integrated circuits is still challenging, especially while considering the comprehensive performance, capacity, and cost.

Second, the materials and devices of photonic-electronic integrated circuits are mainly photonic and optoelectronic devices. At present, silicon-based and indium phosphide (InP) based optical devices have a relatively high level of practicality. Although the functions, performance, and mass manufacturing capabilities of some chips have reached the commercial requirements, there is still much room for improvement in terms of power consumption and the size of lasers, modulators, and detectors. The main R&D institutions performing research in this direction include University of California-Santa Barbara (UCSB), Interuniversity Microelectronics Centre (IMEC), Intel, Fraunhofer Heinrich Hertz Institute (HHI), Photonics Electronics Technology Research Association (PETRA), Singapore Institute of Microelectronics (IME), Chinese Academy of Sciences, etc. Silicon-based and InP-based routes are expected to develop in parallel for a long time. In the short term, InP-based chips should be superior in terms of performance and maturity. In the future, however, silicon-based photonic-electronic integrated circuits have greater development prospects with the increase of communication speed, integration, and demand.

Third, similar to integrated circuit chips, the design tools of photonic-electronic integrated circuits are also evolving in the direction of standardization, modularization, and optoelectronic collaboration. Internationally, manufacturers such as Cadence, Synopsis, Mentor, and ANSYS have established electronic design automation (EDA) software and simulation tools for optoelectronic collaborative design and vertical integration. Meanwhile, China has a relatively weak industrial foundation, as few companies are currently focusing on this field, and there are no mature photonic-electronic integrated circuit design tools available. Photonic-electronic integrated design software is not only an important boost for the hybrid integration of photonic and electronic circuits, but also the core technology for system-level chip integration.

Fourth, the application fields of photonic-electronic integrated circuits are relatively diverse. For optical communications, photonic-electronic integrated circuits are packaged and integrated into standardized and small-sized optical modules to achieve high transmission rates and low cost. For high- performance computing, an optoelectronic transceiver and a calculation engine are manufactured through co- packaged optics (CPO) or monolithic integration to achieve high-density, low-latency interconnection and high- throughput processing. For multi-modal sensing, sensory- storage-computing integrated core particles are constructed through heterogeneous bonding and the three-dimensional integration of photonic and electronic circuits to realize real-time perception and processing for three-dimensional images. In addition, photonic-electronic integrated circuits have successively verified their significant advantages, such as ultra-high speed, ultra-miniaturization, and ultra-large integration, in the fields of quantum communication and quantum computing, AI and neural networks, biological detection, microwave photonics technology, and optical sensing. These applications are regarded as key research directions for new chips in the post-Moore era by most countries or regions such as the USA, Europe, Japan, and China, which have made high investments in this field in all walks of life. Note that the USA currently leads China in the number of papers and citations in this field, and its focus has shifted from device-level R&D to system-level chip integration.

Table 1.2.5 lists the countries with the greatest output of core papers on this research front. It can be seen that the USA and China have obvious advantages, with their core paper outputs ranking first and second, respectively. The international cooperation partners of the USA are mainly China and Germany (Figure 1.2.3). Among the Top 10 institutions with the greatest output of core papers (Table 1.2.2), there are four in the USA, four in Europe, one in China, and one in Canada. Concerning institutional cooperation (Figure 1.2.4), several research institutions in the USA and Europe cooperate closely with each other, while the Massachusetts Institute of Technology and Stanford University are the most active in foreign cooperation and exchanges. As for the number of core citing papers (Table 1.2.7), China ranks first with 31.65%, the USA ranks second with 25.00%, and other countries account for below 7% each. Among the Top 10 institutions with the greatest output of citing papers (Table 1.2.8), four institutions come from China and three are based in the USA, which shows that these two countries have had a lot of focus on this research front.

1.2.3 Integrated microwave photonics

The field of microwave photonics, which has been closely watched by scientific and industrial circles over the past 30 years, was the core driver of the early development of

《Table 1.2.5》

Table 1.2.5 Countries with the greatest output of core papers on “photonic-electronic integrated circuits”

《Table 1.2.6》

Table 1.2.6 Institutions with the greatest output of core papers on “photonic-electronic integrated circuits”

《Figure 1.2.3》

Figure 1.2.3 Collaboration network among major countries in the engineering research front of “photonic-electronic integrated circuits”

《Figure 1.2.4》

Figure 1.2.4 Collaboration network among major institutions in the engineering research front of “photonic-electronic integrated circuits”

《Table 1.2.7》

Table 1.2.7 Countries with the greatest output of citing papers on “photonic-electronic integrated circuits”

《Table 1.2.8》

Table 1.2.8 Institutions with the greatest output of citing papers on “photonic-electronic integrated circuits”

this technology due to the rich processing bandwidth, low- loss optical fiber transmission, and handling flexibility for complex regulatory functions. From the perspective of global frontier research, countries and regions such as the USA, China, Russia, the European Union, and Australia have been dominant. Among them, the USA is in the first echelon in frontier research progress due to its solid research foundation. China’s research activity in the field of integrated microwave photonics progressed almost at the same pace as the international one. In recent years, China has ranked second globally after the USA in terms of the number of frontier publications. Representatives of the progress and breakthroughs include the generation of ultra-broadband signals, the distribution and transmission of RF signals in optical fibers, programmable microwave photonic filters, photonic enhancement radar systems, etc. At present, microwave photon devices and functions used in radar and communication systems are still distinct, and many fundamental scientific problems and technical challenges exist in improving performance indicators such as the linearity of core devices.

With the rapid synchronous development of photonic integration technology, the combination of the two above fields has had a profound impact, resulting in the birth of integrated microwave photonics. In 2007, Nature Photonics published a review of the importance of microwave photonics, combining two worlds and developing microwave photonics. In 2019, a new review in Nature Photonics stated that integration was the future direction of microwave photonics. Integrated microwave photonics enables the maintenance of the high complexity of microwave photonic systems while greatly reducing the size of the whole system, allowing for an ultra-broadband spectrum range and greater instantaneous bandwidth, and providing sufficient frequency freedom and better functionality, higher rates, smaller volumes, high energy efficiency, low power consumption, as well as the expectation of overcoming loss, crosstalk problems and improving device linearity in discrete devices. These benefits make microwave photonics systems superior to RF circuits, and it is expected that the multiplexing and parallel processing of guidance, detection, communication, sensing, storage, computing, and other functions can also be realized at a later stage.

The current trends in integrated microwave photonics are as follows:

1)  In the system architecture, make use of the advantages of microwave photon technology, avoid the current immature integration technology caused by microwave photon device volume, weight and other problems, and give full play to the performance advantages of large bandwidth, easy multiplexing, and low loss and long-distance transmission. For example, through the top-level optimization design of system framework, scholars in Australia have realized the airborne integrated electronic information system with multiple functions including distribution, reconnaissance, navigation, and detection, greatly improving the bandwidth of the information system, and reducing the volume, weight, and power consumption of the entire system.

2)   In terms of functional discrete devices, radar, electronic countermeasures, and network communication are all developing rapidly in the direction of widening the time domain, frequency domain, and airspace. Under the existing system framework, researches are targeting high-performance functions such as broadband filter, frequency conversion, frequency beamforming, and analog-to-digital conversion unit, to obtain a replacement to the original microwave electronic discrete devices, thus greatly enhancing the performance of the existing information system. For example, Russia has improved the resolution of traditional imaging radars by an order of magnitude, based on information processing technologies such as a high-performance microwave photonic integrated radar front end. The Institute of Electronics of the Chinese Academy of Sciences, the 14th Institute of China Electronics Technology Group, the 38th Institute of China Electronics Technology Group, Nanjing University of Aeronautics and Astronautics, and other agencies have conducted relevant radar innovation work and passed the field test verification, in which the bandwidth and imaging resolution of traditional radar has been significantly improved. Based on the function of microwave photon beamforming network, the 29th Institute of China Electronics Technology Group has realized the transformation and improvement of spectrum detection capability.

3)  In the aspect of photonic chip integration based on different materials, the relevant research should achieve breakthroughs in compatibility and matching between materials, realize the low-loss coupling of multiple physical fields, improve unit device efficiency, strengthen process innovation to improve process tolerance, and explore carbon (such as graphene) based new material devices. For example, excimer-laser based direct writing techniques have been presented in Germany to reduce the loss of heterogeneous waveguides to less than 1 dB. In 2018, Harvard University reported a broadband thin- film lithium niobate modulator with a bandwidth greater than 70 GHz, which greatly improves the half-wave voltage, bandwidth, and other key technical indicators. China Shandong NANOLN and Shanghai Institute of Microsystems have made a quantum leap in the wafer bonding technology of silica-based lithium niobate, which belongs to the kind of large-mismatch heterogeneous materials. The Semiconductor Research Institute of the Chinese Academy of Sciences, Sun Yat-sen University, Huazhong University of Science and Technology, and other organizations have broken through the bottleneck of lithium niobate waveguide etching technique and have fabricated lithium niobate chip modulators covering the S-Ka band.

The countries and institutions with the greatest output of core papers on this research front are shown in Tables 1.2.9 and 1.2.10, respectively. The USA has a solid research foundation in microwave photonics, with core papers accounting for about half of the world’s output. The main institutions include Stanford University, California Institute of Technology, and Harvard University. The output of core papers of China is second only to that of the USA, accounting for about 25% of the world’s, with institutions that contribute most being Chinese Academy of Sciences, City University of Hong Kong, and University of Electronic Science and Technology of China. The core papers from Canada, Australia, the Netherlands, and Switzerland are equally distributed within the third tier globally, where the main institutions are the University of Sydney, Royal Melbourne Institute of Technology, Delft University of Technology, and Swiss Federal Institute

《Table 1.2.9》

Table 1.2.9 Countries with the greatest output of core papers on “integrated microwave photonics”

of Technology in Lausanne. International cooperation between major countries is shown in Figure 1.2.5, which is mainly limited to that between China, the USA, Canada, and Australia. In addition to cooperation among the three above-mentioned institutions of China, the Royal Melbourne Institute of Technology has close cooperation with these three Chinese institutions, as shown in Figure 1.2.6. In terms of core citing papers, China and the USA are still the main players, accounting for 31.29% and 26.36% respectively, as shown in Table 1.2.11. Among the institutions, Chinese Academy of Sciences has the largest percentage of citing papers of 19.84%, followed by National Institute of Standards and Technology, California Institute of Technology, and Huazhong University of Science and Technology, each with a percentage of about 10% (Table 1.2.12).

《2 Engineering development fronts》

2 Engineering development fronts

《2.1 Trends in Top 10 engineering development fronts》

2.1 Trends in Top 10 engineering development fronts

The Top 10 engineering development fronts in the field of information and electronic engineering are summarized in Table 2.1.1, encompassing the sub-fields of electronic science and technology, optical engineering and technology, instrument science and technology, information and communication engineering, computer science and technology, and control science. Among these 10 fronts, “high- resolution millimeter-wave radar 4D imaging technology”, “autonomous operation and cooperative control technologies

《Table 1.2.10》

Table 1.2.10 Institutions with the greatest output of core papers on “integrated microwave photonics”

《Figure 1.2.5》

Figure 1.2.5 Collaboration network among major countries in the engineering research front of “integrated microwave photonics”

《Figure 1.2.6》

Figure 1.2.6 Collaboration network among major institutions in the engineering research front of “integrated microwave photonics”

《Table 1.2.11》

Table 1.2.11 Countries with the greatest output of citing papers on “integrated microwave photonics”

《Table 1.2.12》

Table 1.2.12 Institutions with the greatest output of citing papers on “integrated microwave photonics”

for unmanned swarm systems”, “intelligent control of soft robotic systems”, “medical image analysis based on deep learning”, and “technology for securing trustworthy intelligent systems” are published based on the analysis of Derwent Innovation of Clarivate, and the five other fronts are recommended by researchers.

The annual disclosure of core patents involved in the above 10 development fronts for the period of 2015 to 2020 is shown in Table 2.1.2.

(1)  Chiplet design and chip-level three-dimensional stacking microsystem integration technology

Chiplets are small dies with specific functions and standard interconnected interfaces. Chip-level three-dimensional stacking microsystem integration technology means that some chiplets or other components previously produced with different functions, processes, materials, or by different manufacturers, are integrated into a single package according to length, width, and height in three dimensions. The system-

《Table 2.1.1》

Table 2.1.1 Top 10 engineering development fronts in information and electronic engineering

《Table 2.1.2》

Table 2.1.2 Annual number of core patents published for the Top 10 engineering development fronts in information and electronic