1. Introduction
Hyperspectral imaging technology is widely utilized in Earth observation, having evolved from a qualitative detection method to a quantitative identification tool with enhanced instrument performance
[1],
[2],
[3],
[4]. Imaging spectrometers with different spectral ranges are required for various applications. Hyperspectral imagers that cover ultraviolet (UV) wavelengths can detect gases such as SO
2 and NO
2 [5],
[6]. Owing to the maturity of visible near-infrared (VNIR) detectors, the 0.4–1.0 μm wavelength range in the VNIR region is mostly used in hyperspectral sensors. VNIR hyperspectral imaging spectrometers have been applied to vegetation studies using red-edge spectral features
[7],
[8],
[9]. Shortwave infrared (SWIR) hyperspectral sensors are typically used in geologic surveys because carbonates, sulfates, and hydroxyl-bearing minerals exhibit strong spectral features in the 2.0–2.5 μm wavelength range
[10],
[11]. Thermal infrared (TIR) sensors, also known as longwave infrared hyperspectral imagers, have been used to detect hazardous gases with absorption spectral signatures in the 8.0–12.0 μm range and to determine the temperature of objects
[12],
[13],
[14]. Combining multiple hyperspectral sensors with different spectral ranges enhances the performance by extracting more spectral information
[15].
Data users require high-performance hyperspectral imagers, with spatial and spectral resolutions being the most critical parameters
[16]. Similar to general industrial cameras, the spatial resolution is crucial for hyperspectral sensors. A higher spatial resolution aids in extracting textural details or identifying small targets, which is essential for applications such as precise mapping
[17],
[18],
[19]. Hyperspectral sensors provide continuous and fine-grained spectral information, unlike color or multispectral cameras. Adequate spectral resolution is vital for distinguishing objects with similar shapes, such as when classifying and identifying tree species under complex conditions
[20],
[21]. However, it is necessary to balance the spatial and spectral resolutions of hyperspectral imaging systems because of limitations in the signal-to-noise ratio (SNR)
[22]. SNR influences the results in many applications
[22],
[23],
[24]; thus, a trade-off between spatial resolution, spectral resolution, and SNR is essential for developing hyperspectral sensors. Achieving high spectral and spatial resolutions simultaneously is challenging for spaceborne hyperspectral payloads with long observation distances because of the trade-off between these core parameters. For example, spaceborne hyperspectral payloads such as the Environmental Mapping and Analysis Program (EnMAP)
[25], Precursore Iperspettrale della Mission Applicativa (PRISMA)
[26], and Gaofen-5
[27] satellites have a spectral resolution better than 10 nm, whereas their spatial resolution is 30 m. Similar tradeoffs can be observed in other spaceborne hyperspectral imagers, such as Hyperion
[28], the Compact High-Resolution Imaging Spectrometer (CHRIS)
[29], the Compact Reconnaissance Imaging Spectrometer for Mars (CRISM)
[30], the DLR Earth Sensing Imaging Spectrometer (DESIS)
[8], the Hyperspectral Imaging Satellite (HySIS)
[31], the Hyperspectral Infrared Imager (HyspIRI)
[32], and the Copernicus Hyperspectral Imaging Mission for the Environment (CHIME)
[33].
Although spaceborne hyperspectral imagers offer extensive ground coverage, achieving high spatial resolution is challenging. Moreover, the cost of developing spaceborne hyperspectral sensors is significantly higher than that of developing airborne sensors. Therefore, airborne hyperspectral imagers with finer spatial and spectral resolutions than their spaceborne counterparts are often developed to verify essential techniques and explore various applications. Most airborne hyperspectral imagers focus on VNIR and SWIR wavelengths, such as the Airborne Imaging Spectrometer (AIS)
[34], Airborne Imaging Spectrometer for Different Applications (AISA)
[35], Airborne Prism Experiment (APEX)
[36], Airborne Reflective Emissive Spectrometer (ARES)
[37], Airborne Visible/Near-Infrared Imaging Spectrometer (AVIRIS)
[38], AVIRIS-Next Generation (AVIRIS-NG)
[39], AVIRIS-3
[40], Digital Airborne Imaging Spectrometer (DAIS-7915)
[41], Hyperspectral Mapper (HyMap)
[42], Hyperspectral Digital Imagery Collection Experiment (HYDICE)
[43], and Compact Airborne Spectrographic Imager (CASI)/Short-Wave-Infrared Airborne Spectrographic Imager (SASI)
[44]. Moreover, the Hyperspectral Thermal Emission Spectrometer (HyTES)
[12] has been developed to explore thermal wavelength applications. In addition, hyperspectral imagers such as the Airborne Hyperspectral Scanner (AHS)
[45], Multispectral Infrared and Visible Imaging Spectrometer (MIVIS)
[46], Operational Modular Imaging Spectrometer (OMIS)
[47], and Sysiphe
[48] offer wide wavelength coverage from VNIR to TIR.
In recent years, hyperspectral sensors based on unmanned aerial vehicles (UAVs)
[49],
[50],
[51],
[52] and hyperspectral imagers based on ground platforms have been developed
[53]. UAV hyperspectral sensors can achieve centimeter-level spatial resolution, whereas hyperspectral imagers provide higher temporal resolution by continuously observing the same region from a fixed angle. By leveraging the advantages of airborne hyperspectral imagers, an airborne multimodular imaging spectrometer (AMMIS) was designed in 2012 as a critical part of China’s High-Resolution Earth Observation Program (HREOP). AMMIS, China’s new-generation airborne hyperspectral imaging system based on OMIS
[47], has been applied in numerous civilian fields and is used to verify key techniques for future hyperspectral satellite missions.
The main specifications of the AMMIS are shown in Fig. S1 in Appendix A and compared with other hyperspectral imaging systems in
Table 1[35,
36,
39,
44]. It can be noted that the AMMIS excels in spatial resolution, field of view (FOV), spectral coverage, and spectral resolution. To accommodate diverse application requirements, the AMMIS integrates multiple modules covering the UV, VNIR, SWIR, and TIR wavelengths to achieve a wide spectral range. This integration facilitates the exploration of potential applications and supports the feasibility of hyperspectral satellite platforms with multiple optical payloads. By employing FOV stitching technology, we achieved a large FOV of 40°, thereby enhancing efficiency. Moreover, the VNIR module features two operation modes, adjusting high spatial and spectral resolutions based on application requirements. Using a high spatial resolution model, VNIR imagers can obtain centimeter-level ground sample distances at a flight altitude of 2 km. Furthermore, the AMMIS offers high spectral resolution with over 1400 bands and utilizes the push-broom imaging mode to enhance the SNR. Since 2016, we have conducted more than 30 flight campaigns across various provinces in China, accumulating approximately 200 TB of hyperspectral data. These data have been applied to land-cover classification, environmental monitoring, water quality inspection, and crop investigation. Applications of the AMMIS in these fields have demonstrated its superior performance and provided valuable feedback for further instrument optimization.
This study presents the design, performance testing, calibration results, and applications of the AMMIS for the first time. We introduce the key techniques for system integration, optical systems, electronics, and thermal control, along with the calibration methods and performance test results. We also describe the recent flight campaigns and outcomes of using the AMMIS for various applications. In addition, we present three hyperspectral datasets with different scene distributions recorded based on ground investigations, whose classification characteristics have been validated using artificial intelligence models, including traditional machine learning and deep learning. Finally, we discuss the limitations of the AMMIS, our future research plans, and the outlook for next-generation hyperspectral sensors. This study serves as a reference for hyperspectral sensor designers and users of hyperspectral data.
2. System design
2.1. Overview
Table 1 presents the primary specifications of the AMMIS, whereas
Figs. 1(a) and
(b) show the instrument and aircraft platforms, respectively. The AMMIS comprises imaging and control systems, as shown in
Fig. 1(a). The imaging system integrates UV, VNIR, SWIR, and TIR modules, achieving extensive wavelength coverage to satisfy diverse application requirements. The UV module was developed by the Anhui Institute of Optics and Fine Mechanics (AIOFM), Chinese Academy of Sciences (CAS). In contrast, the VNIR, SWIR, and TIR modules were developed by the Shanghai Institute of Technical Physics (SITP), CAS. UV imaging spectrometers are used to detect pollution gas leaks
[54], whereas VNIR and SWIR imagers provide reflectance information on ground objects
[55]. TIR sensors play crucial roles in the detection and identification of minerals
[13]. By combining multiple modules with varying spectral ranges, the system addresses the distinct spectral and spatial resolution requirements across different application domains. For example, atmospheric detection requires a high spectral resolution for gas absorption peak determination, with spatial resolution less critical. In contrast, applications, such as precise mapping, require VNIR and SWIR modules with high spectral and spatial resolutions to differentiate between similar species. However, the trade-off between the SNR and spatial/spectral resolutions must be carefully considered during the design phase
[22]. The design values of the spatial and spectral resolutions for all modules (
Table 1) reflect a balance between core specifications, application demands, and technical feasibility.
We used the push-broom imaging mode to improve the SNR of the AMMIS
[2]. Although this mode offers a narrower FOV than the whiskbroom imaging mode, field-stitching technology is employed to broaden the FOV. Each module incorporates three spectrometers, enabling the FOV to expand to 40° through stitching. The modulation transfer function (MTF) is a metric for camera imaging quality, with high-quality images achievable through imagers boasting a high static MTF. However, the optical system and detector performance affect the MTF of the hyperspectral imaging system. Considering the optical system complexity and detector performance, we designed the static MTFs of the AMMIS to be ≥ 0.2 for the UV, VNIR, and SWIR modules and ≥ 0.15 for the TIR module. The velocity-to-height ratio (VHR)
[55],
[56] is the ratio of aircraft velocity to altitude and matches the imager’s frame frequency. In a push-broom airborne hyperspectral imaging spectrometer, the aircraft progresses forward when integrating two consecutive frames, necessitating an imager frame frequency that aligns with the VHR for seamless frame stitching. The AMMIS is mainly designed for manual aircrafts, such as the Y-5, Y-12, Cessna 208, and XZ-60, as shown in
Fig. 1(b). The flight height and velocity of these aircrafts are in the range of 1–3 km and 150–220 km·h
−1, respectively, indicating that the VHR of the AMMIS is approximately 0.02–0.04 s
−1. The frame frequency and integration time can be adjusted to satisfy the diverse manual aircraft VHR requirements.
We innovatively adopted a multimodular design for the AMMIS to achieve these performance objectives, incorporating the following four modules: UV, VNIR, SWIR, and TIR. We integrated these modules into an upgraded stabilization platform based on the Leica PAV 80, enabling simultaneous imaging across a wide wavelength range, as shown in
Fig. 2(a). Each module contains three spectrometers spliced together to achieve a 40° FOV, with overlapping fields between adjacent spectrometers
[57]. The VNIR and SWIR modules include three VNIR and three SWIR imaging spectrometers, as shown in
Fig. 2(b). As shown in
Fig. 2(c), the TIR, VNIR, SWIR, and UV modules are sequentially aligned along the track direction to achieve an unoccluded FOV through optimization and configuration. The layout of a single VNIR hyperspectral imaging spectrometer is shown in
Fig. 2(d). In the cross-track direction, the height of the VNIR/SWIR module is adjusted to obtain an unoccluded FOV. Furthermore, we utilized the Position and Orientation System (POS) 610 from Trimble Applanix to acquire data on the aircraft attitude measurements for subsequent geometric corrections. All the electronic control devices for the four modules are housed together in a cabinet and connected to the spectrometers via cables. These modules can operate independently or flexibly in combination to complete remote sensing tasks. The UV module primarily comprises optical and electronic submodules
[54]. Owing to space limitations, the three imaging spectrometers have the same optical system assembled in an L-shaped optical box. The data acquisition and transformation of printed circuit boards in electronics have high power consumption, which affects the temperature. Therefore, we installed 5-mm wide fiber-reinforced plastics to prevent heat transfer between the two submodules. Each imaging spectrometer contains a lens hood, fore-optics, transition mirror, collimator, prism-grating dispersive component, re-imager, detector, and electronics
[58]. To reduce the interference from stray light, the support materials of the optical mirrors are blackened. The TIR module comprises three optical components designed for low temperatures
[57]. Two mechanical cryocoolers are used to maintain the spectrometer and focal plane array (FPA) at 100 and 77 K, respectively.
2.2. Optical system and performance
The optical system, which determines the image quality of the hyperspectral data, is one of the most critical parts of the AMMIS.
Figs. 3(a)–(c) show the optical system design and performance of all modules, depicting the optical system of a single spectrometer. The optical layouts of the three spectrometers in the UV
[54], VNIR/SWIR
[58], and TIR
[57] modules are identical.
2.2.1. UV module
In this study, the UV spectrometers, which are essential for high spectral resolution in atmospheric detection applications, are divided into three channels, covering wavelengths of 200–276, 276–380, and 380–500 nm. Each spectrometer achieves a spectral resolution finer than 0.5 nm
[54]. Wavelength segmentation significantly eases the chromatic aberration correction challenges in lens design and enhances the optical system quality by simplifying the convex grating design. Owing to the lower spatial resolution requirements, each spectrometer within the UV module achieves a 40° FOV, eliminating the requirement for FOV stitching while ensuring complete wavelength coverage from 200 nm to 500 nm. As shown in
Fig. 3(a), the UV imaging spectrometer comprises fore-optics, slit, concave mirror, convex grating, and a charge-coupled device (CCD) detector. The convex grating and concave mirror make up an Offner-structure spectrometer, offering the advantages of minimal primary aberration, generous relative aperture, and low smile and keystone
[59]. The response wavelength of the CCD detector spans 200–1000 nm, with dimensions of 1032 × 1072 pixels. In the across-track direction, 1000-pixel samples at the central position of the detector are used to obtain spatial information. Moreover, 1024 pixels along the spectral (band) direction of the detector are used to collect spectral information on the targets
[54]. Pixel binning is implemented to facilitate grating production and enhance spectroscopic performance. A static MTF evaluation yielded a superior result of 0.459 for the UV module, exceeding the design specifications.
2.2.2. VNIR/SWIR module
As shown in
Fig. 3(b), VNIR and SWIR spectrometers exhibit similar optical systems, including dispersive components and re-imagers
[58]. We designed fore-optics using an off-axis three-mirror anastigmatic (TMA) telescope, which has the advantages of low distortion, high throughput, compact volume, and excellent image quality. Furthermore, the TMA can be matched and integrated with spectrometers using appropriate pupils. The distance between the slit and the entrance pupil is 695 mm, and the focal length of the TMA is 128 mm. Simulation results indicated that the root-mean-square (RMS) radii of the telescope spot size are less than 5.7 μm for all the FOVs. To reduce smile and keystone distortions, we developed a novel prism-grating (PG) spectrometer design comprising a prism and a reflective flat-blazed grating. The grating disperses light, whereas the prism compensates for the spectral distortions, offering a low-cost, feasible, and simple solution. To simplify the optical structure, we employed a reflective–refractive configuration in the spectrometer. The collimator uses an on-axis spherical mirror to avoid chromatic aberrations, and the re-imager employs a six-element lens to correct the aberrations introduced by the collimator. This re-imager design reduces the system cost and volume, making it suitable for alignment and aberration correction.
We combined a spectrometer with the TMA telescope to evaluate the performance, and the results indicated that the MTF of the PG imaging spectrometer exceeded 0.3 for all testing positions, surpassing the designed specifications for all testing positions. The smile and keystone distortions were less than ±5% of one pixel
[55],
[58], as shown in
Fig. 4. The spectrometer and re-imager designs of the SWIR spectrometer are similar to those of the VNIR spectrometer. The RMS radii of the SWIR optical system are less than 8 μm for all the FOVs, and the smile and keystone are less than 3 μm. We used a CCD detector of size 2048 × 256 pixels for the VNIR module and a mercury-cadmium-telluride (MCT) detector for the SWIR module.
2.2.3. TIR module
The TIR module comprises three spectrometers
[57], each with an 18.18° FOV, resulting in a total FOV of approximately 40°, excluding overlaps between adjacent spectrometers. We used a refractive telescope for the fore-optics to reduce distortion and cost. The optical simulation showed that the distortion of the telescope is less than 0.6 μm, resulting in high-quality images. We used a reflective triplet configuration in the spectrometer design, incorporating a flat grating to achieve a high optical throughput, compact volume, and high optical speed, as shown in
Fig. 3(c). Unlike UV and VNIR/SWIR modules, which capture the reflected signals of solar radiation, TIR imaging spectrometers mainly capture the self-radiation of the targets. Background radiation significantly affects TIR modules, making its suppression a critical design consideration
[60]. Therefore, we implemented a cryogenic optical design for the TIR spectrometers by maintaining them at 100 K. Specifically, the three spectrometers and Focal Plane Arrays (FPAs) were assembled in a vacuum chamber and cooled to 100 K using two mechanical coolers. The telescope is operated at room temperature, whereas the FPAs are cooled to 77 K using Stirling cryocoolers, as shown in
Fig. 5. We measured the TIR optical system in the laboratory, and the MTFs of all the FOVs exceeded 0.26. The smile and keystone were assessed using filters, black bodies, and slit-perpendicular filters, as shown in
Fig. 6 [57]. We employed an MCT detector with a size of 320 × 256 pixels in the TIR module.
2.3. Electronic control module
As the brain of the AMMIS, the electronic control module (ECM) oversees the self-detection capability of the imaging spectrometers, detector drives, data acquisition, data storage, and the operation of the stabilization platform. As illustrated in
Fig. 7, the ECM comprises a camera control system, data acquisition system, stabilization platform control system, and POS. The ECM sends control commands via Recommended Standard-422 (RS-422) serial communication to manage the power supply, coolers, and parameter configurations.
The camera control system, which is a core part of the ECM, converts the optical signals collected by the optical systems into electronic signals
[55]. The operational statuses of all the cameras are also fed back to the ECM. The image data from the cameras are transmitted to the composite data circuit board via low-voltage differential signal (LVDS) lines, and stored and displayed by the data-acquisition system. For convenience, the stabilization platform and POS control systems operate as independent modules. A stabilization platform control system is used to limit the attitude of the system within a specific range, thereby reducing the drift of the boresight of the imaging spectrometer caused by aircraft vibration. Typically, a three-axis stabilization platform isolates aircraft vibrations in the roll, pitch, and yaw directions and controls the attitude of push-broom cameras. For the AMMIS, we upgraded Leica’s PAV 80 stabilization platform to control the attitude; it was also necessary to upgrade the platform to increase the carrying weight. The POS includes an inertial measurement unit (IMU), global positioning system (GPS) receiver, computer system, and data processing software. We utilized the POS 610 from APPLANIX to obtain high-precision position and attitude information.
2.4. Thermal control and aircraft platforms
The ambient temperature influences the performance of electronic components in a hyperspectral imaging system, which in turn affects the data quality. Consequently, temperature control is a key consideration in the design of AMMIS, particularly TIR imaging spectrometers. We conducted a thermal analysis and designed the AMMIS to operate at flight altitudes of 1–6 km in extreme weather conditions. The simulation results identified heat sources such as coolers that require temperature reduction. The heat generated by the electronic components can maintain the optical components at 283–293 K during winter, eliminating the need for an additional heating system. However, in summer, effective heat dissipation is crucial for preventing instrument failure or performance degradation owing to high temperatures. To address this problem, heat pipes are installed to transfer heat from high-to low-temperature areas.
Fans and water-cooling tubes are mounted on the heat sources to reduce the temperatures of the electronic components and cooler parts. We also developed a mechanical cryocooler to reduce thermal noise in the SWIR detector. The thermal design significantly mitigated the thermal background effect for the UV, VNIR, and SWIR modules. However, this effect was still insufficient for the TIR module, as shown in
Table 2. The optical system of the TIR module requires a low-temperature operating environment to reduce the influence of the background radiance. As shown in
Fig. 5, we designed a low-temperature structure for the optical system of the TIR module. Specifically, we sealed the vacuum chamber interface with a metal knife-edge flange and installed an adsorption pump on the cabin lid to maintain the vacuum in the chamber
[61]. All reflectors, optical brackets, and substrates of the optical components were machined from aluminum alloy to ensure consistent deformation of the cryogenic optical system. We used epoxy to isolate the optical substrate from the vacuum chamber to reduce heat conduction. A multilayer heat shield with gold plating was used on both sides to prevent radiant heat leakage. To mitigate the impact of cooler vibrations on the optical system, we used a flexible thermal strap with high thermal conductivity to connect the optical substrate and coolers. All cryogenic optical system surfaces were blackened except for the mirror surface to satisfy the temperature gradient requirements. In addition, indium was installed on the interface parts to reduce the heat transfer resistance during the assembly of the cryogenic optical system.
We designed the AMMIS primarily for manned aircraft platforms, which offer the advantages of high speed, large load capacity, and extended flight duration. Manned aircraft can accommodate multiple high-performance payloads for comprehensive remote sensing and large-area missions. Airborne remote sensing with human participation is more likely to have real-time capabilities because onboard engineers can process and adjust data according to real-time requirements. This capability is crucial for environmental monitoring, search and rescue operations, and law enforcement inspections. Consequently, airborne hyperspectral imaging spectrometers continue to be essential for various Earth observation fields and satellite hyperspectral missions
[2],
[62].
As shown in
Fig. 1(b), the aircrafts used for the AMMIS include the Y-5, Y-12, XZ-60, and Cessna 208. We analyzed the characteristics of these aircraft platforms to confirm their compatibility with AMMIS, and the results indicated that the Y-12 and XZ-60 met the requirements for carrying all modules. The Y-12 is a light and multipurpose transport aircraft developed from the Y-11 and has been used in cargo transportation, agriculture and forestry operations, and geological exploration. Furthermore, the Y-12 aircraft can provide a voltage direct current of 28 V, which is suitable for powering the AMMIS, and its maximum power consumption of 5 kW, which satisfies the AMMIS requirements. In addition, the Y-12’s optical remote-sensing window is a non-airtight, glass-free window that permits airflow and can accommodate the PAV 80 stabilization platform. The XZ-60 has characteristics similar to those of the Y-12, enabling it to carry all the AMMIS modules. The AMMIS modules can also be flexibly configured for installation on other manned aircraft platforms.
3. Calibration and evaluation
The quality of hyperspectral data significantly influences subsequent applications. In this section, we introduce the preprocessing methods for AMMIS, including laboratory and in-flight calibrations, geometric corrections, and atmospheric corrections. We also report the system performance test and evaluation results.
3.1. Laboratory calibration and performance measurement
In general, radiometric and spectral calibrations of hyperspectral imaging spectrometers are performed in the laboratory before the flight
[28]. Spectral calibration was performed to determine the position of the central wavelength, the spectral response function (SRF), and the spectral resolution. Radiometric calibrations are divided into relative and absolute calibrations; the former improves the non-uniform flat field of the image data, while the latter converts the digital number (DN) value output from the imagers into spectral radiance.
Spectral calibration of the UV, VNIR/SWIR, and TIR modules were conducted in a laboratory, as illustrated in
Figs. 8(a) and (b). We used a tunable laser and mercury-argon lamp as light sources for spectral calibration
[54]. The characteristic emission spectrum of a mercury-argon lamp was employed to calibrate each spectral channel of the UV module, enabling the calculation of the spectral coverage, spectral resolution, and SRF. The mercury-argon lamp was used as the light source to irradiate the focusing lens and diffuse the transmission plate into the imagers, and its characteristic spectrum was recorded. The least-squares method was used to fit the collected data to a Gaussian function to enable us to characterize the spectral curves, assuming a linear spectral response. We calculated the SRF of the UV spectrometers from nine viewing angles and retrieved the spectral resolutions; detailed results are presented in Ref.
[54]. In addition, various calibration sources were used to complete the radiometric calibration of the UV, VNIR/SWIR, and TIR module spectrometers (
Fig. 9(a) and
(b)). A diffuser plate based on a xenon lamp was used for spectrometer 1, whereas an integrating sphere based on a quartz–tungsten–halogen (QTH) lamp was used for spectrometers 2 and 3, as shown in
Fig. 9(a). A well-calibrated spectral radiometer was used to monitor the radiance of the radiometric calibration system. The absolute radiometric calibration accuracies of the three spectrometers were 4.89%, 4.67%, and 4.42%, respectively
[54].
We performed the radiometric calibration of the VNIR and SWIR modules using an integrating sphere in the laboratory, as illustrated in
Fig. 9(b). Atmospheric transmission was considered to manage water vapor and carbon dioxide absorption in laboratory measurements. The integrating sphere was calibrated, and the output radiance was adjusted by changing the number of lights switched on. Calibration data for the output radiance were collected from at least five levels of the integrating sphere, and appropriate camera parameters were set to avoid signal saturation. First, we employed calibration data for relative radiometric correction to reduce the nonuniformity between adjacent pixels. In addition, statistics-based and spatial filtering methods have been applied to remove residual stripe noise
[63],
[64]. We calculated the calibration results and achieved relative radiometric accuracies of 1.19% and 2.4% for the VNIR and SWIR imaging spectrometers, respectively
[55]. Second, absolute radiometric calibration was conducted using laboratory calibration data. The results indicated accuracies of ≤ 5.91% and ≤ 6.54% for the VNIR and SWIR imaging spectrometers, respectively. Spectral calibration of the VNIR and SWIR modules was performed using an illumination source, monochromator, collimator, and control system, as depicted in
Fig. 8(b). A high-power bromine-tungsten lamp served as the light source, and a mercury lamp with a known wavelength was used to correct the monochromator. The spectral calibration accuracies of the VNIR and SWIR imaging spectrometers were 0.109 nm and 0.3 nm, respectively. Furthermore, we developed a spectral calibration method based on the absorptive characteristics of water vapor and a monochromator to enhance spectral calibration accuracy in the laboratory
[65]. The measurement results of the SRF and spectral resolution of the VNIR imaging spectrometers reported in Refs.
[55],
[58] indicate that the developed instrument meets the design objectives. The SRF testing results of the VNIR, SWIR, and TIR imaging spectrometers are shown in
Figs. 10(a)–(c); it can be seen that the fitted spectral curves are almost Gaussian.
We performed a radiometric calibration of the TIR module using a blackbody in the laboratory, as illustrated in
Fig. 9(b) [13]. The calibration experiments were conducted at blackbody temperatures of 293, 313, 333, and 353 K to obtain the radiometric calibration coefficients. Relative and absolute radiometric calibrations were performed to convert the DN value of the imaging spectrometer into radiance. Similar to the VNIR and SWIR modules, destriping methods have been used to remove residual noise from TIR hyperspectral images after radiometric calibration
[63],
[64]. Ultimately, absolute calibration accuracies greater than 1 K were achieved for the three TIR module spectrometers. For the spectral calibration of the TIR module, we used a silicon carbide rod as the light source and an iHR550 monochromator to scan the wavelengths from 8 to 12.5 μm, as illustrated in
Fig. 8(b) [13],
[57]. The scan step of the monochromator was set to 0.5 nm, and the calibration data were recorded and processed to obtain the SRF of the TIR spectrometers, as described in Ref.
[57]. To measure the accuracy of the spectral calibration, we used a carbon dioxide laser with a precisely known wavelength as a light source for comparison with the TIR spectrometer. The spectral shift of the spectrometer was measured and corrected using this method, and a spectral calibration accuracy of ≤ 1 nm was achieved. In addition, we conducted an ammonia gas detection experiment using a TIR imaging spectrometer and compared the results with those obtained using a gas spectrometer. The experimental results demonstrated that the TIR imaging spectrometer can obtain accurate spectral information for ammonia gas
[57].
The performance of hyperspectral imaging spectrometers critically influences their applications. Consequently, we conducted a performance test of the AMMIS, both in the laboratory and outdoors, to verify its compliance with the design requirements listed in
Table 1. In addition, a flight experiment was conducted to test the interface between the stabilization platform and the aircraft. Imaging, spectral, and radiometric tests of the AMMIS were also performed in a laboratory.
The imaging performance measurements included FOV tests, instantaneous field-of-view (IFoV) tests, and MTF assessments. We employed the knife-edge scan method
[66] to measure the MTF of the UV module, and the slit-scan method
[55] for the VNIR, SWIR, and TIR modules. FOV and IFoV tests were performed using a collimator, reflector, and turntable
[55],
[58]. As shown in
Fig. 8, we primarily utilized a light source, monochromator, collimator, reflector, and turntable for the laboratory-based spectral property test. Special light sources, such as mercury lamps with known wavelengths, were employed to improve the accuracy of the spectral calibration. Spectral calibration accuracy is mainly influenced by data fitting errors
[54] and monochromator errors. As shown in
Fig. 9, we evaluated the radiometric performance of AMMIS in the laboratory using an integrating sphere and a blackbody. Radiometric calibration accuracy is primarily affected by the unsteady error of the calibration system, nonlinearity errors, and imager instability
[55]. Moreover, we measured the system sensitivity of the AMMIS in the laboratory based on the results of radiometric calibration, characterized by the SNR for the UV, VNIR, and SWIR modules
[55] and the noise equivalent temperature difference (NETD) for the TIR module
[67]. The radiometric measurement conditions for the UV, VNIR, and SWIR modules were a sun altitude angle of 60° and a ground albedo of 0.3. For the TIR module, the target was a black body at 300 K. Detailed measurement methods are discussed in Ref.
[55]. The performance test results indicated that the developed system satisfied the design requirements, with some parameters exceeding the design values.
Table 3 summarizes the performance test results of AMMIS.
3.2. In-flight calibration
System state changes, such as slight movements of components owing to platform vibrations or temperature changes, can cause spectral shifts in the response of an imaging spectrometer. Thus, in-flight calibration is essential for obtaining accurate radiometric information and enhancing spectral calibration accuracy. We performed an in-flight calibration of the UV module to measure the spectral shift compared with the spectral calibration results in the laboratory
[54]. Slit-function convolution was applied to fit the tested solar spectra to a high-resolution solar reference
[68]. The spectral calibration results obtained in the laboratory served as the initial values for the fitting method, and optimal results were achieved after multiple iterations. The spectral shifts at different positions in the across-track direction were obtained, and the results are provided in Ref.
[54].
To improve the radiometric calibration accuracy of the VNIR and SWIR modules, we conducted an in-flight calibration using five diffuse reflectance panels (Labsphere, Inc., USA) with known reflectance. These targets were placed in the AMMIS flight areas to ensure each panel occupied at least 4 × 4 pixels of the image. We employed an Analytical Spectral Devices (ASD; ASD Field Spec 3; ASD Inc., USA) FieldSpec spectrometer to test the radiation characteristics of the panels synchronously, as shown in Figs. S2(a) and (b) in Appendix A.
[55]. To validate the spectral calibration results, we compared the radiation curves of typical ground objects imaged by AMMIS with those obtained using the ASD spectrometer. The residual error between the two results was ≤ 0.02 W·(m
2·nm·Sr)
−1, as shown in
Fig. S2(c). In addition, we have developed an in-flight spectral calibration method to obtain accurate wavelength positions for VNIR imaging spectrometers
[69]. This method mitigates the influence of the underlying atmospheric surface, receives shifts in the center wavelengths, and improves the uncertainty of spectral calibration to ≤ 0.05 spectral pixels. We have also developed an in-flight calibration algorithm to enhance the spectral calibration accuracy of SWIR imaging spectrometers
[70]. The spectral shifts of all bands were monitored based on the atmospheric absorption characteristics of water vapor at 2000 nm and carbon dioxide at 1269 nm, yielding a 3
σ (
σ means standard deviation) confidence interval of ±0.38 nm.
In contrast to the UV, VNIR, and SWIR modules, the TIR module measures the radiance of ground objects, which is easily affected by background radiance. Moreover, variations in the instrument state can affect the calibration results derived from laboratory data. Consequently, we performed an in-flight calibration of the TIR module to enhance its calibration accuracy. Calibration data from a blackbody mounted on an aircraft were used to convert the DN values of the TIR spectrometers to the target emissivity and temperature. Two modes were used: blackbody calibration and blackbody imaging. In the blackbody calibration mode, the blackbody occupied the full FOV of the TIR spectrometers, and in the blackbody imaging mode, the blackbody was removed. We also used the calibration data to complete the nonuniformity correction and removed the residual stripe noise caused by the nonuniformity of the detector or slit contamination using destriping methods
[13],
[63],
[64]. After the destriping process, the nonuniformities of the VNIR, SWIR, and TIR modules improved from 3.88%, 4.27%, and 4.71% to 1.09%, 0.25%, and 0.41%, respectively
[63]. Furthermore, to improve the calibration accuracy of the TIR module, we conducted an on-site calibration during flight. Ground objects with emissive spectral characteristics similar to those of a blackbody, such as water, were selected as reference targets. The atmospheric conditions and temperature were measured for recalibration to ensure higher precision in the calibration process.
3.3. Geometric and atmospheric correction
Airborne hyperspectral imaging spectrometers are susceptible to platform vibrations and attitude variations, which lead to image distortion across and along the track direction. Consequently, we performed a geometric correction for AMMIS to obtain accurate geometric and geographic information on the ground targets. Geometric calibration was performed using the exterior orientation method
[71]. Data obtained from the POS 610, encompassing parameters such as time, latitude, longitude, altitude, heading, pitch, and roll angles, were used to calculate the exterior orientation elements. Therefore, it was possible to establish a correspondence between the positions of all image pixels and the actual ground coordinates. Image registration among the VNIR, SWIR, and TIR imaging spectrometers was performed using commercial software, such as the Environment for Visualizing Images (ENVI).
Fig. S3 in Appendix A shows the image before and after the geometric correction of the VNIR module, respectively, highlighting the efficacy of the geometric correction in rectifying image distortions. The relative and absolute geometric correction accuracies of AMMIS were 1.0 and 3.9 pixels, respectively, with an image registration accuracy between different modules being better than 1.0 pixel. To enhance the performance of the AMMIS, we jointly performed precise geometric correction for the VNIR and SWIR modules with Wuhan University
[72]. First, homospectral stitching was performed between the three imaging spectrometers to obtain a large-stripe image. Second, a coarse geometric correction was implemented using the POS 610 data to generate P2A-level products. Third, the image registration between the VNIR and SWIR images was completed with an accuracy of more than half a pixel. The geometric correction was refined by combining aerial triangulation with digital elevation map data. Correction results reported in Ref.
[72] indicate a significant improvement in the accuracy of the geometric correction and the correction of detailed distortions.
Atmospheric correction is crucial for obtaining accurate surface reflectance data using hyperspectral imaging spectrometry. We performed atmospheric correction for the AMMIS using several methods based on the application requirements. Cloud effects were disregarded during the flight campaign, which focused on nitrogen dioxide detection with the UV module, owing to clear weather conditions. Given the similar spatial resolution to the Landsat 8 Operational Land Imager (Ball Aerospace & Technologies Corporation, USA)
[73], we used the surface reflectance products of Landsat 8 from the nearest overpass day for atmospheric correction of the UV imaging spectrometers
[54]. Atmospheric correction was conducted for the TIR module based on atmospheric radiative transfer, supplemented by auxiliary data from synchronous ground experiments
[13]. For the VNIR/SWIR module, we collaborated with the Institute of Remote Sensing Applications (IRSA), CAS, and utilized two methods for atmospheric correction. The first method involves synchronous measurements on the ground, capturing the reflectance and radiance of pure pixels in flight areas to establish the relationship between the DN values and reflectance
[55]. The second method utilizes atmospheric models such as Fast Line-of-sight Atmospheric Analysis of Spectral Hypercubes (FLAASH)
[74] to derive the surface reflectance of ground objects. The first method is labor-intensive, whereas the second relies on accurate atmospheric conditions. Therefore, more advanced atmospheric correction methods that are less laborious and more precise are required for the AMMIS to achieve high-precision surface reflectance.
4. Applications
We conducted a series of flight campaigns across various aircraft platforms to validate the performance of the AMMIS and explore its applications. This section provides a detailed account of the flight campaigns and discusses the characteristics of the data obtained from the AMMIS. It also reports on three distinct hyperspectral datasets exhibiting diverse scene distributions generated by the VNIR module for classification and identification. The classification accuracy was evaluated using artificial intelligence (AI) algorithms ranging from traditional machine learning to deep learning methodologies. Finally, we report some primary application results of the AMMIS.
4.1. Flight campaigns and hyperspectral data cube
We conducted the first flight campaign on February 20, 2016 to test the performance of each AMMIS module in Dongfang City, Hainan Province, China. We completed the flight experiment using a Y-12 aircraft at an altitude of 3 km and a speed of 220 km·h−1 from 10:00 am to 2:55 pm (China Standard Time, CST). During this experiment, system cooling was performed at an airport, allowing for the assessment of cooling time and cooler functionality. In addition, the joint debugging of electronic systems and measurement of control modules were performed on an aircraft. The success of this flight campaign was of significant importance, as it provided a reference for enhancing the AMMIS in subsequent stages.
Since 2016, we have performed over 30 flight campaigns in China to validate the performance of the AMMIS and fulfill its application tasks; the detailed information of some flight campaigns is provided in
Table 4. Typically, flight experiments were conducted on sunny days to minimize atmospheric influence, with the flight height set at 0.8–4 km based on the aircraft and task requirements. The number of flight lanes and working time of the system per flight campaign were related to the task specifications. Substantial hyperspectral data exceeding 200 TB were collected between 2016 and 2022. Fig. S4 in Appendix A shows a sample hyperspectral data cube from the AMMIS containing data across various wavelengths for the same ground targets. The numbers of spectral bands for the VNIR, SWIR, and TIR modules were 256, 512, and 140, respectively. A high spectral resolution imaging mode was employed at a flight altitude of 2 km, resulting in spatial resolutions of 0.5 , 1 , and 2 m for the three hyperspectral data cubes. The results shown in Fig. S4 demonstrate the capability of the AMMIS to acquire high-quality hyperspectral data over a wide wavelength range.
4.2. Hyperspectral datasets used in land-cover classification and super-resolution reconstruction
Land-cover classification and identification are crucial for many applications
[75],
[76]. The high spatial and spectral resolutions of the AMMIS-generated hyperspectral data facilitate land-cover classification
[77],
[78],
[79],
[80],
[81] and super-resolution reconstruction
[82],
[83],
[84],
[85],
[86].
Figs. 11 (a) and
(b) show a color image and reference map of the Matiwan village hyperspectral dataset obtained from the 2017 flight campaign in Matiwan village, Xiong’an City, Hebei Province, China. The dataset features a spatial resolution of 0.5 m and a spectral resolution of 2.4 nm within the wavelength range of 0.4–0.95 μm. To create the reference map, we collaborated with the IRSA and CAS to gather and verify the ground-truth data, as shown in
Fig. 11(b). The Matiwan dataset offers a larger scale, higher spatial resolution, and higher spectral resolution than other public hyperspectral datasets, such as Indian Pines and Pavia University
[87]. The image dimensions were 3750 × 1580 pixels with 20 categories, providing 3 677 110 available samples excluding the background. These characteristics make the Matiwan dataset particularly valuable for developing advanced deep-learning algorithms that require large sample sizes for training hyperspectral classification and super-resolution reconstruction models. Because most categories in the Matiwan dataset consist of economic crops, they are suitable for specialized category research
[22].
Most categories in the Matiwan dataset were uniformly distributed, and the samples in each category were clustered into large patches. For comparison, another large-scale hyperspectral dataset was constructed, as shown in
Fig. 12. The dataset was obtained in 2018 from a suburb of Xiong’an City, Hebei Province, China. Owing to the identical flight altitudes of the two campaigns, the Xiong’an suburb hyperspectral dataset exhibits the same spatial resolution, spectral resolution, and wavelength coverage as the Matiwan dataset. The size of the Xiong’an suburb dataset was 2821 × 2840 pixels, with 7 935 209 available samples excluding the background. Six damaged bands were then removed, resulting in a dataset of 250 spectral bands. The Xiong’an suburban dataset was divided into eight categories and contained complex patches characterized by high heterogeneity within each patch.
To further examine the characteristics of the hyperspectral datasets generated by the AMMIS, we collaborated with the Aerospace Information Research Institute, CAS, to produce a third dataset acquired in Shenyang City, Liaoning Province, China, in 2020
[88]. Unlike the previous two datasets, the Shenyang dataset features a spatial resolution of 0.75 m at a flight height of 3 km, comprising 190 spectral bands after the bands with low SNR were removed. The scene distribution of the Shenyang dataset was simple, with an image size of 651 × 837 pixels and 544 887 available spectra. The Shenyang dataset is classified into five categories: rice, trees, corn, grass, and bare land, as shown in
Fig. 13. Owing to its unique properties, the Shenyang dataset was used to evaluate the hyperspectral unmixing and classification
[88].
To analyze the data of different scene distributions obtained from the AMMIS, we performed classification experiments on three hyperspectral datasets using AI algorithms, including traditional machine learning and state-of-the-art deep learning models. We adopted the random forest (RF) classifier, a representative traditional machine learning model, for hyperspectral image classification
[89]. Three convolutional neural network (CNN)-based deep learning methods were used, comprising a two-dimensional (2D) CNN
[80] and two three-dimensional (3D) CNN
[80],
[90] algorithms. Furthermore, we used two state-of-the-art transformer networks, a transformer model vision transformer (VIT)
[91] and a robust vision transformer (RVT)
[92], to compare the performances of these AI algorithms. In the experiment, a random sampling method was adopted with samples drawn without replacement to form the training, testing, and validation sets. The proportions of the training and testing sets for each dataset were 10% and 90%, respectively, and 5% of the samples from the training set were used as validation sets. Using the pre-experimental results, we set the epoch numbers for the Matiwan, Xiong’an suburb, and Shenyang datasets to 50, 50, and 100, respectively. We implemented the classifiers in Python 3.8 and employed Scikit-learn for traditional machine learning and the PyTorch framework for deep learning models. To evaluate the classification results, we used the kappa coefficient (
κ) and overall accuracy (OA) as the main metrics, as delineated in
Table 5 [80],
[90],
[91],
[92]. Notably, the deep-learning-based methods used for the hyperspectral datasets generated by the AMMIS yielded classification accuracies exceeding 90%; in particular, the highest accuracies of the Matiwan and Shenyang datasets exceeded 95%. This result underscores the efficacy and robustness of deep learning approaches for effectively classifying and identifying land-cover categories within hyperspectral imagery, showcasing the potential of advanced AI algorithms in hyperspectral analysis tasks.
4.3. Gas detection
Pollution and hazardous gas detection are crucial for atmospheric monitoring and industrial management
[93],
[94], and the AMMIS plays a significant role in these applications. The UV module of the AMMIS has been used to observe nitrogen dioxide in the troposphere via the differential optical absorption spectroscopy (DOAS) technique
[54]. Moreover, TIR module imaging spectrometers have been employed to detect hazardous gases in both the laboratory and field.
During a flight campaign in Feicheng, China, in June 2018, the UV module of the AMMIS was employed to determine the vertical column density (VCD) of nitrogen dioxide in the troposphere. The mission was executed using a Y-5 aircraft and covered an area of 600 km2. The UV module offered a spatial resolution of approximately 25 m × 22 m. The process of obtaining the VCD of nitrogen dioxide involved four steps.
(1)Preprocessing: this step included relative radiometric correction, geometric correction, spatial pixel binning, and in-flight calibration to ensure high-quality spectral data.
(2)Slant column density (SCD) retrieval: the obtained hyperspectral data were analyzed using the DOAS technique to determine the SCD of nitrogen dioxide.
(3)
SCD to VCD conversion: the radiation transfer model
[95] and surface reflectance data from Landsat 8 were used to calculate the air mass factor, which facilitated the conversion of SCD to VCD.
(4)Spatial resampling: A spatial resampling method was applied to align the data with Google satellite maps.
The results demonstrate the capability of the UV hyperspectral imaging module to identify nitrogen dioxide emission sources, including steel factories, power plants, cement, and carbon factories. The spatial variability of nitrogen dioxide was clearly observed, with the steel factory generating the largest plume. Quantitative analysis and simultaneous ground-based testing validated the efficacy of the UV imaging module in monitoring nitrogen dioxide transmission and emissions
[54].
TIR hyperspectral imaging spectrometers with meter-level spatial resolution offer significant advantages in monitoring room temperature and hazardous gases, allowing the measurement of morphology, category, and concentration over long distances
[96]. Consequently, the gas-detection capabilities of the TIR module in the AMMIS were explored in both the laboratory and the field.
As depicted in
Fig. 14, the absorption spectrum of ammonia gas was measured within close range in the laboratory. Data collection commenced after cooling the optical system to 100 K to mitigate thermal background effects inside the instrument. Blackbody imaging experiments were conducted at 293 and 313 K, followed by radiometric calibration using blackbody data to derive the calibration coefficients. Background data were gathered by imaging a silicon carbide rod and calculating the background radiance. Subsequently, ammonia gas was inserted into a gas pool positioned between the silicon carbide rod and instrument to complete the gas measurement, as illustrated in
Fig. 14. The spectral curve of ammonia gas reported by the National Institute of Standards and Technology (NIST) was compared with the measured results, as shown in
Fig. 15. The comparison results indicated that the TIR module could accurately detect temperature-sensitive gases. Moreover, in a field experiment, sulfur hexafluoride (SF
6) was detected using a TIR module at a distance of 200 m, which revealed variations in the geometric morphology of the gas in the obtained images. These results demonstrate the capability of the TIR module for high-precision gas detection and its potential applications in various environmental monitoring scenarios.
4.4. Other applications
Mineral mapping plays a vital role in the development of society and the global economy
[97],
[98]. TIR hyperspectral imaging is particularly effective for this purpose, owing to its ability to capture rich spectral features in the 8.0–12.0 μm wavelength range
[99]. Consequently, the TIR module in the AMMIS has been employed for mineral identification
[13]. The emission spectra of various minerals, including quartz, epidote, calcite, and fluorite, supplied by the Wuhan Geological Survey Center, China Geological Survey, were measured using a TIR module in the laboratory. As illustrated in
Fig. 16, similar to the ammonia gas detection experiments, the optical system was cooled to 100 K to minimize the thermal background effects within the instrument before measurement. We performed blackbody imaging experiments at 293, 313, 333, and 353 K and recorded the results to calculate the radiometric calibration coefficients. To increase the temperature difference between the samples and their backgrounds, the minerals were heated to approximately 323 K using a heating plate. During the measurements, the temperature of the samples was carefully controlled to maintain stable radiation characteristics. After measuring each sample, a gold panel with a reflectivity of 0.94 in the TIR spectrum was used as the background reference. The gold panel was placed at the same position as the sample to measure the background radiance. Subsequently, data processing algorithms were used to determine the spectral emissivities of the minerals. To validate the measurements, a Fourier-transform infrared spectrometer (FTIS) with high spectral resolution was used to test the spectral emissivity of the minerals. The comparison results shown in
Fig. 17 indicate that the emissivity spectral curves measured by the two sensors were highly similar. In addition, the emission peak positions of the quartz particles matched the spectral laboratory curve published by the Arizona State University
[13]. These experimental results demonstrate that the TIR module of the AMMIS is effective for mineral identification in various applications.
The South–North Water Transfer Project (SNWTP) is a large-scale water conservancy initiative designed to channel water from the water-abundant Jianghuai region to the arid northern and northwestern areas to foster economic and social development
[100]. The eastern branch of the SNWTP extends from the Taihang Mountains to Tianjin City; construction in this region was completed in 2011, as illustrated in Fig. S5(a) in Appendix A. The construction marks are visible compared to the pre-construction scenario (Fig. S5(b)), as shown in Fig. S5(c). Although most canals on the eastern branch are underground, the vegetation above the canals shows signs of recovery. By April 3, 2016, Google Maps displayed almost no signs of construction activity, as shown in Fig. S5(d). However, hyperspectral images captured by the VNIR module of AMMIS on September 17, 2018, revealed abnormal vegetation growth at the construction site, marked by a red rectangle in Figs. S5(e) and (f). Fig. S5(d) presents a composite image using the AMMIS red, green, and blue bands, whereas Fig. S5(f) is a false-color image. Both images present abnormal vegetation growth, a phenomenon that was not visible in Google images from 2017 to 2019. Although this discrepancy may be because of the bandwidth capabilities of the AMMIS, further investigations are required to determine the cause of this anomaly.
As detailed in
Table 4, the AMMIS has conducted over 30 flight campaigns, completing numerous application tasks in various civilian fields. Typically, these missions are conducted in collaboration with research institutes and universities to address specific application requirements. For example, in 2018, we partnered with the Second Institute of Oceanography, Ministry of Natural Resources, to determine coastal water depth using hyperspectral data obtained from the VNIR module during a flight campaign in Qilianyu, Xisha Islands, Sansha City, Hainan Province, China
[101]. The superior performance of AMMIS allowed us to obtain greater retrieved depths (up to 35 m) compared to PRISMA data and UAV hyperspectral data. In 2022, we collaborated with East China Normal University to obtain a chlorophyll concentration map of Dianshan Lake during a flight campaign in Shanghai, China
[102]. Data from other applications are currently being processed and will be reported in future publications.
5. Discussion
5.1. Advantages and limitations
The AMMIS plays a vital role in various fields and is a key component of the HREOP in China. The UV, VNIR, and SWIR modules of the AMMIS were used to verify the essential techniques for spaceborne hyperspectral payloads. The TIR module has been studied and verified for its low-temperature optical system, advanced spectroscopic technology, and large-area infrared focal plane detector, thereby facilitating the future development of satellite TIR hyperspectral imaging spectrometers. The broad-wavelength coverage of the AMMIS enhances its applicability by providing extensive spatial and spectral information. For example, certain minerals exhibit spectral features across the VNIR, SWIR, and TIR bands; thus, broadband spectroscopy can detect more features and increase identification accuracy
[103],
[104],
[105]. The AMMIS exhibits two operational modes: high spatial and spectral resolutions, both of which have been extensively employed in various missions in recent years. The swath coverage of the AMMIS exceeded that of typical push-broom airborne hyperspectral imaging spectrometers, resulting in operational efficiency. The AMMIS can collect hyperspectral data over more than 1000 km
2 within 5 h at a flight altitude of 2 km. Its high operational efficiency makes it the preferred choice for many users. The UV, VNIR, SWIR, and TIR modules can operate collectively and individually, providing flexibility for various tasks. This adaptability allows the AMMIS to be configured for various aircraft platforms, meeting diverse task requirements at minimal cost. The published hyperspectral datasets of the AMMIS are extensive, offering high spatial and spectral resolutions. These datasets are instrumental in developing deep learning AI models that require large samples and real-time satellite processing, necessitating numerous actual hyperspectral datasets for demonstration.
The AMMIS is a highly complex system; assembling all its modules for a single flight campaign can be costly. Nevertheless, the design of the AMMIS is flexible, allowing each module to operate individually based on application requirements. Quantitative applications are an important advantage of hyperspectral remote sensing, which requires high radiometric, geometric, and atmospheric calibration accuracies. Skylight monitoring and in-flight calibrations were performed for AMMIS to enhance calibration accuracy. However, more advanced methods are required to improve the accuracy of preprocessing further. Although the AMMIS can achieve high precision in determining the surface reflectance of targets through synchronous ground measurements, this process is labor-intensive. Therefore, there is a need for highly accurate atmospheric correction methods with low labor costs. In addition, deep-learning-based models applied to hyperspectral datasets over large areas require extensive labels for training to fit the parameters accurately; however, generating these labels is both time-consuming and expensive.
5.2. Future plan
Hyperspectral data from the AMMIS have been used in land-cover surveys, environmental protection, ocean monitoring, water source investigations, agricultural remote sensing, forest identification, and intelligent city construction. Flight campaigns over the past three years have also been affected by the limitations of coronavirus disease 2019 (COVID-19). As these restrictions end, we plan to conduct new flight campaigns for the AMMIS, increasing hyperspectral data availability in the coming years. We will also continue to enhance performance parameters, such as the preprocessing accuracy, sensitivity, and spatial resolution of the AMMIS, to mitigate the limitations imposed by cloudy weather on hyperspectral remote sensing and improve operational efficiency.
Spaceborne hyperspectral sensors provide wide ground coverage and can acquire data for the same target at fixed intervals. In contrast, airborne hyperspectral imaging spectrometers offer high spatial resolution and flexible operation. To combine the advantages of spaceborne and airborne hyperspectral sensors, we plan to conduct flight campaigns of the AMMIS synchronized with hyperspectral satellites such as Gaofen-5. This combined approach can be used to explore new applications and develop the next-generation satellite payloads.
Significant progress has been made in developing midwave infrared (MWIR) hyperspectral imaging spectrometers in recent years
[106]. Based on the application results, hyperspectral imaging spectrometers with wide wavelength coverage are highly effective for exploring new applications. Consequently, a full-wavelength hyperspectral imaging system covering the UV, VNIR, SWIR, MWIR, and TIR bands can be developed in the future.
Different applications impose different performance requirements on the imaging spectrometers. Factors such as target size, geographical environment, and task objectives influence spatial resolution requirements
[17],
[107],
[108]. Although multispectral data may be sufficient for general land-cover classification, species classification and identification in complex environments require hyperspectral data to distinguish objects that are challenging to identify using only spatial texture information. Moreover, a significant trade-off exists between the spatial and spectral resolutions owing to the limitations of the SNR, which significantly affects the applicability of the AMMIS in different fields
[22]. To expand the application scope of the AMMIS, ongoing research will focus on understanding the relationship between the core parameters of hyperspectral sensors and the results obtained using AMMIS across different applications. Developing various AI models
[109] will also be essential, as they may significantly influence both the application outcomes and the design of hyperspectral sensors
[110]. In the future, low-cost dedicated hyperspectral imaging spectrometers are anticipated to be primarily tailored for one or two specific applications. These advancements will enhance the versatility and accessibility of hyperspectral imaging technologies across multiple fields.
6. Conclusions
The AMMIS, a new-generation airborne hyperspectral imaging system developed in China, has completed more than 30 flight campaigns since its inception. This system has been used to verify several critical techniques for spaceborne hyperspectral payloads and has been utilized in various applications owing to its superior performance. This study detailed the design, calibration techniques, flight campaigns, performance tests, and applications of the AMMIS. The module design allows the system to cover a wide spectral range, including the UV, VNIR, SWIR, and TIR wavelengths, with modules that can operate individually or in combination based on task requirements. Advanced optical systems were developed for each module, with particular enhancements to the TIR module through a low-temperature optical system to boost the performance. To ensure high-quality hyperspectral data, various calibration approaches were implemented using both laboratory and in-flight data. Since 2016, numerous flight campaigns have been conducted to verify the performance of this system and explore its applications. The performance test results indicated that the developed instrument met the design requirements, with several indicators exceeding the design specifications.
This study also presented application results and provided access to large-scale hyperspectral datasets with high spatial and spectral resolutions. However, there is still a requirement to improve preprocessing accuracy, particularly in developing advanced atmospheric correction methods that minimize manual intervention. State-of-the-art AI algorithms were applied to the hyperspectral datasets produced by AMMIS, demonstrating excellent performance. A comprehensive understanding of instrument design, data processing methods, and application results is essential for advancing next-generation hyperspectral imaging technologies.
Future efforts will focus on designing hyperspectral imaging spectrometers, developing AI models, and enhancing their applications, with the particular aim of developing next-generation airborne and spaceborne hyperspectral payloads. The anticipated development of a wide-spectrum imager that covers MWIR wavelengths is promising because it can significantly promote downstream applications. Furthermore, improving the preprocessing accuracy, including radiometric, geometric, and atmospheric corrections, may overcome the current limitations of hyperspectral imaging under challenging conditions such as cloudy days or nighttime operations. Advanced AI models such as hyperspectral generative pretrained transformers can significantly enhance the ease of hyperspectral data use. Real-time processing of hyperspectral data and research on processing time prediction can also satisfy the requirements of urgent tasks.
CRediT authorship contribution statement
Jianxin Jia: Writing - review & editing, Writing - original draft, Methodology. Yueming Wang: Writing - review & editing, Writing - original draft. Xiaorou Zheng: Methodology. Liyin Yuan: Data curation. Chunlai Li: Data curation. Yi Cen: Data curation. Fuqi Si: Data curation. Gang Lv: Data curation. Chongru Wang: Data curation. Shengwei Wang: Data curation. Changxing Zhang: Data curation. Dong Zhang: Data curation. Daogang He: Data curation. Xiaoqiong Zhuang: Data curation. Guicheng Han: Data curation. Mingyang Zhang: Methodology. Juha Hyyppä: Writing - review & editing. Jianyu Wang: Writing - review & editing, Data curation.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The authors would like to thank all the engineers from SITP, IRSA, and AIOFM for contributing to developing the AMMIS. We would also like to thank Prof. Yansong Duan’s group at Wuhan University for contributing to the geometric correction. This study was supported by the Shanghai Industrial Collaborative Innovation Fund (HCXBCY-2021-001) and the Academy of Finland (349229).
Appendix A. Supplementary data
Supplementary data to this article can be found online at
https://doi.org/10.1016/j.eng.2024.11.001.
PDF
(4422KB)
