1. Introduction
A great deal of effort has been made to study human-operator issues in human machine systems and human-driven complex systems in the last few decades[
1–
2]. This has included research to improve the analysis and measurement of human performance and the effect of human error on system reliability and effectiveness, and to develop intelligent tools to support human operation[
3–
5]. Despite the increasing levels of automation and the application of advanced technologies and operational concepts, air traffic controllers continue to play a critical role in the air traffic management (ATM) system[
6–
9]. To date, researches have focused on air traffic controllers’ communication, cognitive activities, and mental workload[
10–
12], yet little attention has been given to their eyemovement activities. The ATM system is in the process of being transformed; for example, the shift from radar control to trajectory-based operation is causing the roles, responsibilities, and requirements for both humans and automation to change [
9]. In the future, the main task of the controller will be to monitor the automation and ensure that it functions properly and to quickly resume control in the case of failure or degradation of performance[
9,
13]. Given the unique features of air traffic controllers’ work, understanding their eye movements is of interest in both the ATM domain and other fields. If it is possible to well understand controllers’ eye movements, it might be possible to determine what happens in their decision-making processes.
Eye movement has been the subject of growing research interest within various disciplines, including psychology, ergonomics, and computer science. Kowler [
14] has presented a broad review of the eye-movement literature. Among various behaviors such as blinking, pupil size, and so forth, Kowler emphasizes three oculomotor behaviors of eye movement—namely, gaze control, smooth pursuit, and saccades—and discusses their interactions with vision. In computer science, eye tracking is used to analyze the usability of the computer interface and human–computer interaction [
15]. For example, eye-tracking research undertaken by the MITRE Center for Advanced Aviation System Development (CAASD) evaluated the effects of their newly developed automation concept, the relative position indicator (RPI), on air traffic controllers’ attention-allocation strategies [
16]. The authors found that the RPI changes the controllers’ visual scanning patterns, which may require the information display to be optimized for controllers. Research has also found that eye-movement metrics can be used to measure human performance[
17,
18]. Ahlstrom and Friedman-Berg [
19] investigated the correlations between controllers’ eye movement and cognitive workload. They found that eye-movement measures, including eye blinking and pupil size, provide a more sensitive measure of workload than the commonly used subjective workload. Furthermore, research has shown that eye blink information and saccadic velocity are indicators of arousal levels in natural tasks[
20,
21]. Research on eye movements in the medical domain can be found in Refs.[
22,
23].
Eye-movement research has gained so much attention recently because it can provide insight into information about interests, goals, plans, and cognitive strategies. In Ref. [
24], Jang et al. demonstrated that underlying, hidden cognitive strategies can be inferred through eye movement. They proposed a model to predict human implicit intention based on salient features of eye movements. Although the authors claimed that their model shows plausible performance, this remains to be proved in the context of the performance of complex tasks.
Research on information-seeking behavior has been facilitated by analyzing eye-movement data, since eye movement is the natural indicator of information seeking by the brain. Many studies have been carried out on the topic of task-directed information seeking through eye movements[
25–
29]. A recent study by Gottlieb et al. [
29] highlighted research on the underlying mechanisms of information seeking. Research in three separate fields was reviewed, including machine learning, eye movements, and psychology and neuroscience.
Another school of research on information seeking has focused on the scanning patterns of the human eye that occur when humans solve visual search problems. The Federal Aviation Administration (FAA) has investigated how air traffic control specialists scan the radar screen for information [
30]. Various eye-movement data, including fixations, saccades, blinks, and pupil information, were examined. It was found that controllers tend to focus on the airspace with the highest traffic density, which may lead to the late identification of intrusions, resulting in the occurrence of unsafe events. Similar behavior was reported by Fehd and Seiffert [
31]. Humans tend to focus on a central location, mainly due to the cognitive strategy of grouping multiple targets into a single target. Van Meeuwen et al. [
32] used still pictures of radar screen shots to investigate the cognitive strategies of novice, intermediate, and expert air traffic controllers. Kang et al.[
33,
34] developed an algorithm to quantify controller’s informationseeking and aircraft-selection behaviors.
As illustrated by the studies listed above, eye movements are closely related to cognitive strategies, especially to information seeking. Although researchers have attempted to identify the visual scanning strategies of experienced air traffic controllers, it is still unclear what the effects of training or working experience are on controllers’ information-seeking behaviors.
In this paper, controllers’ eye-movement data are collected and analyzed to provide an initial understanding of the relationships between working experience and eye-movement measures. The rest of the paper is organized as follows: Section 2 presents a brief introduction to the data collection and experiments. Section 3 reports the analytical results on eye-movement behaviors observed in different controllers with different levels of working experience. Conclusions are drawn in Section 4.
2. Method
2.1. Participants
To collect air traffic controllers’ eye-movement data, we invited 25 personnel (22 males, three females) aged 21–40 years (mean (M) = 28.4, standard deviation (SD) = 20.08) to participate in designed simulation experiments. Among them, four students were majoring in air traffic control from Nanjing University of Aeronautics and Astronautics (NUAA) (classified as ‘‘novice”), while the other 21 participants were air traffic controllers from an air traffic control center in Zhejiang Province (including one novice). All the participants were grouped into five levels based on their working experience and personal competency (Table 1).
The working experience for Levels 2, 3, 4, and 5 are (14.0 ± 3.5), (9.3 ± 1.0), (6.5 ± 0.6), and (2.8 ± 1.1) years.
According to the Civil Aviation Administration of China (CAAC), the minimum requirement of working experience for Levels 2, 3, 4, and 5 controllers are 12, 8, 6, and 2 years, respectively. However, controller level is not just determined by years of experience. For example, only the best controllers with 12 years of experience will be accepted into Level 2. Table 1 provides information about the participants in these five levels.
Table 1 Participant information.
According to a statistical report from the Air Traffic Management Bureau (ATMB) of the CAAC, the percentage of female controllers is below 20% in China. The figure for female approach controllers is even lower, at less than 10%. For example, there are two female approach controllers out of 50 approach controllers in the ATMB of Zhejiang Province. Therefore, the gender proportion of participants in our experiments is reasonable.
2.2. Apparatus
faceLABTM 5.0 was used to record the air traffic controllers’ eye movements during real-time simulation. It is an eye-contactless system with a very high ability to track eye movements. Many features can be measured by faceLABTM, including:
• Head pose: This can identify the position and orientation of a subject’s head in three-dimensional (3D) coordinates.
• Gaze: Separate gaze rays can be output for both the left eye and the right eye. Each gaze ray consists of an origin point and a unit vector.
• Saccades: This is defined as the ballistic movement of the eye from one fixation to the next. The saccadic speed can be up to 700 (°)·s-1 .
• Eye closure: A percentage relating to the coverage of the iris is measured to determine the eye closure.
• Eye blinks: This is a binary signal (true or false) record by the system, which reports the occurrence of blink events—that is, a rapid eye closure followed quickly by a rapid eye opening.
• Pupillometry: faceLABTM can measure the diameter of each pupil in meters.
Percentage of eye cosure (PERCLOS): This is a fatigue measure defined as the most reliable single visible indicator for fatigue. The calculation of PERCLOS is mainly based on the eye closure of the subject.
2.3. Simulation scenarios
Due to safety concerns, we could not place any equipment on the controllers’ workstations during their actual working hours to record their eye movements. In order to collect eye-movement data during active complex tasks, a radar control simulation system was prepared to perform high-fidelity simulation exercises. The simulation system has exactly the same functions and interface as the actual ones used by air traffic controllers in their work. Thus, there is no need for the controllers to familiarize themselves with the simulation system. However, a potential limitation of using real-time simulation is that the participant may not feel the same pressures or tensions during simulation as during work. This limitation is addressed in the final part of this paper.
The simulated airspace was an approach sector of Hangzhou (ZSHCAP03), whose route structures are plotted in Fig. 1. The vertical range of the sector is 3000 m and below. The main functions of this sector are sequencing arrival flights before handing over to the Hangzhou tower, and directing departure flights and a few overflying flights.
Fig. 1. The boundaries and route structure of Hangzhou terminal airspace (ZSHCAP03). The highlighted area is an approach sector of the Hangzhou airport (APP03); dark blue lines represent routes; green lines are the sector boundaries; light grey lines indicate the trajectory of eye movements; yellow and light green lines are radar tags. The red dots represent the fixation points of an air traffic controller, as recorded in one simulation exercise.
Three different levels of traffic scenarios—namely, easy, normal, and hard—were specified, based on actual flight data. It should be noted that an analysis of the effect of different traffic scenarios, including unusual traffic scenarios, is beyond the scope of this paper.
The faceLABTM cameras were placed just in front of the controller under the middle of the main screen (Fig. 2). An individual-specific model was built for each participant in order to accurately track his or her eye movement. Calibration was carried out before the simulation started. During the simulation, the participants could use both arms and legs normally, and were permitted to wear eyeglasses. The purpose of the study and the traffic scenarios were briefly presented to the participants before the simulation began. Signed informed consent statements were obtained from all the participants.
Fig. 2. The air traffic control simulation system and eye-tracking device.
2.4. Identification of fixation
To identify fixation from raw data, the velocity-threshold fixation identification (I-VT) algorithm was employed. The main idea of the I-VT is to use the angular velocity to distinguish fixation and saccade points. A predefined parameter, the velocity threshold, must be specified in advance in order to identify fixation points. For example, the threshold can be set to 125 (°)·s-1 .
The first step is to sort the original time series of the eyemovement data into ascending order. Next, point-to-point angular velocities are calculated for each point. The following rules can be used to distinguish fixation and saccade points: If the angular velocity at the point is less than the threshold, it is a fixation point; otherwise, it is a saccadic point. The algorithm then outputs consecutive fixation points into fixation groups. All saccade points are disregarded.
The data format of the output fixation group is given as
where
x and
y are the coordinates of the center of the points in a group,
t is the time of the first point in the group, and
d measures the duration of the fixation group.
Fig. 1 plots an example of one controller’s fixation trajectory. As shown in this figure, the trajectory of the eye movements clearly follows the pattern of the route structure.
3. Results
3.1. Effect of working experience on fixation and saccades
To analyze the effect of working experience on a controller’s fixation behavior, three widely used eye-movement metrics were first examined: area of interest (AOI), fixation duration, and gaze entropy. As discussed in Section 2.1, the eye-movement data were divided into five groups based on the participants’ working experience. The statistical results for the number of AOIs and fixation duration of each group are shown in Fig. 3(a). As shown in the figure, the five groups can be further divided into three categories: CAT I (Level 2), CAT II (Levels 3, 4, and 5), and CAT III (novice).
Fig. 3. Statistical results for (a) AOI and fixation duration, and (b) saccadic velocity. Error bars present the SD for the associated group data.
The eye movement of Level 2 controllers is different from that of the others. Level 2 controllers have the least number of AOIs (M = 1126 s) but the longest fixation duration (M = 1.66 s). Longer fixation and fewer AOIs suggest that Level 2 controllers prefer to focus on certain areas, rather than monitoring all the space in the sector. The reason for this behavior may be associated with their current job responsibility. Level 2 controllers are the most experienced controllers, with at least 12 years of working experience as an air traffic controller. Their skills and competency are outstanding among controllers with the same working experience. However, most Level 2 controllers have been promoted to an administrative position, rather than managing air traffic as a controller. Although they possess the capability for air traffic control, they may be reluctant to screen the whole airspace, especially during a simulation. Apparently, this is a case of misallocation of attention resources.
The fixation duration and the number of AOIs in the CAT II category exhibit opposite trends. The controllers in this category are the main staff in the air traffic control center who are responsible for daily operation. The mean number of AOIs increases slightly from Level 5 to Level 3, whereas the mean fixation duration decreases gradually. The average time required to locate a flight of interest is 1.1 s for the controllers in CAT II, which suggests that they are capable of allocating their attention reasonably. The SD in the number of AOIs is much greater in CAT II than in CAT I. This may indicate strong variability in competence among the controllers. Another reason could be that there are fewer Level 2 controllers than controllers at other levels.
The special case in all five groups is the novice group. We plot the statistical results for AOI and fixation in Fig. 3(a) and saccadic velocity in Fig. 3(b). As shown in Fig. 3(a), the mean fixation is only 0.2 s, which is much lower than that of the other groups. The SD in the number of AOIs is extremely large, although the mean number of AOIs is within the same level as that in CAI II. Here, it should be recalled that the novices had just finished their training, and had no working experience. Thus, it is not surprising to see larger variations among the novices. Compared with the other controllers, novices are not familiar with the airspace and air traffic control procedures. They must allocate their attention resources over the whole airspace. Airspace structures and traffic distribution can definitely affect their cognitive strategies. Their capability to manage air traffic is still under development. However, there could be potentially talented controllers among the novices, as indicated by the large variations in the number of AOIs. These talented novices could locate the correct aircraft and distribute their attention to those areas.
The effect of working experience on fixation can be further revealed through a comparative analysis of the variations in the number of AOIs. As shown in Fig. 3(a), the Level 2 controllers have the smallest SD in AOIs, whereas the novices have the largest. A small variation indicates that seasoned controllers may employ similar information-seeking strategies after years of training and field practice. In contrast, beginners in air traffic control must learn and be trained in information-seeking strategies in the early stage of their career.
A heat map showing the probability of fixation in a given airspace was obtained from the eye-movement data. Fig. 4 present six heat maps for six different controllers from Levels 2, 3, 4, 5, and novice. The hot zones, which are highlighted in red and yellow in the figure, represent the airspace with a higher density of fixation, indicating that the air traffic controllers tended to monitor these areas frequently. The highlighted areas in the images clearly outline the structure of the airspace, with the airport located in the center and major traffic flow to and from the upper left and lower left. Among these images, the airport is always the focus of the controllers. They spent a great deal of time monitoring the traffic handling to and from the airport; this result is in agreement with previous studies [
31]. The heat maps of the novices show that the novices tended to monitor every part of the airspace, which indicates a low efficiency in seeking critical information.
Fig. 4. Heat maps of fixation generated from six controllers’ data. (a) Level 2; (b) Level 3; (c) Level 4; (d) Level 5; (e, f) novices.
3.2. Saccadic velocity
The saccadic velocity was calculated in order to investigate the saccade behavior of the controllers. The results are plotted in Fig. 3(b). The average saccadic velocity of the novices was faster than that of the licensed controllers, whose saccadic velocities were nearly the same. As a result of professional training over many years, the seasoned controllers have gained a special ability to manage air traffic. Research has shown that saccadic velocity is related to the capability to track moving targets. A faster saccadic velocity may indicate better capability in response to a rapidly changing environment. In this case, the novices had the fastest saccadic velocity; however, that does not mean that the novices were better than the other controllers at searching for necessary information. Instead, the novices’ velocity could be caused by their random selection of aircraft, since the novices were unfamiliar with controlling. Novices cannot estimate the position of aircraft, so they need to look at each aircraft frequently.
To further investigate the differences in eye-movement behavior among the air traffic controllers, we compared the distribution of the number of fixation points during a gaze, the duration of fixation, and the saccadic speed. In studies on the voice communication activities of air traffic controllers, scholars found that the inter-communication times of air traffic controllers exhibit a heavy-tailed distribution[
35,
36]. We wanted to determine whether or not the controllers’ eye-movement behavior showed a similar pattern. Fig. 5 present the respective distributions of the eye-movement measures listed above. It can be seen that the distribution of fixation duration and that of the fixation count have similar patterns. The distributions of the data from the experienced controllers are characterized by exponential decay with a powerlaw tail, whereas the distributions of the novices are characterized by a power-law distribution (fitted by a straight line in the log–log plot). The distributions of saccadic speed, however, are centered along a single line, suggesting that all the controllers’ saccadic speeds follow a power-law distribution. These results need to be further explored.
Fig. 5. Probability (P) distribution of (a) fixation count, (b) fixation duration, and (c) saccadic velocity.
3.3. Gaze entropy
The prevalence of data-driven approaches has facilitated quantitative measurements of information-seeking behavior through eye movements. The development of gaze-related measures based on information theories is of particular interest in human factors research. The concept of entropy, which originated in physics, aims to describe the degree of chaos in a system. Entropy has been successfully applied in various fields, including physics, social science, and—most recently—eye-movement studies [
37]. Given recorded eye-movement data, the stationary gaze entropy (STE) and gaze transition entropy (GTE) can be calculated based on the equations of Shannon’s entropy and conditional entropy. A recent review on gaze entropy measures can be found in Ref. [
38]. In the present work, we used Shannon’s entropy to measure the controller’s gaze entropy, leaving GTE for further study. The gaze entropy (
Hg) is defined as follows:
Where X is the sequence of fixation with total length n, while i is the location of each fixation contained in X. p(x, y) represents the probability that a controller’s fixation falls at the (x, y) position. This equation gives a measure of the average uncertainty of the position of the gaze at an instant in time during the experiment. Therefore, it can determine the discreteness of the controller’s gaze. To calculate the gaze entropy, we first discretized the radar screen into 64 × 36 parts, resulting in a total of 2304 squares. It was then straightforward to obtain the frequencies of the gaze falling onto each of these squares at on any given time sample.
The gaze entropy of each controller was calculated using Eq. (1), with an average of (6.94 ± 0.79) across all the controllers. Fig. 6 presents the gaze entropy for different levels of controllers. The box on the far right of the figure shows the entropy of all the controllers. It is interesting to note that the gaze entropy exhibits a similar trend compared with the saccadic velocity (Fig. 3(b)). The Level 3 controllers have higher saccadic velocities but lower entropy, suggesting the most efficient information-gathering capability. While the novice controllers also exhibit lower entropy and the fastest saccadic velocity, their mean fixation is much lower than that of the others. This could indicate an inability to acquire the necessary information from a complicated traffic situation.
Fig. 6. The gaze entropy of different levels of controllers. +: outliers.
Due to the limited size of the sample data for each level of controller, we focus here on a comparison between the novices and experts. We chose 7.2 as a threshold to divide the controllers into two groups: the high gaze entropy (HGE) group and the low gaze entropy (LGE) group. The results are presented in Table 2. Since the number of experimental samples is less than 40, Fisher’s exact test was employed to examine the statistical test [
39]. The results show statistically significant differences between the novice and expert gaze entropy (
p = 0.042). Nearly 80% of the novice controllers belong to the LGE group, while four out of 11 of expert controllers are in the LGE group. In general, the more gaze points, the greater the gaze entropy, which is consistent with the results obtained in previous sections. However, a conclusion can be drawn from the above results that could lead to a misinterpretation of air traffic controllers’ information-seeking strategies by relying too strongly on a single eye-movement measure.
Table 2 A comparison of gaze entropy differences among groups with different levels of experience.
Note: Controllers from Level 2, 3, and 4 are classified into expert group, while Level 5 and novice are classified into novice group.
4. Conclusions
Eye-movement data can be used as a proxy for human perception and allocation of attention. This paper presented research on air traffic controllers’ eye movements, with an emphasis on the differences in eye-movement behavior among air traffic controllers with different levels of working experience. Our objective was to understand the underlying mechanisms of the observed eyemovement behavior using a data-driven approach. This work focused on the statistical patterns of the eye movements of all the studied air traffic controllers. In order to collect more reliable eye-movement data, we designed real-world radar control scenarios. Eye-movement data were collected during human-in-the-loop simulation. We developed an I-VT algorithm to extract fixation information from the recorded eye-movement data. Three commonly investigated metrics were thoughtfully explored: the number of AOIs, fixation duration, and saccadic velocity. The results suggest that a qualified controller can employ an efficient information-seeking strategy and allocate attention resources reasonably. Further efforts are needed to improve the training efficiency of visual search behavior in air traffic control. Although the probability distributions of the eye-movement data show heavy-tailed characteristics, the shapes are useful in discriminating between novice and experienced controllers. A comparison of gaze entropy was also carried out; however, the findings could lead to a misinterpretation of air traffic controllers’ information-seeking strategies by relying too strongly on a single eye-movement measure.
In conclusion, we find that working experience has a notable effect on controllers’ eye-movement behavior. AOI, fixation duration, saccadic velocity, and gaze entropy were demonstrated to be reasonable indicators of information-seeking behavior for judging whether a trainee controller possesses the ability to collect information efficiently. However, a combination of different eyemovement measures is recommended to reveal the fundamental differences in air traffic controllers’ cognitive strategies.
This work has several limitations. One main challenge, which may also be encountered by other human behavior studies, is that the tension and pressure experienced by air traffic controllers could not be captured. During real-world operation, air traffic controllers must make quick, correct decisions in order to main safe and orderly aircraft flow. No mistakes can be made during real operations. During a real-time simulation, however, controllers’ working attitude will change, since they know that they can make mistakes. Thus, the controllers will be more relaxed during humanin-the-loop simulation than in their daily work, and their behavior many change due to the different environmental and physiological circumstances. The second limitation of this work is that only the main tasks of controllers were considered. Secondary tasks, such as coordination with other controllers, were not simulated. During real-world operation, air traffic controllers are required to handle flights coming in from and going out to adjacent sectors. Thus, they must monitor the traffic situation in the surrounding sectors in order to prepare a tactical plan. To capture these kinds of behaviors, a large-scale simulation experiment must be designed that involves several connected sectors. Last but not least, the effects of the simulation scenarios were not considered in the present work. Instead of using still pictures, we captured the controllers’ eye-movement data using real-time traffic. The density and complexity of traffic scenarios might influence the controller’s information-seeking behavior. Given the unique characteristics of air traffic control work, further interdisciplinary research by collaborators from the fields of psychology, computer engineering, and data science is recommended.
Acknowledgements
We thank air traffic controllers and students for participating the experiments. This research was supported by the National Natural Science Foundation of China (U1833126, U2033203, 61773203, and 61304190).
Compliance with ethics guidelines
Yanjun Wang, Liwei Wang, Siyuan Lin, Wei Cong, Jianfei Xue, and Washington Ochieng declare that they have no conflict of interest or financial conflicts to disclose.
PDF
(1440KB)
