《1 Introduction》

1 Introduction

In 2016, the Institute for Defense Analyses (IDA), a wellknown think tank in the United States of America, released a report titled “Current and Potential Use of Technology Forecasting Tools in the Federal Government.” The report aimed at achieving two major objectives, the first of which involved understanding the current state of technology-forecasting techniques employed in federal government agencies and the role that technology forecasting plays in decision-making processes. The second objective was to understand what kind of technology-forecasting tools are desired by the federal agencies for future use, including the characteristics of the said tools, approaches, and information that concerned personnel would like to use [1].

The aforementioned investigation by IDA and subsequent report were performed and published, respectively, under the support of the Intelligence Advanced Research Projects Activity (IARPA). IARPA is presently exploring current and future applications of technology forecasting in science and technology (S&T) decision-making processes and is also working towards development of automated technology-forecasting tools. The IDA report would be of great help to IARPA in predicting future technology characteristics and/or applications, thereby improving the development and subsequent implementation of S&T analytical capabilities. It would also help IARPA identify obstacles that need to be overcome to facilitate development of automated technology-forecasting tools and accelerate their transition from the design phase to service-operation phase.

《2 Current state of available technology-forecasting tools》

2 Current state of available technology-forecasting tools

《2.1 Use of technology forecasting is not ideal, and government agencies involved in interviews have different technology-forecasting requirements》

2.1 Use of technology forecasting is not ideal, and government agencies involved in interviews have different technology-forecasting requirements

As defined in the IDA report, technology forecasting implies prediction of future characteristics or applications of current technologies within a specific timescale [2]. Agencies perform the technology-forecasting exercise—as a consequence of routine surveillance and reporting requirements, and as a reaction to occurrence of a specific event or rise a certain need.

Specific applications of technology forecasting include gathering intelligence information, identifying threats and opportunities emerging from potential future applications of technologies, managing research portfolios, understanding how future scenarios might be shaped or affected by today’s long-term technology investments, and understanding economic and policy implications of the evolution and global availability of commercial technologies and/or products.

In order to understand how the federal government uses technology-forecasting tools to assist decision-making, IDA researchers interviewed and enquired a large number of people associated with federal agencies involved in fields of emerging technologies and related applications. Presently, only two such government agencies—the CIA Directorate of Science and Technology and Energy Information Administration—employ automated tools for their daily work, and only one of these agencies—the Energy Information Administration—uses automated tools for technology forecasting. However, as many as 11 government and other agencies involved in the interview expressed different requirements for technology forecasting, as listed in Table 1.

《Table 1》

Table 1. Interview respondents’ agency affiliation, the agency goal, and its relationship with technology forecasting.

《2.2 Government agencies demonstrate different pursuits of technology-forecasting timescales and corresponding automation levels》

2.2 Government agencies demonstrate different pursuits of technology-forecasting timescales and corresponding automation levels

Agencies with which the interviewed personnel were affiliated were grouped by their forecasting timescale, and divided into three bins: short-term (0–5 years in the future), mid-term (5–20 years in the future), and long-term (20–30 years in the future).

The type of technology-forecasting technique adopted by concerned agencies could be related to the forecasting timescale. As per the IDA report, agencies interested in the state of technologies or their applications that are within 0–5 years of being realized are prepared to respond quickly to technologies that have already reached the later stages of their readiness. On the other hand, agencies interested in technological developments or applications that are expected to come to fruition in the next 5–20 years are making funding decisions, such as infrastructure and long-term organizational strategies. Lastly, agencies expecting long-term technology forecasting are most likely to prefer technology forecasting with higher automatization levels, as summarized in Table 2.

《Table 2》

Table 2. Agencies grouped by forecasting timescales, and whether they currently are using, pursuing, or not pursuing automated methods.

《2.3 Government agencies prefer short-term and nonautomated technology forecasting》

2.3 Government agencies prefer short-term and nonautomated technology forecasting

In accordance with Table 2, government agencies involved in the interview appeared to be interested in performing short-term technology forecasting owing to a combination of two factors— (1) analysts are overwhelmed by the sheer volume of information that they need to be aware of; (2) technology-development cycles may end up being too fast for analysts to keep up with latest technological trends.

In addition, government agencies seemed relatively uninterested in technology forecasting with high automatization levels or use of technology-forecasting tools owing to the lack of suitability of available tools to agency requirements, comfort in the use of current processes, lack of available time to train personnel in the use of new tools, and prohibitively high tool costs. In addition, agencies cited a lack of trust in automated methods with regards to analysts as well as decision-makers.

An output from an automated tool, as per the interviewed agency personnel, would most likely be further interpreted, contextualized, and explained by analysts prior to being elevated to the level of decision-makers. Any automated tool must, therefore, be designed to aid rather than supplant the role of analysts, and it must not be operated in a stand-alone manner.

《3 Desired capabilities of automated tools》

3 Desired capabilities of automated tools

The interviewed government-agency personnel expressed hope that technology-forecasting tools would possess certain automatization capabilities, which could be divided into three types—forecasting capabilities, alert capabilities, and tracking and summarization capabilities—as listed in Table 3.

《Table 3》

Table 3. Automated capabilities of technology-forecasting tools and government agencies interested in the capacities

Tools that possess forecasting capabilities can implement mid-term and post-phase forecasting (extrapolation and trend analysis), short-term forecasting (prediction markets) [3], and even perform dual-use probability calculations. When an indicator exceeds a pre-determined threshold, automated tools with alert capability issue an alert for researchers. Researchers, therefore, can focus on specific issues and understand potential threats and opportunities available with the use of technology. Tracking and summarization capabilities assist in information summarization and situational awareness in the given research field, technology, or interpersonal network status.

《3.1 Forecasting capabilities》

3.1 Forecasting capabilities

Only few government agencies expressed interest in capabilities that would provide users with qualitative information regarding a future state. However, agencies that did express some interest were more concerned about capabilities that could identify trends in current technologies and provide a basis for predicting future characteristics of a given technology and its applications. Such capabilities are discussed in greater detail below.

3.1.1 Extrapolation and trend analysis

In extrapolation and trend analysis, a user or tool makes employ past and/or present information to forecast a future state. Such tools are likely be most useful for agencies that are focused on forecasting technologies that are presently in their middle to late development stages. For agencies attempting to forecast technologies and/or applications that would witness fruition 20–30 years into the future, enough data might not be available to establish any trend, since the technology would only be in its early development stages with little evidence of its developmental pathway.

3.1.2 Dual-use probability calculation

Variations in the ability to forecast the future state of a technology by extrapolating data from historical trends also refers to the ability of projecting possible trajectories of a technology for given varying sets of driving forces. One particular application of interest in this case would be a tool, which when used in the early developmental stages of a technology, would be capable of determining the probability of the technology developing into one of the dual-use type.

3.1.3 Prediction markets

In prediction markets, collective judgment of a group of individuals is used to generate forecasts concerning an event or a parameter. Predictions are considered as assets to be traded within a virtual market accessed by a number of individuals. Prediction markets could be used to help agencies with a wide range of forecasting needs, although in order to gain interest from participants, it could likely be best used for short-term forecasts, which could be proven or disproven over a relatively short timescale. Within short-term forecasting needs, prediction markets would be best suited to fast-moving technology-development cycles, which possess relatively few trackable indicators.

《3.2 Alert capabilities》

3.2 Alert capabilities

An interview respondent expressed interest in the development of an alert tool that raises “red flags” when a certain pre-determined threshold is crossed, thereby directing analysts towards a need to perform in-depth analyses. Such a tool, in general, would require analysts to first determine future states that they would be concerned with. Subsequently, analysts could determine indicators, which would provide information regarding the likelihood of occurrence of the said future states, as well as their threshold values at which they would want to be alerted.

This class of red-flag capabilities could support a range of analysis by allowing the user to specify the type of input data and threshold. In basic research applications, a program manager may require tools capable of extracting certain scientific journals and/or conference proceedings. The thresholds, in this case, could be set as the number of keywords, contributing authors, or the co-occurrence of keywords and authors. Analysts that look at technologies in the later stages of their development might be interested in tools capable of extracting information from patent applications or appearance of keywords in trade journals. Because of the range of possible customizations, this class of tools could potentially be used to support short-, medium-, and longterm forecasting efforts.

《3.3 Tracking and summarization capabilities》

3.3 Tracking and summarization capabilities

Technology-forecasting tools equipped with tracking and summarization capabilities are suitable for government agencies with a desire to be more aware of all information relevant to their fields without specifying any desire for predictive technologies. Such capabilities could further be classified into technology tracking through name changes, identifying and tracking distinguished people in a given field of research, and organization visualization information.

3.3.1 Technology tracking through name changes

One of the challenges associated with tracking technologies over a long period of time is to be cognizant of the name and terminology changes associated with it over its entire development cycle. Techniques to automate the tracking of such information would be very useful.

3.3.2 Identifying and tracking distinguished people in a field of research

The said tool can be used to identify leading scientists, associated with a novel and/or emerging area of research, as a means to not only understand the state of cutting-edge research in the given field but also identifying promising people to invest in. In addition, a social network map of researchers, wherein collaborations between academic and industrial scientists are highlighted, could be used to help analysts identify researchers and innovations to track, thereby accelerating the commercialization of technology.

3.3.3 Organization and visualization of information

A tool capable of organizing and displaying information would be extremely useful, especially if it possessed intuitive visualization capabilities. A fully developed organization and visualization tool useful to federal-government-agency personnel would be the one that allows users to select data sources of interest and suggest additional, complementary data sources to supplement the user-selected sources. The tool would continuously receive updates from the selected data sources, and subsequently possess a number of different analytical capabilities, such as—(1) summarizing the information within a single data source or across data sources via topic modeling or generating word clouds; (2) trend analysis, which would allow users to observe changes in the usage of a term or topic over time; (3) highlighting new terms or topics; and (4) allowing users to examine co-occurrence.

《4 Conclusions》

4 Conclusions

The IDA report titled “Current and Potential Use of Technology Forecasting Tools in the Federal Government” reviewed the status quo of technology forecasting in federal agencies and analyzed technology-forecasting tool capabilities desired by the different agencies. The author summarizes the application situation of technology forecasting in federal agencies and presents the following opinions.

(1) Existing technology-forecasting tools do not meet the requirements of federal agencies owing to lack of universality. Available technology-forecasting tools can help users integrate, summarize, and highlight key pieces of information and scan the horizon for indicators to keep analysts abreast with new developments. However, these tools do not customize customer requirements. At the same time, the functional singleness of existing tools fails to meet the requirements of government agencies. It is unrealistic to develop a technology-forecasting tool that meets the requirements of all government agencies. Even when certain agencies are interested in broad-technology horizon scanning activities, they are yet merely examining the technologies within the context of their mission and goals.

(2) Technology-forecasting tools with visualization capabilities are favored by federal agencies. The tool that organizes and displays information in an intuitive way would help analysts in government agencies to better analyze data and make decisions,regardless of whether the tool possesses any predictive components. Visualization is a key component of tracking and summarization as well as extrapolation and trend-analysis tools.

(3) U.S. government agencies invest little trust in automated technology-forecasting tools. On one hand, their respondents mentioned that there exists the fear that forecasting tools would replace human judgment. Data analysts and researchers are accustomed to collecting information and using their expert judgment to determine connotations and values of information. If technology forecasting can automatically determine future impact of technology, personnel involved in the interview were concerned that this would weaken the core role of humans in analysis and decision-making. On the other hand, the respondents expressed concerns over the fact that automated forecasting tools, even with their powerful forecasting capabilities, could reduce the degree of acceptance of automation tools by government agencies.

(4) When developing technology-forecasting tools, our country can draw on the experience of U.S. federal government agencies in using technology-forecasting tools. To this end, we should first fully survey the requirements of all parties and focus on customization and flexibility of technology- forecasting tools. Second, we should balance the complexity, transparency, and functionality of the technology tools. Third, we should not rely too much on the technology forecasting tools. Human should in the loop of technology decision-marking. Fourth, we can consider visualization as an important design indicator.