Search scope:
排序： Display mode:
Pre-Trained Language Models and Their Applications Review
Haifeng Wang, Jiwei Li, Hua Wu, Eduard Hovy, Yu Sun
Engineering 2023, Volume 25, Issue 6, Pages 51-65 doi: 10.1016/j.eng.2022.04.024
Pre-trained language models have achieved striking success in natural language processing (NLP), leading to a paradigm shift from supervised learning to pre-training followed by fine-tuning. The NLP community has witnessed a surge of research interest in improving pre-trained models. This article presents a comprehensive review of representative work and recent progress in the NLP field and introduces the taxonomy of pre-trained models. We first give a brief introduction of pre-trained models, followed by characteristic methods and frameworks. We then introduce and analyze the impact and challenges of pre-trained models and their downstream applications. Finally, we briefly conclude and address future research directions in this field.
Software development in the age of intelligence: embracing large language models with the right approach Perspective
Xin PENG
Frontiers of Information Technology & Electronic Engineering 2023, Volume 24, Issue 11, Pages 1513-1519 doi: 10.1631/FITEE.2300537
Keywords： 大语言模型；ChatGPT；软件工程；软件开发
Progress in Neural NLP: Modeling, Learning, and Reasoning Review
Ming Zhou, Nan Duan, Shujie Liu, Heung-Yeung Shum
Engineering 2020, Volume 6, Issue 3, Pages 275-290 doi: 10.1016/j.eng.2019.12.014
Natural language processing (NLP) is a subfield of artificial intelligence (AI) that focuses on enabling computers to understand and process human languages. In the last five years, we have witnessed the rapid development of NLP in tasks such as machine translation, question-answering, and machine reading comprehension based on deep learning and an enormous volume of annotated and unannotated data. In this paper, we will review the latest progress in the neural network-based NLP framework (neural NLP) from three perspectives: modeling, learning, and reasoning. In the modeling section, we will describe several fundamental neural network-based modeling paradigms, such as word embedding, sentence embedding, and sequence-to-sequence modeling, which are widely used in modern NLP engines. In the learning section, we will introduce widely used learning methods for NLP models, including supervised, semi-supervised, and unsupervised learning; multitask learning; transfer learning; and active learning. We view reasoning as a new and exciting direction for neural NLP, but it has yet to be well addressed. In the reasoning section, we will review reasoning mechanisms, including the knowledge, existing non-neural inference methods, and new neural inference methods. We emphasize the importance of reasoning in this paper because it is important for building interpretable and knowledge-driven neural NLP models to handle complex tasks. At the end of this paper, we will briefly outline our thoughts on the future directions of neural NLP.
Keywords： Natural language processing Deep learning Modeling learning and Reasoning
Indeterminacy Causal Inductive Automatic Reasoning Mechanism Based On Fuzzy State Describing
Yang Bingru,Tang Jing
Strategic Study of CAE 2000, Volume 2, Issue 5, Pages 44-50
New framework of knowledge representation of fuzzy language field and fuzzy language value structure is shown in this paper. Then the generalized cell automation that can synthetically process fuzzy indeterminacy and random indeterminacy and the generalized inductive logic causal model are brought forward. On this basis, the new logic indeterminate causal inductive automatic reasoning mechanism which is based on fuzzy state describing is brought forward. At the end of this paper its application in the development of intelligent controller is discussed.
Keywords： language field language value structure generalized cell automation generalized inductive logic causal model automatic reasoning intelligent controller
Research of the Organization Structure of Large Construction Companies Based on the Entropy Theory
Wang Xing,Zhan Wei,Wang Guoqing
Strategic Study of CAE 2014, Volume 16, Issue 10, Pages 84-88
This paper analyzes the significance of organization management to the companies and the relationship between the enterprise organization structure design and information transfer. The differences of organization structures between construction companies at home and abroad are compared. The information entropy model is used to evaluate the structure from two aspects as efficiency and effectiveness of time transmission. Based on this model, Kajima Construction Group and China Railway Construction Corporation are evaluated.
Keywords： organization structure large construction companies entropy theory degree of order
The Mathematical Model for Large-sized Agro-ecological Engineering
Bian Yousheng,Wang Tianxi,Chen Zhenglong,Cui Bin
Strategic Study of CAE 2002, Volume 4, Issue 7, Pages 17-22
This paper has set up a mathematical model of economic development, which combines the linear program model and the input-output model, based on analyzing the systematic structure and program objectives in the farm of Shengli Oil Fields. It has achieved satisfactory results in eight years by means of the mathematical model to guide the economic production in the farm and proved that the established model is right.
Keywords： agro-ecological engineering mathematical model Shengli Oil Fields
Incorporating target language semantic roles into a string-to-tree translation model Article
Chao SU, Yu-hang GUO, He-yan HUANG, Shu-min SHI, Chong FENG
Frontiers of Information Technology & Electronic Engineering 2017, Volume 18, Issue 10, Pages 1534-1542 doi: 10.1631/FITEE.1601349
Keywords： Machine translation Semantic role Syntax tree String-to-tree
Zhang Guozong,Wang Yonghua,Liu Xiong
Strategic Study of CAE 2014, Volume 16, Issue 10, Pages 106-112
The life cycle integration management of the large-scale public utility construction projects refers to the process integration of every stage of the project from decisions, planning, design, implementation, operation, maintenance to the end. In this paper, in order to realize the full life-cycle target system, we studied the whole process integration management and the interrelation of different assignments in different phases. The full life-cycle process integration model of large-scale public utility construction projects was proposed. And on this basis we discussed the support conditions of full life-cycle process integration of large-scale public utility construction projects so that the project can achieve balance and harmony in the whole life cycle as well as the investment benefit and social service function can be further improved.
Keywords： large-scale public utility construction project full life cycle process integration support conditions
Emerging topic identification from app reviews via adaptive online biterm topic modeling Research Article
Wan ZHOU, Yong WANG, Cuiyun GAO, Fei YANG,yongwang@ahpu.edu.cn
Frontiers of Information Technology & Electronic Engineering 2022, Volume 23, Issue 5, Pages 678-691 doi: 10.1631/FITEE.2100465
Keywords： App reviews Emerging topic identification Topic model Natural language processing
Liu Zhao,Meng Shaoping,Lü Zhitao
Strategic Study of CAE 2003, Volume 5, Issue 12, Pages 48-54
Two model tests for pylon´s anchorage zone of two long-span cable-stayed bridges, the Second Nanjing Yangtze River Bridge and the Runyang Yangtze River Bridge, are expounded in this paper. Safety factors for cracking and ultimate strength are obtained. The optimal configuration of prestress tendons in anchorage zone, and the emulation effects of segment model test to integral pylon are investigated. The results can be a reference for the design of pylon of cable-stayed bridges.
Keywords： cable-stayed bridge pylon model test design research
Evacuation Analysis of a Large Shopping Mall
Song Weiguo,Yu Yanfei,Zhang Heping
Strategic Study of CAE 2005, Volume 7, Issue 10, Pages 78-83
Along with the development of the society, it flows out more and more high-rise, underground and large-space buildings, of which the fire protection designs are sometimes beyond the requirements of existing national fire protection codes of China. Therefore, the performance-based fire protection design of buildings has been getting more and more chances of application. Evacuation analysis is one of the key problems in the performance-based design. In this paper, the performance-based design of a large shopping mall is introduced, and a cellular automata (CA) based evacuation model, i.e. the CAFE model is used to analyze the efficiency of evacuation. Because the interactions among pedestrians and those between pedestrians and environment are quantified in CAFE model, the values of evacuation time obtained through the model are slightly larger than those of an evacuation software, Simulex, indicating that the CAFE model is to some extent more conservative and the analysis results are with higher reliability.
Keywords： evacuation performance-based design cellular automata
Strategies and Principles of Distributed Machine Learning on Big Data Review
Eric P. Xing,Qirong Ho,Pengtao Xie,Dai Wei
Engineering 2016, Volume 2, Issue 2, Pages 179-195 doi: 10.1016/J.ENG.2016.02.008
The rise of big data has led to new demands for machine learning (ML) systems to learn complex models, with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics (such as high-dimensional latent features, intermediate representations, and decision functions) thereupon. In order to run ML algorithms at such scales, on a distributed cluster with tens to thousands of machines, it is often the case that significant engineering efforts are required—and one might fairly ask whether such engineering truly falls within the domain of ML research. Taking the view that “big” ML systems can benefit greatly from ML-rooted statistical and algorithmic insights—and that ML researchers should therefore not shy away from such systems design—we discuss a series of principles and strategies distilled from our recent efforts on industrial-scale ML solutions. These principles and strategies span a continuum from application, to engineering, and to theoretical research and development of big ML systems and architectures, with the goal of understanding how to make them efficient, generally applicable, and supported with convergence and scaling guarantees. They concern four key questions that traditionally receive little attention in ML research: How can an ML program be distributed over a cluster? How can ML computation be bridged with inter-machine communication? How can such communication be performed? What should be communicated between machines? By exposing underlying statistical and algorithmic characteristics unique to ML programs but not typically seen in traditional computer programs, and by dissecting successful cases to reveal how we have harnessed these principles to design and develop both high-performance distributed ML software as well as general-purpose ML frameworks, we present opportunities for ML researchers and practitioners to further shape and enlarge the area that lies between ML and systems.
Keywords： Machine learning Artificial intelligence big data Big model Distributed systems Principles Theory Data-parallelism Model-parallelism
Design and Application of Clinical Big Data Management System for Oncology
Ma Lin, Bao Chenlu, Li Qing,Wu Jingyi, Pan Hong’an, Li Pengfei, Zhang Luxia, Zhan Qimin
Strategic Study of CAE 2022, Volume 24, Issue 6, Pages 127-136 doi: 10.15302/J-SSCAE-2022.06.011
Cancer is a serious threat to human life and health. Along with the development of medical informatization in China, healthcare institutions have cumulated a great quantity of clinical data in oncology; however, these data have not been fully explored owing to the disunity of data standards and great difficulties in data management. Hence, establishing a national clinical big data management system for oncology based on artificial intelligence could potentially promote the application of clinical data in oncology, further improving the quality and efficiency of clinical management for oncology. This study conducted an in-depth analysis of the problems and challenges of clinical data management and application for oncology and presented the significant values of an oncology clinical data management system. Considering the complexity of multi-source and multi-modal data in oncology, we explored the possible mechanisms and pathways of applying artificial intelligence to the management and research of clinical data for oncology Furthermore, a full-circle solution was designed, and the construction framework and technology systems were promoted for the clinical data management system for oncology, including the development of common data models for oncology, data collection and security management, data standardization and structuring, data analysis and application, and data quality control. Besides, we validated the feasibility and benefits of the promoted system in clinical practice by taking the clinical data management for lung cancer in a tertiary hospital as an example. Finally, we proposed some suggestions on the research directions of the clinical big data management system for oncology.
Keywords： clinical big data management system oncology artificial intelligence common data model natural language processing
The processing technology of Longlin Temple Tunnel through a large cavern
Huang Hongjian,Xue Bin
Strategic Study of CAE 2009, Volume 11, Issue 12, Pages 35-40
Longlin Temple Tunnel is one of the 26 risk II tunnels on Yichang-Wanzhou Railway. The whole tunnel locates in limestone strata, and the standard height of the whole tunnel is through a vertical seepage zone. In construction process, several large caverns were revealed, whose karst and development characteristics were introduced, and the processing methods of two super-large karsts were discussed. DK232+467 is the biggest cavern in the construction history of China railways, and cavern protection and construction are extremely difficult. Subgrade filling and opencut tunnel were adopted. The large span roof of DK231+796 cavern is very difficult to process and its construction risk is high. Supporting columns were creatively used to support cavern roof. The successfully processing experience of the two super-large caverns will provide reference for similar projects.
Keywords： Longlin Temple Tunnel super-large cavern processing
Strategic Management of Large Projects
Wang Yingluo,Liu Yi,Li Yuan
Strategic Study of CAE 2004, Volume 6, Issue 2, Pages 28-32
The strategic management of large projects is both theoretically and practically important. Some scholars have advanced flexible strategy theory in China. Strategic flexibility and flexible strategy, and the basic system and characteristics of flexible strategy coupled with the changes of flexible strategy and integration of strategic management are discussed in this paper.
Keywords： flexible strategy strategic management large projects
Title Author Date Type Operation
Pre-Trained Language Models and Their Applications
Haifeng Wang, Jiwei Li, Hua Wu, Eduard Hovy, Yu Sun
Journal Article
Software development in the age of intelligence: embracing large language models with the right approach
Xin PENG
Journal Article
Progress in Neural NLP: Modeling, Learning, and Reasoning
Ming Zhou, Nan Duan, Shujie Liu, Heung-Yeung Shum
Journal Article
Indeterminacy Causal Inductive Automatic Reasoning Mechanism Based On Fuzzy State Describing
Yang Bingru,Tang Jing
Journal Article
Research of the Organization Structure of Large Construction Companies Based on the Entropy Theory
Wang Xing,Zhan Wei,Wang Guoqing
Journal Article
The Mathematical Model for Large-sized Agro-ecological Engineering
Bian Yousheng,Wang Tianxi,Chen Zhenglong,Cui Bin
Journal Article
Incorporating target language semantic roles into a string-to-tree translation model
Chao SU, Yu-hang GUO, He-yan HUANG, Shu-min SHI, Chong FENG
Journal Article
Research on the full life-cycle process integration model of large-scale public utility construction project and its support conditions
Zhang Guozong,Wang Yonghua,Liu Xiong
Journal Article
Emerging topic identification from app reviews via adaptive online biterm topic modeling
Wan ZHOU, Yong WANG, Cuiyun GAO, Fei YANG,yongwang@ahpu.edu.cn
Journal Article
Model Tests and Comparative Study of the Pylon's Anchorage Zone of Two Long-span Cablae-stayed Bridges
Liu Zhao,Meng Shaoping,Lü Zhitao
Journal Article
Strategies and Principles of Distributed Machine Learning on Big Data
Eric P. Xing,Qirong Ho,Pengtao Xie,Dai Wei
Journal Article
Design and Application of Clinical Big Data Management System for Oncology
Ma Lin, Bao Chenlu, Li Qing,Wu Jingyi, Pan Hong’an, Li Pengfei, Zhang Luxia, Zhan Qimin
Journal Article
The processing technology of Longlin Temple Tunnel through a large cavern
Huang Hongjian,Xue Bin
Journal Article
京公网安备 11010502051620号
