Since the release of the conversational artificial intelligence (AI) model by OpenAI in November 2022, large language model (LLM) technology has developed rapidly over the past 2 years. Today, LLMs can not only understand and generate language, images, and videos, but also solve complex problems through deep reasoning [
1] and tool invocation [
2]. With the application of LLMs in the field of systems engineering, industrial production efficiency and flexibility in scenarios such as intelligent question and answer (Q&A), data analysis [
3], code testing [
4], model-driven engineering [
5], and dynamic decision-making [
6] have been significantly improved.
Reliability systems engineering (RSE) is a complex engineering discipline that focuses on studying the entire product life-cycle and addressing failures. Its goal is to reduce failure rates, improve system reliability, and extend product lifespans. Based on this concept, Wang [
7] proposed model-based reliability systems engineering (MBRSE), which centers on the V-model and employs a model-driven approach to comprehensively consider a product’s functionality, performance, and general quality characteristics in order to achieve failure prevention, diagnosis, and management. New possibilities could be unlocked through the integration of LLMs and RSE, given the powerful analytical and learning capabilities of LLMs.
As shown in
Fig. 1, this paper provides a preliminary survey of a typical complex engineering scenario in terms of LLM application and the RSE process, summarizes and analyzes key application scenarios and technologies, identifies existing challenges, and provides recommendations for future research directions.
1. Current development of large language models in RSE
In this study, we systematically collected literature on LLMs and their applications in RSE from the Web of Science database. “LLMs OR ‘large language models’” were employed as primary keywords, along with additional RSE-related terms such as “systems engineering,” “requirements,” “design,” “manufacturing,” “verification OR verify,” “PHM OR ‘Prognostics Health Management’,” “maintenance,” “fault OR failure,” “reliability,” and “control OR management.” All papers published from 2022 onwards were included. An initial manual screening of titles and abstracts resulted in 1636 potentially relevant articles.
The subsequent in-depth analysis involved full-text reviews to confirm relevance to LLMs in RSE. We applied an RSE V-model-based coding framework to categorize LLM applications across different phases and conducted a thematic analysis to identify emerging trends and technologies. This process enabled us to summarize and reflect on the development of LLMs in application scenarios and key technologies.
1.1. Application scenarios
Based on our understanding of RSE and the synthesis of retrieved information, we constructed an RSE V-model that centers on the identification, mitigation, testing, and validation of failure. The model, which is shown in
Fig. 2, achieves multidimensional and multi-level reliability forward design in five dimensions—namely, requirements, design, manufacturing, verification, and maintenance—and at three levels—namely, system, subsystem, and assembly. In relation to the five dimensions of the V-model and the capabilities of LLMs for perception and understanding, reasoning and decision-making, and generation and interaction, potential application scenarios for LLMs in RSE are summarized below the V-model in
Fig. 2. In addition to supporting traditional scenarios such as design modeling, requirements analysis, intelligent control, and decision support, LLMs find practical applications in frontier fields, including perceptual enhancement, intelligent manufacturing, and embodied intelligence.
1.2. Key technologies
As illustrated in
Fig. 3, the technologies applied in LLMs follow a hierarchical structure comprising pre-training, fine-tuning, and prompt-based interaction, reflecting a shift from high-data-dependency foundation building to low-resource interaction with improved perceptual and interactive capacities.
1.2.1. Pre-training
Pre-training enables LLMs to acquire the foundational structure and semantics of language from large-scale natural language data. However, in industrial settings, technical constraints, limited data availability, and high training costs have hindered progress, with few novel advancements being made. In Ref. [
8], domain-specific multimodal training and industry-informed strategies are proposed to enhance LLM outputs via self-supervised and cross-modal learning.
1.2.2. Fine-tuning
Fine-tuning incorporates domain-specific data and human preferences into pre-trained LLMs, shaping the direction and style of their reasoning and improving their performance in tackling specific engineering tasks. This approach, which has been effectively applied in RSE [
9], has demonstrated that fine-tuning an LLM gives it high accuracy (97%) in detecting vulnerabilities in Ethereum smart contracts. In addition, a two-stage LLM-based framework has been proposed for the reliability assessment of rotating machinery under partial information, achieving 93.6% accuracy and outperforming other methods by 5.2% to 40.7% [
10]. Moreover, fine-tuning has been applied within a digital twin multi-agent framework, boosting the fault diagnosis accuracy for bridge cranes by 30.9% [
11]. Furthermore, a framework has been introduced for aligning crowd-sourced human feedback to enhance reinforcement learning for code generation [
12].
1.2.3. Prompt-based interaction
Prompt-based interactions, such as retrieval-augmented generation (RAG) and agent-based approaches, are expanding the capabilities of LLMs through context-guided interactions, prompt design, and tool empowerment. These advancements are being applied to tasks such as operations and maintenance knowledge Q&A [
13], fault localization and explanation [
14], and root cause analysis (RCA) [
15]. Moreover, a notable progression in leveraging LLMs for complex engineering tasks can be observed in AI-assisted software engineering [
16], collaborative agent architectures [
17], and interactive human-AI frameworks [
18], each contributing to more robust and adaptable industrial solutions.
2. Challenges
Given the interdisciplinary and complex nature of RSE, the challenges in applying LLMs can be summarized into the following three aspects.
2.1. Systemic deficiencies in domain data
Currently, there is a systemic flaw in the data within the RSE domain: The data involved in complex engineering tasks is often imbalanced and incomplete, particularly in terms of failure data, leading to biases in failure prediction. These systematic deficiencies in domain data hinder the summarization of design experiences; moreover, due to inadequate data collection during operational scenarios and maintenance phases, they limit the training effectiveness and inference capabilities of LLMs and constrain their application in RSE.
2.2. Complexity of RSE analysis
RSE analysis employs a life-cycle management approach, which is substantially complicated by the phased and diverse structure of the V-model. This complexity is exemplified by failure modes and effects analysis (FMEA), as shown in
Fig. 4. In such high-complexity analyses, it is insufficient to merely comprehend product design. To achieve a comprehensive evaluation, an LLM must also integrate physical laws, statistical patterns, and domain-specific knowledge. The integrated framework proposed in Ref. [
19] effectively addresses cross-level component dependencies, multi-timescale variables, and multidimensional maintenance resource constraints. This enables robust decision-making within dynamic system limitations. Crucially, this systematic integration of interdependencies and operational constraints is key in overcoming current LLM limitations in applied engineering settings, where thoroughness and accuracy are paramount.
2.3. Difficulties in explaining and evaluating output
Although LLM outputs, based on probabilistic inference, may be logically consistent, they often fail to meet the high standards of complex engineering tasks and can even deviate from expectations. Moreover, the “black-box” nature of LLMs results in a lack of transparency and explainability in the reasoning process. The accuracy, feasibility, and reproducibility of the output critically rely on maintaining a scientifically sound balance between automated inference and human oversight in real-world applications. In RSE, causality, interpretability, and stability are essential requirements, but the performance of current LLMs is still insufficient to meet these standards.
3. Future development directions
3.1. Pre-trained industrial LLMs for solving complex engineering issues
To tackle the challenge of scarce and imbalanced failure data in extreme RSE scenarios, it is essential to explore failure and degradation data-generation techniques under low-data conditions, including optimizing knowledge extraction, proofreading, and retrieval methods; integrating expert knowledge and experience; and generating a reliability knowledge network to refine high-quality datasets. These efforts will improve the scale, quality, and systematization of domain data, thereby enhancing LLMs’ training effectiveness and application ability in solving complex engineering problems, laying a solid foundation for reliability analysis. Additionally, improvements are still expected in the basic architecture and training methods of LLMs to address limitations in context windows, attention computation, multimodal fusion, and resource consumption.
3.2. Adaptability for solving complex analysis processes
In RSE, the occurrence patterns, control methods, and handling techniques of failures are highly complex and uncertain. Current LLMs struggle to address this complexity and lack effective means to evaluate their outputs. Therefore, it is necessary to develop technologies related to LLMs along the three dimensions of data, capability, and architecture, which include multimodal processing, problem understanding, task planning, logical reasoning, and causal analysis. A key direction involves integrating LLMs with probabilistic models, Bayesian inference, and reinforcement learning to enhance their ability to manage uncertainty and make decisions in complex, real-world scenarios. In addition, an appropriate evaluation framework must be established to quantify and assess LLM outputs, allowing them to understand and respond to real-world scenarios and thereby equipping them with the capability to address complex reliability issues. This will enable LLMs to be applied across the stages of equipment argumentation, design, manufacturing, operation and maintenance, providing support for modeling, simulation, and integration, and promoting the development of failure prevention, diagnosis, prediction, and repair technologies.
3.3. Organic integration of human, machine, and intelligence for coordinated development
After LLM technology shifts its focus from training to reasoning, these models will inevitably evolve toward richer capabilities and more seamless human-AI interaction in the near future. To address higher-level issues in RSE, especially in safety-critical domains such as aerospace, healthcare, and automotive systems, it is essential to design a seamlessly integrated human-machine-intelligence composite AI system that complies with regulatory and standardization requirements. By integrating multimodal LLM agents, advancing embodied intelligence, and enabling iterative collaboration between AI models, tools, and human experts, we can continuously evaluate and refine outputs through structured human-AI interaction. This approach fosters adaptability and trust, leading to the development of an active, dynamic composite AI system capable of supporting reliable decision-making in regulated environments.
4. Conclusions
LLMs have been preliminarily applied in RSE, demonstrating unique advantages in knowledge extraction and application compared with traditional methods. However, LLMs still fall short in areas such as data resources, performance, and explainability. This paper proposes future development directions for LLMs in RSE to address these shortcomings. To fully leverage the capabilities of LLMs, future efforts should focus on exploring more practical engineering application scenarios, promoting collaborative interaction with other intelligent models, expanding their scope of application, and developing systematic engineering frameworks. These actions will increase the intelligence of design, production, and other processes, ultimately achieving the goal of empowering RSE with LLMs.
PDF
(3127KB)
