Modern industrial systems have grown increasingly extensive, complex, and hierarchical, with operations relying on numerous knowledge-based queries. These queries necessitate considerable human resources while also requiring high levels of accuracy, subjectivity, and consistency, all of which critically influence operational efficiency. To overcome these challenges, this study proposes an industrial retrieval-augmented generation (RAG) method designed to enhance large language models (LLMs) using domain-specific knowledge, thereby improving the precision of question answering. A comprehensive industrial knowledge base was constructed from diverse sources, including journal articles, theses, books, and patents. A Text classification model based on bidirectional encoder representations from transformers (BERTs) was trained to accurately classify incoming queries. Furthermore, the general text embedding-dense passage retrieval (GTE-DPR) model was employed to perform word embedding and vector similarity retrieval, facilitating the alignment of query vectors with relevant entries in the knowledge base to obtain initial responses. LLMs subsequently refined these initial results to produce accurate final answers. Experimental evaluations confirm the effectiveness of the proposed approach. In particular, when applied to ChatGLM2-6B, the RAG method increased the ROUGE-L score from 32.52% to 55.04% and improved accuracy from 50.52% to 73.92%. Comparable improvements were also observed with LLaMA2-7B, underscoring the RAG framework’s capability to significantly enhance the accuracy and relevance of industrial question-answering (QA) systems.
Smart manufacturing has been established as a strategic national priority aimed at strengthening the global competitiveness of the manufacturing sector. The rapid advancement of artificial intelligence technologies, driven by the proliferation of big data, along with changes in research paradigms, such as the emergence of cyber-physical systems and interdisciplinary knowledge integration, has accelerated the transformation of manufacturing toward digitalization, interconnectivity, and intelligent automation. The fourth industrial revolution is anticipated to facilitate the automation and intelligent execution of knowledge-intensive tasks in manufacturing [1,2]. Nevertheless, current technologies remain inadequate in fully and efficiently exploiting industrial knowledge, thereby hindering the realization of timely intelligent analysis, accurate control, and autonomous decision-making capabilities [3,4]. Consequently, the effective integration and application of industrial knowledge have become essential to improving the performance and competitiveness of industrial systems.
Industrial question-answering (QA) systems are designed to retrieve targeted industrial knowledge and deliver precise answers, rather than merely presenting a ranked list of documents. These systems play a vital role in the integration and practical utilization of industrial knowledge [5]. Since the 1960s, a variety of QA methodologies have developed, which can generally be grouped into three main categories: corpus-based, knowledge graph-based, and large language model (LLM)-based approaches [6]. Corpus-based methods represent the earliest form of QA systems and typically rely on rule-based techniques or similarity vector computations to extract information from large text corpora. For instance, Masood et al. [7] developed an expert system for the selection of rapid prototyping technologies, incorporating data from 39 commercially available systems produced by 21 different manufacturers. Similarly, Ruiz et al. [8] introduced a system that enables hands-free information retrieval for manufacturing personnel, utilizing processed and annotated PDF documents along with similarity vector matching to ensure high classification accuracy and reliable responses. In another example, Liu et al. [9] applied natural language processing (NLP) methods to construct a decision-support system capable of accurately addressing technical queries in pressure vessel design. Although such traditional approaches have demonstrated conceptual effectiveness, they limit efficiency in semantic understanding, conceptual association, and retrieval efficiency [10,11].
Recently, knowledge graph-based methods have emerged as a structured and interpretable approach for industrial QA, facilitating reasoning over entities and their interrelationships [12]. For instance, Han et al. [13] proposed a multi-hop fault diagnosis framework that employs knowledge graph reasoning and decision-making to support fault detection in hot rolling line equipment. In the field of aviation assembly, Liu et al. [14] developed a joint reasoning framework that integrates a named entity recognition model with subgraph embedding techniques, and their findings include a comparative analysis between this approach and LLM-based QA systems. In the chemistry domain, Zhou et al. [15] introduced a novel knowledge graph QA system that utilizes hybrid embeddings to support fact-based information retrieval for chemistry-related research and industrial use cases. Similarly, Lee et al. [16] presented a general-purpose QA model that combines a knowledge graph and QA methods to ensure consistency in time and quality during the review of purchase order documents. Zhou et al. [17] also applied an injection molding knowledge graph in combination with fine-tuned bidirectional encoder representations from Transformers (BERT) model to interpret user intent and retrieve semantically relevant knowledge. Over the past several decades, knowledge graph-based methods have attracted considerable interest due to their interpretability and well-structured representation. Nonetheless, challenges persist, particularly in managing large-scale datasets, adapting to dynamic environments, handling incomplete or inaccurate information, and establishing precise relationships among semantic entities [18,19].
LLMs have been trained on extensive text corpora comprising billions of parameters to enhance natural and interactive communication between humans and machines [[20], [21], [22]]. Hostetter et al. [23] employed advanced chatbot technologies to perform QA tasks in the field of fire engineering, demonstrating the transformative potential of LLMs within this domain. Similarly, Rivera et al. [24] applied LLMs in the coal mining sector, achieving accurate and contextually appropriate answers through the use of tailored prompting strategies. Addressing QA tasks involving tabular data, Mo et al. [25] introduced a novel few-shot table prompting method to mitigate the generation of invalid Structured Query Language (SQL) or Not only SQL (NoSQL) queries by LLMs, particularly in cases involving complex questions and tables with numerous columns. However, LLMs often exhibit limited comprehension of highly specialized industrial knowledge, which can result in responses that appear fluent yet contain factual inaccuracies or fabricated content [26]. To address these limitations, retrieval-augmented generation (RAG) has emerged as a promising approach that combines the generalization ability of pre-trained LLMs with the retrieval of relevant domain-specific knowledge [[27], [28], [29], [30], [31], [32]]. For instance, Wang and Li [33] integrated LLMs with industrial knowledge bases to overcome technical challenges arising from gaps in domain expertise in operations and maintenance. Despite such progress, RAG methods based on LLMs continue to encounter two major challenges: insufficient incorporation of specialized domain knowledge and the persistence of hallucinated outputs in generated responses.
To address the limitations of conventional RAG frameworks, particularly issues related to hallucination and insufficient integration of domain-specific knowledge, this study proposes several enhancements tailored for industrial QA scenarios, such as manufacturing, equipment maintenance, and fault diagnosis. First, the industrial knowledge base is divided into specialized sub-knowledge bases to enable more accurate and efficient retrieval. Second, a BERT-based industrial question classifier is introduced to filter out non-industrial queries and direct relevant questions to the corresponding sub-knowledge base. These improvements aim to reduce hallucinations and bridge domain knowledge gaps, thereby improving the precision and contextual relevance of the responses. The main contributions of this work are as follows:
(1) A dedicated industrial knowledge base is constructed by collecting five representative types of documents, including journal articles, dissertations, books, patents, and other relevant materials. These documents undergo preprocessing steps, such as recognition, cleansing, deduplication, and segmentation, to create a structure and focused knowledge base that supports efficient retrieval and utilization.
(2) A BERT-based text classifier is developed and trained to filter questions before retrieving information from LLMs. By screening queries in advance, the classifier helps to ensure that generated answers remain within the bounds of verifiable human knowledge, thereby reducing the likelihood of hallucination outputs.
(3) A knowledge-enhanced LLM is implemented, which retrieves relevant information by processing both user queries and knowledge base content. This approach leverages the general text embedding-dense passage retrieval (GTE-DPR) model for word embedding and uses Facebook artificial intelligence similarity search (FAISS) for vector similarity search, enabling the language model to integrate retrieved domain-specific knowledge into fluent and accurate answers.
(4) Extensive comparative experiments were conducted to evaluate the effectiveness of the proposed industrial RAG framework. The results demonstrate notable improvements across modules, including question classification, knowledge retrieval, and answer generation. In addition, ablation studies highlight the contributions of each component, confirming that their integration significantly enhances the quality and relevance of industrial question answering.
The remainder of this paper is organized as follows: Section 2 reviews relevant literature on LLM, knowledge retrieval techniques, and RAG. Section 3 introduces the proposed QA approach that incorporates industrial knowledge to enhance the performance of LLMs. Section 4 presents the experimental setup and results. The conclusion and potential directions for future research are provided in the final section.
2. Related work
This section reviews the existing literature relevant to the present study, focusing on three primary areas: pre-trained LLMs (Section 2.1), knowledge retrieval methods (Section 2.2), and RAG (Section 2.3). Each of these areas contributes critically to enhancing the performance and accuracy of QA systems within specialized industrial domains.
2.1. Pre-trained LLMs
Pre-trained LLMs refer to large-scale autoregressive models in NLP that are trained on extensive text corpora and contain billions or even trillions of parameters. These models are designed to predict the next token in a sequence based on preceding tokens, with representative examples including PanGu-α [34] and ChatGPT [35]. Given a word sequence $X=\left\{{x}_{1},{x}_{2},...,{x}_{n}\right\}$ and its probability distribution $p\left(X\right)$, the training objective of LLMs involves maximizing the log-likelihood, as defined in Eq. (1). These models support a wide range of applications, such as machine translation, question answering, and text generation [2,34]:
where $p\left(X\right)=p\left({x}_{n}\mid {x}_{1},{x}_{2},...,{x}_{n-1};\theta \right)$ denotes the probability of the nth token ${x}_{n}$ conditioned on the preceding tokens ${x}_{1:n-1}$, and $\theta $ represents the model parameters to be learned during training.
In 2022, OpenAI’s release of ChatGPT drew widespread attention from the global research community and marked a significant shift toward the era of general-purpose generative artificial intelligence [[35], [36], [37], [38]]. This development endowed LLMs with impressive capabilities in general-domain question answering, primarily enabled by the Transformer architecture, which comprises multi-head attention (MHA) mechanisms and fully connected (FC) feed-forward networks (FFN), as illustrated in Fig. 1.Fig. 1(a) illustrates the encoding and propagation process of each token using the sentence “The capital of China is Beijing” as an example, through the stacked Transformer layers of the PanGu-α model to predict the next word. Fig. 1(b) details a single Transformer layer, comprising two key sub-modules: multi-head self-attention and the FFN, both encapsulated with LayerNorm and residual connections.
MHA(·): Each self-attention module within a Transformer includes four projection matrices, as depicted in Fig. 1(a): ${W}_{h}^{k},{W}_{h}^{q},{W}_{h}^{v},{W}_{h}^{m}\in {ℝ}^{d\times d/{N}_{h}}$ where d denotes the hidden dimension, h is the index of the attention head, and ${N}_{h}$ is the total number of heads. Given the output from the previous layer ${H}_{l-1}\in {ℝ}^{N\times d}$(with N being the sequence length), the three key components (query ${Q}_{h}$, key ${K}_{h}$, and value ${V}_{h}$) are computed as shown in Eq. (2).
where ${A}_{h}$ is the attention score matrix for head h, $Attentio{n}_{h}\left(·\right)$ denotes the weighted output for each token, and $\text{softmax}\left(\text{·}\right)$ normalizes scores across the sequence.
Finally, the output from all attention heads is aggregated and computed using Eq. (4).
where ${H}_{l}^{\text{MHA}}$ represents the residual-enhanced output of the multi-head attention block at layer l, and $\text{LayerNorm}\left(\text{·}\right)$ is the layer normalization function.
FFN(·): It consists of two linear layers, represented by ${W}^{1}\in {ℝ}^{d\times {d}_{ff}},{b}^{1}\in {ℝ}^{{d}_{ff}},{W}^{2}\in {ℝ}^{{d}_{ff}\times d},{b}^{2}\in {ℝ}^{d},$ where ${d}_{ff}$ denotes the dimension of the inner layer, as shown in Fig. 1(b). The output of MHA(·) is passed into FFN(·) to compute the corresponding output, as given by Eq. (5).
where $\text{GeLU}\left(\text{·}\right)$ denotes the Gaussian error linear unit activation function.
Despite the remarkable advancements of large models in language understanding and generation, challenges persist, particularly in domain-specific applications. LLMs often encounter difficulties when handling specialized or knowledge-intensive queries. In scenarios involving topics beyond their training data or requiring current information, these models may generate inaccurate or fabricated outputs. Such limitations underscore the necessity for enhancing the accuracy and reliability of LLMs, especially within specialized and rapidly evolving fields [39,40].
2.2. Knowledge retrieval
Traditional knowledge retrieval methods primarily rely on sparse retrieval techniques, such as best match 25 (BM25) [41] and term frequency-inverse document frequency (TF-IDF) [42]. These approaches build inverted indexes based on word frequency statistics, enabling efficient retrieval and demonstrating strong recall performance in practical applications, as shown in Eq. (6).
where $\text{TF}(t,{d}_{i})$ represents the frequency of a given term t within a specific document ${d}_{i}$, while $\text{IDF}(t,D)$ captures the rarity of the term t across the entire document set D. The function ${f}_{tf}(t,{d}_{i})$ counts the occurrences of term t in document ${d}_{i}$, and ${\displaystyle \sum }_{{t}^{\prime }\in {d}_{i}}{f}_{tf}({t}^{\prime },{d}_{i})$ gives the total number of terms (${t}^{\prime }$) in ${d}_{i}$. The variable n indicates the total number of documents in D, and $|{d}_{i}\in D:t\in {d}_{i}|$ denotes the number of documents in which the term t appears. Despite their effectiveness, traditional sparse retrieval methods primarily rely on term frequency statistics and often overlook the semantic relationships between words [43,44].
Pre-trained language models, such as BERT [45], have significantly improved text understanding in recent years, effectively addressing the limitations of traditional methods that overlook semantic relationships between words. For instance, Karpukhin et al. [46] proposed the dense passage retriever, a BERT-based model that generates dense vector representations through separate query and passage encoders, enabling relevance scoring between query and document vectors. This approach demonstrated notable gains over traditional retrieval models. Furthermore, Wang et al. [47] introduced a hybrid retriever framework that integrates BM25 with dense retrieval models. By applying a convex linear combination (Convex-fusion) to merge sparse and dense retrieval scores, the hybrid retriever effectively compensates for the limitations of each individual method and achieves improved recall performance in document retrieval tasks.
2.3. RAG
RAG is a technique that enhances the performance of LLMs by integrating information retrieval with generative capabilities [48]. As described by Lewis et al. [49], this approach strengthens the reasoning ability of LLMs, particularly in domains requiring specialized knowledge. By combining indexing, retrieval, and generation modules, RAG significantly improves the accuracy and reliability of responses, making it especially effective for knowledge-intensive tasks [50,51].
With advancements in technology, RAG has developed into more complex systems that have improvements in indexing techniques, such as sliding windows, fine-grained text segmentation, and metadata fusion [48,50]. For instance, Eibich et al. [52] employed hypothesis document embeddings (HyDEs) combined with LLM-based re-ranking to enhance retrieval accuracy. Similarly, Zhou et al. [53] introduced meta-knowledge summaries to personalize queries and increase retrieval depth. Despite its effectiveness, RAG continues to face challenges, such as limited retrieval efficiency and a lack of diversity in generated content.
Recent studies have increasingly emphasized modular RAG architectures, which offer improved flexibility and scalability by refining individual components, such as integrating similarity search modules and fine-tuning retrievers [54]. Common strategies include reorganizing RAG modules [55] and reconfiguring retrieval pipelines [56]. The adoption of modular RAG methods is growing, enabling sequential processing and facilitating end-to-end integration. Representative examples include FlashRAG [57], Telco-RAG [58], and AutoRAG [59]. Nonetheless, current RAG-based systems continue to encounter limitations, primarily due to the inclusion of irrelevant or conflicting information and difficulties in handling complex query formulations [50].
To mitigate the impact of irrelevant or contradictory external information, which can result in inaccurate or hallucinated responses, Yan et al. [60] proposed the corrective RAG (CRAG) framework. This method incorporates lightweight retrieval evaluators to assess the quality of retrieved content and expands retrieval coverage through large-scale web searches, thereby improving the robustness and reliability of the generated responses. Despite these advancements, RAG-based systems still face challenges in addressing complex queries with high precision. To improve performance in multi-hop question answering tasks, Chan et al. [61] proposed refining queries through explicit rewriting, decomposition, and disambiguation. In addition, Hei et al. [62] tackle the issue of dynamic relevance by improving document retrieval recall and answer accuracy using a two-stage retrieval framework supported by compact classifiers.
In industrial QA systems, the objective is not to rely on LLMs for generating conversationally fluent responses. Instead, these systems prioritize the retrieval of highly accurate and reliable answers by leveraging human-validated external cognitive knowledge bases. This involves the comprehension, application, and transformation of factual data, along with structured summarization and interpretation through LLMs. Therefore, industrial applications necessitate access to specialized and domain-specific knowledge that may not be present in the pre-training data of LLMs. It is also crucial to prevent hallucinated or fabricated outputs commonly associated with generative models.
3. Methodology
This section first presents an overview of the proposed framework. It then provides a detailed description of the industrial knowledge base. Subsequently, query analysis is presented, followed by the design of the classification module. The final subsection describes the knowledge retrieval and ranking mechanism designed to support industrial QA.
3.1. Framework overview
Inspired by the interactive response capabilities of LLMs and the flexibility of integrating domain-specific external knowledge, this work proposes an industrial QA method. This method enhances LLMs by utilizing industrial knowledge to enable accurate, efficient, and automated knowledge retrieval. As illustrated in Fig. 2, the framework consists of four main components: industrial knowledge base construction; industrial question classifier and retrieval interface; industrial knowledge retrieval; and response generation using LLMs.
Construction of industrial knowledge base (Fig. 2(a)): Industrial queries often span domains, such as electrical engineering, materials science, chemistry, construction, and related fields. This type of knowledge reflects human understanding, utilization, and adaptation of the natural environment. To support efficient retrieval and QA, relevant information from these five core domains is systematically summarized and organized. The knowledge base is developed by cleaning, tokenizing, and segmenting content from journal articles, dissertations, and production-related data, thereby refining the scope of retrieval and improving access to relevant information.
Industrial question classifier (Fig. 2(b)): This component classifies incoming industrial questions to define the response scope of the system and further refine the knowledge retrieval process. A BERT-based text filter is used to screen and categorize questions, evaluating their relevance to the industrial domain. It also aligns the questions with the appropriate sub-knowledge base.
Industrial knowledge retrieval (Fig. 2(c)): This module employs dense retrieval methods to identify the most relevant information fragments from the knowledge base, ensuring both accuracy and relevance. Initially, related texts are extracted from a broad corpus, which is then used to enhance the ranking performance during retrieval.
LLM prompted responses (Fig. 2(d)): This component converts retrieved knowledge chunks into coherent and accurate responses suited to human comprehension. The knowledge fragments are combined with user questions to form a prompt template, which is input into LLMs using prompt engineering techniques. The generated responses are then delivered to the user.
3.2. Industrial knowledge base construction
A precise and domain-specific knowledge base is crucial for effective industrial QA. The construction process begins by converting documents into editable text, followed by the use of regular expressions to remove elements, such as tables of contents, images, annotations, and blank spaces. Subsequent steps include data cleaning and the removal of redundant content to ensure the textual material is complete and coherent. A predefined stop-word list is employed to eliminate special symbols and non-informative words, such as “some” and “many.” After this, tokenization is applied to divide the text into individual words while maintaining sentence-level meaning. Text chunking is then carried out to group semantically related content, using newline characters, specifically “\n” and “\n\n”, to segment the text accordingly.
Given that industrial texts are rich in technical terminology and generally composed of short sentences, the maximum chunk length is limited to 256 characters, with a maximum overlap of 32 characters between chunks. This overlapping technique ensures semantic continuity between segments. The final knowledge base is stored in an editable format using the structure “Title: Chunk.” Furthermore, based on the source content, the knowledge base is divided into five sub-domains corresponding to major industrial sectors: chemical engineering (${K}_{\text{C}}$), petroleum (${K}_{\text{P}}$), metallurgy (${K}_{\text{M}}$), electricity (${K}_{\text{E}}$), and building materials (${K}_{\text{B}}$), as shown in Eq. (7).
where K represents the industrial knowledge base, L is the chunk length, O is the character overlap between chunks, and $f_{p}(\cdot)$ denotes the processing function applied to the industrial corpus. The parameter L=256 specifies the maximum chunk length, while O=32 indicates the maximum character overlap between chunks.
3.3. Question classifier for narrowing retrieval scope
LLMs, such as ChatGPT [35], often generate inaccurate or misleading responses when addressing queries from specialized or interdisciplinary domains. Let ${Q}_{\text{a}}$ denote the complete set of input questions, R represent the subset of questions that fall within a specific domain’s scope, and C indicate the subset of questions for which the model can provide professional and accurate answers. It follows that $C\le R\le {Q}_{\text{a}}$. To constrain queries within the domain-relevant subset R, a question text filter is employed to map the broader set ${Q}_{\text{a}}$ into R, thereby increasing the proportion of relevant and answerable queries. This process is formalized in Eq. (8).
$R={\aleph }_{t}({Q}_{\text{a}})$
where $\aleph_{t}(\cdot)$ denotes the question text filter, which determines whether a given user query is relevant to the industrial domain. Questions identified as irrelevant are excluded from further processing to prevent ambiguity and hallucinations commonly associated with LLMs. If a query is deemed relevant, it is further classified to determine its corresponding industrial subdomain, such as chemical, petroleum, metallurgy, electricity, or building materials, and is then mapped to the appropriate sub-knowledge base ${K}_{T}$. The process of industrial question classification is further defined by Eq. (9).
where ${K}_{T}$ denotes the sub-knowledge base associated with the relevant query category, $k\in K$ represents an individual knowledge item within the industrial knowledge base, and $\text{Ctg}\left(k\right)$ denotes its assigned category label, and the function ${f}_{\text{flt}}$ is responsible for filtering questions based on industrial relevance, while ${f}_{\text{cls}}$ classifies the filtered questions into their corresponding industrial subdomains. Let Q represent a user-submitted query, and the transition ${Q}_{\text{a}}\to R$ indicates the mapping of all input questions to the subset that pertains to the industrial domain. The label T identifies the specific category to which the question belongs, namely, metallurgy, petroleum, chemical, electricity, or building materials.
Fig. 3 illustrates the process involving the industrial question filter ${f}_{\text{flt}}$ and the classifier ${f}_{\text{cls}}$, which determine whether a question is relevant to the industrial domain and identify its corresponding category. The industrial text filter ${f}_{\text{flt}}$ is built on a BERT-based model, which assesses the necessity of a response and assists the language model in generating suitable answers. Within this framework, the BERT network functions either as a filter (upper path) or as a classifier (lower path), enabling text vectorization of the input questions. The primary distinction between the two lies in their classification objectives: the filter performs binary classification, whereas the classifier addresses a multi-class classification task. Final classification outcomes are derived by feeding the question embeddings into an FC layer, as defined in Eq. (10).
where W denote the parameter matrix of the classification network (an FC network), which is fine-tuned by maximizing the log-likelihood of the correct label. G is the predicted category label, H denotes the hidden representation from the “[CLS]” token, and “[CLS]” is a special token in BERT used to summarize the input sequence for classification. The classification output is derived from the “[CLS]” token representation, based on two contributing elements that enhance the model’s interpretability. The classifier distinguishes among six categories: non-relevant, metallurgy, petroleum, chemistry, building materials, and electricity. During training, only the parameters of the FC layer are updated. Although the filter and classifier can be unified into a single multi-class classification model, they are discussed separately in this work to improve clarity and interpretability.
In this study, the dimension of the BERT-based question text embedding is set to 768. The FC network comprises two hidden layers with dimensions of 384 and 768. The output layer has a dimension of 6, corresponding to the industrial categories, along with an additional class for non-relevance. The non-relevance class serves to constrain responses within the boundaries of human-validated cognitive knowledge bases, focused on the understanding, utilization, and transformation of nature, rather than enabling unrestricted, human-like interactive responses.
3.4. Industrial knowledge retrieval
Knowledge retrieval is a key component of the proposed industrial QA framework. This process involves mapping both user queries and knowledge base content into a continuous vector space using a pre-trained LLM. The similarity between the query and individual knowledge fragments is then computed to identify initial candidate segments. These candidates are subsequently ranked based on their similarity scores, as defined in Eq. (11).
where sim(·) denotes the similarity function, such as FAISS or cosine similarity. The variable $q\in {Q}_{a}$ represents a question within the set of answerable queries ${Q}_{a}$, and ${K}_{j}\in K$ refers to the jth knowledge chunk identified as relevant to q, where j is the index of the chunk. ${\mathrm{arg}}_{j}$max(·) denotes the index j at which the maximum value is attained. ${K}_{Q}$ denotes the final set of knowledge fragments retrieved from the corresponding sub-knowledge base ${K}_{T}$ that matches the query domain. This retrieval process aims to maximize recall of relevant information and extract the top-K most informative context segments.
DPR is employed using a dual-encoder architecture to support industrial knowledge retrieval [46], as shown in Fig. 4. This architecture enables efficient retrieval of large volumes of text chunks by replacing the traditional BERT model with the general text embedding (GTE-large-zh) model, which generates 1024-dimensional word embeddings [63]. GTE-large-zh demonstrates enhanced capability in capturing complex semantic information and accurately representing word relationships. To improve similarity matching across high-dimensional text vectors, a similarity matrix $s\in {ℝ}^{m}$ is computed between the DPR-based query embeddings and the embeddings of knowledge chunks using the FAISS library, based on ${L}_{2}$ distance [64]. A smaller ${L}_{2}$ distance corresponds to a higher degree of similarity.
Given the varying precision levels of different retrieval engines, the quality of the retrieved results can be inconsistent. Therefore, as expressed in Eq. (12), it is essential to perform an additional ranking step to accurately filter and organize the final set of relevant contextual information.
where ${K}_{Q}^{r}$ denotes the rankedd set of knowledge chunks ${K}_{Q}^{i}$ that are most relevant to the query q, obtained through a sorting function $\text{Sort()}$. ${K}_{i}$ denotes the ith knowledge fragment in the retrieved set, and ${\mathrm{arg}}_{i}\text{sort()}$ denotes the index i corresponding to the sorted order of values. In this study, the BGE-reranker-large model is employed for this purpose. It is trained on a multilingual dataset and based on the XLM-RoBERTa architecture, offering robust performance across both Chinese and English language scenarios.
3.5. Industrial QA and evaluation
After retrieving the relevant contextual knowledge, each knowledge chunk is combined with the user’s original question and sequentially input into the LLM. The model integrates the provided knowledge to generate accurate and controlled responses aligned with the query. Consequently, the answers are grounded in validated cognitive knowledge bases rather than relying on generic, conversational outputs that imitate human expression. This process is formally described in Eq. (13).
$A_{k}=\operatorname{LLMs}\left(M \bullet K_{Q}^{r}\right), M \propto \overrightarrow{\text { Scores }}$
where ${A}_{k}$ denotes the final answer generated by the LLM. The term M represents virtual weight matrices used to emphasize shared or overlapping knowledge across the ranked knowledge chunks ${K}_{Q}^{r}$. The variable $\overrightarrow{\text{Scores}}$ refers to the sorted relevance scores between the query and the retrieved knowledge chunks. $\propto $ denotes element-wise proportionality between vectors.
As shown in Table 1, the effectiveness and reliability of the proposed method are evaluated using several standard metrics. For the question classification module, precision (P), recall (R), and F1 score are adopted as evaluation indicators [8,11,16,26]. The performance of the knowledge retrieval module is assessed using mean reciprocal rank (MRR) [65]. Finally, for the answer generation module, recall-oriented understudy for gisting evaluation-longest common subsequence (ROUGE-L) and accuracy (ACC) are used to evaluate the quality and correctness of the generated responses [66].
4. Experimental study
Industrial QA emphasizes the retrieval of validated cognitive knowledge, as opposed to generating conversational, human-like responses. Accordingly, the experimental study is designed to evaluate key components of the proposed method, including knowledge base construction, question classification, knowledge retrieval, and answer generation performance. An ablation study is also conducted to verify the effectiveness of each module within the framework.
4.1. Description and setup
The industrial knowledge base comprises approximately 12 410 documents accumulated over the past two decades. These include 3462 journal articles, 1689 theses, 210 books, 5477 patents, and various other materials related to industrial production. All documents were converted into editable text, with elements, such as tables of contents, images, annotations, blank spaces, and redundant content removed. Special characters and non-informative words were processed using a stopword list. To preserve sentence integrity, the texts were tokenized into individual words, and newline characters (“\n” and “\n\n”) were identified to ensure continuity of content. The knowledge base was structured using a chunking method with a maximum chunk length of 256 characters and an overlap of 32 characters between chunks. Texts were stored in a “Title: Chunk” format, resulting in a total of 107 045 retrievable segments.
Industrial QA datasets were derived from the constructed knowledge base. In this process, questions were generated by LLMs based on corresponding answers and subsequently reviewed and refined by domain experts to ensure their ACC. To support the evaluation of industrial question classification, five sub-domain corpora were compiled, encompassing metallurgy, petroleum, chemicals, electricity, and building materials.
In the question classification experiment, the design adhered to the principle of controlling computational complexity without compromising the validity of the experimental conclusions. The classification model was configured with a maximum chunk length of 128, a batch size of 16, a dropout rate of 0.1 to mitigate overfitting, cross-entropy as the loss function, and the Adam optimizer with an initial learning rate of 1×10-5. The model was trained for 3 epochs using a batch size of 16. For performance comparison and validation, four baseline models were employed: FastText [67], TextCNN [68], TextRNN [69], and TextRCNN [70], using the parameter configurations specified in their respective original studies.
For knowledge retrieval and re-ranking, the parameter k is set to five or ten, and three models are used for the corresponding experiments: General DPR [46], Domain DPR [63], and Domain + GTE-DPR.
For the industrial QA task, two publicly available LLMs, ChatGLM2-6B and LLaMA2-7B, are selected as the base models. The generation parameters are configured with a temperature of 0.7 and a maximum output length of 500 tokens. All experiments are conducted on the same hardware setup, which includes two NVIDIA GeForce RTX 3090 GPUs (Peng Cheng Laboratory, China) and a 104-core Intel(R) Xeon(R) Gold 6230R CPU (Peng Cheng Laboratory) running at 2.10 GHz.
4.2. Results of question classification
Question classification is an essential function in industrial QA systems, as it determines the appropriate handling of user queries. As previously noted, the classification model is implemented using BERT. A classification dataset was constructed by randomly selecting 7800 question samples from the full dataset, with equal representation across six categories: metallurgy, petroleum, chemicals, electricity, building materials, and non-industrial. The datasets were divided into three parts: 6000 samples for training, 1200 for testing, and 600 for validation.
Fig. 5 presents the performance of the proposed industrial question classification system. In Fig. 5(a), the classification results on the validation set are shown. The confusion matrix reveals three misclassifications of petroleum as chemical and three misclassifications of chemicals as petroleum. Overall, the validation set achieves over 95 correct predictions out of 100 samples for each category. Fig. 5(b) illustrates the results on the test set, where confusion is again observed between petroleum and chemical categories, with 11 petroleum questions misclassified as chemicals and 8 chemicals questions misclassified as petroleum. This misclassification suggests a degree of similarity between content in these two domains. On the test set, the classification model achieves more than 185 correct predictions out of 200 samples per category, with particularly strong performance in the metallurgy and electricity categories. These results confirm that the BERT-based classification model performs effectively for industrial classification.
To further assess the performance of the BERT-based question classification model, comparative experiments were conducted against four baseline models: FastText [67], TextCNN [68], TextRNN [69], and TextRCNN [70]. Each experiment was executed ten times to ensure the consistency and reliability of the classification outcomes. Table 2 presents the comparison results in terms of precision, recall, and F1-score. On the test set, the proposed model consistently outperformed all baseline models. In particular, it achieved a mean F1-score of 97.79%, which is approximately 20.17% higher than the next best-performing model, TextRCNN. Furthermore, across ten independent runs, the BERT classifier exhibited a standard deviation of only 0.23% in F1-score, whereas all neural baseline models showed standard deviations of at least 0.8%. This low variance demonstrates the model’s robustness to random initialization and data shuffling, an important attribute for industrial environments where frequent retraining may be required.
4.3. Results of knowledge retrieval
The knowledge retrieval dataset was divided into 100 000 queries for training and 7045 queries for testing. To evaluate the effectiveness of the proposed retrieval strategy, performance was assessed using top-k retrieval metrics with k=5 and k=10. Three models were considered for comparison: General DPR [46], Domain DPR [63], and the proposed Domain + GTE-DPR model. Each experiment was conducted five times to ensure the consistency and reliability of the results.
Table 3 summarizes the comparative performance of the models based on MRR@5 and MRR@10. The proposed Domain + GTE-DPR model outperformed both baselines across all metrics. Specifically, it achieved a mean MRR@5 of 90.12% and a mean MRR@10 of 90.28%, surpassing the retrieval accuracy of the other approaches. Moreover, the model exhibited the lowest deviations among all methods, indicating excellent stability and robustness. The findings underscore the effectiveness of domain-specific training, as evident from the performance improvements of the domain DPR model over the Domain DPR baseline.
Further enhancement was achieved by replacing the BERT-base-Chinese encoder with the GTE-base-zh embedding model, which improved both the accuracy and consistency of retrieval in the Domain + GTE-DPR configuration. This result highlights the importance of domain adaptation and the use of semantic embeddings in optimizing industrial knowledge retrieval. In addition, the near-zero variance (standard deviation = 0) observed for the proposed model reflects its exceptional consistency across repeated trials. By contrast, although the Domain DPR model achieved relatively strong results, it displayed minor performance variability, whereas the General DPR baseline showed lower retrieval accuracy and higher consistency. These observations confirm that general-purpose models are inadequate for handling domain-specific knowledge tasks in industrial settings without appropriate domain adaptation.
4.4. Industrial QA generation performance
Due to limitations in computational resources, the QA generation experiments were performed using two LLMs: ChatGLM2-6B and LLaMA2-7B. The dataset used for these experiments was consistent with that of the knowledge retrieval task and comprised the same set of queries and documents. From this dataset, a subset of 1000 QA pairs was randomly selected for evaluation. To ensure the reliability of the results, each experiment was conducted five times.
The objective of this experiment was to compare the answer generation performance of the LLMs under two different settings: without and with the integration of the RAG framework. The first approach, referred to as the LLM-based method, involved directly inputting questions into the LLM without any supporting retrieved content, thereby serving as the baseline. The second approach, termed the RAG-based method, provided the LLMs with additional contextual information in the form of relevant documents, which were obtained through the proposed classification and retrieval modules. This setup was designed to evaluate the extent to which the incorporation of retrieved knowledge improves the accuracy and quality of generated answers in industrial domains.
Table 4 presents a comparative analysis of the industrial QA performance of two LLMs, ChatGLM2-6B and LLaMA2-7B, under LLM-based and RAG-based configurations. The results clearly demonstrate that the integration of the RAG framework significantly enhances both the quality and accuracy of the generated responses. Specifically, for ChatGLM2-6B, the average ROUGE-L score increased from 32.52% to 55.04%, while the average ACC improved from 50.52% to 73.92%. Similarly, for LLaMA2-7B, the ROUGE-L score rose from 29.54% to 54.00%, and average ACC increased from 42.92% to 70.60%. These improvements indicate that incorporating relevant retrieved documents into the prompt context enables the models to generate more accurate and contextually appropriate responses. Furthermore, the reduction in standard deviation across metrics suggests enhanced stability and consistency in output quality.
The RAG framework yields considerable absolute gains, exceeding 22% in ROUGE-L and between 23% and 28% in ACC across both models, underscoring its effectiveness in improving factual ACC. The marked reduction in standard deviation, particularly in ROUGE-L (from 1.45% to 0.41% for ChatGLM2-6B), further validates that RAG not only enhances average performance but also ensures more consistent and dependable answer generation.
4.5. Ablation study
To assess the individual contributions of the question classification and knowledge retrieval modules within the proposed RAG framework for industrial knowledge integration, an ablation study was conducted. Three configurations were evaluated: the baseline ChatGLM2-6B model without any auxiliary modules; ChatGLM2-6B augmented with the question classification module; ChatGLM2-6B enhanced with both the question classification and knowledge retrieval modules.
The results of this study are summarized in Table 5. The baseline configuration achieved a mean ROUGE-L score of 32.52% and a mean ACC of 50.52%. Introducing the question classification module led to marked improvements, raising the ROUGE-L and ACC scores to 46.40% and 66.06%, respectively. These gains are attributed to the classifier’s ability to filter out irrelevant queries and direct appropriate inputs to the retrieval process. When both the question classification and knowledge retrieval modules were applied, the model achieved a further increase in performance, with the ROUGE-L mean score reaching 55.04% and ACC improving to 73.92%.
These results demonstrate that each module contributes incrementally and significantly to overall system performance. The classifier alone enhanced ACC by over 15 percentage points, highlighting its role in refining input relevance. The addition of the retrieval module further improved factual consistency and completeness. Moreover, the standard deviation of the ROUGE-L score decreased from 1.45% in the baseline model to 0.41% in the full configuration, indicating increased output stability. This progressive improvement underscores the value of modular architecture in developing robust industrial and accurate industrial QA systems.
5. Conclusion
This study presents a comprehensive approach to integrating domain-specific industrial knowledge into a robust and accurate QA system, with the goal of reducing hallucination and enhancing answer reliability in industrial contexts. The proposed framework combines a structured domain knowledge base with modules for question classification, knowledge retrieval, and response generation powered by LLMs. A BERT-based classifier was developed to accurately identify industrial queries, achieving a high F1-score of 97.79%, thereby demonstrating its effectiveness in filtering relevant questions. For knowledge retrieval, the system incorporates the Domain + GTE-DPR model, which achieved strong performance metrics, with MRR@5 and MRR@10 values of 90.12% and 90.28%, respectively, underscoring the benefit of domain-specific adaptation.
Furthermore, the integration of the RAG method significantly improved the quality and relevance of the generated answers. Notably, the ChatGLM2-6B model exhibited a rise in the mean ROUGE-L score from 32.52% to 55.04%, and in ACC from 50.52% to 73.92%. Similarly, the LLaMA2-7B model showed an increase in the ROUGE-L score from 29.54% to 54.00% and in ACC from 42.92% to 70.60%. The effectiveness of the proposed framework was further validated through ablation studies, which confirmed the complementary contributions of the question classification and knowledge retrieval components. Overall, the results indicate that the proposed method offers a reliable and efficient solution for intelligent question answering in industrial applications, establishing a strong foundation for further advancements in this field.
Despite its effectiveness, the current framework has certain limitations. Although the industrial knowledge base covers multiple domains, the processes of segmentation and preprocessing may not capture all domain-specific nuances. In addition, conventional evaluation metrics may fall short in accurately reflecting the consistency between AI-generated answers and expert assessments. Future work will focus on expanding and regularly updating the knowledge base to improve its comprehensiveness. It will also explore the integration of evaluation tools, such as quality assessment of machine-generated answers (QAMAI) [71], to better assess the alignment of model outputs with expert-level responses.
LuJ, WangX, ChengX, YangJ, KwanO, WangX. Parallel factories for smart industrial operations: from big AI models to field foundational models and scenarios engineering. IEEE/CAA J Autom Sinica2022; 9(12):2079-86.
[2]
WuT, HeS, LiuJ, SunS, LiuK, HanQL, et al. A brief overview of ChatGPT: the history, status quo and potential future development. IEEE/CAA J Autom Sinica2023; 10(5):1122-36.
[3]
YuanX, XuW, WangY, YangC, GuiW. A deep residual PLS for data-driven quality prediction modeling in industrial process. IEEE/CAA J Autom Sinica2024; 11(8):1777-85.
[4]
SunPZ, BaoY, MingX, ZhouT. Knowledge-driven industrial intelligent system: concept, reference model, and application direction. IEEE Trans Comput Soc Syst2023; 10(4):1465-78.
[5]
OjokohB, AdebisiE. A review of question answering systems. J Web Eng2018; 17(8):717-58.
[6]
YangT, MeiY, XuL, YuH, ChenY. Application of question answering systems for intelligent agriculture production and sustainable management: a review. Resour Conserv Recycling2024; 204:107497.
[7]
MasoodSH, SooA. A rule based expert system for rapid prototyping system selection. Robot Comput-Integr Manuf2002; 18(3-4):267-74.
[8]
RuizE, TorresMI, del PozoA. Question answering models for human-machine interaction in the manufacturing industry. Comput Ind2023; 151:103988.
[9]
LiuX, ChengZ, ShenZ, ZhangH, MengH, XuX, et al. Building a question answering system for the manufacturing domain. IEEE Access2022; 10:75816-24.
[10]
Calijorne SoaresMA, ParreirasFS. A literature review on question answering techniques, paradigms and systems. J King Saud Univ Comput Inf Sci2020; 32(6):635-46.
[11]
TianD, LiM, RenQ, ZhangX, HanS, ShenY. Intelligent question answering method for construction safety hazard knowledge based on deep semantic mining. Autom Construct2023; 145:104670.
[12]
XiongH, WangS, TangM, WangL, LinX. Knowledge graph question answering with semantic oriented fusion model. Knowl Base Syst2021; 221:106954.
[13]
HanH, WangJ, WangX. Leveraging knowledge graph reasoning in a multihop question answering system for hot rolling line fault diagnosis. IEEE Trans Instrum Meas2023; 73:3505014.
[14]
LiuP, QianL, LuH, XueL, ZhaoX, TaoB. The joint knowledge reasoning model based on knowledge representation learning for aviation assembly domain. Sci China Technol Sci2024; 67(1):143-56.
[15]
ZhouX, ZhangS, AgarwalM, AkroydJ, MosbachS, KraftM. Marie and BERT—a knowledge graph embedding based question answering system for chemistry. ACS Omega2023; 8(36):33039-57.
[16]
LeeSH, ChoiSW, LeeEB. A question-answering model based on knowledge graphs for the general provisions of equipment purchase orders for steel plants maintenance. Electronics2023; 12(11):2504.
[17]
ZhouZW, JongWR, TingYH, ChenSC, ChiuMC. Retrieval of injection molding industrial knowledge graph based on transformer and BERT. Appl Sci2023; 13(11):6687.
[18]
WenP, MaY, WangR. Systematic knowledge modeling and extraction methods for manufacturing process planning based on knowledge graph. Adv Eng Inf2023; 58:102172.
[19]
YuP, GongW, BaiZ, ZhaoH, DengW. Knowledge graph civil aviation question answering based on deep learning. In: Proceedings of the 2022 China Automation Congress (CAC), 2022 Nov 25-27; Xiamen, China. Piscataway: IEEE; 2022. p. 600-4.
[20]
von RuedenL, MayerS, BeckhK, GeorgievB, GiesselbachS, HeeseR, et al. Informed machine learning—a taxonomy and survey of integrating prior knowledge into learning systems. IEEE Trans Knowl Data Eng2021; 35(1):614-33.
[21]
TopsakalO, AkinciTC. Creating large language model applications utilizing langchain: a primer on developing LLM apps fast. Int Conf Appl Eng Nat Sci2023; 1(1):1050-6.
[22]
TallatR, HawbaniA, WangX, Al-DubaiA, ZhaoL, LiuZ, et al. Navigating industry 5.0: a survey of key enabling technologies, trends, challenges, and opportunities. IEEE Commun Surv Tutor2023; 26(2):1080.
[23]
HostetterH, NaserM, HuangX, GalesJ. The role of large language models (AI chatbots) in fire engineering: an examination of technical questions against domain knowledge. Nat Hazards Res2024; 4(4):669-88.
ZhengS, PanK, LiuJ, ChenY. Empirical study on fine-tuning pre-trained large language models for fault diagnosis of complex systems. Reliab Eng Syst Saf2024; 252:110382.
[27]
JiZ, YuT, XuY, LeeN, IshiiE, FungP. Towards mitigating LLM hallucination via self reflection. Findings of the association for computational linguistics. In: Bouamor H, Pino J, Bali K, editors. Findings of the Association for Computational Linguistics: EMNLP 2023. Kerrville: Association for Computational Linguistics; 2023. p. 1827-43.
[28]
WangC, LongQ, XiaoM, CaiX, WuC, MengZ, et al. Biorag: a RAG-LLM framework for biological question reasoning. 2024. arXiv:2408.01107.
[29]
JiangX, FangY, QiuR, ZhangH, XuY, ChenH, et al. TC-RAG:turing-complete RAG’s case study on medical LLM systems. In:Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics; 2025 Jul 27-Aug 1; Vienna, Austria. Kerrville: Association for Computational Linguistics; 2025. p. 11400-26.
[30]
LiJ, YuanY, ZhangZ. Enhancing LLM factual accuracy with rag to counter hallucinations: a case study on domain-specific queries in private knowledgebases. 2024. arXiv:2403.10446.
[31]
MansurovaA, MansurovaA, NugumanovaA. QA-RAG: exploring LLM reliance on external knowledge. Big Data Cog Comput2024; 8(9):115.
[32]
LiuJ, LinJ, LiuY. How much can rag help the reasoning of LLM? 2024. arXiv:2410.02338.
[33]
WangH, LiYF. Large language model empowered by domain-specific knowledge base for industrial equipment operation and maintenance. In:Proceedings of the 2023 5th International Conference on System Reliability and Safety Engineering (SRSE); 2023 Oct 20-23; Beijing, China. Piscataway: IEEE; 2023. p. 474-9.
[34]
ZengW, RenX, SuT, WangH, LiaoY, WangZ, et al. Pangu-a: large-scale autoregressive pretrained chinese language models with auto-parallel computation. 2021. arXiv:2104.12369.
[35]
BrownT, MannB, RyderN, SubbiahM, KaplanJ, DhariwalP, et al. Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems; 2020 Dec 6-12; Vancouver, BC, Canada. New York City: Curran Associates Inc.; 2020. p. 1877-901.
[36]
ZhouB, LiX, LiuT, XuK, LiuW, BaoJ. CausalKGPT: industrial structure causal knowledge-enhanced large language model for cause analysis of quality problems in aerospace product manufacturing. Adv Eng Inf2024; 59:102333.
[37]
ZhangX, ZhouZ, MingC, SunYY. GPT-assisted learning of structure-property relationships by graph neural networks: application to rare-earth-doped phosphors. J Phys Chem Lett2023; 14(50):11342-9.
[38]
SitapureN, KwonJSI. CrystalGPT: enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers. Comput Chem Eng2023; 177:108339.
[39]
KandpalN, DengH, RobertsA, WallaceE, RaffelC. Large language models struggle to learn long-tail knowledge. In:Proceedings of the 40th International Conference on Machine Learning; 2023 Jul 23-19; Honolulu, HI, USA. Cambridge: MIT Press; 2023. p. 15696-707.
[40]
ZhangY, LiY, CuiL, CaiD, LiuL, FuT, et al. Siren’s song in the AI ocean: a survey on hallucination in large language models. 2023. arXiv:2309.01219.
[41]
RobertsonS, ZaragozaH. The probabilistic relevance framework: BM25 and beyond. Found Trends Inf Retr2009; 3(4):333-89.
[42]
RobertsonSE, WalkerS. On relevance weights with little relevance information. In: Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1997 Jul 27-31; Philadelphia, PA, USA. New York City: Association for Computing Machinery; 1997. p. 16-24.
[43]
MariottiL, GuidettiV, MandreoliF, BelliA, LombardiP. Combining large language models with enterprise knowledge graphs: a perspective on enhanced natural language understanding. Front Artif Intell2024; 7:1460065.
[44]
SiriwardhanaS, WeerasekeraR, WenE, KaluarachchiT, RanaR, NanayakkaraS. Improving the domain adaptation of retrieval augmented generation (RAG) models for open domain question answering. Trans Assoc Comput Linguist2023; 11:1-17.
[45]
KentonJDMWC, ToutanovaLK. BERT:pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies; 2019 Jun 2-7; Minneapolis, MN, USA. Kerrville: Association for Computational Linguistics; 2019. p. 4171-86.
[46]
KarpukhinV, Og˘uzB, MinS, LewisP, WuL, EdunovS, et al. Dense passage retrieval for open-domain question answering. In:Proceedings of the Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2020 Nov 16-20; Online. Kerrville: Association for Computational Linguistics; 2020. p. 6769-81.
[47]
WangS, ZhuangS, ZucconG. BERT-based dense retrievers require interpolation with BM25 for effective passage retrieval. In: Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval; 2021 Jun 21; Online. New York City: Association for Computing Machinery; 2021. p. 317-24.
[48]
FanW, DingY, NingL, WangS, LiH, YinD, et al. A survey on rag meeting LLMs:towards retrieval-augmented large language models. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25-29; Barcelona, Spain. New York City:Association for Computing Machinery; 2024. p. 6491-501.
[49]
LewisP, PerezE, PiktusA, PetroniF, KarpukhinV, GoyalN, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Proceedings of the 34th International Conference on Neural Information Processing Systems; 2020 Dec 6-12; Vancouver, BC, Canada. New York City: Curran Associates Inc.; 2020. p. 9459-74.
[50]
GaoY, XiongY, GaoX, JiaK, PanJ, BiY, et al. Retrieval-augmented generation for large language models: a survey. 2023. arXiv:2312.10997.
[51]
MaX, GongY, HeP, ZhaoH, DuanN. Query rewriting for retrieval-augmented large language models. In:Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; 2023 Dec 6-10; Singapore. Kerrville: Association for Computational Linguistics; 2023. p. 5303-15.
ZhouY, LiuZ, JinJ, NieJY, DouZ.Metacognitive retrieval-augmented large language models. In: Proceedings of the ACM Web Conference 2024; 2024 May 13-17; Singapore. New York City: Association for Computing Machinery; 2024. p. 1453-63.
[54]
MuludiK, FitriaKM, TrilokaJ, Sutedi. Retrieval-augmented generation approach: document question answering using large language model. Int J Adv Comput Sci Appl2024; 15(3):776-85.
[55]
YuW, IterD, WangS, XuY, JuM, SanyalS, et al. Generate rather than retrieve: large language models are strong context generators. 2022. arXiv:2209.10063.
[56]
ShaoZ, GongY, ShenY, HuangM, DuanN, ChenW. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. 2023. arXiv:2305.15294.
[57]
JinJ, ZhuY, DouZ, DongG, YangX, ZhangC, et al. FlashRAG:a modular toolkit for efficient retrieval-augmented generation research. In:Companion Proceedings of the ACM on Web Conference 2025; 2025 Apr 28-May 2; Sydney, Australia. New York City:Association for Computing Machinery; 2025. p. 737-40.
[58]
BorneaAL, AyedF, De DomenicoA, PiovesanN, MaatoukA. Telco-RAG: navigating the challenges of retrieval augmented language models for telecommunications. In: Proceedings of the GLOBECOM 2024— 2024 IEEE Global Communications Conference; 2024 Dec 8-12; Cape Town, South Africa. Piscataway: IEEE; 2024. p. 2359-64.
[59]
KimD, KimB, HanD, EibichM. AutoRAG: automated framework for optimization of retrieval augmented generation pipeline. 2024. arXiv:2410.20878.
ChanCM, XuC, YuanR, LuoH, XueW, GuoY, et al. RQ-RAG: learning to refine queries for retrieval augmented generation. 2024. arXiv:2404.00610.
[62]
HeiZ, LiuW, OuW, QiaoJ, JiaoJ, SongG, et al. DR-RAG: applying dynamic document relevance to retrieval-augmented generation for question-answering. 2024. arXiv:2406.07348.
[63]
LiZ, ZhangX, ZhangY, LongD, XieP, ZhangM. Towards general text embeddings with multi-stage contrastive learning. 2023. arXiv:2308.03281.
[64]
JohnsonJ, DouzeM, JégouH. Billion-scale similarity search with GPUs. IEEE Trans Big Data2021; 7(3):535-47.
[65]
ZhangW, WangM, HanG, FengY, TanX. A knowledge graph completion algorithm based on the fusion of neighborhood features and vBiLSTM encoding for network security. Electronics2024; 13(9):1661.
[66]
LinCY. Rouge:a package for automatic evaluation of summaries. In: Text summarization branchesout. Kerrville:Association for Computational Linguistics; 2004. p. 74-81.
[67]
JoulinA, GraveE, BojanowskiP, MikolovT. Bag of tricks for efficient text classification. In:Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics; 2017 Apr 3-7; Valencia, Spain. Kerrville: Association for Computational Linguistics; 2017. p. 427-31.
[68]
KimY. Convolutional neural networks for sentence classification. In:Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2014 Oct 25-29; Doha, Qatar. Kerrville: Association for Computational Linguistics; 2014. p. 1746-51.
[69]
LiuP, QiuX, HuangX. Recurrent neural network for text classification with multi-task learning. In:Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence; 2016 Jul 9-15; New York City, NY, USA. Palo Alto:AAAI Press; 2016. p. 2873-9.
[70]
LaiS, XuL, LiuK, ZhaoJ. Recurrent convolutional neural networks for text classification. In:Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence; 2015 Jan 25-30; Austin, TX, USA. Palo Alto: AAAI Press; 2015. p. 2267-73.
[71]
VairaLA, LechienJR, AbbateV, AlleviF, AudinoG, BeltraminiGA, et al. Validation of the quality analysis of medical artificial intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms. Eur Arch Otorhinolaryngol2024; 281:6123-31.
PDF
(1643KB)
