Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction

Yaqing Sheng; Jiuyang Tang; Weixin Zeng; Xiang Zhao; Yuejin Tan

doi:10.1016/j.eng.2026.02.017

Engineering ›› 2026, Vol. 63 ›› Issue (8) :221 -231. DOI: 10.1016/j.eng.2026.02.017

Research

research-article

Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction

Author information +

History +

PDF (2200KB)

Abstract

Cognitive diagnosis is a fundamental task in online education and serves as the basis for subsequent educational tasks, focusing on measuring students’ mastery levels across various knowledge concepts. Existing methods mainly model the interactions between students and exercises, while largely neglecting the relations among the underlying knowledge concepts, thus failing to fully model the level of student mastery. Although some works propose to characterize the prerequisite relation among knowledge concepts, such modeling is still limited. To fill in the gap, in this work, we present a heterogeneous concept graph augmented cognitive diagnosis model (HCGCDM), a cognitive diagnosis method that strives to characterize and model the relationships among knowledge concepts with large language models (LLMs), hence benefiting the measurement of student mastery levels. First, we propose a heterogeneous concept graph construction module to automatically detect and mine the complex relations among knowledge concepts by using the power of LLMs. Specifically, we extract pairs of knowledge concepts with high similarity and then use retrieval-augmented generation to systematically detect and validate triples to construct a reliable heterogeneous concept graph. Subsequently, we develop a heterogeneous concept graph modeling and aggregation module that adaptively identifies important features and then integrates graph representations into knowledge concepts and exercises for more accurate assessment. We empirically evaluate our proposal on several tasks, and the results demonstrate that HCGCDM and its components are effective, surpassing the state-of-the-art methods.

Graphical abstract

Keywords

Large language model / Generate artificial intelligence / Intelligent education / Cognitive diagnosis / Heterogeneous graph neural network

Cite this article

Download citation ▾

Yaqing Sheng, Jiuyang Tang, Weixin Zeng, Xiang Zhao, Yuejin Tan. Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction. Engineering, 2026, 63 (8) : 221-231 DOI:10.1016/j.eng.2026.02.017

登录浏览全文

4963

注册一个新账户忘记密码

1. Introduction

Cognitive diagnosis (CD), as a pivotal component of online education systems, is the basis for subsequent educational tasks [1], such as computerized adaptive testing (CAT) [2], [3]. CD aims to assess a learner’s mastery levels across knowledge concepts based on their current performance in answering questions. A case in point can be found in Fig. 1, where students Ava and Bob give responses to several exercises (e.g., e_i, i = 1, 2, 3, 4). Besides, there is an expert-built Q-matrix characterizing relationships between exercises and knowledge concepts. Based on the response records and the Q-matrix, CD methods would infer students’ mastery levels of knowledge concepts. The mastery level not only reflects the students’ proficiency in knowledge concepts but also assists in providing targeted advice for their subsequent learning.

Initially developed from psychometrics, there are several representative CD models, such as item response theory (IRT) [4] and multidimensional item response theory (MIRT) [5]. IRT and its variant models are scalable mathematical frameworks. While they are straightforward to implement, their scalability and interpretability remain constrained. With the advancement of deep learning, CD methods have also incorporated deep learning to enhance accuracy and explainability, such as neural cognitive diagnosis model (NCDM) [6], knowledge-association neural cognitive diagnosis model (KaNCD) [7], and the oversmoothing-resistant cognitive diagnosis framework (ORCDF) [8]. These deep learning-based methods mostly leverage the multi-layer perceptron to improve the accuracy of CD based on response records and relations between students and exercises.

Intuitively, students need to master several knowledge concepts in order to answer questions correctly. Hence, the relationships among knowledge concepts are vital. The successful comprehension of a knowledge concept increases the likelihood of a positive transfer of learning to the associated knowledge concepts. Most of the methods construct the dependency among knowledge concepts through feature engineering and deep learning [9], [10], [11], [12]. These works almost employ the prerequisite relation to model the relationship between knowledge concepts. However, relying solely on prerequisite relations oversimplifies the relations between knowledge concepts [13], [14], [15]. For instance, although there is no prerequisite relation between matrix row operation and matrix row echelon, these two knowledge concepts can enhance each other’s understanding, indicating the existence of other relation types, for instance, synergistic relation. Consequently, accurately depicting the relationships between knowledge concepts is essential for a more precise assessment of students’ mastery.

To mitigate the aforementioned problems, we propose a heterogeneous concept graph augmented cognitive diagnosis model, HCGCDM. Specifically, we first design the Heterogeneous Concept Graph Construction module to build a comprehensive heterogeneous concept graph by leveraging the power of large language models (LLMs) [16]. This module first identifies pairs of knowledge concepts with a high possibility of forming triples. Then, we acquire and evaluate triples through retrieval-augmented generation (RAG) under three representative relation types, that is, prerequisite, parallel, and synergistic relations. Notably, by using LLMs to build the heterogeneous concept graph, we not only automate the discovery of relations among knowledge concepts but also eliminate the need for massive training data. To effectively capture the heterogeneous graph structures, we propose the Heterogeneous Concept Graph Modeling and Aggregation module, which involves three key operations, that is, subgraph encoder, adaptive selector, and graph information aggregator. On account of the different interactions among knowledge concepts, three subgraphs are decomposed based on three relations. Then, the subgraph encoder generates expressive representations for each subgraph using suitable graph neural networks (GNNs). For instance, the graph attention network (GAT) is leveraged to model the prerequisite relation as different levels of child knowledge concepts contribute to parent knowledge concepts differently. To address the distinct relational priorities of knowledge concepts within an exercise, an adaptive selector is developed for dynamic subgraph embedding integration. Next, a graph information aggregator is devised to merge graph information with the original graph representations with weights. The final step in HCGCDM is to jointly leverage response records and heterogeneous concept graph information to infer students’ mastery levels. We have conducted several experiments on three real-world datasets to validate the effectiveness of our proposal.

In summary, the contributions of this work are as follows:

• We propose to enhance cognitive diagnosis via LLM-driven heterogeneous concept graph construction, which can automatically mine complex interconnections among knowledge concepts by using the power of LLMs.

• We present a heterogeneous concept graph modeling and aggregation module that adaptively identifies essential subgraph features and integrates graph representations.

• Experimental results demonstrate the effectiveness of our proposal, including prediction performance, analysis on different LLMs, and analysis on the number of knowledge concepts.

The rest of the paper is organized as follows. Section 2 discusses related works. Section 3 provides the problem definition and elaborates on the whole model. Section 4 introduces experiment settings, and then illustrates and analyzes the results of the experiments. Finally, Section 5 concludes the article and introduces future prospects.

2. Related work

2.1. Cognitive diagnosis models

CD is a basic task in online education systems that aims to assess students’ current proficiency levels regarding knowledge concepts by analyzing their responses. Current CD methods are mainly divided into non-deep learning-based and deep learning-based models [4], [5], [6], [7], [15], [17]. The most representative model with non-deep learning is the IRT [4]. The core assumption of IRT is that the relationship between students’ responses and their cognitive level can be modeled by a continuous function. However, as this assumption relies on complex mathematical formulations, IRT and its variants are difficult to extend and lack interpretability. Hence, deep learning-based CD methods have emerged. Wang et al. [6] is the first to propose a neural network-based framework (NCDM), which leverages neural networks for modeling complex nonlinear interactions, thereby eliminating the dependence on manually defined functions, such as IRT.

Following the previous survey [1], we categorize recent deep learning-based CD works into the following scenarios: limited data scenario [18], [19], [20], [21], [22], group-level scenario [23], [24], and efficiency problem [25]. In the limited data scenario, there are works exploring how to quickly diagnose the un-interaction students’ mastery level [18], [19]. In the group-level scenario [23], [24], they tend to mine the similarity among students at the group level. For efficiency problems, the incremental cognitive diagnosis (ICD) [25] employs an incremental CD method to update method. Furthermore, several studies [26], [8], [27] concentrate on modeling interactive relationships. HyperCDM [26] employs a hypergraph to mine homogeneous influence among students, exercises, and knowledge concepts. ORCDF [8] designs a response-aware graph network that captures the crucial response signals within the response graph.

2.2. CD methods based on hierarchical modeling

Attribute hierarchy presents the hierarchical relations between knowledge concepts. CD methods based on attribute hierarchy are a class of models under the hierarchical cognitive assumption (HCA) [28]. HCA assumes that there are sequential relations between two concepts. For instance, “addition” is the parent of “arithmetic.” Intuitively, the mastery of “addition” is a prerequisite for mastering the “four fundamental arithmetic operations.” Therefore, HCA posits that the parent concept is a precondition for mastering the child concept. Building on HCA, the hierarchical cognitive diagnosis (HCD) [29] is proposed to model relations between knowledge concepts into CD methods. However, it is time-consuming. Hence, Li et al. [14] utilize a Bayesian network to model students’ cognitive states in the attribute hierarchy. Though previous methods consider the attribute hierarchy, the attribute hierarchy solidifies the relationships between knowledge concepts and eliminates the possibility of developing diversified relations. Our work differs from the existing approaches in that ① we break the hierarchical modeling and propose the heterogeneous concept graph to explore and model the complex relations among knowledge concepts comprehensively; ② we propose a complete pipeline to construct a heterogeneous concept graph automatically without expert annotations; and ③ we design a module that models heterogeneous concept graphs and adaptively learns to focus on essential subgraph features while aggregating graph representations.

2.3. Graph representation

Graph representation is a technique used for converting graph data into low-dimensional vector representations, preserving features of nodes, edges, and overall structures to support downstream tasks such as node classification and link prediction [30]. With the development of deep learning, GNNs have become its core tool, which can generate node embeddings by recursively aggregating node and neighbor information, and can capture local structures while understanding global structures through multi-layer architectures [31]. Recently, researchers have proposed a variety of improved models, such as graph convolutional networks (GCNs) [32] and GATs [33], which have performed well in processing complex data such as weighted graphs, dynamic graphs, and heterogeneous graphs [34]. Moreover, with the development of LLMs, numerous studies have explored enhancing graph representations using LLMs [35], [36].

3. Methodology

3.1. Definition

Task overview. Assume that there are students $S=\left\{s_{1}, s_{2}, \ldots, s_{N}\right\}$, exercises $\mathscr{E}=\left\{e_1, e_2, \ldots, e_M\right\}$, and knowledge concept $C=\left\{c_{1}, c_{2}, \ldots, c_{K}\right\}$. N denotes the total number of students. M denotes the total number of exercises. K denotes the total number of knowledge concepts. Denote $\operatorname{Res}=\{(s, e, y) \mid s \in S, e \in \mathscr{E}, y \in\{0,1\}\}$ as the set of response records, where y is the response score, that is, 1 represents that the student s answers the exercise e correctly and 0 otherwise. Q-matrix presents the relationships between exercises and knowledge concepts, defined as $\mathscr{Q}=\left(q_{i j}\right)_{M \times K}$, where $q_{i j}=1$ if the exercise i involves a knowledge concept j and $q_{i j}=0$ otherwise.

Heterogeneous concept graph. We define knowledge concepts and their relations as a heterogeneous concept graph. A heterogeneous concept graph consists of finite triples $\mathscr{T}=\left\{\left(c_i, r_i, c_j\right) \mid c_i, c_j \in C, r_i \in \mathscr{R}\right\}$, where $\mathscr{R}$ is the set of relations among knowledge concepts. The heterogeneous concept graph is also denoted as a directed graph $\mathscr{G}$, where $\mathscr{G}=(C, D)$, C is the node set and D is the edge set. Next, we divide the $\mathscr{G}$ into three subgraphs based on the different edge types $\mathscr{R}$, denoted as $\mathscr{G}_{1}, \mathscr{G}_{2}, \mathscr{G}_{3}$.

Problem definition. Given a students’ response records Res, a matrix $\mathscr{Q} \in \mathbb{R}^{M \times K}$ and a heterogeneous concept graph $\mathscr{G}$, the goal is to obtain the mastery level of students on each knowledge concept, denoted as $\text { Mastery } \in \mathbb{R}^{N \times K}$.

Table 1 lists the notations used in this paper.

3.2. Framework outline

As shown in Fig. 2, HCGCDM consists of two parts. In the heterogeneous concept graph construction module, we first select pairs of highly similar knowledge concepts and identify interconnections among them, and construct an accurate heterogeneous concept graph through triple generation and triple evaluation with LLMs. Then, we divide the heterogeneous concept graph into three subgraphs according to relation types. The heterogeneous concept graph modeling and aggregation module comprises a subgraph encoder, an adaptive selector, and a graph information aggregator. We initially model three subgraphs based on different GNNs to obtain the basic representations through the subgraph encoder. Then, we develop an adaptive selector to identify the vital subgraph features dynamically. Using a graph information aggregator, the knowledge concepts and exercise features are updated with graph information. HCGCDM ultimately generates diagnostic predictions of students’ knowledge mastery levels based on response records and graph information.

3.3. Heterogeneous concept graph construction

To enhance the interconnections among knowledge concepts, we build a heterogeneous graph based on knowledge concepts, referred to as the heterogeneous concept graph. This heterogeneous concept graph not only enriches the interactions among knowledge concepts but also visualizes the relations among them, which provides valuable insights for the subsequent analysis of student performance. In the heterogeneous concept graph, knowledge concepts are represented as nodes, and relations among them are described as edges. Since various types of relations between knowledge concepts are difficult to acquire automatically, following existing literatures [37], [38], [39], [12], we focus on modeling three representative types of relations in this study:

• If mastering knowledge concept a is necessary to master another knowledge concept b, then there is a prerequisite relation between a and b.

• The parallel relation indicates that two knowledge concepts exist at the same hierarchical level, such as “addition” and “subtraction.”

• The synergistic relation is characterized by two knowledge concepts that mutually enhance each other’s understanding and application.

Though there are deep learning-based methods [11], [39], [12] to explore the prerequisite relation, they fail to obtain triples involving the prerequisite relation without a training set. Moreover, current methods lack automated frameworks to extract triples for parallel and synergistic relations. Inspired by the powerful abilities of LLMs [40], [41], we propose leveraging LLMs to generate triples across all three relation types.

Pair extraction. To accurately capture the relations among knowledge concepts, we input pairwise knowledge concepts as input to LLMs. Given that not all pairs are useful, and considering the computational efficiency of LLMs when handling $C_{K}^{2}$ pairs, we devise the pair extraction step, which selects pairs that have a higher likelihood of forming meaningful triples. Specifically, we embed knowledge concepts with bert-base-nli-mean-tokens model (the base model on huggingface) into vectors and compute their cosine similarity. Concept pairs whose similarity exceeds a threshold are identified as the potential candidates for relation detection (Eq. (1)). Regarding the selection of the threshold, the value is determined by the model performance on the validation set.

(1)$P_{1}=\left\{\left(c_{i}, c_{j}\right) \mid \operatorname{sim}\left(c_{i}, c_{j}\right)>\theta\right\}.$

θ refers to a predefined similarity threshold. Only sample pairs (c_i, c_j) whose similarity score sim(c_i, c_j) is larger than θ are collected into the set P₁.

Triple generation. Considering that knowledge concepts have different semantics, it may lead to misunderstanding if triples are directly generated by LLMs using the prompt Pgen and the extracted pairs P1, as illustrated in Fig. 3(a). Hence, we exploit RAG [42], [43] to improve the reliability of generated content by effectively utilizing external knowledge. For example, “mean” has many meanings, but by utilizing the external knowledge of mathematics, it is obvious that “mean” generally refers to the average number.

Specifically, we first establish an external knowledge base EK related to knowledge pairs to facilitate more accurate RAG, which is sourced from knowledge concepts in web pages and mathematical knowledge bases. Using the embedding similarity based on the bert-base-nli-mean-tokens model, we retrieve the top-k most relevant knowledge $\mathscr{P}_{\text {KB }}$ from EK. Finally, we use the retrieved external knowledge, along with the extracted pairs and the original prompt $\mathscr{P}_{\text {gen }}$ to form the final prompt $\mathscr{P}$ fed to the LLM, as shown in Fig. 3(b):

(2)$\mathscr{P}=\mathscr{P}_{\text {gen }} \oplus P_{1} \oplus \mathscr{P}_{\mathrm{KB}}$

where ⊕ is the concatenation operation.

After the instructions $\mathscr{P}$ are fed into the LLM, it processes them and generates the corresponding triples, denoted as $\mathscr{T}_{1}=\left\{\left(c_{i}, r_{i}, c_{j}\right) \mid c_{i}, c_{j} \in C, r_{i} \in \mathscr{R}\right\}$.

Triple evaluation. The quality of generated triples directly influences the construction of a heterogeneous concept graph. To guarantee the precision of triples obtained via triple generation, again, we use an LLM to validate the effectiveness of triples. In this step, we request the LLMs to analyze whether the triple in $\mathscr{T}_{1}$ is correct to further enhance the credibility of the heterogeneous concept graph using RAG, which follows a procedure similar to that of triple generation. First, we calculate the similarity between the input query and the external knowledge base EK under the encoder bert-base-nli-mean-tokens model. Then, compared to the original prompt $\mathscr{P}_{\text {eva }}$, the most relevant external knowledge k (the highest similarity) $\mathscr{P}_{\mathrm{KB}}^{\prime}$ is retrieved. Then, the final prompt $\mathscr{P}^{\prime} $ is composed of the original prompt $\mathscr{P}_{\text {eva }}$ (as shown in Fig. 4(a)), the external knowledge $\mathscr{P}_{\text {KB }}^{\prime}$ and $P_{2}$ (triple), as presented in Fig. 4(b):

(3)$\mathscr{P}^{\prime}=\mathscr{P}_{\mathrm{eva}} \oplus P_{2} \oplus \mathscr{P}_{\mathrm{KB}}^{\prime}$

The ultimate triples are denoted as $\mathscr{T}=\left\{\left(c_{i}, r_{i}, c_{j}\right) \mid c_{i}, c_{j} \in C, r_{i} \in \mathscr{R}\right\}$.

We treat each triple in $\mathscr{T}$ as an edge in the heterogeneous concept graph $\mathscr{G}$, thereby converting $\mathscr{T}$ into $\mathscr{G}$. On account of the distinct information of relation types, we decompose $\mathscr{G}$ into three relation-specific subgraphs, denoted as $\mathscr{G}_{1}, \mathscr{G}_{2}, \mathscr{G}_{3} $, involving the prerequisite relation, the parallel relation, and the synergistic relation, respectively.

3.4. Heterogeneous concept graph modeling and aggregation

Subgraph encoder. Owing to the distinct patterns of interaction among knowledge concepts, we adopt dedicated GNN architectures for each subgraph. Initially, we use a pretrained language model (PLM) [44] to capture the semantic features of knowledge concepts as the initial representations of nodes:

(4)$\boldsymbol{x}_{i}=\operatorname{PLM}\left(\operatorname{text}\left(c_{i}\right)\right)$

In $\mathscr{G}_{1}$, the prerequisite relation requires modeling varying contributions of child concepts to their parent concepts, which is naturally handled by two-layer GATs. Owing to the inherently directed nature of prerequisite relations, where each child node contributes varying levels of importance to its parent node, GAT is employed to effectively capture these asymmetric dependencies. Thus, the representations of the nodes in $\mathscr{G}_{1}$ can be calculated as

(5)$\boldsymbol{h}_{1}=\sigma\left(\sum_{j \in \mathscr{N}(i)} \alpha_{i j} \cdot \boldsymbol{W}_{1} \boldsymbol{x}_{i}\right)$

where $\alpha_{i j}=\frac{\exp \left(\operatorname{LeakyReLU}\left(\boldsymbol{a}^{\mathrm{T}}\left(\boldsymbol{W} \boldsymbol{x}_{i} \| \boldsymbol{W} \boldsymbol{x}_{j}\right)\right)\right)}{\sum_{k \in \mathcal{M}(i)} \exp \left(\operatorname{LeakyReLU}\left(\boldsymbol{a}^{\mathrm{T}}\left(\boldsymbol{W} \boldsymbol{x}_{i} \| \boldsymbol{W} \boldsymbol{x}_{k}\right)\right)\right)}$ is the set of neighbors of node i, σ is the activation function, such as ELU (exponential linear units). $\boldsymbol{W}_{1}$ is the learnable matrix.

In $\mathscr{G}_{2}$, the parallel relation benefits from the GCN’s neighborhood aggregation, which captures the local structure similarity among knowledge concepts. For example, “addition” and “subtraction” might both be connected to “arithmetic,” and GCN can leverage this to create more informative representations. Furthermore, the GCN propagates information across layers to maintain this hierarchical consistency, ensuring that parallel nodes are represented similarly. Thus the representations of the nodes in $\mathscr{G}_{2}$ are computed using

(6)$\boldsymbol{h}_{2}=\sigma\left(\widehat{\boldsymbol{D}}^{-\frac{1}{2}} \widehat{\boldsymbol{A}} \widehat{\boldsymbol{D}}^{-\frac{1}{2}} \cdot \boldsymbol{W}_{2} \boldsymbol{x}_{i}\right)$

where $\widehat{\boldsymbol{A}}=\boldsymbol{A}+\boldsymbol{I}$ is the new adjacency matrix after the adjacency matrix A has been added from the loop I. $\widehat{\boldsymbol{D}}$ is the degree matrix of $\widehat{\boldsymbol{A}}$. $\boldsymbol{W}_{2}$ is the learnable matrix.

In $\mathscr{G}_{3}$, for the synergistic relation, we adopt a CoAttention-GNN [45], [33] to characterize how two knowledge concepts enhance each other’s understanding. The bidirectional flow inherent in the synergistic relation necessitates that our model accounts for propagation in both directions. This approach enables a more robust capture of the synergistic information gain resulting from the interaction. Thus the representations of the nodes in $\mathscr{G}_{3}$ can be obtained through:

(7)$\boldsymbol{h}_{3}=\boldsymbol{W}_{3} \cdot\left[\sum_{j \in \mathscr{N}(i)} \alpha_{i j} \cdot \boldsymbol{W}_{31} \boldsymbol{x}_{i} \| \sum_{j \in \mathscr{N}(i)} \alpha_{i j} \cdot \boldsymbol{W}_{32} \boldsymbol{x}_{i}\right]$

where $\sum_{j \in \mathscr{N}(i)} \alpha_{i j} \cdot \boldsymbol{W}_{31} \boldsymbol{x}_{i}$ presents the forward flow and $\sum_{j \in \mathscr{N}(i)} \alpha_{i j} \cdot \boldsymbol{W}_{32} \boldsymbol{x}_{i} $ means the backward flow.

Adaptive selector. Intuitively, distinct exercises focus on different relation types. For instance, when students solve algebra problems, the prerequisite relations between knowledge concepts contribute more significantly to addressing this issue. In other words, different exercises exhibit varying dependencies on specific subgraph structures. Therefore, we develop an adaptive selector to automatically assign weights for subgraphs, enhancing the overall representation of exercises. The weights for each graph are learned through an attention network and used to compute a weighted combination of the three subgraphs’ features, resulting in adaptively adjusted graph features. We first acquire the attention weights of three subgraphs αi as follows:

(8)$\alpha_{i}=\frac{\exp \left(f\left(\boldsymbol{h}_{i}\right)\right)}{\sum_{j=1}^{3} \exp \left(f\left(\boldsymbol{h}_{j}\right)\right)}$

where $f(\cdot)$ is an attention function.

Then, we multiply subgraph features with their corresponding weights to obtain the representation of a node $\boldsymbol{h}^{\prime}$:

(9)$\boldsymbol{h}^{\prime}=\sum_{i=0}^{3} \alpha_{i} \cdot \boldsymbol{W}_{4} \boldsymbol{h}_{i}$

Graph information aggregator. To organically integrate the graph information into knowledge concepts and exercises, we devise the graph information aggregator. The aggregator assigns a weight μi to the graph information

(10)$\mu_{i}=\sigma\left(\boldsymbol{f}_{1} \cdot\left[\boldsymbol{h}^{\prime}, \boldsymbol{h}_{\mathrm{e}}\right]\right)$

where $\boldsymbol{h}_{\mathrm{e}}$ presents exercise attribute representations, such as exercise difficulty representation $\boldsymbol{h}_{\text {diff }}$ and exercise difference representation $\boldsymbol{h}_{\text {disc }}$. $\boldsymbol{f}_{1}$ is the linear layer. σ is the sigmoid activation function.

Then, the representations with graph information are obtained:

(11)$\boldsymbol{h}_{\mathrm{e}}^{\prime}=\boldsymbol{h}_{\mathrm{e}}+\mu_{i} \odot \boldsymbol{h}^{\prime}$

Hence, the final exercise difficulty representation $\boldsymbol{h}_{\text {diff }}^{\prime}$ and exercise difference representation $\boldsymbol{h}_{\text {disc }}^{\prime}$ are obtained:

(12)$\begin{array}{l}\boldsymbol{h}_{\mathrm{diff}}^{\prime}=\boldsymbol{h}_{\mathrm{diff}}+\mu_{i 1} \odot \boldsymbol{h}^{\prime} \\\boldsymbol{h}_{\mathrm{disc}}^{\prime}=\boldsymbol{h}_{\mathrm{disc}}+\mu_{i 2} \odot \boldsymbol{h}^{\prime}\end{array}$

3.5. Model training

Following prior works [6], [7], [14], we employ deep-learning based interaction function to predict the student’s response $\hat{y}_{i}$:

(13)$\hat{y}_{i}=\boldsymbol{f}_{2}\left(\boldsymbol{\Theta} \circ\left(\boldsymbol{f}\left(\boldsymbol{h}_{\mathrm{s}}, \boldsymbol{h}_{\mathrm{k}}\right)-\boldsymbol{h}_{\mathrm{diff}}^{\prime}\right) \times \boldsymbol{h}_{\mathrm{disc}}^{\prime}\right)$

where $\boldsymbol{h}_{\mathrm{s}}$ is the student’s representations and $\boldsymbol{h}_{\mathrm{k}}$ is the knowledge concept representations. $\boldsymbol{h}_{\text {diff }}^{\prime}$ and $\boldsymbol{h}_{\text {disc }}^{\prime}$ are the exercise’s difficulty and discrimination features. $\boldsymbol{f}_{2}(\cdot)$ and $\boldsymbol{f}(\cdot)$ denote multilayer perceptrons (MLP).

The loss function $\mathscr{L}$ of HCGCDM is the CrossEntropy between the output $\hat{y}_{i}$ and the true label $y_{i}$:

(14)$\mathscr{L}=-\sum_{i} y_{i} \log \hat{y}_{i}+\left(1-y_{i}\right) \log \left(1-\hat{y}_{i}\right)$

The pseudo-code of the whole method is shown in Algorithm 1.

4. Experiments

4.1. Experimental setting

Datasets. We conduct experiments on three real-world datasets, including Assist2009-2010 [46], Junyi [47], and Math1 [48]. Following the literatures [14], [15], for Junyi, we remove students with fewer than 15 response records to guarantee the full interaction. Then we randomly select 10 000 students and retain 666 knowledge concepts that are included in these students’ response records. In order to demonstrate the scalability of our model, we employ a larger dataset called XES3G5M [49]. The details of three datasets are shown in Table 2.

Baselines. To fully validate the performance of HCGCDM, we select several baselines for comparison:

• MIRT [5] is selected as the presentative model of traditional IRT.

• NCDM [6] is a work that employs neural networks to model interactive functions.

• RCD [13] is a relation map-driven CD model.

• KANCD [7] considers the knowledge concept association to improve the prediction performance of students.

• HyperCDM [26] proposes a hypergraph cognitive diagnosis model to capture the homogeneous influence and mitigate over-smoothing.

• ISG-CD [27] aims to address the edge heterogeneity and edge uncertainty that are overlooked in existing graph-based CD models.

Implementation details. In the heterogeneous concept graph construction module, for the selection of LLMs in triple generation and validation, we use GPT-4o. In model training, we set learning rate = 0.002; dim = 20 (the feature dimension). All experiments are conducted using PyTorch 1.12.0 and on a GeForce RTX 3090 with 24 GB of random access memory (RAM). The value θ in pair extraction is determined by the model performance in the validation set.

Metrics. For the prediction performance task, we exploit the common metrics to evaluate the performance of CD methods, including area under the curve (AUC), accuracy (ACC), and mean squared error (MSE) [14], [15], [1]. Following the literatures [6], [1], we adopt the prediction performance task, which aims to predict the students’ response to the exercise and validate the performance of CD methods.

4.2. Prediction performance

As shown in Table 3, HCGCDM achieves superior performance than other baselines on the Assist2009-2010 and Math1 datasets. It concludes that a heterogeneous concept graph strengthens interconnections among knowledge concepts and enhances their integration with exercise representations, thereby improving the accuracy of diagnosis. However, HyperCDM obtains better results than HCGCDM on the Junyi dataset. We attribute the superior performance of HyperCDM to the following two reasons: ① In the Junyi dataset, knowledge concepts and exercises are entirely identical, which causes interference in the graph construction and leads to lower performance. For example, both the textual descriptions of knowledge concepts and exercises are the same, such as counting. ② HyperCDM leverages hypergraphs to uncover more relationships, thereby enriching the information. However, HyperCDM suffers from a critical issue, excessive time consumption, which hinders its practical application in online education. For other baselines, the psychometrics-based method achieves the worst results. In general, deep learning-based methods perform better than the psychometrics-based method (MIRT). This might be due to the fact that MIRT’s modeling is simplistic and it’s hard to fully capture the features of students, exercises, and knowledge concepts.

4.3. Ablation study

To examine the usefulness of our proposal’s components, we conduct an ablation study on the prediction performance task. Specifically, as listed in Table 4, “w/o HCGC” denotes removing the heterogeneous concept graph construction module. “w/o LLM” denotes the generation of triples via with data-driven [37], [38], [39] methods. “w/o AS” denotes the replacement of the adaptive selector with the average and “w/o GIA” represents replacing the graph information aggregator with the average.

It reads from Table 4 that all components contribute positively to overall performance, validating the effectiveness of the modules proposed in HCGCDM. Evidently, owing to the powerful capabilities of LLMs, generation using LLMs performs considerably better than using data-driven methods by comparing HCGCDM with “w/o LLM.” Furthermore, the adaptive selector dynamically chooses subgraph features for each exercise, enhancing exercise representations by adjusting feature weights. Notably, the metrics drop the most after removing the heterogeneous concept graph construction module (“w/o HCGC”). The considerable decline in performance can be attributed to the fact that a heterogeneous concept graph captures the complex interconnections among knowledge concepts, thereby assisting in correctly inferring students’ mastery levels.

4.4. Case study

As shown in Figs. 5(a) and (b), we provide a student’s mastery level on knowledge concepts and a partial heterogeneous concept graph to analyze the diagnosis results of HCGCDM. As shown in Fig. 5(a), there is a prerequisite relation between percent of and percent increase or decrease, thus the mastery level of percent of is higher than percent increase or decrease. Moreover, though there is no prerequisite relation between addition and subtraction integers and subtraction whole numbers, the parallel relation leads to the similarity of the mastery level. Compared with NCDM, the mastery level obtained by HCGCDM is more accurate than that of NCDM. Furthermore, the heterogeneous concept graph and the mastery level of knowledge concepts are able to provide some valuable guidance. For example, this student has a relatively weak grasp of percent increase or decrease and should focus on practicing with related exercises in future learning. Similarly, as shown in Fig. 5(b), application of odd and even numbers and constructing numbers using parity rules share a common parent node understanding of odd and even numbers and have a synergistic relation. Therefore, the student's knowledge mastery of application of odd and even numbers and constructing numbers using parity rules should be very similar.

4.5. Analysis on different LLMs

In the heterogeneous concept graph construction module of HCGCDM, LLMs can be replaced. In HCGCDM, we employ GPT-4o as the base LLM, abbreviated as HCGCDM-GPT. GPT-4o is a closed-source LLM with huge parameters and has powerful abilities. To explore the influence of LLMs’ ability on our proposal, we employ Llama3-8B-Instruct with smaller parameters as the base LLM in the heterogeneous concept graph construction module, abbreviated as HCGCDM-Llama3. As presented in Table 5, the results of HCGCDM-GPT are better than HCGCDM-Llama3, which can be inferred that LLMs influence the diagnostic performance by impacting the accuracy of constructing triples.

4.6. Analysis on the number of knowledge concepts

To investigate the impact of the number of knowledge concepts on diagnostic performance, we compare the enhancement of AUC, ACC, and MSE of our proposal with other baselines. For three datasets, we select the psychometrics-based method (MIRT) for comparison.

As shown in Fig. 6, the enhancement of AUC, ACC, and MSE is most obvious in Math1 (Math), followed by those in Assist2009-2010 (Assist) and Junyi. The Math1 dataset contains fewer knowledge concepts and less data than the other two datasets, leading to limited information availability. Hence, the heterogeneous concept graph construction enriches interactions, allowing models to utilize the sparse data more effectively and improve the accuracy of inference. By contrast, the Junyi dataset suffers from semantic overlap between knowledge concepts and exercise descriptions, leading to information redundancy during training. This fundamental data limitation leads to limited enhancement of the Junyi dataset.

4.7. Analysis on different subgraph encoders

To further illustrate the importance of the subgraph encoder adapted to the relation, we conduct additional experiments by replacing the subgraph encoder. R1, R2, and R3 denote replacing the subgraph encoders that model the prerequisite relation, parallel relation, and synergistic relation encoders, respectively. For instance, “HCGCDM-R1-GCN” replaces the original GAT-based prerequisite relationship encoder with a two-layer GCN while maintaining the parallel and synergistic relation encoders unchanged. As summarized in Table 6, our method achieves optimal performance, indicating that the choice of subgraph encoder influences the accuracy of cognitive diagnosis. We conclude that a GNN better aligned to the relational characteristics can more effectively capture its features. For instance, GAT models the prerequisite relations by leveraging attention mechanisms to determine the importance of different child nodes relative to a parent node, thereby enabling more precise feature representation.

4.8. Examples of heterogeneous concept graph

As listed in Table 7, we provide some triples in the heterogeneous concept graph. The first and second rows of each relation are extracted from Assist2009-2010 and XES3G5M, respectively. Notably, the prerequisite relation is defined as unidirectional, while the other relations are bidirectional.

4.9. Limitations

In this study, although the description of HCGCDM is relatively complete and comprehensive, and experiments demonstrate good effectiveness, there are still some limitations that warrant consideration. First, our current method expands relations among knowledge concepts following existing studies, including the parallel, synergistic, and prerequisite relations, but fails to automatically extract relations. Moreover, the heterogeneous concept graph construction module requires textual descriptions for each knowledge concept, thereby restricting our approach to some datasets and hindering its generalizability. In future studies, we expect to achieve fully automated heterogeneous graph generation driven by LLMs. Secondly, we use the two-stage RAG method for automatic generation, including triple generation and triple evaluation, to automatically build the heterogeneous concept graph. Nevertheless, there are still better multi- agent approaches, such as experiential learning agent [50]. Finally, CD is influenced by numerous factors that extend beyond answer history, including behavior data such as time spent and facial expressions of students when answering questions. However, owing to the limitations of dataset, these factors are not incorporated into this study.

5. Conclusions

In this work, we propose HCGCDM, which improves the accuracy of CD by modeling knowledge concepts as a heterogeneous concept graph to mine deep and complex connections. Specifically, we design the heterogeneous concept graph construction module to identify complex interconnections among knowledge concepts automatically by LLMs. Through heterogeneous concept graph modeling and aggregation, knowledge concepts and exercise representations are incorporated with graph information.

We demonstrate the effectiveness of HCGCDM through several experiments. By modeling the intricate interconnections between knowledge concepts, the heterogeneous concept graph substantially provides adaptive information for each triple by facilitating the transfer of knowledge mastery levels and thereby improving the accuracy of cognitive diagnosis. Ablation experiments demonstrate that our proposal with the heterogeneous concept graph performs marginally superior, achieving increases of 1.64% in AUC and 1.87% in accuracy. Therefore, incorporating these prior relationships is a necessary prerequisite for accurate assessment.

A promising direction for future studies is to explore additional relations among knowledge concepts to further enhance the explainability of CD. Moreover, with the development of online education, more and more unseen exercises and knowledge concepts have emerged. A critical challenge is how to diagnose the mastery level of students without student-exercise interactions.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Wang F, Gao W, Liu Q, Li J, Zhao G, Zhang Z, et al. A survey of models for cognitive diagnosis: new developments and future directions. 2024. arXiv:2407.05458.

[2]

Zhuang Y, Liu Q, Huang Z, Li Z, Jin B, Bi H, et al. A robust computerized adaptive testing approach in educational question retrieval. In:Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022 Jul 11-15, Madrid, Spain. New York City: Association for Computing Machinery (ACM); 2022. p. 416-26.

[3]

Wang H, Long T, Yin L, Zhang W, Xia W, Hong Q, et al. GMoCAT:a graph-enhanced multi-objective method for computerized adaptive testing. In:Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023 Aug 6-10, Long Beach, CA, USA. New York City: Association for Computing Machinery (ACM); 2023. p. 2279-89.

[4]	Reise SP, Ainsworth AT, Haviland MG. Item response theory: fundamentals, applications, and promise in psychological research. Curr Dir Psychol Sci 2005; 14(2):95-101.

[5]	Reckase MD. The past and future of multidimensional item response theory. Appl Psychol Meas 1997; 21(1):25-36.

[6]

Wang F, Liu Q, Chen E, Huang Z, Chen Y, Yin Y, et al. Neural cognitive diagnosis for intelligent education systems. In:Proceedings of the AAAI Conference on Artiﬁcial Intelligence, 2020 Feb 7-12, New York City, NY, USA. New York City: Association for the Advancement of Artiﬁcial Intelligence (AAAI); 2020. p. 6153-61.

[7]	Wang F, Liu Q, Chen E, Huang Z, Yin Y, Wang S, et al. NeuralCD: a general framework for cognitive diagnosis. IEEE Trans Knowl Data Eng 2023; 35 (8):8312-27.

[8]

Qian H, Liu S, Li M, Li B, Liu Z, Zhou A. ORCDF:an oversmoothing-resistant cognitive diagnosis framework for student learning in online education systems. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024 Aug 25-29, Barcelona, Spain. New York City: Association for Computing Machinery (ACM); 2024. p. 2455-66.

[9]

Gordon J, Zhu L, Galstyan A, Natarajan P, Burns G. Modeling concept dependencies in a scientiﬁc corpus. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016 Aug 12- 17, Berlin, Germany. Stroudsburg: Association for Computational Linguistics (ACL); 2016. p. 866-75.

[10]

Pan L, Li C, Li J, Tang J. Prerequisite relation learning for concepts in MOOCs. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017 Jul 30-Aug 4, Vancouver, BC, Canada. Stroudsburg: Association for Computational Linguistics (ACL); 2017. p. 1447-56.

[11]	Zhu Y, Zamani H. Predicting prerequisite relations for unseen concepts. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022 Dec 7- 11, Abu Dhabi, United Arab Emirates. Stroudsburg: Association for Computational Linguistics (ACL); 2022. p. 10974-85.

[12]

Sun J, He Y, Xu Y, Sun J, Sun G. A learning-path based supervised method for concept prerequisite relations extraction in educational data. In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, ID USA. New York City: Association for Computing Machinery (ACM); 2024. p. 2168-77.

[13]

Gao W, Liu Q, Huang Z, Yin Y, Bi H, Wang MC, et al. RCD:relation map driven cognitive diagnosis for intelligent education systems. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval; 2021 Jul 11-15; virtual event. New York City: Association for Computing Machinery (ACM); 2021. p. 501-10.

[14]

Li J, Wang F, Liu Q, Zhu M, Huang W, Huang Z, et al. HierCDF:a Bayesian network-based hierarchical cognitive diagnosis framework. In:Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2022 Aug 14-18; Washington, DC, USA. New York City: Association for Computing Machinery (ACM); 2022. p. 904-13.

[15]	Liu S, Qian H, Li M, Zhou A. QCCDM:a Q-augmented causal cognitive diagnosis model for student learning. In: Proceedings of the ECAI 2023: 26th European Conference on Artiﬁcial Intelligence; 2023 Sep 30-Oct 4; Kraków, Poland. Amsterdam: IOS Press; 2023. p. 1536-43.

[16]	Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, et al. GPT-4 technical report. 2023. arXiv:2303.08774.

[17]

Li J, Liu Q, Wang F, Liu J, Huang Z, Yao F, et al. Towards the identiﬁability and explainability for personalized learner modeling:an inductive paradigm. In:Proceedings of the ACM on Web Conference, 2024 May 13-17, Singapore. New York City: Association for Computing Machinery (ACM); 2024. p. 3420-31.

[18]

Ma H, Wang C, Zhu H, Yang S, Zhang X, Zhang X. Enhancing cognitive diagnosis using un-interacted exercises:a collaboration-aware mixed sampling approach. In:Proceedings of the AAAI Conference on Artiﬁcial Intelligence, 2024 Mar 25-31, Vancouver, BC, Canada. New York City: Association for the Advancement of Artiﬁcial Intelligence (AAAI); 2024. p. 8877-85.

[19]	Liu S, Shen J, Qian H, Zhou A. Inductive cognitive diagnosis for fast student learning in web-based intelligent education systems. In:Proceedings of the ACM on web conference, 2024 May 13-17, Singapore City, Singapore. New York City: Association for Computing Machinery (ACM); 2024. p. 4260-71.

[20]

Gao W, Wang H, Liu Q, Wang F, Lin X, Yue L, et al. Leveraging transferable knowledge concept graph embedding for cold-start cognitive diagnosis. In:Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023 Jul 23-27, Taipei, China. New York City: Association for Computing Machinery (ACM); 2023. p. 983-92.

[21]

Gao W, Liu Q, Wang H, Yue L, Bi H, Gu Y, et al. Zero-1-to-3:domain-level zero-shot cognitive diagnosis via one batch of early-bird students towards three diagnostic objectives. In:Proceedings of the AAAI Conference on Artiﬁcial Intelligence, 2024 Mar 25-31, Vancouver, BC, Canada. New York City: Association for the Advancement of Artiﬁcial Intelligence (AAAI); 2024. p. 8417-26.

[22]	Liu S, Zhou Z, Liu Y, Zhang J, Qian H. Language representation favored zero-shot cross-domain cognitive diagnosis. 2025. arXiv:2501.13943.

[23]

Liu S, Yu X, Ma H, Wang Z, Qin C, Zhang X. Homogeneous cohort-aware group cognitive diagnosis:a multi-grained modeling perspective. In:Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023 Oct 21-25, Birmingham, UK. New York City: Association for Computing Machinery (ACM); 2023. p. 4094-98.

[24]

Ma H, Song S, Qin C, Yu X, Zhang L, Zhang X, et al. DGCd:an adaptive denoising GNN for group-level cognitive diagnosis. In: Proceedings of the International Joint Conference on Artiﬁcial Intelligence, 2024 Aug 3- 9, Jeju, Republic of Korea. Stockholm: International Joint Conferences on Artiﬁcial Intelligence Organization (IJCAI); 2024. p. 5492-500.

[25]

Tong S, Liu J, Hong Y, Huang Z, Wu L, Liu Q, et al. Incremental cognitive diagnosis for intelligent education. In:Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022 Aug 14-18, Washington, DC, USA. New York City: Association for Computing Machinery (ACM); 2022. p. 1760-70.

[26]

Shen J, Qian H, Liu S, Zhang W, Jiang B, Zhou A. Capturing homogeneous influence among students:Hypergraph cognitive diagnosis for intelligent education systems. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024 Aug 25-29, Barcelona, Spain. New York City: Association for Computing Machinery (ACM); 2024. p. 2628-39.

[27]

Shao P, Yang Y, Gao C, Chen L, Zhang K, Zhuang C, et al. Exploring heterogeneity and uncertainty for graph-based cognitive diagnosis models in intelligent education. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining ( Volume 1), Toronto, ON, Canada. New York City: Association for Computing Machinery (ACM); 2025. p. 1233-43.

[28]	Leighton JP, Gierl MJ, Hunka SM. The attribute hierarchy method for cognitive assessment: a variation on Tatsuoka’s rule-space approach. J Educ Meas 2004; 41(3):205-37.

[29]	Templin J, Bradshaw L. Hierarchical diagnostic classiﬁcation models: a family of models for estimating and testing attribute hierarchies. Psychometrika 2014; 79(2):317-39.

[30]	Khoshraftar S, An A. A survey on graph representation learning methods. ACM Trans Intell Syst Technol 2024; 15(1):1-55.

[31]	Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Trans Neural Netw 2009; 20(1):61-80.

[32]	Kipf TN, Welling M. Semi-supervised classiﬁcation with graph convolutional networks. 2016. arXiv:1609.02907.

[33]	Velicˇkovic´ P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y. Graph attention networks. 2017. arXiv:1710.10903.

[34]	Zhang C, Song D, Huang C, Swami A, Chawla NV. Heterogeneous graph neural network. In:Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019 Aug 4-8, Anchorage, AK, USA. New York City: Association for Computing Machinery (ACM); 2019. p. 793-803.

[35]	Tang J, Yang Y, Wei W, Shi L, Xia L, Yin D, et al. HiGPT:heterogeneous graph language model. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024 Aug 25-29, Barcelona, Spain. New York City: Association for Computing Machinery (ACM); 2024. p. 2842-53.

[36]

Wang F, Zhu G, Yuan C, Huang Y. LLM-enhanced cascaded multi-level learning on temporal heterogeneous graphs. In:Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024 Jul 14-18, Washington, DC, USA. New York City: Association for Computing Machinery (ACM); 2024. p. 512-21.

[37]	Nakagawa H, Iwasawa Y, Matsuo Y.Graph-based knowledge tracing: modeling student proﬁciency using graph neural network. In: IEEE/WIC/ACM International Conference on Web Intelligence; 2019 Oct 14-17; Thessaloniki, Greece. Los Alamitos: IEEE Computer Society; 2019. p. 156-63.

[38]	Tong S, Liu Q, Huang W, Huang Z, Chen E, Liu C, et al. Structure-based knowledge tracing: an influence propagation view. In: 2020 IEEE International Conference on Data Mining (ICDM), 2020 Nov 17-20, Sorrento, Italy, Sorrento, Italy. Los Alamitos: IEEE Computer Society; 2020. p. 541-50.

[39]	Mazumder D, Paik JH, Basu A. A graph neural network model for concept prerequisite relation extraction. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, UK. New York City: Association for Computing Machinery (ACM); 2023. p. 1787-96.

[40]	Kasneci E, Seßler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ 2023; 103:102274.

[41]	Huang L, Yu W, Ma W, Zhong W, Feng Z, Wang H, et al. A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Trans Inf Syst 2025; 43(2):1-55.

[42]	Zhao P, Zhang H, Yu Q, Wang Z, Geng Y, Fu F, et al. Retrieval-augmented generation for AI-generated content: a survey. 2024. arXiv:2402.19473.

[43]

Fan W, Ding Y, Ning L, Wang S, Li H, Yin D, et al. A survey on RAG meeting LLMs:towards retrieval-augmented large language models. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024 Aug 25-29, Barcelona, Spain. New York City: Association for Computing Machinery (ACM); 2024. p. 6491-501.

[44]	Devlin J. Bert: pre-training of deep bidirectional transformers for language understanding. 2018. arXiv:1810.04805.

[45]	Xiong C, Zhong V, Socher R. Dynamic coattention networks for question answering. 2016. arXiv:1611.01604.

[46]	Feng M, Heffernan N, Koedinger K.Addressing the assessment challenge with an online system that tutors as it assesses. User Model User-adapt Interact 2009; 19(3):243-66.

[47]	Chang HS, Hsu HJ, Chen KT. Modeling exercise relationships in e-learning:a uniﬁed approach. In:Proceedings of the 8th International Conference on Educational Data Mining (EDM), 2015 Jun 26-29, Madrid, Spain. New York City: International Educational Data Mining Society; 2015. p. 532-5.

[48]	Liu Q, Wu R, Chen E, Xu G, Su Y, Chen Z, et al. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Trans Intell Syst Technol 2018; 9 (4):1-26.

[49]

Liu Z, Liu Q, Guo T, Chen J, Huang S, Zhao X, et al. XES3G5M:a knowledge tracing benchmark dataset with auxiliary information. In: Proceedings of the 37th International Conference on Neural Information Processing Systems, 2023 Dec 10- 16, New Orleans, LA, USA. Red Hook: Curran Associates Inc.; 2023. p. 57612-24.

[50]	Zhao A, Huang D, Xu Q, Lin M, Liu YJ, Huang G. EXPEL:LLM agents are experiential learners. In: Proceedings of the AAAI Conference on Artiﬁcial Intelligence, 2024 Mar 25-Apr 4, Vancouver, BC, Canada. Menlo Park: Association for the Advancement of Artiﬁcial Intelligence (AAAI); 2024. p. 19632-42.