Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction

Yaqing Sheng; Jiuyang Tang; Weixin Zeng; Xiang Zhao; Yuejin Tan

doi:10.1016/j.eng.2026.02.017

Engineering ›› :202602017 DOI: 10.1016/j.eng.2026.02.017

Article

research-article

Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction

Author information +

History +

PDF

Abstract

Cognitive diagnosis is a fundamental task in online education and serves as the basis for subsequent educational tasks, focusing on measuring students’ mastery levels across various knowledge concepts. Existing methods mainly model the interactions between students and exercises, while largely neglecting the relations among the underlying knowledge concepts, thus failing to fully model the level of student mastery. Although some works propose to characterize the prerequisite relation among knowledge concepts, such modeling is still limited. To fill in the gap, in this work, we present a heterogeneous concept graph augmented cognitive diagnosis model (HCGCDM), a cognitive diagnosis method that strives to characterize and model the relationships among knowledge concepts with large language models (LLMs), hence benefiting the measurement of student mastery levels. First, we propose a heterogeneous concept graph construction module to automatically detect and mine the complex relations among knowledge concepts by using the power of LLMs. Specifically, we extract pairs of knowledge concepts with high similarity and then use retrieval-augmented generation to systematically detect and validate triples to construct a reliable heterogeneous concept graph. Subsequently, we develop a heterogeneous concept graph modeling and aggregation module that adaptively identifies important features and then integrates graph representations into knowledge concepts and exercises for more accurate assessment. We empirically evaluate our proposal on several tasks, and the results demonstrate that HCGCDM and its components are effective, surpassing the state-of-the-art methods.

Cite this article

Download citation ▾

Yaqing Sheng, Jiuyang Tang, Weixin Zeng, Xiang Zhao, Yuejin Tan. Enhancing Cognitive Diagnosis via LLM-Driven Heterogeneous Concept Graph Construction. Engineering 202602017 DOI:10.1016/j.eng.2026.02.017

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Wang F, Gao W, Liu Q, Li J, Zhao G, Zhang Z, et al. A survey of models for cognitive diagnosis: new developments and future directions. 2024. arXiv:2407.05458.

[2]

Zhuang Y, Liu Q, Huang Z, Li Z, Jin B, Bi H, et al. 2022 Jul 11-15; Madrid, Spain. A robust computerized adaptive testing approach in educational question retrieval. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval; New York City:Association for Computing Machinery (ACM); 2022. p. 416-26.

[3]

Wang H, Long T, Yin L, Zhang W, Xia W, Hong Q, et al. GMoCAT:a graph-enhanced multi-objective method for computerized adaptive testing. In:Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2023 Aug 6-10; Long Beach, CA, USA. New York City:Association for Computing Machinery (ACM); 2023. p. 2279-89.

[4]	Reise SP, Ainsworth AT, Haviland MG. Item response theory: fundamentals, applications, and promise in psychological research. Curr Dir Psychol Sci 2005; 14(2):95-101.

[5]	Reckase MD. The past and future of multidimensional item response theory. Appl Psychol Meas 1997; 21(1):25-36.

[6]

Wang F, Liu Q, Chen E, Huang Z, Chen Y, Yin Y, et al. Neural cognitive diagnosis for intelligent education systems. In:Proceedings of the AAAI Conference on Artificial Intelligence; 2020 Feb 7-12; New York City, NY, USA. New York City:Association for the Advancement of Artificial Intelligence (AAAI); 2020. p. 6153-61.

[7]	Wang F, Liu Q, Chen E, Huang Z, Yin Y, Wang S, et al. NeuralCD: a general framework for cognitive diagnosis. IEEE Trans Knowl Data Eng 2023; 35(8):8312-27.

[8]

Qian H, Liu S, Li M, Li B, Liu Z, Zhou A. ORCDF:an oversmoothing-resistant cognitive diagnosis framework for student learning in online education systems. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25-29; Barcelona, Spain. New York City:Association for Computing Machinery (ACM); 2024. p. 2455-66.

[9]

Gordon J, Zhu L, Galstyan A, Natarajan P, Burns G.Modeling concept dependencies in a scientific corpus. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers); 2016 Aug 12-17; Berlin, Germany. Stroudsburg:Association for Computational Linguistics (ACL); 2016. p. 866-75.

[10]

Pan L, Li C, Li J, Tang J.Prerequisite relation learning for concepts in MOOCs. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers); 2017 Jul 30-Aug 4; Vancouver, BC, Canada. Stroudsburg:Association for Computational Linguistics (ACL); 2017. p. 1447-56.

[11]	Zhu Y, Zamani H.Predicting prerequisite relations for unseen concepts. In:Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing; 2022 Dec 7-11; Abu Dhabi, United Arab Emirates. Stroudsburg:Association for Computational Linguistics (ACL); 2022. p. 10974-85.

[12]

Sun J, He Y, Xu Y, Sun J, Sun G. 2024 Oct 21-25; Boise ID, USA. A learning-path based supervised method for concept prerequisite relations extraction in educational data. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management; New York City:Association for Computing Machinery (ACM); 2024. p. 2168-77.

[13]

Gao W, Liu Q, Huang Z, Yin Y, Bi H, Wang MC, et al. RCD:relation map driven cognitive diagnosis for intelligent education systems. In:Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval; 2021 Jul 11-15; virtual event. New York City:Association for Computing Machinery (ACM); 2021. p. 501-10.

[14]

Li J, Wang F, Liu Q, Zhu M, Huang W, Huang Z, et al. HierCDF:a Bayesian network-based hierarchical cognitive diagnosis framework. In:Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2022 Aug 14-18; Washington, DC, USA. New York City:Association for Computing Machinery (ACM); 2022. p. 904-13.

[15]	Liu S, Qian H, Li M, Zhou A. QCCDM:a Q-augmented causal cognitive diagnosis model for student learning. In:Proceedings of the ECAI 2023: 26th European Conference on Artificial Intelligence; 2023 Sep 30-ct 4; Krak? ?w, Poland. Amsterdam: IOS Press; 2023. p. 1536-43.

[16]	Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, et al. GPT-4 technical report. 2023. arXiv:2303.08774.

[17]

Li J, Liu Q, Wang F, Liu J, Huang Z, Yao F, et al. Towards the identifiability and explainability for personalized learner modeling:an inductive paradigm. In:Proceedings of the ACM on Web Conference; 2024 May 13-17; Singapore City, Singapore. New York City:Association for Computing Machinery (ACM); 2024. p. 3420-31.

[18]

Ma H, Wang C, Zhu H, Yang S, Zhang X, Zhang X. Enhancing cognitive diagnosis using un-interacted exercises:a collaboration-aware mixed sampling approach. In:Proceedings of the AAAI Conference on Artificial Intelligence; 2024 Mar 25-31; Vancouver, BC, Canada. New York City:Association for the Advancement of Artificial Intelligence (AAAI); 2024. p. 8877-85.

[19]	Liu S, Shen J, Qian H, Zhou A. Inductive cognitive diagnosis for fast student learning in web-based intelligent education systems. In:Proceedings of the ACM on Web Conference; 2024 May 13-17; Singapore City, Singapore. New York City:Association for Computing Machinery (ACM); 2024. p. 4260-71.

[20]

Gao W, Wang H, Liu Q, Wang F, Lin X, Yue L, et al. 2023 Jul 23-27; Taipei, China. Leveraging transferable knowledge concept graph embedding for cold-start cognitive diagnosis. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval; New York City:Association for Computing Machinery (ACM); 2023. p. 983-92.

[21]

Gao W, Liu Q, Wang H, Yue L, Bi H, Gu Y, et al. Zero-1-to-3:domain-level zero-shot cognitive diagnosis via one batch of early-bird students towards three diagnostic objectives. In:Proceedings of the AAAI Conference on Artificial Intelligence; 2024 Mar 25-31; Vancouver, BC, Canada. New York City:Association for the Advancement of Artificial Intelligence (AAAI); 2024. p. 8417-26.

[22]	Liu S, Zhou Z, Liu Y, Zhang J,Qian H. Language representation favored zero-shot cross-domain cognitive diagnosis. 2025. arXiv:2501.13943.

[23]

Liu S, Yu X, Ma H, Wang Z, Qin C, Zhang X. Homogeneous cohort-aware group cognitive diagnosis:a multi-grained modeling perspective. In:Proceedings of the 32nd ACM International Conference on Information and Knowledge Management; 2023 Oct 21-25; Birmingham, United Kingdom. New York City:Association for Computing Machinery (ACM); 2023. p. 4094-98.

[24]

Ma H, Song S, Qin C, Yu X, Zhang L, Zhang X, et al. DGCd:an adaptive denoising GNN for group-level cognitive diagnosis. In:Proceedings of the International Joint Conference on Artificial Intelligence; 2024 Aug 3-9; Jeju, South Korea. Stockholm: International Joint Conferences on Artificial Intelligence Organization (IJCAI); 2024. p. 5492-500.

[25]

Tong S, Liu J, Hong Y, Huang Z, Wu L, Liu Q, et al. 2022 Aug 14-18; Washington DC, USA. Incremental cognitive diagnosis for intelligent education. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; New York City:Association for Computing Machinery (ACM); 2022. p. 1760-70.

[26]

Shen J, Qian H, Liu S, Zhang W, Jiang B, Zhou A. Capturing homogeneous influence among students:Hypergraph cognitive diagnosis for intelligent education systems. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25-29; Barcelona, Spain. New York City:Association for Computing Machinery (ACM); 2024. p. 2628-39.

[27]

Shao P, Yang Y, Gao C, Chen L, Zhang K, Zhuang C, et al. 2025 Aug 3-7; Toronto ON, Canada. Exploring heterogeneity and uncertainty for graph-based cognitive diagnosis models in intelligent education. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Volume 1); New York City:Association for Computing Machinery (ACM); 2025. p. 1233-43.

[28]	Leighton JP, Gierl MJ, Hunka SM. The attribute hierarchy method for cognitive assessment: a variation on tatsuoka’s rule-space approach. J Educ Meas 2004; 41(3):205-37.

[29]	Templin J, Bradshaw L. Hierarchical diagnostic classification models: a family of models for estimating and testing attribute hierarchies. Psychometrika 2014; 79(2):317-39.

[30]	Khoshraftar S, An A. A survey on graph representation learning methods. ACM Trans Intell Syst Technol 2024; 15(1):1-55.

[31]	Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Trans Neural Netw 2009; 20(1):61-80.

[32]	Kipf TN,Welling M. Semi-supervised classification with graph convolutional networks. 2016. arXiv:1609.02907.

[33]	Veličković P, Cucurull G, Casanova A, Romero A, Lio P,Bengio Y. Graph attention networks. 2017. arXiv:1710.10903.

[34]	Zhang C, Song D, Huang C, Swami A, Chawla NV. 2019 Aug 4-8; Anchorage AK, USA. Heterogeneous graph neural network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; New York City:Association for Computing Machinery (ACM); 2019. p. 793-803.

[35]	Tang J, Yang Y, Wei W, Shi L, Xia L, Yin D, et al. HiGPT:heterogeneous graph language model. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25-29; Barcelona, Spain. New York City:Association for Computing Machinery (ACM); 2024. p. 2842-53.

[36]

Wang F, Zhu G, Yuan C, Huang Y. 2024 Jul 14-18; Washington DC, USA. LLM-enhanced cascaded multi-level learning on temporal heterogeneous graphs. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval; New York City:Association for Computing Machinery (ACM); 2024. p. 512-21.

[37]	Nakagawa H, Iwasawa Y, Matsuo Y.Graph-based knowledge tracing: modeling student proficiency using graph neural network. In: IEEE/WIC/ACM International Conference on Web Intelligence; 2019 Oct 14-17; Thessaloniki, Greece. Los Alamitos: IEEE Computer Society; 2019. p. 156-63.

[38]	Tong S, Liu Q, Huang W, Huang Z, Chen E, Liu C, et al. Structure-based knowledge tracing: an influence propagation view. In: 2020 IEEE International Conference on Data Mining (ICDM); 2020 Nov 17-20; Sorrento, Italy. Los Alamitos: IEEE Computer Society; 2020. p. 541-50.

[39]

Mazumder D, Paik JH, Basu A. 2023 Oct 21-25; Birmingham, United Kingdom. A graph neural network model for concept prerequisite relation extraction. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management; New York City:Association for Computing Machinery (ACM); 2023. p. 1787-96.

[40]	Kasneci E, Seßler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? on opportunities and challenges of large language models for education. Learn Individ Differ 2023; 103:102274.

[41]	Huang L, Yu W, Ma W, Zhong W, Feng Z, Wang H, et al. A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Trans Inf Syst 2025; 43(2):1-55.

[42]	Zhao P, Zhang H, Yu Q, Wang Z, Geng Y, Fu F, et al. Retrieval-augmented generation for ai-generated content: a survey. 2024. arXiv:2402.19473.

[43]

Fan W, Ding Y, Ning L, Wang S, Li H, Yin D, et al. A survey on RAG meeting LLMs:towards retrieval-augmented large language models. In:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25-29; Barcelona, Spain. New York City:Association for Computing Machinery (ACM); 2024. p. 6491-501.

[44]	Devlin J.Bert: Pre-training of deep bidirectional transformers for language understanding. 2018. arXiv:1810.04805.

[45]	Xiong C, Zhong V,Socher R. Dynamic coattention networks for question answering. 2016. arXiv:1611.01604.

[46]	Feng M, Heffernan N, Koedinger K. Addressing the assessment challenge with an online system that tutors as it assesses. User Model User-adapt Interact 2009; 19(3):243-66.

[47]	Chang HS, Hsu HJ, Chen KT. Modeling exercise relationships in e-learning:a unified approach. In:Proceedings of the 8th International Conference on Educational Data Mining (EDM); 2015 Jun 26-29; Madrid, Spain. New York City:International Educational Data Mining Society; 2015. p. 532-5.

[48]	Liu Q, Wu R, Chen E, Xu G, Su Y, Chen Z, et al. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Trans Intell Syst Technol 2018; 9(4):1-26.

[49]

Liu Z, Liu Q, Guo T, Chen J, Huang S, Zhao X, et al.XES3G5M:a knowledge tracing benchmark dataset with auxiliary information. In:Proceedings of the 37th International Conference on Neural Information Processing Systems; 2023 Dec 10-16; New Orleans, LA, USA. Red Hook:Curran Associates Inc.; 2023. p. 57612-24.

[50]	Zhao A, Huang D, Xu Q, Lin M, Liu YJ, Huang G. EXPEL:LLM agents are experiential learners. In:Proceedings of the AAAI Conference on Artificial Intelligence; 2024 Mar 25-Apr 4; Vancouver, BC, Canada. Menlo Park: Association for the Advancement of Artificial Intelligence (AAAI); 2024. p. 19632-42.