Journal Home Online First Current Issue Archive For Authors Journal Information 中文版

Frontiers of Information Technology & Electronic Engineering >> 2019, Volume 20, Issue 9 doi: 10.1631/FITEE.1800467

Anon-group parallel frequent pattern mining algorithm based on conditional patterns

1. College of Computer Science and Technology, Changchun University, Changchun 130022, China
2. School of Economics, Changchun University, Changchun 130022, China
3. School of Information Science and Technology, Northeast Normal University, Changchun 130117, China
4. Division of Engineering Science, University of Toronto, Ontario M5S2E8, Canada
5. School of Computer Science and Electronic Engineering, University of Essex, Colchester CO43SQ, UK

Available online: 2019-11-07

Next Previous

Abstract

Frequent itemset mining serves as the main method of association rule mining. With the limitations in computing space and performance, the association of frequent items in large data mining requires both extensive time and effort, particularly when the datasets become increasingly larger. In the process of associated data mining in a big data environment, the MapReduce programming model is typically used to perform task partitioning and parallel processing, which could improve the execution efficiency of the algorithm. However, to ensure that the associated rule is not destroyed during task partitioning and parallel processing, the inner-relationship data must be stored in the computer space. Because inner-relationship data are redundant, storage of these data will significantly increase the space usage in comparison with the original dataset. In this study, we find that the formation of the frequent pattern (FP) mining algorithm depends mainly on the conditional pattern bases. Based on the parallel frequent pattern (PFP) algorithm theory, the grouping model divides frequent items into several groups according to their frequencies. We propose a non-group PFP (NG-PFP) mining algorithm that cancels the grouping model and reduces the data redundancy between sub-tasks. Moreover, we present the NG-PFP algorithm for task partition and parallel processing, and its performance in the Hadoop cluster environment is analyzed and discussed. Experimental results indicate that the non-group model shows obvious improvement in terms of computational efficiency and the space utilization rate.

Related Research