Mining high utility itemsets in massive transactional datasets

Abstract

Mining High Utility Itemsets from a transaction database is to find itemsets that have utility beyond an user-specified threshold. Existing High Utility Itemsets mining algorithms suffer from many problems when being applied to massive transactional datasets. One major problem is the high memory dependency: the gigantic data structure built is assumed to fit in the computer main memory. This paper proposes a new disk-based High Utility Itemsets mining algorithm, which achieves its efficiency by applying three new ideas. First, transactional data is converted into a new database layout called Transactional Array that prevents multiple scanning of the database during the mining phase. Second, for each frequent item, a relatively small independent tree is built for summarizing co-occurrences. Finally, a simple and non-recursive mining process reduces the memory requirements as minimum candidacy generation and counting is needed. We have tested our algorithm on several very large transactional databases and the results show that our algorithm works efficiently.