Research output: Contribution to journal › Article › peer-review
Approximate approach for frequent itemsets mining on massive distributed data beyond computing capacity. / Ngueilbaye, Alladoumbaye; Ratmir, Sibagatullin; Cai, Yongda et al.
In: Expert Systems with Applications, Vol. 318, 132043, 01.07.2026.Research output: Contribution to journal › Article › peer-review
}
TY - JOUR
T1 - Approximate approach for frequent itemsets mining on massive distributed data beyond computing capacity
AU - Ngueilbaye, Alladoumbaye
AU - Ratmir, Sibagatullin
AU - Cai, Yongda
AU - Mahmud, Mohammad Sultan
AU - Sun, Xudong
AU - Nechesov, Andrey
AU - Goncharov, Sergey S.
AU - Huang, Joshua Zhexue
N1 - This research has been supported by the National Engineering Laboratory for Big Data System Computing Technology under Grant No.SZU-BDSC-IF2024-03 and the National Natural Science Foundation of China under Grant number 61972261.
PY - 2026/7/1
Y1 - 2026/7/1
N2 - Frequent itemsets mining (FIM) is a fundamental task in data mining; however, traditional methods struggle with massive distributed data that exceeds available memory and computing resources. Mining frequent itemsets (FIs) from a massive static distributed data file (MSDDF) on a cluster with limited memory is therefore a challenging problem. In this paper, we propose Approximate Frequent Itemsets Mining (ApproxFIM), a novel two-stage solution that combines a new sampling method and an approximation approach to reduce computational cost under strict resource constraints. In the first stage, a bounded number of data blocks are randomly selected from the MSDDF and converted into representative random sample partitions. Theoretical guarantees are derived to bound the number of selected data blocks and to ensure the quality of the random sample, and prove that each constructed sample remains representative of the entire dataset. In the second stage, frequent itemsets are mined independently and in parallel from the sampled partitions using FP-Growth, and the resulting patterns are aggregated into a final approximate FIs set. ApproxFIM is implemented in Apache Spark using the Local Operations with Global Operations (LOGO) computing paradigm and evaluated on both real-world and synthetic datasets. Experimental results demonstrate that ApproxFIM scales effectively, significantly reduces memory and execution time requirements, and produces accurate approximations, making it well-suited for practical massive static distributed data mining on small clusters with limited resources.
AB - Frequent itemsets mining (FIM) is a fundamental task in data mining; however, traditional methods struggle with massive distributed data that exceeds available memory and computing resources. Mining frequent itemsets (FIs) from a massive static distributed data file (MSDDF) on a cluster with limited memory is therefore a challenging problem. In this paper, we propose Approximate Frequent Itemsets Mining (ApproxFIM), a novel two-stage solution that combines a new sampling method and an approximation approach to reduce computational cost under strict resource constraints. In the first stage, a bounded number of data blocks are randomly selected from the MSDDF and converted into representative random sample partitions. Theoretical guarantees are derived to bound the number of selected data blocks and to ensure the quality of the random sample, and prove that each constructed sample remains representative of the entire dataset. In the second stage, frequent itemsets are mined independently and in parallel from the sampled partitions using FP-Growth, and the resulting patterns are aggregated into a final approximate FIs set. ApproxFIM is implemented in Apache Spark using the Local Operations with Global Operations (LOGO) computing paradigm and evaluated on both real-world and synthetic datasets. Experimental results demonstrate that ApproxFIM scales effectively, significantly reduces memory and execution time requirements, and produces accurate approximations, making it well-suited for practical massive static distributed data mining on small clusters with limited resources.
KW - Big data analytics
KW - Distributed data files
KW - Frequent itemsets mining
KW - Parallel and distributed algorithms
KW - Sampling techniques
KW - Spark
UR - https://www.scopus.com/pages/publications/105034621405
UR - https://www.elibrary.ru/item.asp?id=91433759
UR - https://www.mendeley.com/catalogue/b95cdd51-896a-329e-897f-992a9288fc1a/
U2 - 10.1016/j.eswa.2026.132043
DO - 10.1016/j.eswa.2026.132043
M3 - Article
VL - 318
JO - Expert Systems with Applications
JF - Expert Systems with Applications
SN - 0957-4174
M1 - 132043
ER -
ID: 81008205