A Novel Entropy-Based Framework for Hybrid Sampling in Imbalanced Learning
Ren-Jieh Kuo1,*, Muhammad Rizki1, Ferani Eva Zulvia2, Eddy Roflin3
1 Department of Industrial Management, National Taiwan University of Science and Technology, Taipei, Taiwan
2 Department of Industrial Engineering, Institut Teknologi Bandung, Bandung, Indonesia
3 Faculty of Medicine, Sriwijaya University, Palembang, Indonesia
* Corresponding Author: Ren-Jieh Kuo. Email: 
Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.084436
Received 22 April 2026; Accepted 01 July 2026; Published online 31 July 2026
Abstract
Imbalanced data remain a critical challenge in classification, as skewed distributions bias models toward majority classes and diminish sensitivity to minority classes, which are often the most critical. To address this issue, this paper proposes the Information Filtered Hybrid Algorithm (IF-HA), a novel entropy-based sampling method that integrates undersampling and oversampling guided by information theory. IF-HA quantifies instance importance through an instance-wise difference statistic. In the undersampling stage, majority of instances with low difference statistics in the border area are eliminated, while in the oversampling stage, synthetic samples are generated from two minority core points or two minority instances with high difference statistics located in the border area. This process removes noise, eliminates redundant majority border points, and generates synthetic minority samples in informative regions until an entropy-based imbalance threshold is reached. The proposed algorithm is evaluated on 20 benchmark datasets from the UCI and KEEL repositories. Results demonstrate that IF-HA consistently improves minority detection and achieves higher F1 Scores, recall, and AUC (Area Under the Curve) than other methods, including SMOTE, Borderline-SMOTE, ADASYN (Adaptive Synthetic Sampling), and SMOTE-TLNN-DEPSO. A real-world tuberculosis (TB) dataset from Indonesia was further used to validate the practical applicability of IF-HA using KNN, Random Forest, and XGBoost (eXtreme Gradient Boosting) classifiers. The results show consistent improvements after applying IF-HA. These findings indicate that entropy-based hybrid sampling is a promising approach for structured tabular imbalanced classification, while further validation on high-dimensional text and image datasets remains necessary to establish broader generalizability.
Keywords
Imbalanced data; hybrid sampling; information theory; entropy-based algorithm; minority class detection; tuberculosis classification
Open Access
Login
Register
Submit a Paper
Propose a Special lssue
Download PDF
Downloads
Citation Tools
