TY  - EJOU
AU  - Halabaku, Erblin 
AU  - Bytyçi, Eliot 

TI  - Overfitting in Machine Learning: A Comparative Analysis of Decision Trees and Random Forests
T2  - Intelligent Automation \& Soft Computing

PY  - 2024
VL  - 39
IS  - 6
SN  - 2326-005X

AB  - Machine learning has emerged as a pivotal tool in deciphering and managing this excess of information in an era of abundant data. This paper presents a comprehensive analysis of machine learning algorithms, focusing on the structure and efficacy of random forests in mitigating overfitting—a prevalent issue in decision tree models. It also introduces a novel approach to enhancing decision tree performance through an optimized pruning method called Adaptive Cross-Validated Alpha CCP (ACV-CCP). This method refines traditional cost complexity pruning by streamlining the selection of the alpha parameter, leveraging cross-validation within the pruning process to achieve a reliable, computationally efficient alpha selection that generalizes well to unseen data. By enhancing computational efficiency and balancing model complexity, ACV-CCP allows decision trees to maintain predictive accuracy while minimizing overfitting, effectively narrowing the performance gap between decision trees and random forests. Our findings illustrate how ACV-CCP contributes to the robustness and applicability of decision trees, providing a valuable perspective on achieving computationally efficient and generalized machine learning models.
KW  - Artificial intelligence; decision tree; random forest; prune; overfitting

DO  - 10.32604/iasc.2024.059429