Open Access

ARTICLE

A Hybrid Approach for Heavily Occluded Face Detection Using Histogram of Oriented Gradients and Deep Learning Models

Thaer Thaher^1,*, Muhammed Saffarini², Majdi Mafarja³, Abdulaziz Alashbi⁴, Abdul Hakim Mohamed⁵, Ayman A. El-Saleh⁶

1 Department of Computer Systems Engineering, Arab American University, Jenin, P.O. Box 240, Palestine
2 Department of Computer Systems Engineering, Faculty of Engineering and Technology, Palestine Technical University—Kadoorie, Tulkarm, P.O. Box 305, Palestine
3 Department of Computer Science, Birzeit University, Birzeit, P.O. Box 14, Palestine
4 NexoPrima Sdn Bhd, UKM-MTDC, Bangi, 43650, Selangor, Malaysia
5 Information Systems and Business Analytics Department, A’Sharqiyah University (ASU), Ibra, 400, Oman
6 Department of Electrical Engineering and Computer Science, College of Engineering, A’Sharqiyah University (ASU), Ibra, 400, Oman

* Corresponding Author: Thaer Thaher. Email:

(This article belongs to the Special Issue: Machine Learning and Deep Learning-Based Pattern Recognition)

Computer Modeling in Engineering & Sciences 2025, 144(2), 2359-2394. https://doi.org/10.32604/cmes.2025.065388

Received 11 March 2025; Accepted 10 July 2025; Issue published 31 August 2025

Abstract

Face detection is a critical component in modern security, surveillance, and human-computer interaction systems, with widespread applications in smartphones, biometric access control, and public monitoring. However, detecting faces with high levels of occlusion, such as those covered by masks, veils, or scarves, remains a significant challenge, as traditional models often fail to generalize under such conditions. This paper presents a hybrid approach that combines traditional handcrafted feature extraction technique called Histogram of Oriented Gradients (HOG) and Canny edge detection with modern deep learning models. The goal is to improve face detection accuracy under occlusions. The proposed method leverages the structural strengths of HOG and edge-based object proposals while exploiting the feature extraction capabilities of Convolutional Neural Networks (CNNs). The effectiveness of the proposed model is assessed using a custom dataset containing 10,000 heavily occluded face images and a subset of the Common Objects in Context (COCO) dataset for non-face samples. The COCO dataset was selected for its variety and realism in background contexts. Experimental evaluations demonstrate significant performance improvements compared to baseline CNN models. Results indicate that DenseNet121 combined with HOG outperforms other counterparts in classification metrics with an F1-score of 87.96% and precision of 88.02%. Enhanced performance is achieved through reduced false positives and improved localization accuracy with the integration of object proposals based on Canny and contour detection. While the proposed method increases inference time from 33.52 to 97.80 ms, it achieves a notable improvement in precision from 80.85% to 88.02% when comparing the baseline DenseNet121 model to its hybrid counterpart. Limitations of the method include higher computational cost and the need for careful tuning of parameters across the edge detection, handcrafted features, and CNN components. These findings highlight the potential of combining handcrafted and learned features for occluded face detection tasks.

---

Keywords

---