TY - EJOU AU - Rehman, Amjad AU - Mujahid, Muhammad AU - Damasevicius, Robertas AU - Alamri, Faten S AU - Saba, Tanzila TI - Densely Convolutional BU-NET Framework for Breast Multi-Organ Cancer Nuclei Segmentation through Histopathological Slides and Classification Using Optimized Features T2 - Computer Modeling in Engineering \& Sciences PY - 2024 VL - 141 IS - 3 SN - 1526-1506 AB - This study aims to develop a computational pathology approach that can properly detect and distinguish histology nuclei. This is crucial for histopathological image analysis, as it involves segmenting cell nuclei. However, challenges exist, such as determining the boundary region of normal and deformed nuclei and identifying small, irregular nuclei structures. Deep learning approaches are currently dominant in digital pathology for nucleus recognition and classification, but their complex features limit their practical use in clinical settings. The existing studies have limited accuracy, significant processing costs, and a lack of resilience and generalizability across diverse datasets. We proposed the densely convolutional Breast U-shaped Network (BU-NET) framework to overcome the mentioned issues. The study employs BU-NET’s spatial and channel attention methods to enhance segmentation processes. The inclusion of residual blocks and skip connections in the BU-NEt architecture enhances the process of extracting features and reconstructing the output. This enhances the robustness of training and convergence processes by reducing the occurrence of vanishing gradients. The primary objective of BU-NEt is to enhance the model’s capacity to acquire and analyze more intricate features, all the while preserving an efficient working representation. The BU-NET experiments demonstrate that the framework achieved 88.7% average accuracy, 88.8% F1 score for Multi-Organ Nuclei Segmentation Challenge (MoNuSeg), and 91.2% average accuracy, 91.8% average F1 for the triple-negative breast cancer (TNBC) dataset. The framework also achieved 93.92 Area under the ROC Curve (AUC) for TNBC. The results demonstrated that the technology surpasses existing techniques in terms of accuracy and effectiveness in segmentation. Furthermore, it showcases the ability to withstand and recover from different tissue types and diseases, indicating possible uses in medical treatments. The research evaluated the efficacy of the proposed method on diverse histopathological imaging datasets, including cancer cells from many organs. The densely connected U-NEt technology offers a promising approach for automating and precisely segmenting cancer cells on histopathology slides, hence assisting pathologists in improving cancer diagnosis and treatment outcomes. KW - Breast cancer; histopathology; BU-NET; deep learning DO - 10.32604/cmes.2024.056937