The size, shape, and physical characteristics of the human skull are distinct when considering individual humans. In physical anthropology, the accurate management of skull collections is crucial for storing and maintaining collections in a cost-effective manner. For example, labeling skulls inaccurately or attaching printed labels to skulls can affect the authenticity of collections. Given the multiple issues associated with the manual identification of skulls, we propose an automatic human skull classification approach that uses a support vector machine and different feature extraction methods such as gray-level co-occurrence matrix features, Gabor features, fractal features, discrete wavelet transforms, and combinations of features. Each underlying facial bone exhibits unique characteristics essential to the face's physical structure that could be exploited for identification. Therefore, we developed an automatic recognition method to classify human skulls for consistent identification compared with traditional classification approaches. Using our proposed approach, we were able to achieve an accuracy of 92.3–99.5% in the classification of human skulls with mandibles and an accuracy of 91.4–99.9% in the classification of human skills without mandibles. Our study represents a step forward in the construction of an effective automatic human skull identification system with a classification process that achieves satisfactory performance for a limited dataset of skull images.
Researchers in digital forensics commonly deal with a series of activities, including collecting, examining, identifying, and analyzing the digital artefacts required for obtaining evidence regarding physical object authenticity [
However, the utilization of ink streaks on skulls to apply an alphanumeric code can damage the authenticity of the skull as a study material. Hence, skull collection management necessitates a certain approach to maintain the authenticity of the collection and avoid damage through the use of chemicals. Attaching stickers with the call number is an alternative. However, this method also has drawbacks because stickers can become loose, fall off, and become fixed to other skulls.
Therefore, it is challenging to increase the number of new skull collections because of difficulties associated with their storage and collection. Skulls can include those separated from the mandible. Labelling errors are a major problem when human skulls and other skeletal collections in an anthropology forensics laboratory are ink out. Apart from the loss of labels attached to new bone collections, the mixing of old bone collections with new bones and high usage factors are challenges that must be overcome in skull collection management.
The use of digital cameras by anthropologists and other researchers to classify human bones is currently limited to manual investigation and comparison. Although some previous studies have applied automatic methodologies, such as machine learning, to identify human skulls, the majority of the samples were obtained via computerized tomography (CT) scans of living participants. These samples have limited relevance with respect to the analysis of skulls of dead subjects, as required in physical anthropology forensics.
In this study, we investigate a digital forensics approach for the physical anthropological investigation of skulls of dead humans based on their specific characteristics. Our main contributions are as follows. First, the significance of this work lies in the application of machine learning and data analytics knowledge to the new domain of physical anthropology collection management and addressing its unique challenges. Second, given the aforementioned problems introduced by manual labeling techniques, this study aims to evaluate the relevant contrasting features of human skulls and build skull-based identities from various positions via automatic classification. Third, our work proposes automatic classification of the skull beneath the human face that would allow curators to identify features based on skeletal characteristics. This technique would potentially assist in the management of museum collections or the laboratory storage of skulls; skulls could be identified without being manually marked or labeled, thereby maintaining their authenticity.
This study is inspired by face recognition technology. The structure of the mandible, mouth, nose, forehead, and the overall features associated with the human skull can be recognized using various means and properties. Based on the availability of these properties, face recognition can be conducted by comparing different facial images and classifying the faces using a support vector machine (SVM). The study [
As observed in the present study, the effective combination of different feature filters is a step forward in using machine learning to conduct investigations in physical anthropology and its sub-areas. Researchers in the physical anthropology field often focus on analyzing the data characteristic obtained from the skulls of dead humans; this characteristic has rarely been found in previous studies on automatic face recognition. Therefore, this work offers a new perspective on the application of machine learning to physical anthropology and tackling its challenges, i.e., the limited physical collection of skulls of dead humans, variation in the completeness of skull construction, and deterioration of the skull condition over time. All these challenges are obstacles to the training of appropriate machine learning techniques and obtaining appropriate feature extraction is the key to achieve the learning objective, successful facial classification.
The remaining sections of this manuscript are as follows. Section 2 presents related works. Section 3 discusses the skull structure that provides the initial information for skull classification. Section 4 presents our main research approach and contribution to developing a machine learning-based automatic classification platform for classifying human skulls in physical anthropology. Section 5 reports our experimental results and validates our research approach. Finally, we summarize the main results of this research and directions of our future work in Section 6.
There is increasing demand for an image classification system that can perform automatic facial recognition tasks [
Elmahmudi et al. [
Furthermore, Chen et al. [
A previous anthropology study [
The so-called computational forensics method is a specific to the forensic anthropology approach [
As observed in most of the above studies, automatic face recognition is mainly focused on the analysis of data obtained from living humans, be it in the form of digital camera or CT images. Even though the physical characteristics for facial identification and computational forensics for gender classification have been investigated in the anthropology literature, automated digital tools that are robust in terms of facial identification appear to be lacking. Thus, this work is a step forward in developing an automated tool by incorporating machine learning and knowledge about robust features.
In principle, the facial skeleton or viscerocranium comprises the anterior, lower, and skull bones, namely, facial tissue, and other structures that form the human face. It comprises various types of bones, which are derived from the branchial arches interconnected among the bones of the eyes, sinuses, nose, and oral cavity and are in unity with the calvarias bones [ Frontal—This bone comprises the squamous, which tends to be vertical, and the orbital bones, which are oriented horizontally. The squamous forms part of the human forehead, and the orbital part is the part of the bone that supports the eyes and nose. Nasal—The paired nasal bones have different sizes and shapes but tend to be small ovals. These bones unite the cartilage located in the nasofrontal and upper parts of the lateral cartilages to form the human nose and consists of two neurocraniums and two viscerocraniums. Vomer—The vomer bone is a single facial bone with an unpaired midline attached to an inferior part of the sphenoid bone. It articulates with the ethmoid, namely, the two maxillary bones and two palatine bones, forming the nasal septum. Zygomatic—The zygomatic bone is the cheekbone positioned on the lateral side and forms the cheeks of a human. This bone has three surfaces, i.e., the orbital, temporal, and lateral surfaces. It articulates directly with the remaining four bones, i.e., the temporal, sphenoid, frontal, and maxilla bones. Maxilla—This is often referred to as the upper jaw bone and is a paired bone that has four processes, i.e., the zygomatic, alveolar, frontal, and palatine processes. This bone supports the teeth in the upper jaw but does not move like the lower jaw or mandible. Mandible—The mandible is the lower jaw bone or movable cranial bone, which is the largest and strongest facial bone. It can open and close a human's mouth. The mandible has two basic bones, i.e., the alveolar part and the mandible base, located in the anterior part of the lower jaw bone. Furthermore, it has two surfaces and two borders [
In the following subsections, we describe the systematic design steps adopted for developing an automatic intelligent human skull recognition system using data collection and processing, feature extraction filters, and skull classification to obtain maximum prediction accuracy.
We used hardware and software platforms that would allow us to meet the objectives of this study and conduct forensic tests on human skulls. First, we used a DSC-HX300 digital camera (Sony Corp., Japan) equipped with high-resolution Carl Zeiss lenses for obtaining the skull images. Then, we applied Matlab software version R2013a to convert the image data into a numeric form. Finally, we implemented an SVM classifier with Eclips SDK in Java for skull classification. To run the aforementioned software, we used a personal computer with the following specifications: Intel Core i5 Processor equipped with 8 GB of RAM, using the Windows XP operating system.
Digitizing human skulls: In the first step, skulls were digitized by taking their photos from various angles using a digital camera. Thus, images of the face or front, left, right, bottom, and top areas could be obtained. The obtained results were then documented and saved as digital image files. Feature extraction: This step was conducted to obtain certain values from skull images via feature filtering or extraction based on pixel characteristics and other criteria. Various feature filters were applied to compare the accuracy rates of the implemented filters. This was the major image processing activity prior to the segmentation and classification steps. We considered four different feature-filtering techniques to determine the relevant features and extract their corresponding values from the images. We conducted a texture analysis approach using this feature filter before classifying the human skulls. Four feature filters were separately applied to obtain a different accuracy rate for classification. For this study, we used 22 feature-level, co-occurrence matrices (GLCM), 12 features of the discrete wavelet transform (DWT), 48 Gabor features, and 24 fractal features or segmentation-based fractal texture analyses (SFTA). In total, we used 106 features. The filters were applied to analyze 24 images of skulls at various rotation angles (from 1° to 360°); each image was extracted with these filters to obtain a different statistical decomposition. Therefore, each skull image produced a minimum of 360 images to be extracted through the deployment of various filters before classification. Classification: The support vector machine (SVM) is a widely applied method developed by Awad and Khanna [
Feature extraction involves the transformation of data. The derivative values from original data are transformed into variable data with statistical values that can be further processed. Here, we used the following techniques for feature extraction.
GLCM is a popular filter for texture analysis. It captures information regarding the gray-value spatial distribution in an image and the image texture's corresponding frequency at given specified angles and distances. Feature extraction using GLCM is conducted based on the estimated probability density function of a pixel using a co-occurrence matrix along with its pixel pairs, where features can be statistically and numerically quantified [ Thus, pixels are labeled as “1” if they belong to the ROI and “0” otherwise. From Here, (i,j) denotes the index of the pixel in the image, and Img (i,j) denotes the probability of the pixel index (i,j). GLCM can generate 22 texture features, as explained in detail by Tsai et al. Wavelet features. A digital image comprises many pixels that can be represented in a two-dimensional (2D) matrix. Outside the spatial domain, an image can be represented in the frequency domain using a spectrum method called the DWT. In several studies (e.g., [ Gabor features. Gabor filters are shaped through dilation and rotation in a single kernel with several parameters. The corresponding filter function is used as a kernel to obtain a dictionary filter for analyzing the texture images. The 2D Gabor filter has several benefits in a spatial domain, such as a number of different scales and orientations allows for feature extraction and also, invariance for rotation, illumination, and translation involving the Gaussian kernel function [ Here, parameter Fractal features are considered when evaluating images with similar textures. Features are obtained from the fractal dimensions of the transformed images obtained from the boundary of segmented image structures and grayscale images. Fractal features can be used to compute the fractal dimension for any surface roughness. Furthermore, they can be used to evaluate the gray image and compare various textures. Fractal dimensions can be realized as a measure of irregularity or heterogeneity. If an object has self-similarity properties, then the entire set of minimized subsets will have the same properties. In this study, the boundaries of the feature vector were used to measure fractals. The measurement is represented as Δ (x, y), and can be expressed as follows:
This measurement function is similar to the one in Costa et al. [
In this study, human skulls were categorized based on their mandibles. We validated and compared the samples’ unique characteristics (not only skulls with mandibles but also those without mandibles), as shown in
We experimented with seven different angles for the images of skulls with and without mandibles: front, top, and back angles, as well as 45° right-angle, 45° left-angle, 90° right-angle, and 90° left-angle rotations. Then, we rotated the image step-by-step by 360°; each degree of rotation produced one sample image that was stored as the input sample for machine learning. For example, the front angle was rotated by 360°, and thus we analyzed 360 data samples. Subsequently, we converted all the images to grayscale in jpeg (jpg) format, set a pixel size of 53 × 40 for each image, and set the file size to 4 kb.
Class | Sample | Pixels | Size | Total Data |
---|---|---|---|---|
Skull with mandible | |
53 × 40 | 4.00 kb | 8,640 |
Skull without mandible | |
53 × 40 | 4.00 kb | 8,640 |
In this experiment, it was conducted by dividing into training and testing data with a ratio of 2:1. There were ten sets and each set comprised 300 images selected for training data and another 150 images for test data.
The limitations of this study were difficulty in obtaining experimental data and using camera settings to ensure the same resolution when capturing skull images. Another limitation was that seven different angles were considered to perform comparisons between skulls with and without mandibles. Because of the difficulty associated with finding research objects, this study focused on the classification of 24 skulls, which were all in an incomplete condition, especially those that had teeth attached.
As described previously, we considered two different digital skull images: skulls with mandibles and skulls without mandibles. We first applied each feature extraction filter separately to clearly understand the factors influencing the experimental results. This process was followed by combining all the feature extraction filters. The following subsections discuss the application of filters and obtained classification accuracy.
In Experiment I, we considered the images of human skulls with mandibles and examined them from different angles as shown in
The detailed steps of this experiment were as follows.
Step 1: We used 24 sets of images extracted using various extraction filters. Each resulting set of images contained 360 transformed images obtained by rotating the original image via one-degree rotation per step. From all the available images, we selected 200 skull images as training data and 100 skull images as testing data. Our four extraction filtering techniques were then applied for feature extraction. Step 2: We ran the SVM to predict human skulls with mandibles using the four filtering techniques individually and then a combination of all four filters. Step 3: We conducted a series of image testing steps on the basis of the appropriate model constructed in Step (2) for human skulls with mandibles. Finally, we repeated Steps (1)–(3) nine more times (for a total of ten replicates) and obtained the average performance.
The classification of skulls differed in accuracy across the seven angles of interest. Evidently, each filter had a different accuracy even though the within-filter results were numerically stable. Gabor feature extraction was stable, i.e., higher than 90%, making it the superior feature filter among the four considered techniques. In contrast, the DWT filter resulted in an accuracy rate as low as 89.73%. Conversely, the GLCM, Gabor, and fractal filters consistently achieved a classification accuracy >98%. With prediction accuracies that were mostly >90%, all four filters are promising tools for assisting the SVM in automatically classifying human skulls for physical anthropology applications.
Filter | Front | −45° left | 45° right | −90° left | 90° right | Back | Top |
---|---|---|---|---|---|---|---|
GLCM | 98.07 | 99.90 | 99.75 | 99.76 | 99.81 | 99.74 | 99.99 |
DWT | 92.37 | 94.05 | 89.73 | 94.01 | 93.77 | 94.97 | 97.05 |
Gabor | 99.24 | 99.55 | 99.21 | 99.43 | 99.45 | 99.44 | 99.69 |
SFTA | 99.33 | 99.21 | 99.00 | 98.98 | 98.57 | 98.68 | 99.51 |
All | 99.52 | 99.57 | 99.53 | 99.57 | 99.39 | 99.46 | 99.80 |
We also conducted identifications of skulls without mandibles to evaluate the robustness of our classification system.
Filter | Front | −45° Left | 45° Right | −90° Left | 90° Right | Back | Top |
---|---|---|---|---|---|---|---|
GLCM | 99.95 | 99.92 | 99.88 | 99.87 | 99.86 | 99.87 | 99.95 |
DWT | 91.45 | 88.36 | 92.18 | 90.58 | 93.43 | 95.18 | 96.24 |
Gabor | 99.29 | 99.19 | 99.39 | 99.27 | 99.34 | 99.65 | 99.63 |
SFTA | 98.97 | 99.00 | 98.46 | 98.82 | 98.69 | 99.42 | 99.48 |
All | 99.61 | 99.56 | 99.56 | 99.32 | 99.46 | 99.50 | 99.72 |
In automatic human skull classification, the implementation of feature extraction and the combination of different feature filters play a significant role in the accumulation of relevant features. Each filter can produce several features. A classification system with diverse results can be produced by using four different filters and combining all generated features. For example, in this study, the use of GLCM comprising 22 features resulted in a classification accuracy rate of 99.86–99.95% depending on the angular position of the skull. Conversely, DWT feature extraction had a much lower accuracy rate of 88.36–96.24%.
We also used different electronic imaging devices to compare and validate the results of the previous experiments in which we used a high-resolution camera; however, in Experiment III, we used a mobile camera (NOKIA 3.1 plus) with a lower resolution. We used the same experimental approach but captured the skull front angle images with different lens sizes for camera resolutions of 2, 4, and 9 MP.
Filter | 2 MP | 4 MP | 9 MP |
---|---|---|---|
GLCM | 91.41 | 93.17 | 97.83 |
DWT | 67.38 | 67.89 | 70.07 |
Gabor | 88.48 | 91.50 | 93.55 |
SFTA | 79.32 | 80.20 | 90.67 |
All | 83.75 | 86.98 | 94.62 |
Our experimental results indicate that the classification of skulls with mandibles was as accurate as that of skulls without mandibles. However, the required calculation time for processing the images of skulls with mandibles was shorter than that for skulls without mandibles.
This study extends the analysis and framework for the identification of human faces reported in previous studies [
Research | Research object | Approach | Accuracy rate (%) |
---|---|---|---|
[ |
Live human face | PCA, sub-block processing | 97.60 |
[ |
Live human face | PCA, dual-tree complex wavelet transform (DT-CWT), and single-tree complex wavelet transform (ST-CWT) | 94.67 |
[ |
Live human face | Principal component analysis (PCA), particle swarm optimization (PSO)–SVM (PSO–SVM) | 98.00 |
[ |
Live human face | PCA, Euclidean, Gaussian mixture model (GMM) | 97.04 |
[ |
Live human face | CNNs | 98.43 |
Our work | SVM | 99.50 |
Unlike human face recognition research, one of the major challenges associated with the present study was the acquisition of human skull data. This is because the skull is an inanimate object that must be moved to obtain data from various angles. This movement was achieved by manually turning the skull to appropriate angles to obtain images from various positions. This is highly challenging, especially when the skull is in an incomplete condition.
Moreover, variation in the amount of training data can impact the accuracy of the classification task. It is thus of interest to investigate how various training dataset sizes can affect the performance of SVM classification. The prediction accuracy rates for skulls with and without mandibles show that the amount of training and testing data affects the prediction accuracy. For example, with the GLCM filter, when we used only one training data item to predict skulls with mandibles, we obtained an accuracy rate of 18.33%. However, when we used 100 training data items, the accuracy rate was 97.03%. Thus, a greater amount of applied training data will result in a higher accuracy.
Skulls generally have one dominant texture and color but may have different shapes and sizes even if the skulls share ancestry. However, if the bones are buried in different soils (for example, clay or calcareous soils), they will have different colors.
In this forensic study, we applied a digital camera to digitize the skulls. The implementation of different digitizing tools will affect the level of accuracy, especially regarding image resolution. Therefore, in further research, we recommend the use of advanced digital technology capabilities such as, postmortem computed tomography (PMCT) and angiography, as well as X-rays.
This study focused on only 24 human skulls with mandibles and 24 skulls without mandibles because of the limitations and difficulties in obtaining sample data in physical anthropology. However, we also conducted experiments on other skulls without mandibles (99 skulls) even though with some bone structures were incomplete when they were discovered. Therefore, we only focused on the classification of skull faces. Our results were similar to those obtained from Experiments III, although the level of accuracy was slightly higher than those in previous experiments.
We developed an automatic computerized digital forensics approach for human skull identification using feature extraction in tandem with an SVM. We applied a digital forensics framework to classify human skulls with and without mandibles. We tested four different feature extraction filters for feature extraction that resulted in different classification accuracies. GCLM achieved the maximum accuracy with features generated from Gabor and fractal features (>99%). In contrast, DWT features resulted in identification prediction accuracies <95%. The combination of the four feature extraction techniques produced an accuracy rate >99% for skulls both with and without mandibles. Thus, every human skull has unique features that can be used to distinguish its identity in forensics applications, especially in physical anthropology collection management.
We can identify several future directions for research related to skull identification. For future work, it will be necessary to optimize the combined feature extraction and classification method and to explore other feature extraction techniques and classification methods for performance comparisons. Utilizing additional skull data when using the CNN method could be the main focus for such future research. Furthermore, the determination of the age and gender associated with the skulls will greatly assist researchers in identifying humans who disappeared due to natural disasters or who were victims of criminal activities.