|Computer Systems Science & Engineering |
A Holographic Diffraction Label Recognition Algorithm Based on Fusion Double Tensor Features
1Hangzhou Dianzi University, Hangzhou, 310018, China
2Zhejiang Police College, Hangzhou, 310018, China
3University of Warwick, Coventry, CV4 7AL, UK
*Corresponding Author: Shanqing Zhang. Email: firstname.lastname@example.org
Received: 30 December 2020; Accepted: 26 February 2021
Abstract: As an efficient technique for anti-counterfeiting, holographic diffraction labels has been widely applied to various fields. Due to their unique feature, traditional image recognition algorithms are not ideal for the holographic diffraction label recognition. Since a tensor preserves the spatiotemporal features of an original sample in the process of feature extraction, in this paper we propose a new holographic diffraction label recognition algorithm that combines two tensor features. The HSV (Hue Saturation Value) tensor and the HOG (Histogram of Oriented Gradient) tensor are used to represent the color information and gradient information of holographic diffraction label, respectively. Meanwhile, the tensor decomposition is performed by high order singular value decomposition, and tensor decomposition matrices are obtained. Taking into consideration of the different recognition capabilities of decomposition matrices, we design a decomposition matrix similarity fusion strategy using a typical correlation analysis algorithm and projection from similarity vectors of different decomposition matrices to the PCA (Principal Component Analysis) sub-space , then, the sub-space performs KNN (K-Nearest Neighbors) classification is performed. The effectiveness of our fusion strategy is verified by experiments. Our double tensor recognition algorithm complements the recognition capability of different tensors to produce better recognition performance for the holographic diffraction label system.
Keywords: Label recognition; holographic diffraction; fusion double tensor; matrix similarity
With the rapid development of printing technology, new types of product labels are used. Holographic diffraction labels have been chosen by many manufacturers due to their unique anti-counterfeiting feature. With the popularity of smartphones, there is an increasing demand for image recognition using mobile phones. Different image features are shown in different illumination environments due to the unique physical feature of holographic diffraction labels. Traditional image recognition algorithms are not ideal for holographic diffraction label recognition.
In this study, tensor is used to represent data to preserve the optically variable data of a diffraction image. Tensor has been widely used in signal and image processing [1–3], factor analysis [4,5], and voice communication . Since a tensor can maintains the structure of the original data, tensor analysis has appealed to researchers. Most datasets can be represented by matrices and efficiently analyzed by singular value decomposition (SVD) . However, some specific datasets such as sequence images, video and text cannot be represented by matrices directly, so additional operations are required. For example, when SVD decomposition cannot be used directly, tensor decomposition such as Tucker decomposition is required . Vasilescu et al.  constructed face images into two-dimensional tensors for face recognition.
Low-dimension sub-space learning methods have been expanded to tensor representation, such as tensor principal component analysis , tensor linear discriminant analysis , and multilinear discriminant analysis . Stoudenmire et al.  proposed a supervised tensor-learning framework that can directly process high order tensor data.
Information from changing illumination of holographic diffractive labels is lost if it is represented using matrix . In addition, smartphone camera captures the jitter and rotational interference. A double tensor is used herein to represent the features of holographic diffraction labels. Most tensor-based image recognition methods directly represent the original data of the image as a tensor and do not include feature extraction. An appropriate feature extraction method makes it possible that the original image is represented as a tensor through new features , which results in better recognition performance. Moreover, the original data and the extracted features can complement each other, further improving the accuracy of classification . Taking into consideration of the features of holographic images under different illumination, we propose a holographic label classification method that combines an HSV (Hue Saturation Value) tensor with a HOG (Histogram of Oriented Gradient) feature tensor. Accurate classification and identification of label images are achieved by similarity measurement of both tensors.
2 Proposed Method
A color image has three channels of RGB and can be represented as a tensor intrinsically. A holographic label has different color information for different illuminations because of its light-varying feature. In order to preserve the color information of an image, the holographic image is converted from the RGB to the HSV color space and is further represented as an HSV tensor . The HOG tensor of a label is constructed on the extracted HOG features . In order to measure the similarity between the tensors, high-order singular value decomposition (HOSVD) is used to obtain the decomposition matrix of tensor expansion matrices , and the typical correlation coefficients of the decomposition matrix are calculated using Canonical correlation analysis (CCA) to obtain the similarity vector. Because different decomposition matrices have different classification capabilities, we propose a fusion strategy to perform principal component analysis (PCA) dimension reduction for the similarity vector. Finally, the nearest neighbor algorithm is used for classification. The flowchart of our algorithm is shown in Fig. 1.
2.1 Image Preprocessing
The tilt and rotation of an image taken by a mobile phone always causes incorrect recognition. In order to ensure the accuracy of the classification, rotation correction using edge detection and the Hough transformation are performed for all input images.
In order to remove interference from the background in a label image, the grayscale image is converted into a binary image using the maximum OTSU. Then Canny edge detection is performed on the binary image. The traditional Canny operator performs Gaussian smoothing on the original image in the process of edge detection. However, the influence of noise is related to the distance of the noise point from the center after Gaussian smoothing. It causes image edge blurring  and impacts the image correction effect . Median filtering is used to preserve the edge information of an image. The test results are shown in Fig. 2c. The binary image obtained by edge detection goes through the Hough rotation correction, and the results are shown in Fig. 2.
After the rotation correction, the original RGB color space is converted into a HSV space and normalized into a third-order HSV tensor , where and represent the width and height of the image, respectively.
2.2 Generation of HOG Tensor
Image features are extracted using HOG descriptors. In contrast to traditional HOG feature extraction algorithms, a faster HOG feature extraction method  is used to obtain the same descriptors as the original HOG.
The size of a normalized image is given as , and the image is divided into sub-blocks, where is the size of each sub-block. The gradient of the image is calculated and the gradient direction histogram of each sub-block is constructed using the four-way normalization method. This normalization method is shown in Fig. 3. Each block is generated by the normalization of the four adjacent sub-blocks using vector as superposition of the positive direction in the histogram. The normalization of the block is defined as follows:
where is the second norm of vector , .
Each block yields four different normalization results, is the number of directions (bins) in the histogram, and each block gets one HOG descriptor of in length, as shown in Fig. 4.
A third-order tensor is obtained using HOG feature extraction on the original image, where , , respectively.
2.3 Similarity Between the Double Tensors
The obtained HOG tensor and HSV tensor are the primary features of an image. These primary features are decomposed into orthogonal matrices using HOSVD algorithm. The similarity between the decomposition matrices of the test sample and the training sample are measured using CCA [23,24].
2.3.1 Generation of Decomposition Matrix.
A tensor is decomposed into decomposition matrices using HOSVD. First, a high-order tensor is expanded into a two-dimensional matrix. An order tensor is expanded into a series of two-dimensional matrices using the modulo-N expanding, where the size of is . For example, a third-order tensor is expanded into matrix , as shown in Fig. 5.
HOSVD decomposition of an expanded matrix is represented as follows:
where represents a diagonal matrix, and are the orthogonal matrices that can be spanned to column space and row space of , respectively. Factor matrix is an orthogonal matrix and correlated to the non-zero singular values of . The decomposition matrix is regarded as a point in the Glesman manifold, so it represents the mapping of the primary feature tensor to the Glesman manifold, and the tensor similarity can be calculated in the Glesman manifold for tensor classification identification. Three different mappings in the manifold are obtained for the HSV tensor and HOG tensor, respectively.
2.3.2 CCA Similarity Measurement.
CCA is used to measure the similarity between tensor decomposition matrices. For random vector and , optimization goal of CCA  is used to find vector and , so that the correlation between and is maximized. It is defined in Eq. (3).
where and represent two typical variables, is the typical correlation, and is the correlation between and . The typical correlation between matrices and is defined in Eq. (4). The typical correlation of two decomposition matrices is calculated using MATLAB function .
Six decomposition matrices are obtained for the HSV and HOG tensors, and the similarity between the two samples is represented as a 6-dimensional vector .
2.3.3 Similarity Fusion
Six typical correlations are obtained based on calculating the similarity of holographic labels described in the previous sections. The summation of all six typical correlations may be simply used as the similarity between the samples. However, different decomposition matrices of a tensor contain different information and have different distinctive capabilities. Therefore, each decomposition matrix serves as an independent unit, and an effective method is proposed to fuse these similarities. The process is shown in Fig. 6.
The similarity vectors between the test sample and the training samples are represented as respectively. Principal component analysis is used to find the best projection sub-space in order to determine the space within the largest fusion vector. The select training is implemented on PCA sub-space. For a series of similarity vectors, their mean values are defined using Eq. (5):
The scatter matrix is calculated using Eq. (6):
The scatter matrix is decomposed using Eq. (7):
The diagonal matrix consists of eigenvalues of , the column vectors of are the corresponding eigenvectors. The PCA sub-space consists of the corresponding feature vectors of the largest eigenvalues. A test similarity vector is generated using Eq. (8):
where represents the maximum value of the first column of the training vector. Because a larger represents higher similarity, the maximum value of each dimension serves as the test vector. All vectors and are projected to the PCA sub-space and classified using the nearest neighbor algorithm.
3 Experimental Results
The dataset used in this study contains 200 holographic diffraction labels with an image size of . Meanwhile, since different image features are shown in different illumination environments due to the unique physical feature of holographic diffraction labels, we expand the dataset to 800 by applying an affine transformation of 90 degrees to the original labels. These original images are decreased in size to 256 × 256 and converted to HSV color space as the original tensor of the samples. HOG tensor extraction is performed using the original image. The diffraction labels in the dataset are classified into 8 categories according to lighting environment, with each category containing 800 labels. Some test images with different lighting environments are shown in Fig. 7. The algorithm is realized using MATLAB R2012a.
3.1 The Advantages of the Fusion Decomposition Matrix
The advantages of the similarity algorithm of the fusion decomposition matrix are analyzed in this study. First, a classification experiment is performed using the decomposition matrices for the HSV tensor, and the recognition results are shown in Tab. 1. Then, the summation strategy was used in their experiments . As shown in Tab. 1, the recognition rate using the fusion strategy is 87.96% and better than that of the decomposition matrix alone. The recognition rate of our fusion algorithm is 93.61%, which indicates good recognition effect.
The recognition abilities of different decomposition matrices are not the same but it was not considered in their study. Our strategy overcomes this shortcoming. As shown in Tab. 1, the sample recognitions of decomposition matrices 1, 2, and 3 are different among each other. The typical correlation coefficients of each decomposition matrix are summed into one similarity vector. The similarity vector is projected to the PCA sub-space and classified by the K-nearest neighbor algorithm. The similarity of the decomposition matrices between our algorithm and the summation method is shown in Fig. 8. There are five training samples and one test sample.
Calculating the typical correlation between the test sample and the training samples, we obtain five three-dimensional similarity vectors . Each element in the vectors is the similarity between the test sample and the training sample of an individual decomposition matrix. As shown in Fig. 8, the similarity results, obtained by using the summation strategy of decomposition matrix are very close. The same similarity results are obtained in 5 training samples using the fusion strategy. On the other hand, our algorithm uses the PCA to project the similarity vector to another sub-space and enlarges the distance between different samples. The similarity vector of the test sample is represented as the largest similarity between the set of samples and the test sample. represents the sample vectors in the sub-space, represents the maximum similarity vector in the sub-space. The projected sample vectors do not overlap with each other. Our fusion strategy has good performance and its recognition rate reaches 93.61% for the dataset.
3.2 The Complementarity of the Double Tensor Feature
The complementarity of the double tensor is tested. First, only the HSV tensors of the original data are used for holographic image recognition in the dataset. The confusion matrix of the HSV tensor recognition is shown in Fig. 9a. The identification of the HSV tensor is not good in distinguishing between “Label 10” and other holographic labels. Many “Label 10” labels are mis-identified as “Label 6” or “Label 7”. Second, the tensors constructed using the HOG features are tested with the same dataset, and the confusion matrix is shown in Fig. 9b. The HOG tensor has a high recognition rate (95%) for “Label 10”. The two tensors have complementary effect although the HOG tensor is worse than HSV tensor in other categories.
The typical correlation coefficients of the HSV tensor and the HOG tensor are combined based on the above experiment. A confusion matrix is obtained using this double tensor, as shown in Fig. 9c. The double tensor balances the inconsistent recognition results of holographic labels, to some extent, the misidentification of one tensor may be masked by the other tensor, resulting in the complementary effect being generally better than each effect alone. The recognition results using each tensor and the double tensor are shown in Tab. 2. The recognition accuracy of the double tensor is improved greatly.
3.3 Algorithm Comparison
Our algorithm is compared with the algorithm proposed in [17,26] with the same dataset after rotation, cropping, and illumination change of the sample, respectively. The recognition results of the holographic labels are shown in Tabs. 3–5. The experimental results show that our algorithm is robust to rotation and illumination changes. The HSV tensor in the double tensor contains the color information of the sample, and the HOG tensor represents the gradient information. A higher recognition rate is achieved because of their complementation. However, cropping causes loss of sample information, it results in misjudgment. If the cropping parameters are small, the recognition accuracy remains higher than the expanded SIFT in [17,26].
An algorithm for holographic diffraction label recognition using a complementary double tensor is proposed. First, an approach is proposed to generate the HOG feature tensor that combines the HSV tensor of the original data to obtain the double tensor. Then, the double tensor is decomposed using HOSVD to obtain the double tensor decomposition matrix. Finally, typical correlation analysis is used to calculate the similarity between the decomposition matrices. The similarity of the decomposition matrix is fused according to different recognition capabilities, and the similarity vectors are projected to a PCA sub-space for classification. The algorithm makes up for the deficiency of the original data tensor, improves recognition rate, does not require advanced training process, and has high computational efficiency. The experimental results have shown that the double tensor fusion algorithm is capable of performing efficient recognition for holographic diffraction labels.
Acknowledgement: We thank the anonymous reviewers and editors for their very constructive comments.
Funding Statement: This work was mainly supported by Public Welfare Technology and Industry Project of Zhejiang Provincial Science Technology Department. (No. LGG18F020013, No. LGG19F020016, LGF21F020006).
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
|This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.|