Computer Systems Science & Engineering DOI:10.32604/csse.2021.016340
Article

A Holographic Diffraction Label Recognition Algorithm Based on Fusion Double Tensor Features

Li Li1, Chen Cui1,2, Jianfeng Lu1, Shanqing Zhang1,*and Ching-Chun Chang3

1Hangzhou Dianzi University, Hangzhou, 310018, China
2Zhejiang Police College, Hangzhou, 310018, China
3University of Warwick, Coventry, CV4 7AL, UK
*Corresponding Author: Shanqing Zhang. Email: sqzhang@hdu.edu.cn
Received: 30 December 2020; Accepted: 26 February 2021

Abstract: As an efficient technique for anti-counterfeiting, holographic diffraction labels has been widely applied to various fields. Due to their unique feature, traditional image recognition algorithms are not ideal for the holographic diffraction label recognition. Since a tensor preserves the spatiotemporal features of an original sample in the process of feature extraction, in this paper we propose a new holographic diffraction label recognition algorithm that combines two tensor features. The HSV (Hue Saturation Value) tensor and the HOG (Histogram of Oriented Gradient) tensor are used to represent the color information and gradient information of holographic diffraction label, respectively. Meanwhile, the tensor decomposition is performed by high order singular value decomposition, and tensor decomposition matrices are obtained. Taking into consideration of the different recognition capabilities of decomposition matrices, we design a decomposition matrix similarity fusion strategy using a typical correlation analysis algorithm and projection from similarity vectors of different decomposition matrices to the PCA (Principal Component Analysis) sub-space , then, the sub-space performs KNN (K-Nearest Neighbors) classification is performed. The effectiveness of our fusion strategy is verified by experiments. Our double tensor recognition algorithm complements the recognition capability of different tensors to produce better recognition performance for the holographic diffraction label system.

Keywords: Label recognition; holographic diffraction; fusion double tensor; matrix similarity

1  Introduction

With the rapid development of printing technology, new types of product labels are used. Holographic diffraction labels have been chosen by many manufacturers due to their unique anti-counterfeiting feature. With the popularity of smartphones, there is an increasing demand for image recognition using mobile phones. Different image features are shown in different illumination environments due to the unique physical feature of holographic diffraction labels. Traditional image recognition algorithms are not ideal for holographic diffraction label recognition.

In this study, tensor is used to represent data to preserve the optically variable data of a diffraction image. Tensor has been widely used in signal and image processing [1–3], factor analysis [4,5], and voice communication [6]. Since a tensor can maintains the structure of the original data, tensor analysis has appealed to researchers. Most datasets can be represented by matrices and efficiently analyzed by singular value decomposition (SVD) [7]. However, some specific datasets such as sequence images, video and text cannot be represented by matrices directly, so additional operations are required. For example, when SVD decomposition cannot be used directly, tensor decomposition such as Tucker decomposition is required [8]. Vasilescu et al. [9] constructed face images into two-dimensional tensors for face recognition.

Low-dimension sub-space learning methods have been expanded to tensor representation, such as tensor principal component analysis [10], tensor linear discriminant analysis [11], and multilinear discriminant analysis [12]. Stoudenmire et al. [13] proposed a supervised tensor-learning framework that can directly process high order tensor data.

Information from changing illumination of holographic diffractive labels is lost if it is represented using matrix [14]. In addition, smartphone camera captures the jitter and rotational interference. A double tensor is used herein to represent the features of holographic diffraction labels. Most tensor-based image recognition methods directly represent the original data of the image as a tensor and do not include feature extraction. An appropriate feature extraction method makes it possible that the original image is represented as a tensor through new features [15], which results in better recognition performance. Moreover, the original data and the extracted features can complement each other, further improving the accuracy of classification [16]. Taking into consideration of the features of holographic images under different illumination, we propose a holographic label classification method that combines an HSV (Hue Saturation Value) tensor with a HOG (Histogram of Oriented Gradient) feature tensor. Accurate classification and identification of label images are achieved by similarity measurement of both tensors.

2  Proposed Method

A color image has three channels of RGB and can be represented as a tensor intrinsically. A holographic label has different color information for different illuminations because of its light-varying feature. In order to preserve the color information of an image, the holographic image is converted from the RGB to the HSV color space and is further represented as an HSV tensor [17]. The HOG tensor of a label is constructed on the extracted HOG features [18]. In order to measure the similarity between the tensors, high-order singular value decomposition (HOSVD) is used to obtain the decomposition matrix of tensor expansion matrices [19], and the typical correlation coefficients of the decomposition matrix are calculated using Canonical correlation analysis (CCA) to obtain the similarity vector. Because different decomposition matrices have different classification capabilities, we propose a fusion strategy to perform principal component analysis (PCA) dimension reduction for the similarity vector. Finally, the nearest neighbor algorithm is used for classification. The flowchart of our algorithm is shown in Fig. 1.

Figure 1: Flowchart of double tensor recognition algorithm

2.1 Image Preprocessing

The tilt and rotation of an image taken by a mobile phone always causes incorrect recognition. In order to ensure the accuracy of the classification, rotation correction using edge detection and the Hough transformation are performed for all input images.

In order to remove interference from the background in a label image, the grayscale image is converted into a binary image using the maximum OTSU. Then Canny edge detection is performed on the binary image. The traditional Canny operator performs Gaussian smoothing on the original image in the process of edge detection. However, the influence of noise is related to the distance of the noise point from the center after Gaussian smoothing. It causes image edge blurring [20] and impacts the image correction effect [21]. Median filtering is used to preserve the edge information of an image. The test results are shown in Fig. 2c. The binary image obtained by edge detection goes through the Hough rotation correction, and the results are shown in Fig. 2.

Figure 2: The image with rotation correction (a) Grayscale (b) Traditional Canny algorithm (c) Improved Canny algorithm (d) Binary map (e) Hough detection line (f) Rotation correction

After the rotation correction, the original RGB color space is converted into a HSV space and normalized into a third-order HSV tensor C∈iw×h×3 , where w and h represent the width and height of the image, respectively.

2.2 Generation of HOG Tensor

Image features are extracted using HOG descriptors. In contrast to traditional HOG feature extraction algorithms, a faster HOG feature extraction method [22] is used to obtain the same descriptors as the original HOG.

The size of a normalized image is given as w×h , and the image is divided into (w/bsize)×(h/bsize) sub-blocks, where bsize is the size of each sub-block. The gradient of the image is calculated and the gradient direction histogram of each sub-block is constructed using the four-way normalization method. This normalization method is shown in Fig. 3. Each block is generated by the normalization of the four adjacent sub-blocks using vector v as superposition of the positive direction in the histogram. The normalization of the block is defined as follows:

v=v/||v||22+ε2 (1)

where ||v||2 is the second norm of vector v , ε>0 .

Figure 3: The normalization of blocks in the image

Each block yields four different normalization results, nOrients is the number of directions (bins) in the histogram, and each block gets one HOG descriptor of nOrients×4 in length, as shown in Fig. 4.

Figure 4: HOG feature extraction

A third-order tensor G∈iW×H×(nOrients×4) is obtained using HOG feature extraction on the original image, where W=w/bsize , H=h/bsize , respectively.

2.3 Similarity Between the Double Tensors

The obtained HOG tensor and HSV tensor are the primary features of an image. These primary features are decomposed into orthogonal matrices using HOSVD algorithm. The similarity between the decomposition matrices of the test sample and the training sample are measured using CCA [23,24].

2.3.1 Generation of Decomposition Matrix.

A tensor is decomposed into decomposition matrices using HOSVD. First, a high-order tensor is expanded into a two-dimensional matrix. An Nth order tensor A∈is1×s2×s3×⋯×sN is expanded into a series of two-dimensional matrices {A(k)}k=1N using the modulo-N expanding, where the size of A(k) is sk×∏i≠k⁡Si . For example, a third-order tensor A∈i4×3×5 is expanded into matrix A(1)∈R4×15,A(2)∈R3×20,A(3)∈R5×12 , as shown in Fig. 5.

Figure 5: Modulo-N unfold of a third-order tensor

HOSVD decomposition of an expanded matrix is represented as follows:

A(k)=U(k)∑(k)⁡V(k)T (2)

where ∑(k) represents a diagonal matrix, U(k) and V(k) are the orthogonal matrices that can be spanned to column space and row space of A(k) , respectively. Factor matrix V(k) is an orthogonal matrix and correlated to the non-zero singular values of A(k) . The decomposition matrix V(k) is regarded as a point in the Glesman manifold, so it represents the mapping of the primary feature tensor to the Glesman manifold, and the tensor similarity can be calculated in the Glesman manifold for tensor classification identification. Three different mappings in the manifold are obtained for the HSV tensor and HOG tensor, respectively.

2.3.2 CCA Similarity Measurement.

CCA is used to measure the similarity between tensor decomposition matrices. For random vector x∈im and y∈in , optimization goal of CCA [23] is used to find vector u∈im and v∈in , so that the correlation between uTx and vTy is maximized. It is defined in Eq. (3).

ρ=maxcorr(uTx,vTy) (3)

where u and v represent two typical variables, ρ is the typical correlation, and corr(X,Y) is the correlation between X and Y . The typical correlation between matrices X∈iN×m1 and Y∈iN×m2 is defined in Eq. (4). The typical correlation of two decomposition matrices is calculated using MATLAB function canoncorr() .

ρ=maxX′TY′,whereX′=Xu,Y′=Yv (4)

Six decomposition matrices {V(i)}i=16 are obtained for the HSV and HOG tensors, and the similarity between the two samples is represented as a 6-dimensional vector ψ=(φ1,φ2,⋯,φ6) .

2.3.3 Similarity Fusion

Six typical correlations are obtained based on calculating the similarity of holographic labels described in the previous sections. The summation of all six typical correlations may be simply used as the similarity between the samples. However, different decomposition matrices of a tensor contain different information and have different distinctive capabilities. Therefore, each decomposition matrix serves as an independent unit, and an effective method is proposed to fuse these similarities. The process is shown in Fig. 6.

Figure 6: The process of the decomposition matrix similarity fusion

The similarity vectors between the test sample and the training samples are represented as ψ1,ψ2,⋯,ψt respectively. Principal component analysis is used to find the best projection sub-space in order to determine the space within the largest fusion vector. The select training is implemented on PCA sub-space. For a series of similarity vectors, their mean values are defined using Eq. (5):

ψ¯=1t∑k=1t⁡ψk (5)

The scatter matrix is calculated using Eq. (6):

S=∑k=1t⁡(ψk−ψ¯)(ψk−ψ¯)T (6)

The scatter matrix is decomposed using Eq. (7):

S=ΦAΦT (7)

The diagonal matrix A consists of eigenvalues of S , the column vectors of Φ are the corresponding eigenvectors. The PCA sub-space consists of the corresponding feature vectors of the ρ largest eigenvalues. A test similarity vector is generated using Eq. (8):

ψT={max(ψj,1),max(ψj,2),⋯,max(ψj,6)|j=1,2,⋯,t} (8)

where max(ψj,1) represents the maximum value of the first column of the training vector. Because a larger ψ represents higher similarity, the maximum value of each dimension serves as the test vector. All vectors ψ1,ψ2,⋯,ψt and ψt are projected to the PCA sub-space and classified using the nearest neighbor algorithm.

3  Experimental Results

The dataset used in this study contains 200 holographic diffraction labels with an image size of 512×512 . Meanwhile, since different image features are shown in different illumination environments due to the unique physical feature of holographic diffraction labels, we expand the dataset to 800 by applying an affine transformation of 90 degrees to the original labels. These original images are decreased in size to 256 × 256 and converted to HSV color space as the original tensor of the samples. HOG tensor extraction is performed using the original image. The diffraction labels in the dataset are classified into 8 categories according to lighting environment, with each category containing 800 labels. Some test images with different lighting environments are shown in Fig. 7. The algorithm is realized using MATLAB R2012a.

Figure 7: Samples with different lighting environments

3.1 The Advantages of the Fusion Decomposition Matrix

The advantages of the similarity algorithm of the fusion decomposition matrix are analyzed in this study. First, a classification experiment is performed using the decomposition matrices for the HSV tensor, and the recognition results are shown in Tab. 1. Then, the summation strategy was used in their experiments [25]. As shown in Tab. 1, the recognition rate using the fusion strategy is 87.96% and better than that of the decomposition matrix alone. The recognition rate of our fusion algorithm is 93.61%, which indicates good recognition effect.

Table 1: Comparison of recognition rate

The recognition abilities of different decomposition matrices are not the same but it was not considered in their study. Our strategy overcomes this shortcoming. As shown in Tab. 1, the sample recognitions of decomposition matrices 1, 2, and 3 are different among each other. The typical correlation coefficients of each decomposition matrix are summed into one similarity vector. The similarity vector is projected to the PCA sub-space and classified by the K-nearest neighbor algorithm. The similarity of the decomposition matrices between our algorithm and the summation method is shown in Fig. 8. There are five training samples and one test sample.

Figure 8: The similarity of the decomposition matrices between two algorithms

Calculating the typical correlation between the test sample and the training samples, we obtain five three-dimensional similarity vectors ψ1,ψ2,⋯,ψ5 . Each element in the vectors is the similarity between the test sample and the training sample of an individual decomposition matrix. As shown in Fig. 8, the similarity results, obtained by using the summation strategy of decomposition matrix are very close. The same similarity results are obtained in 5 training samples using the fusion strategy. On the other hand, our algorithm uses the PCA to project the similarity vector to another sub-space and enlarges the distance between different samples. The similarity vector of the test sample is represented as the largest similarity between the set of samples and the test sample. ψ1 represents the sample vectors in the sub-space, ψt represents the maximum similarity vector in the sub-space. The projected sample vectors do not overlap with each other. Our fusion strategy has good performance and its recognition rate reaches 93.61% for the dataset.

3.2 The Complementarity of the Double Tensor Feature

The complementarity of the double tensor is tested. First, only the HSV tensors of the original data are used for holographic image recognition in the dataset. The confusion matrix of the HSV tensor recognition is shown in Fig. 9a. The identification of the HSV tensor is not good in distinguishing between “Label 10” and other holographic labels. Many “Label 10” labels are mis-identified as “Label 6” or “Label 7”. Second, the tensors constructed using the HOG features are tested with the same dataset, and the confusion matrix is shown in Fig. 9b. The HOG tensor has a high recognition rate (95%) for “Label 10”. The two tensors have complementary effect although the HOG tensor is worse than HSV tensor in other categories.

Figure 9: The fusion matrix for identification using different tensors (a) Confusion matrix using HSV tensor (b) Confusion matrix using HOG tensor (c) Confusion matrix using the double tensor

The typical correlation coefficients of the HSV tensor and the HOG tensor are combined based on the above experiment. A confusion matrix is obtained using this double tensor, as shown in Fig. 9c. The double tensor balances the inconsistent recognition results of holographic labels, to some extent, the misidentification of one tensor may be masked by the other tensor, resulting in the complementary effect being generally better than each effect alone. The recognition results using each tensor and the double tensor are shown in Tab. 2. The recognition accuracy of the double tensor is improved greatly.

Table 2: Recognition results using single tensor and the double tensor

3.3 Algorithm Comparison

Our algorithm is compared with the algorithm proposed in [17,26] with the same dataset after rotation, cropping, and illumination change of the sample, respectively. The recognition results of the holographic labels are shown in Tabs. 3–5. The experimental results show that our algorithm is robust to rotation and illumination changes. The HSV tensor in the double tensor contains the color information of the sample, and the HOG tensor represents the gradient information. A higher recognition rate is achieved because of their complementation. However, cropping causes loss of sample information, it results in misjudgment. If the cropping parameters are small, the recognition accuracy remains higher than the expanded SIFT in [17,26].

Table 3: Comparison of recognition after sample scaling

Table 4: Comparison of recognition after sample cropping

Table 5: Comparison of recognition after sample illumination angle change

4  Conclusions

An algorithm for holographic diffraction label recognition using a complementary double tensor is proposed. First, an approach is proposed to generate the HOG feature tensor that combines the HSV tensor of the original data to obtain the double tensor. Then, the double tensor is decomposed using HOSVD to obtain the double tensor decomposition matrix. Finally, typical correlation analysis is used to calculate the similarity between the decomposition matrices. The similarity of the decomposition matrix is fused according to different recognition capabilities, and the similarity vectors are projected to a PCA sub-space for classification. The algorithm makes up for the deficiency of the original data tensor, improves recognition rate, does not require advanced training process, and has high computational efficiency. The experimental results have shown that the double tensor fusion algorithm is capable of performing efficient recognition for holographic diffraction labels.

Acknowledgement: We thank the anonymous reviewers and editors for their very constructive comments.

Funding Statement: This work was mainly supported by Public Welfare Technology and Industry Project of Zhejiang Provincial Science Technology Department. (No. LGG18F020013, No. LGG19F020016, LGF21F020006).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.