Compressed sensing (CS), as an efficient data transmission method, has achieved great success in the field of data transmission such as image, video and text. It can robustly recover signals from fewer Measurements, effectively alleviating the bandwidth pressure during data transmission. However, CS has many shortcomings in the transmission of hyperspectral image (HSI) data. This work aims to consider the application of CS in the transmission of hyperspectral image (HSI) data, and provides a feasible research scheme for CS of HSI data. HSI has rich spectral information and spatial information in bands, which can reflect the physical properties of the target. Most of the hyperspectral image compressed sensing (HSICS) algorithms cannot effectively use the inter-band information of HSI, resulting in poor reconstruction effects. In this paper, A three-stage hyperspectral image compression sensing algorithm (Three-stages HSICS) is proposed to obtain intra-band and inter-band characteristics of HSI, which can improve the reconstruction accuracy of HSI. Here, we establish a multi-objective band selection (Mop-BS) model, a multi-hypothesis prediction (MHP) model and a residual sparse (ReWSR) model for HSI, and use a staged reconstruction method to restore the compressed HSI. The simulation results show that the three-stage HSICS successfully improves the reconstruction accuracy of HSICS, and it performs best among all comparison algorithms.
The emergence of hyperspectral image (HSI) is a key progress in the remote sensing field. It is usually captured by spectral imagers and spectral sensors mounted on various platforms, which have rich spectral and spatial information. The spectral information of HSI means it has a huge number of continuous spectral bands from dozens to hundreds that represent a different spectrum, which simultaneously image the same target area. The spatial information means it has a lot of pixels in one band, especially in the characterization of large area ground. HSI is a key part in the classification of ground features, water quality inversion and other fields because of plentiful information. However, it also leads to a lot of redundant information, including redundant bands and pixels, which greatly reduces the efficiency of HSI data acquisition and processing. Hyperspectral image compression sensing has received extensive attention to solving this problem. Compressed sensing theory is different from Shannon's theory, which can recover robust solutions of signals from lower sampling rates (SR) than the Nyquist theorem. The sparse signal processing in hyperspectral image compressed sensing (HSICS) can reduce the cost of HSI's expensive acquisition and processing, and ease the data transmission bandwidth. Abedi et al. [
Various data forms have different requirements for the CS reconstruction algorithm. In HSICS, using inter-band and intra-band information can improve the reconstruction effect of CS. Wang et al. [
The above research shows that multi-band joint reconstruction can effectively improve the reconstruction effect of HSI, which considers the spectral information between various bands in HSICS. However, there are a lot of redundant bands in HSI. In multi-band joint reconstruction, the redundant band has a great influence on the effect of compressed reconstruction. Therefore, it is necessary to select the optimal band subsets for multi-band joint reconstruction in the CS reconstruction process. The band selection process can be analogous to a feature selection process. The optimal subset is affected by two most important issues, such as the evaluation criteria and search algorithm [
The HSI band selection can be constructed as a multi-objective optimization problem, and a stochastic optimization algorithm can be used to solve it to obtain the best band subset. In recent years, research on multi-objective optimization algorithms has made great breakthroughs [
Aiming at the problem of low-precision reconstruction of HSICS, this paper proposes a three-stage HIS-CS algorithm to reduce the redundancy of HSI. In the first phase, the Mop-BS to obtain the optimal band subsets. Then, a reweighted sparse residual (ReWSR) model and a multi-hypothesis prediction (MHP) model are established to iteratively obtain a reconstructed image. In the second phase, Intra-band reconstruction with the spatial information of the band subsets is used to obtain a preliminary restored image with the ReWSR model and MHP model. The third phase performs inter-band reconstruction with inter-spectral information of the band subsets with the ReWSR and MHP model.
In this section, we specifically introduce the three-stages HSICS. Inspired by the video compression sensing reconstruction method in [
In this section, the HSI band selection is modeled as a multi-objective combination optimization problem as the first phase. In the multi-band joint reconstruction of HSICS, the similarity between the bands will greatly improve the effect of joint reconstruction. However, it is not appropriate to simply use the similarity of the bands to select the band subset of HSICS. It is also necessary to consider the amount of information in the bands. Only a band with a large amount of information can provide a better joint reconstruction effect in the second and third stages. Meanwhile, the number of band subsets is uncertain. We cannot directly judge the influence of the number of band subsets on the second and third stages. Based on the above factors, this article comprehensively considers the amount of information, band similarity and reconstruction error to establish the Mop-BS model.
The HSI compressive sensing algorithm proposed in this paper is expanded on the video block compressive sensing. In video compressed sensing, the images of each frame in a video have a time series relationship, so the images of adjacent frames have a strong correlation. After blocking the image between adjacent frames, better performance can be obtained by searching for similar blocks. However, in HSI compressed sensing, adjacent bands do not necessarily have a strong correlation. We analyze the factors that affect the band correlation to establish the Mop-BS model, and use the multi-objective optimization method to select the optimal bands. This aims to select a band combination with strong correlation, which is called the optimal band subset. Compressed sensing in subsets can greatly improve the reconstruction accuracy of HSI. The overall flow of the Mop-BS model is shown in
HSI contains different amounts of information on different wavebands. This is due to the environments in which the spectral images are collected. the high-information band presents higher image quality, clearer structure, and higher accuracy in HSI processing. This article uses information entropy to measure the amount of band information. The formula is shown in
The
Among them,
The correlation between bands is particularly important for three-stage HSICS. In the third step, the algorithm will search for similar blocks in the band subset. Here, we use the mean spectral angle (MSA) to measure the similarity between the bands. MSA measures the similarity between bands by calculating the angle of two vectors in HSI. The similarity of the band is inversely proportional to the angle.
To solve the above model, we have several inspire-algorithms to compare. A population (
The dimension of each individual is the total number of bands in HSI. A real number between [0, 1] is randomly generated for each band in the initialization operation. If the value is greater than 0.5, it means that this band is selected. In encoding, the random numbers between [0, 1] are uniformly distributed, and the selection probability is set to 0.5 to ensure that each band has the same probability of selection and non-selection.
The offspring population is regarded as the solution space. The optimization process is the process of guiding the solution space convergence according to the model. When the maximum number of iterations is reached, the optimal band subset will be obtained. The process of solving the Mop-BS model is the first phase of the HSICS algorithm.
The MHP model and ReWSR model are introduced in detail in this part. These two models were originally proposed in [
The BCS has been proved to be effective in CS [
This overlapping block method can effectively eliminate image artifacts and suppress possible block effects. The band X is divided into
This section mainly introduces the second and third phase reconstructions in the Three-stage HSICS algorithm. In fact, the second and third stages are both based on the two models introduced in the previous section. The difference is that the second stage is intra-band reconstruction and the third stage is inter-band reconstruction. That is, the second phase searches for similar blocks in a single band, and the third stage searches for similar blocks from a subset of bands.
The bands in the first phase are initially reconstructed. The first and last two bands of the band subset are selected as the key bands, and the SR of the key band is set to 0.7. The selection of key bands is used to increase the information of the band subset after compressing. Through the first stage of the algorithm, there is a strong correlation between the selected optimal band subsets. However, in order to improve more reconstruction accuracy of HSI, we set two bands as key bands and perform a smaller compression ratio, which can retain more information of band subset. In fact, the key bands can be any two bands in the subset, which has little effect on the performance of the algorithm.
When reconstructing, the images of each band are reconstructed in sequence. In the case of multi-hypothesis prediction, similar blocks are found from within the band image. For the method of searching for similar blocks within the band, we refer to the intra-frame reconstruction method of video compressed sensing in [
In the third stage, the initial image reconstructed is reconstructed again. When the band
The calculation of the optimal weight in the third stage, uses the steepest descent method. The steepest gradient descent method is given in
The ReWSR model |
---|
4) |
5) |
6) End |
We will research and verify the advantages of the three-stage HSICS through three experiments. The comparative experiments were carried out on 3 public HSI datasets, Indian pines (IN), Salinas (SA) and Pavia University (PU). The simulation experiment environment is Intel(R) Core (TM) i7-10710U CPU @ 1.10 GHz, 1.61 GHz, 16.0 GB.
The IN dataset consists of 224 spectral reflection bands, and it has 145145 pixels in every band. The SA dataset consists of 224 spectral reflection bands, which cover 83 × 86 pixels in every band. The PU dataset has 115 spectral reflection bands and its size includes 610 × 340 pixels. The datasets is
tailored to meet the experimental needs.
Stage 1 | Stage 2 | Stage 3 | |||||||
---|---|---|---|---|---|---|---|---|---|
IN | SA | PU | IN | SA | PU | IN | SA | PU | |
200 | 204 | 115 | \ | \ | |||||
50 | \ | \ | |||||||
\ | 10 | ||||||||
\ | 6*6 | 9*9 | 11*11 | 6*6 | 9*9 | 11*11 | |||
20 | 10 | 40 | |||||||
100 | 30 | 30 | 30 | 20 | 20 | 20 |
This article sets up three comparison experiment. In the first stage, four advanced inspire-algorithms are used to solve on Mop-BS. In the second and third stages, this article takes IN and SA as examples, and conducts experiments in four cases of sampling 0.2, 0.3, 0.4, and 0.5. Finally, we compare with three HSI-CS algorithms to verify the performance of the algorithm proposed in this article.
We modeled the HSI band selection as a multi-objective combination optimization model, and conducted experiments using NSGA-III [
Indicators | M | GrEA | KnEA | NSGAIII | ISDENew |
---|---|---|---|---|---|
HV | 3 | 5.5725e-2 (1.61e-2) + | 3.1009e-2 (1.14e-2)- | 2.0526e-2 (4.90e-3)- | 3.3981e-2 (6.77e-3)- |
IGD | 3 | 1.9586e-1 (4.31e-2) + | 2.7364e-1 (4.60e-2)- | 3.1606e-1 (2.52e-2)- | 2.6202e-1 (2.57e-2)- |
+/-/≈ | 0/2/0 | 0/2/0 | 0/2/0 |
From
Then, we randomly select one of the solution sets for the next two stages. The optimal band subsets of the three data sets are given in
[27, 28, 30, 31, 44, 45, 46, 55, 70, 120] | |
---|---|
[41, 46, 58, 59, 61, 65, 76, 117, 198] | |
[21, 35, 42, 53, 60, 70, 83, 97, 101] |
In
In the second stage, we used the band subset selected in the first stage to perform intra-band reconstruction. On the IN dataset, the optimal band subset includes ten bands, and the 27th and 120th bands were regarded as the key bands, which are sampled as 0.7. On the SA dataset, there are nine bands selected as the optimal band subset, and the 41th and 198th bands are regarded as the key bands, which are sampled as 0.7. On the PU dataset, nine bands were selected as the optimal band subset, and the 21th and 101th bands were regarded as the key bands, which are sampled as 0.7. We will introduce the simulation results on the three data sets below.
In the second phase, the iterations are set to 20.
The SR ranges from 0.2–0.5, and the PSNR of the restored image is shown in
27 key | 28 | 30 | 31 | 44 | 45 | 46 | 55 | 70 | 120 key | |
---|---|---|---|---|---|---|---|---|---|---|
0.2 | 35.31 | 25.48 | 24.74 | 24.41 | 26.64 | 26.82 | 26.36 | 25.59 | 27.24 | 42.75 |
0.3 | 35.31 | 27.67 | 26.68 | 26.35 | 28.72 | 28.78 | 28.43 | 27.51 | 29.04 | 42.75 |
0.4 | 35.31 | 29.45 | 28.44 | 28.16 | 30.63 | 30.73 | 30.37 | 29.28 | 31.54 | 42.75 |
0.5 | 35.31 | 31.35 | 30.35 | 30.06 | 32.35 | 32.43 | 31.98 | 31.09 | 32.98 | 42.75 |
The SR ranges from 0.2–0.5, and the PSNR of the restored image is shown in
41 key | 46 | 58 | 59 | 61 | 65 | 76 | 117 | 198 key | |
---|---|---|---|---|---|---|---|---|---|
0.2 | 44.19 | 31.61 | 31.68 | 31.90 | 32.52 | 32.04 | 30.13 | 31.54 | 50.39 |
0.3 | 44.19 | 31.35 | 30.35 | 30.06 | 32.35 | 32.43 | 31.98 | 31.09 | 50.39 |
0.4 | 44.19 | 35.40 | 35.51 | 35.68 | 37.25 | 37.14 | 33.50 | 34.50 | 50.39 |
0.5 | 44.19 | 41.05 | 41.16 | 41.26 | 41.36 | 41.88 | 38.86 | 38.28 | 50.39 |
The SR ranges from 0.2–0.5, and the PSNR of the restored image is shown in
21key | 35 | 42 | 53 | 60 | 70 | 83 | 97 | 101 key | |
---|---|---|---|---|---|---|---|---|---|
0.2 | 43.06 | 31.87 | 31.87 | 31.65 | 31.52 | 31.65 | 28.72 | 28.98 | 39.73 |
0.3 | 43.06 | 34.29 | 34.27 | 34.08 | 33.98 | 34.05 | 31.18 | 31.62 | 39.73 |
0.4 | 43.06 | 36.46 | 36.34 | 36.09 | 35.95 | 35.99 | 33.26 | 33.74 | 39.73 |
0.5 | 43.06 | 38.38 | 38.27 | 38.00 | 37.73 | 37.76 | 34.95 | 35.46 | 39.73 |
In the third stage, we use the initial image reconstructed in the second stage. During the reconstruction, firstly, the algorithm searches for images with a higher reconstruction quality than the current band in the initial restored image, and weight coefficients are added to these images. The key band is set to ensure that a band with a higher reconstruction quality than the current band can be searched.
In the third stage, the band with the best reconstruction quality is the 70 band in
27 key | 28 | 30 | 31 | 44 | 45 | 46 | 55 | 70 | 198 key | |
---|---|---|---|---|---|---|---|---|---|---|
0.2 | 35.31 | 27.34 | 26.15 | 26.03 | 27.79 | 27.92 | 27.57 | 26.66 | 28.18 | 42.75 |
0.3 | 35.31 | 29.80 | 28.25 | 28.26 | 30.08 | 30.20 | 29.91 | 28.94 | 30.50 | 42.75 |
0.4 | 35.31 | 31.80 | 30.24 | 30.25 | 32.29 | 32.37 | 32.08 | 30.97 | 32.81 | 42.75 |
0.5 | 35.31 | 33.95 | 32.28 | 32.26 | 34.14 | 34.21 | 33.94 | 32.99 | 34.72 | 42.75 |
Similarly, the band with the best reconstruction quality is the 65 band in
41 key | 46 | 58 | 59 | 61 | 65 | 76 | 117 | 198 key | |
---|---|---|---|---|---|---|---|---|---|
0.2 | 44.19 | 32.00 | 32.07 | 32.64 | 32.76 | 32.88 | 30.89 | 31.73 | 50.39 |
0.3 | 44.19 | 33.95 | 32.28 | 32.26 | 34.14 | 34.21 | 33.94 | 32.99 | 50.39 |
0.4 | 44.19 | 39.70 | 39.87 | 41.00 | 40.13 | 41.03 | 36.99 | 37.36 | 50.39 |
0.5 | 44.19 | 41.35 | 41.50 | 42.30 | 41.78 | 42.68 | 39.28 | 38.71 | 50.39 |
The band with the best reconstruction quality is the 97 band in
21 key | 35 | 42 | 53 | 60 | 70 | 83 | 97 | 101 key | |
---|---|---|---|---|---|---|---|---|---|
0.2 | 43.06 | 35.53 | 34.2 | 33.63 | 33.41 | 32.93 | 33.83 | 35.35 | 39.66 |
0.3 | 43.06 | 37.29 | 35.9 | 35.26 | 34.97 | 35.25 | 35.51 | 37.15 | 39.66 |
0.4 | 43.06 | 38.84 | 38.47 | 37.78 | 38.48 | 37.66 | 37 | 38.61 | 39.66 |
0.5 | 43.06 | 40.47 | 39.07 | 38.39 | 39.08 | 38.29 | 39.4 | 41.02 | 39.66 |
Next, we compared four hyperspectral image compressed sensing algorithms, MT-BCS [
In
To sum up, taking SR = 0.5 as an example, we compare the average improved PSNR from Stage 2 to Stage 3 of the three datasets, where IN is 1.99, SA is 0.54, and PU is 2.16. This also reflects a drawback of our work. When the size of the dataset is small, the information obtained by the algorithm through inter-band and intra-band will be less, which leads to the performance of the MHP model and the ReWSR model, which reduces the accuracy of the final reconstructed image.
A three-stage HSICS algorithm is proposed to recover HSI. In the first phase, we suggest an HSI band selection model, which makes the HSI band selection as Mop-BS, and uses advanced inspire-algorithms to solve it. In the second and third phases, this paper improves the MHP model and ReWSR model, optimizes the block indexing method in the BCS of HSI. Through the most similar block, intra-band and inter-bands. The similar blocks are modeled with residuals, and the weights are re-iterated. In the HSICS, the algorithm can effectively improve the reconstruction accuracy from experience results.
In the future, we will consider modifying the weights of the ReWSR model and designing more advantageous algorithms to optimize the weights to fit the relationship of band similarity and further improve the performance of HSICS reconstruction. On the other hand, we will consider reducing the time consumption in HSICS, and improving the practicality of the algorithm.