In order to solve the rainfall estimation error caused by various noise factors such as clutter, super refraction, and raindrops during the detection process of Doppler weather radar. This paper proposes to improve the rainfall estimation model of radar combined with rain gauge which calibrated by common Kalman filter. After data preprocessing, the radar data should be classified according to the precipitation intensity. And then, they are respectively substituted into the improved filter for calibration. The state noise variance
In recent years, various industries in our country have highly affirmed the economic and social values contained in meteorological data. Meteorological early warning and forecasting play a vital role in the development of all walks of life in our country. With the innovation and development of meteorological technology, the requirements of meteorological users for weather forecasting and early warning products are also increasing [
With the continuous improvement of remote sensing technology, the use of radar to complete meteorological work has become one of the more common and respected methods nowadays. The echo pictures, data, charts and other information of weather phenomena obtained by Doppler radar detection provide important data basis for short-term or very short-term weather forecasts of meteorological forecasting and early warning research departments. It is a very good tool for sexual weather inspection, forecasting and early warning [
However, the meteorological data information obtained in actual measurement is often mixed with other noises. If you want to obtain accurate weather forecasts from the data with noise interference, you need to eliminate useless noise interference as much as possible [
So far, Kalman filter has been well applied in the field of meteorological business and has achieved certain success. In 2004, Zheng et al. [
The Doppler radar measurement obtains the radar reflectivity factor Z that effectively illuminates the precipitation particles in the body. Its value is directly related to the droplet spectrum, and the droplet spectrum directly determines the precipitation intensity I, which means that there is also a certain correlation between the radar reflectivity factor Z and the precipitation intensity I [
Precipitation depends on the geometry, number density, and phase state of the precipitation particles within the detection range. The intensity of precipitation depends on the number density and the final velocity of the precipitation particles. The geometry and form of the precipitation particles directly affect the falling speed of precipitation particles. Therefore, the principle of Doppler weather radar forecasting precipitation is to measure the intensity of precipitation I in the relevant area or a single point through the echo intensity Z received by the Doppler radar.
The relationship between Z-I is shown in the following formula
In formula
However, due to the distribution of raindrop spectrum is different for different precipitation types and different regions, the coefficients A and b also have large changes. In addition, because the method of using the Z-I relationship to estimate the precipitation completely relies on the radar reflectivity factor Z measured by the Doppler radar, and various external factors such as the height and distance measured by the radar lead to the Z cannot be able to fully objectively reflect the actual precipitation situation. Therefore, we can conclude that although the method of realizing rainfall estimation through the Z-I relationship is simple and easy, the accuracy of the rainfall estimation is not high.
In order to solve the problem that the radar rainfall estimation accuracy obtained by the Z-I relation method is not high. Somebody proposed to calibrate it. At present, the methods to calibrate Doppler weather radar rainfall estimation are mainly divided into two categories. The first category is calibration without a rain gauge, which is mainly to calibrate the coefficients A and b according to the above-mentioned Z-I relationship to obtain the best coefficient value. The second category is calibration with rain gauges. This method of real-time calibration using rain gauges includes single-point calibration, average calibration, distance-weighted calibration and Kalman filter calibration, etc. This method makes up for the deficiency that the precipitation intensity obtained by using Doppler weather radar alone is affected by the uncertainty of the Z-I relationship and radar parameters.
In the study of rainfall estimation using Doppler weather radar combined with rain gauge, it is usually necessary to select a set of values of A and b based on experience. Normally, the Z-I relationship is set to [
The precipitation rate can be calculated according to the Z-I relationship. And the initial value field is reversed. We can obtain the calibration factor f from the rain gauge data of the automatic weather station. The product of the calibration factor f and the initial value field is used as the final rainfall estimate.
The establishment of automatic weather stations can help people better and more accurately grasp the changes of weather. They can automatically detect multiple elements according to the needs of a certain area without manual intervention, and automatically generate messages to transmit the detected data to the central station regularly [
The establishment of automatic weather stations provides data in places that are difficult for people to access or are not suitable for living, enhances the density of the observation station network and can provide some measurement data outside the normal manual measurement time. What's more, automatic weather stations can effectively increase the reliability of observation data by new technologies. Compared with manual measurement, it can effectively reduce human error. To a large extent, the consumption and expenditure of human resources for measuring data are reduced.
At present, the construction of automatic weather stations around the world has been very extensive and mature. Corresponding weather stations are also set up in various regions of our country. As shown in
Kalman Filtering is an estimation method first proposed by the mathematician R.E.Kalman for the problem of stochastic process state estimation [
So far, Kalman filter has been widely used in robot navigation, target tracking, signal processing, aerospace and other fields. In the rainfall estimation, Kalman filtering is also an effective means to eliminate random interference and improve the accuracy of weather forecasting [
The Kalman filter algorithm defines the state variable as X and the observation variable as Z. The state of the system is as follows
The state equation is to infer the current state based on the state and control variables at the previous moment, where A which is an
The Kalman filter observation equation is as formula
The realization of the Kalman filter algorithm mainly includes two parts: the prediction process and the correction process [
The formula for the prediction process is shown in
The correction process formula is as follows
Aiming at the error problem between the rainfall estimation result obtained by ordinary Kalman filter and the actual precipitation, we decide to improve on the basis of the original Kalman filter calibration radar combined with rain gauge rainfall estimation model [
First of all, after the initial quality control and data processing of the Doppler weather radar data, it will no longer be directly combined with the pre-processed automatic weather station rain gauge data and then calculate the deviation calibration factor according to formula
In order to effectively improve the accuracy of rainfall estimation, the common Kalman filter algorithm needs to be improved. When calibrating radar rainfall estimation data, for the convenience, the two parameters of Kalman filter (state noise variance
The overall technical route of the radar rainfall estimation model based on improved Kalman filter is shown in
When using the ordinary Kalman filter algorithm to process the rainfall estimation of Doppler radar data, we can know from the literature [
In formulas
In general, for calculation simplicity, the state noise variance
Therefore, in order to better adapt to the more complicated rainfall estimation in the precipitation process, we decide to optimize the values of the state noise variance
It is conceivable that the value and update of the state noise variance
In view of the large amount of calculation mentioned above and the deviation of the value update process, we decide when estimating the precipitation, the data will be substituted into the improved Kalman filter calibrated rainfall estimation model batch by batch according to the result of the above classification which depends on the precipitation intensity. Take several sets of data for each type of deviation calibration factor into the improved Kalman filter that can adaptively calculate and update the state noise variance
The Kalman filter is adaptive, which means that the ordinary Kalman filter algorithm can automatically calculate and update the state noise variance
The semi-adaptive Kalman filter is to first calculate the parameter value of each step based on the values of the previous observation data. The current calibrated optimal state noise variance
Define the “innovation” sequence according to the theory of Kalman filtering:
According to formula
Combine it with the state
From the basic principle of Kalman filter, we know that
D represents variance. According to the deviation calibration factor at time k − 1, the deviation value of the deviation calibration factor at time k is estimated as:
The error variance of the forecast bias is:
The variance of the “innovation” sequence
According to the theoretical characteristics of the Kalman filter algorithm, we can know:
Simultaneous formulas
According to formulas
So far, we have basically realized the improvement of the ordinary Kalman filter algorithm. The traditional Kalman filter algorithm is adaptive and the two parameters used in the filtering process can be calculated according to the input calibration deviation to obtain the optimal parameter assignment for the subsequent filtering observation data.
First, we use the Doppler weather radar data from Nanjing, Jiangsu Province on August 16, 2021 and the data from the automatic weather station in Nanjing, Jiangsu Province at the same time period to analyze the influence of the value of the Kalman filter parameter state noise variance Q on the filtering results [
Subsequently, several times of the same intensity radar data obtained from the same site in Nanjing, Jiangsu Province and the data from the automatic weather station in Nanjing, Jiangsu Province during the same period were used to analyze the experimental results of the rainfall estimation based on the improved semi-automatic Kalman filter algorithm.
The maximum observation data range of Nanjing Doppler weather radar echo is 230 km, and the main data extracted is the scan data with an elevation angle of 1.5 that is completed every 6 min. The automatic weather station data mainly extracts the measured data statistics of the precipitation per hour in Nanjing.
In the rainfall estimation based on the improved Kalman filter, X is the one-dimensional variable precipitation, and the variances of
The observed noise variance R is also called the measurement noise variance, which is a statistical parameter. Its value is closely related to the accuracy of the data returned by the Doppler weather radar. The measurement variance can be obtained through long-term probability statistics on the data measured. In the experiment to test whether the parameter Q has an effect on the rainfall estimation result, the measurement noise variance R is set to 0.1.
It is difficult to obtain accurate data for the process noise variance Q. Therefore, in this paper, a comparative experiment is carried out on the value of the process noise parameter Q suitable for the rainfall estimation model.
Take the Doppler weather radar data on August 16, 2021 as sample data to analyze and determine the optimal value of the process noise variance Q. In this experiment, when other parameter conditions are the same, Kalman filter is performed on the parameter Q with values of 0.01, 0.1, 0.25, 0.5, and 0.75. The filtering results are shown in the following
Taking the measured precipitation at two points as reference in the above figure, and the specific data statistics are shown in
Parameter Q | Time 1 error/mm | Time 2 error/mm | Mean error/mm |
---|---|---|---|
Unfiltered | 0.2000 | 0.3000 | 0.25000 |
0.01 | 0.2575 | 0.5970 | 0.42725 |
0.1 | 0.0810 | 0.2450 | 0.16300 |
0.25 | 0.0290 | 0.2580 | 0.14350 |
0.5 | 0.0950 | 0.2720 | 0.18350 |
0.75 | 0.1240 | 0.2790 | 0.20150 |
We can see that the assumed value of the parameter value Q has a significant impact on the results of the rainfall estimation. The best value of the state noise variance Q under different conditions at different times will also change to a certain extent as time changes. Therefore, we can conclude that
For the ever-changing and complex weather in practical applications, its parameters cannot always be consistently adapted to various environments and time periods. Therefore, the radar precipitation in the area is subjected to an improved semi-adaptive Kalman filter, and the calibrated results are compared and analyzed with the actual measurement value of the automatic weather station rain gauge in the area.
In this experiment, Several Doppler weather radar data of the same type of precipitation intensity in Nanjing City, Jiangsu Province were selected as sample data to be substituted into the rainfall estimation model for experimentation. First, the radar data obtains the rainfall estimation result according to the Z-I relationship, that is, the rainfall estimation data of the unfiltered wave. Compare the intensity of the unfiltered rainfall estimation with the actual measured value of the rain gauge in the area to obtained f which will be substituted into the rainfall estimation model based on the traditional Kalman filter and the improved Kalman filter.
In the experiment, the parameters of the rainfall estimation based on the traditional Kalman filter are set to the optimal value obtained from the above experiment. Take the data obtained from the automatic weather station rain gauge in the area as the actual precipitation. The final comparison of experimental results is shown in
Data order | Unfiltered | Traditional Kalman filter | Improved semi-adaptive Kalman filter | Rain gauge | |||
---|---|---|---|---|---|---|---|
Estimated value/mm | Relative error | Estimated value/mm | Relative error | Estimated value/mm | Relative error | Measured value/mm | |
202101 | 1.90 | 13.6% | - | - | - | - | 2.20 |
202102 | 1.80 | 10% | 1.9812 | 0.94% | 1.9782 | 1.09% | 2.0 |
202103 | 1.00 | 9.1% | 1.1018 | 1.64% | 1.0990 | 0.09% | 1.1 |
202104 | 4.00 | 4.8% | 4.4510 | 5.98% | 4.4430 | 5.79% | 4.2 |
202105 | 1.30 | 7.1% | 1.4313 | 2.24% | 1.4287 | 2.05% | 1.4 |
202106 | 2.20 | 4.3% | 2.4222 | 5.31% | 2.4178 | 5.12% | 2.3 |
202107 | 5.30 | 1.9% | 5.8350 | 8.06% | 5.8247 | 7.86% | 5.4 |
202108 | 2.08 | 5.5% | 2.2900 | 4.09% | 2.2850 | 3.86% | 2.2 |
202109 | 2.80 | 9.7% | 3.0828 | 0.55% | 3.0772 | 0.74% | 3.1 |
202110 | 2.60 | 7.1% | 2.8626 | 2.24% | 2.8574 | 2.05% | 2.8 |
Mean error | 0.1800 | 7.31% | 0.0620 | 3.45% | 0.0567 | 3.18% | Mean |
2.67 |
From the data in the table, we can see that the ordinary Kalman filter has significantly improved the rainfall estimation results. The average relative error of the rainfall estimation drops to 3.45%. Compared with the unfiltered rainfall estimation error, it only accounts for 47.2% of its average relative error value. The improved semi-adaptive Kalman filter's average relative error is further reduced to 3.18%, which only accounts for 43.5% of the precipitation estimate without filter wave calibration. It can be seen that the improved semi-adaptive Kalman filter algorithm has further improved the accuracy of radar rainfall estimation based on the calibration error of the ordinary Kalman filter.
In this paper, we propose a rainfall estimation model based on improved semi-adaptive Kalman filter to calibrate the rainfall estimation results, and realize the optimization of the state noise variance
This work was supported by the National Natural Science Foundation of China (Grant No. 42075007), the Open Grants of the State Key Laboratory of Severe Weather (No. 2021LASW-B19).