The Role of Deep Learning in Parking Space Identification and Prediction Systems

1  Introduction

Smart city promises to achieve sustainable development goals, particularly the sustainable cities and communities goal, which focuses on making cities and human settlements inclusive, safe, resilient, and sustainable. The evolution of the Internet of Things (IoT) and big data has been playing a vital role in the feasibility of smart city initiatives [1–3]. Finding an available parking space is one of the most challenging tasks in today’s smart city transportation system [4]. According to [5], approximately 40% of the traffic congestion in cities is caused by vehicles seeking parking spaces. Consequently, traffic congestion, pollution emission, and traffic accidents are becoming severe. Identifying parking spaces using real-time parking occupancy data provided by sensors has been proposed; however, identification alone is insufficient because an available parking space identified for a vehicle may have been taken by another vehicle when it arrives, resulting in the driver’s frustration. Searching for another parking space can aggravate traffic jams. This explains the need to predict the availability of parking spaces [6,7].

Research has been undertaken to predict and identify the availability of parking spaces within a short time to facilitate drivers to find parking spaces efficiently. Deep learning (DL) has been the preferred approach. Traditionally, DL has a deep neural network (DNN), such as multilayer perceptron (MLP), and it comprises three main types of layers, namely the input, hidden, and output layers [8,9]. Fig. 1 presents an example of the identification of the availability of parking spaces in open parking lots using the DL approach. DL, specifically a convolutional neural network (CNN), identifies whether or not a vehicle is present in each parking space. There are various DL architectures applied to parking identification and prediction systems (see Section 2.1).

Figure 1: An example of the identification of the availability of parking spaces in open parking lots using DL [10]. The green and red rectangles show the availability and unavailability of parking spaces

1.1 Impact of Parking Identification and Prediction Systems on Smart Cities

Improving mobility and transportation is one of the major initiatives with significant contributions to the successful implementation of smart cities. Drivers looking for a parking space may cause traffic congestion, so parking identification and prediction systems are related to route planning, traffic management, and parking space management. According to [11], drivers who use information related to the availability of parking spaces in making decisions are 45% more successful in availing of parking spaces than those without. Hence, parking identification and prediction systems have significant impacts on the efficiency of smart cities. DL has been achieving promising results in the identification and prediction of the availability of parking spaces [12–18], which makes significant impacts on smart cities.

1.2 Potential Applications of Parking Identification and Prediction Systems

Parking space identification and prediction systems can be applied in a variety of applications. Parking space identification has been applied in: (a) autonomous valet parking systems to find a parking space without human involvement [19]; (b) traffic light control systems to identify traffic density on roads [20]; (c) autonomous electric cars to seek parking spaces equipped with the charging facility [21]; (d) smart cars or robots to seek safer places during a disaster (e.g., flood) [22]; (e) smart port to seek available places for cargo storage [23]; and (f) industry 4.0 to provide automatic inventory storage [24,25]. Meanwhile, parking space prediction has been applied in driverless cars to autonomously predict the availability of parking spaces which is useful in route planning [26,27].

1.3 Comparison with Existing Surveys

While there are some surveys in the literature focusing on different aspects of parking systems, this is the first paper presenting a review of DL-based parking identification and prediction systems. In this section, existing surveys in the domain of parking systems are reviewed.

Majority of surveys in the literature focus on smart parking systems rather than DL techniques. For instance, various types of smart parking systems and vehicle detection techniques, which mainly use sensing outcomes from sensors, are presented in [28]. On the other hand, this paper presents a survey of the literature based on four main aspects: (a) the types of available smart parking systems in the literature; (b) the types of vehicle detection techniques used in smart parking systems; (c) methods and algorithms; and (d) three stages of the implementation of smart parking systems, namely proposal, simulation, and implementation. Based on the real-world implementation perspective, the review of prediction models focusing on design factors and enabling technologies (e.g., sensors) has been presented in [29,30]. In [20], smart parking systems are classified, and various sensors used in smart parking systems are presented. Security and privacy are considered in interoperability and data exchange. There are two types of design factors affecting the smart parking system performance: (a) soft design factors include software, data processing, security, and privacy; and (b) hard design factors include communication networks, errors, and reliability. In [21], the architecture of smart parking systems is presented, and their components (e.g., sensors, communication protocols, and software solutions) are explained and compared (e.g., in terms of usage trends). In [4], a survey of smart parking systems focusing on three main topics, namely data collection, parking system deployment, and service dissemination, is presented. Data collection is made through information sensing which depends on sensor connectivity, parking meters, crowdsourcing, and shared parking. Parking system deployment includes software systems, large-scale deployment, and the prediction of available parking spaces. Service dissemination disseminates information based on drivers’ behaviors and their competition in getting a parking space. In [31], a survey of automated valet parking systems focusing on enabling technologies, including platforms, sensors, localization, maps, and environmental models, is presented. A comparison between the Carnegie Mellon platform and the V-charge test vehicle is made covering the features of LIDARs, radars, and cameras. The platforms include 3D geometric and semantic maps, LIDAR- and video-based localizations, and environmental models with static and dynamic obstacles. Finally, in [32], a short survey focusing on enabling technologies to monitor, reserve, and manage dynamic pricing in smart parking systems is presented. The survey includes categories, functionalities, and the latest developments of smart parking systems, as well as a comparison of existing solutions in terms of sensors, deployment, connectivity, services, and processing. None of the surveys focuses on the DL-based parking identification and prediction system, which is the main focus of this paper. In general, this paper discusses DL techniques, datasets, performance metrics, open-source implementations, and open research directions for DL-based parking identification and prediction systems.

Inspired by previous review papers [4,28–31], this paper follows their structures and organizations. In general, the problems (i.e., parking space identification and prediction systems) and general solutions (i.e., DL) are first introduced, followed by the specific description of proposed schemes in the literature. Next, open issues and challenges are identified and elaborated. Finally, concluding remarks are presented. More details about the organization of this paper are presented in Section 1.3. This paper presents a review of state-of-the-art schemes proposed in research papers mainly published from 2019 to 2021.

In short, while general reviews of designing parking management systems using IoT [29,30], information and communication technology [4], algorithms [28] (e.g., static, dynamic, and real-time algorithms), and automated valet parking [31], have been presented, this paper complements the existing works by focusing on the DL approach, particularly on how DL models can be applied to formulate the parking identification and prediction problems, and how the strengths of various DL approaches can provide added advantages in addressing the challenges brought about by parking management systems. Hence, this is the first review paper that provides a systematic and extensive synthesis, analysis, and summary of DL approaches applied to parking identification and prediction problems. This helps to identify research gaps in existing works and explore future research directions.

1.4 Novelty, Contributions, and Organization of this Paper

Based on the literature search, there is a lack of research papers focusing on DL approaches applied to solving parking identification and prediction problems. This paper is novel in two ways. Firstly, this is the first review paper that extensively covers the application of DL approaches to parking systems to solve identification and prediction problems by presenting a comprehensive taxonomy. Secondly, this paper identifies various research gaps and challenges that open doors for future research and advancements in the area of parking identification and prediction systems.

Hence, the contributions of this work are as follows:

•   The roles played by DL in parking identification and prediction systems are highlighted, and a review of recent premier advances in DL approaches applied to predict and identify the availability of parking spaces is presented.

•   A taxonomy is devised to categorize and classify proposed schemes in the literature.

•   Open-source implementations of DL-based approaches for parking identification and prediction systems are presented.

•   Open research challenges are identified and elaborated.

This paper is organized as follows. Based on DL-based parking identification and prediction systems, Sections 2 and 3 present a taxonomy and the role of DL, respectively. Section 4 presents recent works. Section 5 presents open-source implementations. Section 6 presents open issues and research challenges. Section 7 concludes this paper.

2  Taxonomy of DL-Based Parking Identification and Prediction Systems

Fig. 2 presents the taxonomy explained in the rest of this section.

Figure 2: Taxonomy of DL-based parking identification and prediction systems

2.1 DL Architectures

There are seven architectures for DL-based parking identification and prediction systems as follows:

A.1   Multilayer perceptron (MLP) provides a continuous function to process data for performing pattern classification, recognition, prediction, and approximation tasks [9,18]. MLP has been adopted in [13] to predict the availability of parking spaces. In [13], the MLP architecture consists of an input layer with four neurons, three fully-connected (FC) hidden layers with five neurons each, and an output layer with five neurons as shown in Fig. 3. Data flows from the input layer to the output layer.

A.2   Convolutional neural network (CNN) uses convolutional layers to process visual data for performing image processing, classification, and segmentation tasks [33]. CNN has been adopted in [12] to identify the availability of parking spaces. CNN has two types of layers, namely the convolutional layer and the traditional FC layer. In [12], the CNN architecture consists of an input layer, three convolutional layers, an FC layer, and an output layer as shown in Fig. 4. Each convolutional layer consists of convolution, pooling, and the activation function, such as rectified linear unit (ReLU). Data flows from the input layer to the output layer.

A.3   Region-based convolutional neural network (RCNN), which is based on CNN (A.2), provides region proposals from images to perform object detection efficiently [34,35]. The region proposals identify a set of candidate detections present in an image, such as a car and a truck in an image. The RCNN architecture consists of an input layer, region proposals, a CNN architecture, the FC layer, which is split into two separate streams, and an output layer for the two streams as shown in Fig. 5. The input layer represents an image, which is first divided into approximately 2,000 region proposals, and then CNN is applied to each region respectively. Subsequently, bounding boxes, which are rectangles that serve as points of reference to detect objects, are generated, and objects, such as a car or a truck, are classified in the output layer. RCNN has been adopted in [18] to identify the availability of parking spaces. Data flows from the input layer to the output layer.

A.4   Graph neural network (GNN) provides graphs to process data and perform node-, edge-, and graph-level prediction tasks [36]. GNN has been adopted in [15] to identify the availability of parking spaces. In [15], the GNN architecture consists of an input layer, two hidden layers, and an output layer, as shown in Fig. 6. There are two parts in each hidden layer, namely graph and activation function (i.e., ReLU). Each graph is represented by vertices and links, whereby each link is either directed or undirected between the vertices. In each graph, each vertex learns information about the vertices in its neighborhood. Data flows from the input layer to the output layer.

A.5   Recurrent neural network (RNN) provides an internal memory to store previous inputs and perform sequence prediction tasks. However, RNN can process short sequences only. The traditional RNN architecture consists of an input layer It, hidden layers H, and an output layer Ot as shown in Fig. 7. In Fig. 7, the input I0 is processed to generate output O0 in the current time step. Subsequently, the output O0 is combined with the input I1 in the next time step. The output O1 is combined with the input I2 in the subsequent time step and so on. By doing so, the current input and output are obtained based on the previous input during training. RNN has been adopted in [37–39] to predict the availability of parking spaces. Data flows from the input layer to the output layer.

A.6   Long short-term memory (LSTM), which is based on RNN (A.5), also provides memory with gates to store previous inputs efficiently and perform sequence prediction tasks [40]. LSTM is suitable for processing long sequences. The LSTM architecture consists of an input layer, an LSTM layer, and an output layer as shown in Fig. 8. The LSTM layer has a memory cell that consists of an input gate g, an output gate h, two multiplication nodes X, a summation node +, and two activation function gates δ. The multiplication and summation nodes perform multiplication and addition computation tasks, respectively. The activation function gates transform data into a value within the range of 0 and 1. LSTM has been adopted in [16] to predict the availability of parking spaces. Data flows from the input layer to the output layer.

A.7   Dilated convolutional neural network (DCNN), which is based on CNN (A.2), replaces the simple convolutional layer with the dilated convolutional layer in its architecture. The dilated convolutional layer expands the input (or kernel) by inserting holes between its consecutive elements. Compared to the traditional CNN (A.2) architecture, DCNN has a larger receptive field covering a large area of input images. The difference between a simple convolutional layer and a dilated convolutional layer is depicted in Fig. 9. DCNN has been adopted in [41] to identify the availability of parking spaces.

Figure 3: An example of an MLP architecture [13]

Figure 4: An example of a CNN architecture [12]

Figure 5: Traditional RCNN architecture consists of a CNN architecture (see Fig. 4) [34]

Figure 6: An example of a GNN architecture [15]

Figure 7: Traditional RNN architecture [39]

Figure 8: Traditional LSTM architecture [40]

Figure 9: The illustration of the convolutional and dilated convolutional layers [42]. The dilated convolutional layer has a larger receptive field

2.2 Systems

There are two systems used to find the availability of parking spaces in the context of DL as follows:

S.1   The identification system identifies and updates the availability of parking spaces in a real-time manner. For instance, the system identifies an available parking space when a vehicle arrives at a parking lot [12,15].

S.2   The prediction system predicts the availability of parking spaces. For instance, the system predicts the availability of every single parking space in a parking lot [13].

2.3 Datasets

A parking lot is a reserved area with parking spaces, which are slots for vehicles to park. There are eleven main parking lot datasets used in the literature to evaluate and validate the proposed schemes based on DL, which are as follows:

D.1   PKLot1 [10] is a dataset composed of 12,417 images of parking lots and 695,899 images of parking spaces, which were captured using three different camera views at two parking lots. These parking lots were located at the Pontifical Catholic University of Parana (PUCPR) and the Federal University of Parana (UFPR) in Curitiba, Brazil.

D.2   PS2.02 [43] is a dataset composed of 9,827 training images of parking lots and 9,476 images of parking spaces. It has 2,338 testing images of parking lots and 2,168 images of parking spaces. The parking lot is located at Tongji University, Shanghai, China.

D.3   PSV is a dataset composed of 4,200 images of parking spaces. The parking lot was located at Tongji University in Shanghai, China.

D.4   Santander’s dataset is a dataset composed of the measurements collected by almost 400 parking sensors deployed in the main parking lot of Santander in Spain.

D.5   Sii-Mobility [37] is a dataset collected from 12 parking spaces located at three different locations (i.e., downtown, hospitals, and outskirts such as parks and ride systems) within 81 days between 5th January 2017 and 26th March 2017 in Florence, Italy. The status of each parking space was updated every 15 min.

D.6   CNRPark + EXT3 [12] is a dataset composed of around 150,000 labeled images of 164 available and unavailable parking spaces in a parking lot. The images were collected using 9 cameras under different weather conditions from November 2015 to February 2016.

D.7   Birmingham [38] is a dataset composed of the occupancy of 29 parking spaces operated by National Car Parks in Birmingham, UK. Data was updated every 30 min from 8:00 am to 4:30 pm. The dataset is licensed under the Open Government Licenses v3.0.

D.8   San Francisco4 is a dataset composed of the sensing outcomes of 8,200 parking sensors deployed at on-street metered parking lots and gate-controlled off-street locations in San Francisco. The dataset was collected from 12th August 2013 to 22nd September 2013, and it was updated every 15 min. There are 3,948 records for each of the 572 street blocks totaling 3,948 × 572 = 2,258,256 records.

D.9   Melbourne [44] is a dataset composed of the sensing outcomes of 7,114 parking sensors (i.e., in-ground sensors) deployed along 75 streets (or around 270 street blocks) throughout 24 areas in the Melbourne central business district. The dataset was collected from 1st October 2011 to 30th September 2012 totaling 12,208,178 records of parking events.

D.10   Santa Monica5 [45] is a dataset collected from 9 parking lots in Santa Monica, California from 7:00 am on 11th May 2018 to 9:50 pm on 11th June 2018. Data was collected every 5 min.

D.11   Self-created dataset is a dataset composed of a large number of images or measurements of sensors, which were generated and annotated manually rather than using any publicly available datasets.

2.4 Performance Measures

There are eight performance measures achieved by various DL approaches to predict and identify the availability of parking spaces. The self-explanatory equations for the performance measures are as follows:

P.1   Accuracy

Accuracy=npns(1)

where np represents the total number of correct predicted values, and ns represents the total

P.2   Precision

Precision=TpTp+Fp(2)

where Tp represents the number of true positive values, and Fp represents the number of false positive values.

P.3   Recall

Recall=TpTp+Fn(3)

where Fn represents the number of false negative values.

P.4   Mean absolute error (MAE)

MAE=∑|Pv−Av|ns(4)

where Pv represents the predicted value, and Av represents the actual value.

P.5   Root mean square error (RMSE) is a quadratic scoring function as follows:

RMSE=∑(Pv−Av)2ns(5)

P.6   Mean absolute percentage error (MAPE)

MAPE=1ns∑|Pv−Av|Av(6)

P.7   Root relative squared error (RRSE)

RRSE=RMSE∑(Pv−v)2ns(7)

where v represents the last known value.

P.8   Mean absolute scaled error (MASE)

MASE=mean(|Ov−Pv1np−1∑npi=2Ovi−Ovi−1|)(8)

where Ov represents the observed value

3  The Roles of DL Architectures in Parking Identification and Prediction Systems

DL plays an important role in parking identification and prediction systems. There are seven main DL architectures, namely MLP (A.1), CNN (A.2), RCNN (A.3), GNN (A.4), RNN (A.5), LSTM (A.6), and DCNN (A.7), applied to identify (S.1) and predict (S.2) the availability of parking spaces in the literature. The DL architectures are essential to cater to the continuous and high-dimensional data related to the system. Fig. 10 presents the general framework of DL applied to parking identification and prediction systems. Table 1 summarizes various enhancements of DL architectures to address the challenges of parking identification and prediction systems.

Figure 10: The general framework of DL applied to parking identification and prediction systems. The input has a data processing module processing dataset and raw data. The input is fed into the DL architecture for identification and prediction. The output module provides an available parking space. If there is no parking space available, the car may either look at the same or another parking lot

3.1 The DL Approach Based on the MLP Architecture

The DL approach based on the MLP architecture (A.1) (see Fig. 3) has been applied in [13] to predict parking space availability (S.2). The MLP architecture consists of: (a) the input layer represents input data comprised of the measurements collected by sensors; (b) the hidden layer(s) have FC layers and activation functions (e.g., ReLU); and (c) the output layer predicts the availability of every single parking space in a parking lot.

3.2 The DL Approach Based on the CNN and RCNN Architectures

The DL approach, which is based on the CNN (A.2) (see Fig. 4) and RCNN (A.3) (see Fig. 5) architectures, has been applied in [12,14,17] to identify parking space availability (S.1). The CNN architecture consists of: (a) the input layer represents an input image of a parking lot (or parking spaces); (b) the convolutional layer has the convolution and pooling layers, and activation functions (e.g., ReLU); (c) the FC layers; and (d) the output layer classifies objects, such as a car and an empty parking space.

3.2.1 Enhancements

There are two main enhancements to the RCNN (A.3) (see Fig. 5) architecture. Firstly, region proposals are provided to identify a set of candidate detections present in an image. Region proposals allow the DNN network to identify where to look in an image and detect the exact objects. By using region proposals, DNN enhances the accuracy of object detection. Secondly, the output layer is split into two separate streams to generate bounding boxes and classify objects, such as a car and an empty parking space. This enhancement allows DNN to classify objects accurately and tell DNN where objects are located in an image by generating bounding boxes around objects. In contrast, the traditional CNN only predicts the class of an object. This enhancement enables DNN to increase its accuracy in object classification. While these enhancements increase the accuracy of detection and classification, particularly when the number of classes (e.g., a car, a truck, an empty parking space, and so on) is large, they increase computational cost and processing time. This limitation, which is associated with the RCNN architecture, has been solved using Fast RCNN although it has not been investigated in the context of parking identification and prediction systems. Compared with RCNN, the Fast RCNN architecture has convolutional layers before the region proposals [46]. Fast RCNN architecture uses convolutional, region proposals, and FC layers. It performs classification and detection tasks with higher speed and accuracy as compared to RCNN [46]. The main objective of Fast RCNN is to reduce computational cost and processing time, and hence it can predict the locations of different objects in an image quickly and accurately.

3.2.2 Applications

The enhancements have been applied to RCNN in [18] to identify parking space availability (S.1), whereby the RCNN architecture consists of five types of layers: (a) the input layer; (b) the region proposals layer divides an input image into approximately 2,000 region proposals and identifies a set of candidates detected in an image; (c) the convolutional layers; (d) the FC layers; and (e) the output layer has two streams to generate the bounding boxes and classify the objects, such as a car and an empty parking space.

3.3 The DL Approach Based on the GNN Architecture

The DL approach based on the GNN architecture (A.4) (see Fig. 6) has been applied in [15] to identify parking space availability (S.1). The GNN architecture consists of: (a) the input layer represents an input image of a parking lot (or parking spaces); (b) the hidden layer(s) have graphs and activation functions (e.g., ReLU); and (c) the output layer classifies objects, such as a car and an empty parking space.

3.4 The DL Approach Based on the RNN Architecture

The DL approach based on the RNN architecture (A.5) (see Fig. 7) has been applied in [38] to predict parking space availability (S.2). The RNN architecture consists of: (a) the input layer represents input data comprised of the measurements collected by sensors; (b) the hidden layer(s) have FC layers and activation functions (e.g., ReLU); and (c) the output layer predicts the availability of every single parking space in a parking lot.

3.5 The DL Approach Based on the LSTM Architecture

The DL approach based on the LSTM architecture (A.6) (see Fig. 8) has been applied in [16] to predict parking space availability (S.2). The LSTM architecture consists of: (a) the input layer represents input data comprised of the measurements collected by sensors; (b) the LSTM layer consists of a memory cell; and (c) the output layer predicts the availability of every single parking space in a parking lot.

3.6 The DL Approach Based on the DCNN Architecture

The DL approach based on the DCNN architecture (A.7) (see Fig. 9) has been applied in [41] to identify parking space availability (S.1). The DCNN architecture consists of: (a) the input layer represents an input image of a parking lot (or parking spaces); (b) the dilated convolutional layer consists of the dilated convolution and pooling layers, and activation functions (e.g., ReLU); (c) the FC layer; and (d) the output layer classifies objects, such as a car and an empty parking space.

4  Recent Works on the Deployment of DL in Parking Identification and Prediction Systems

This section presents a review of some recent works on DL applied to parking identification and prediction systems. Table 2 presents a comparison of the attributes of DL-based approaches in parking identification and prediction systems. Table 3 presents a comparison of key contributions and future directions of parking identification and prediction system investigations.

4.1 Deployment of the MLP Architecture

In [13], a comparative analysis of the MLP approach (A.1) and other machine learning techniques (i.e., K-nearest neighbors, decision tree, random forest, and voting classifier) is presented for the prediction (S.2) of parking space availability. Analysis was conducted for a validity period of 10 to 20 min using the Santander’s dataset (D.4), and it has been shown to achieve a high accuracy rate (P.1), high precision rate (P.2), and high recall rate (P.3) in predicting the availability of parking spaces.

In [47], two types of prediction approaches (S.2) are proposed to predict parking space availability. The first approach is based on the probability that an available parking lot continues to be available in the coming time intervals. The second approach is based on the prediction of the short-term occupancy of parking lots in specific regions. Using MLP (A.1), predictions for the next 1, 5, 15, and 30 min were based on the 1-min parking data of Santander’s dataset (D.4). It has been shown to achieve better MAE (P.4), RMSE (P.5), MAPE (P.6) and RRSE (P.7) in predicting the availability of parking spaces.

4.2 Deployment of the CNN Architecture

In [12], a CNN-based approach (A.2) runs on a Raspberry Pi platform equipped with a smart camera module for the identification (S.1) of parking space availability. The main deployment challenge is the visual classification problem caused by shadows and obstacles (i.e., trees, lamps, or people), which occupy parking spaces. The proposed scheme uses the redundancy of overlapping parking spaces to reduce the effects of obstacles (e.g., trees). Based on the PKLot (D.1) dataset, the approach has been shown to achieve a high accuracy rate (P.1) in identifying the availability of parking spaces.

The work presented in [17] is an extension of [12]. In [17], a CNN-based approach (A.2) acquires local and global information simultaneously and integrates them for the identification (S.1) of parking space availability. The local information includes the orientation and location of junctions in parking lots, and the global information includes the occupancy (e.g., a car or a truck) and types (e.g., a parallel or perpendicular parking space). Based on the PS2.0 (D.2) dataset, the approach achieves a high accuracy rate (P.1), precision rate (P.2), and recall rate (P.3) in identifying the availability of parking spaces.

The work presented in [14] is an extension of [12]. In [14], a CNN-based approach (A.2) converts the empty parking space identification task into a two-step problem: (a) parking space identification; and (b) occupancy classification for the identification (S.1) of parking space availability. The approach also updates the PS2.0 (D.2) and PSV (D.3) datasets by labeling the type of parking space in each image. Based on the datasets, the approach has been shown to achieve a high precision rate (P.2) and recall rate (P.3) in identifying the availability of parking spaces.

Other CNN-based approaches have been proposed. In [48], CNN (A.2) performs the identification (S.1) of parking space availability using the PS2.0 dataset (D.2). It detects the corners and line features of parking lots. It has been shown to achieve a high accuracy rate (P.1) of around 98% in identifying the availability of parking spaces. In [49], CNN (A.2) performs the identification (S.1) of parking space availability using directional marking-point regression (DMPR-PS) and the PS2.0 dataset (D.2). It has been shown to achieve a high precision rate (P.2) and recall rate (P.3) in identifying the availability of parking spaces. In [43], CNN (A.2) performs the identification (S.1) of parking space availability, namely DeepPS. The investigation uses a new large-scale labeled dataset, namely PS2.0 (D.2), which has been released for public use. DeepPS has been shown to achieve a high precision rate (P.2) and recall rate (P.3) in identifying the availability of parking spaces. In [44], CNN (A.2) is incorporated with two other machine learning techniques, namely regression tree and support vector regression, for the prediction (S.2) of parking space availability using the San Francisco (D.8) and Melbourne (D.9) datasets. It has been shown to achieve better MAE (P.4) and RMSE (P.5) in predicting the availability of parking spaces.

4.3 Deployment of the RCNN Architecture

The work presented in [18] is an extension of [17] (see Section 4.2). In [18], the RCNN-based approach (A.3) is suitable for handling generic quadrilaterals and rectangular bounding boxes [12] while detecting parking spaces for the identification (S.1) of parking space availability. Specifically, it can process images of parking spaces, which are in both quadrilateral and rectangular shapes, captured from different angles. Self-created dataset (D.11), which is composed of around 400 bird’s eye view (BEV) images, was used and it has been shown to achieve a high accuracy rate (P.1) and precision rate (P.2) in identifying the availability of parking spaces.

4.4 Deployment of the GNN Architecture

In [15], a GNN-based approach (A.4) uses the link information (e.g., location) of the marking points in an image to identify image features (e.g., empty parking spaces). The marking points (or vertices (A.4)) are represented as graph-structured data, which consists of vertices and links processed using GNN. Based on the PS2.0 (D.2) and PSV (D.3) datasets, the approach has been shown to achieve high precision (P.2) and recall (P.3) rates in identifying the availability of parking spaces.

4.5 Deployment of the RNN Architecture

In [38], an RNN-based approach (A.5) for the prediction (S.2) of parking space availability has been conducted using the Birmingham dataset (D.7). It has been shown to achieve a better MAE (P.4) in predicting the availability of parking spaces. In [37], a comparative analysis of RNN (A.5) and a machine learning technique called support vector regression using the Sii-Mobility dataset (D.5) is performed for the prediction (S.2) of parking space availability. It has been demonstrated that RNN outperforms others and achieves better RMSE (P.5) and MASE (P.8) in predicting the availability of parking spaces.

4.6 Deployment of LSTM Architecture

In [16], an LSTM-based approach (A.6) uses a mobile application to check the live status of parking spaces and book parking spaces in advance for the prediction (S.2) of parking space availability. The approach uses a self-created dataset (D.11), which is comprised of the number of parking spaces, occupancy status, date, and time, and it has been shown to achieve a high accuracy rate (P.1) in predicting the availability of parking spaces. In [45], a DL model, which is based on CNN (A.2) and LSTM (A.6), is proposed to predict (S.2) the availability of parking spaces using the Santa Monica dataset (D.10). By combining two DL architectures (i.e., CNN (A.2) and LSTM (A.6)), the proposed scheme can accurately make short-term (i.e., ≤30 min) and long-term (i.e., >30 min) predictions of the parking spaces. The proposed scheme has been shown to achieve better MAE (P.4), RMSE (P.5), and MAPE (P.6) in predicting the availability of parking spaces.

4.7 Deployment of DCNN Architecture

In [41], a DCNN-based approach (A.7) for the identification (S.1) of parking space availability has been conducted using the PKLoT (D.1) and CNRPark + EXT (D.6) datasets. It has been shown to achieve a high accuracy rate (P.1) in identifying the availability of parking spaces.

5  Open-Source Implementations of DL-Based Parking Identification and Prediction Systems

Open-source implementations are mainly useful for two purposes. Firstly, researchers who plan to extend existing works can use open-source codes and work on their extensions without the need of reinventing the wheel. Secondly, researchers who are new to the field can use open-source codes as baselines for their understanding before starting to implement their own approaches, which may be based on open-source codes. Due to the importance of open-source implementations, such implementations related to DL-based parking identification and prediction systems are discussed.

For parking space identification (S.1), in [43], DeepPS is proposed, and it has been widely used in the literature for CNN-based approaches (A.2). An open-source implementation of DeepPS in MATLAB and PS2.0 (D.2), and a large-scale labeled dataset, are provided in the website6. The work presented in [49] is an extension of [43]. In [49], using directional marking-point, DMPR-PS, which is a CNN-based (A.2) multi-attribute regression model, is proposed to identify parking spaces based on the PS2.0 dataset (D.2). Its source code in Python is available at GitHub7.

For parking space prediction (S.2), in [38], an open-source implementation of DLOPT, which is an RNN-based approach (A.5), is proposed to predict the availability of parking spaces based on the Birmingham dataset (D.7). Its source code in Python is available at GitHub8.

6  Looking Forward: Open Research Challenges

This paper has presented a review of some recent premier advances in DL approaches applied to predict and identify the availability of parking spaces. Based on the recent works, open research challenges in the deployment of DL in parking identification and prediction systems are highlighted in this section.

6.1 Considering Environmental Conditions

Considering real-world environmental conditions while training DL models helps to reflect real-world scenarios. In [14], the effects of environmental conditions, such as rainfall and shadows, have been considered. However, other environmental conditions, such as snowfall, darkness, and foggy conditions, should be considered in the training of DL models for more accurate results. The main challenge brought about by these environmental conditions is the reduced accuracy of object detection and classification due to the need to detect and classify a diverse range of objects, including cars, trucks, and empty spaces, under dark, snowfall, and foggy conditions.

In real-world scenarios, there are various types of parking spaces, such as on-street, off-street, parallel, perpendicular, 30-degree angle, and 45-degree angle parking spaces [50]. In [17], parallel and perpendicular parking spaces have been considered. However, other types of parking spaces should be considered to reflect real-world scenarios. The main challenge is that considering a diverse range of parking spaces takes a longer time in object detection and classification.

6.2 Integration of Future Generation Technologies

Future-generation technologies, such as autonomous vehicles equipped with cameras, sensors, lidars, and radars [51], can be integrated with DL to provide performance enhancement in automation. For instance, in the literature, autonomous vehicles have not been considered while deploying DL models to identify and predict parking space availability. The use of autonomous vehicles, instead of vehicles with human drivers, can reduce human errors leading to improvement in overall performance and safety [52]. Nevertheless, autonomous vehicles may not make optimal decisions in all real-world environmental conditions, such as darkness, snowfall, and fog [53]. Hence, DL must address the challenge of object detection and classification, such as a reduced accuracy of detecting and classifying a diverse range of objects, such as cars, trucks, and empty spaces, under unpredictable real-world environments. However, training DL models under such circumstances often require a dataset consisting of a large collection of images covering all kinds of states, which may also increase the training duration of the DL models. Fast RCNN and Faster RCNN techniques, which are based on CNN [12], have been introduced to increase the speed and accuracy of DL models. Based on Fast RCNN, the Faster RCNN technique enhances both RCNN and Fast RCNN, whereby its architecture uses a separate network to predict the region proposals of images [54]. While these techniques can be adopted for the deployment of parking identification and prediction systems to achieve higher learning speed and accuracy of object detection and classification among autonomous vehicles, further investigation can be pursued to integrate autonomous vehicles with DL models in parking identification and prediction systems.

6.3 Integration of Recommendation Systems and IoT Systems

A recommendation system can be integrated with the DL model to improve driver experience. For instance, in [13,16], the parking prediction system (S.2) has been used. In these works, recommendation systems are adopted to predict parking spaces to provide performance enhancement, such as finding the nearest parking space to the driver’s destination within the shortest time. Thus, a recommendation system understands a driver’s needs and provides recommendations accordingly. Nevertheless, the recommendation system requires a large amount of data to facilitate different kinds of drivers’ needs and make effective recommendations, so it must address the challenge of an increased computational cost. The proposed solution in [55] uses binary neural networks to reduce the computational cost, and it has been shown to achieve promising results in reducing the computational cost of DL. Future investigations could be pursued to reduce the computational cost of DL while being deployed in parking identification and prediction systems. Moreover, privacy can be considered. Some existing works on privacy preservation in parking recommender systems [56] can serve as a guide to improve privacy in DL-based parking space identification and prediction systems.

Additionally, it is also interesting to integrate the DL model with IoT systems (e.g., smart parking) because DL alone may not be sufficient in providing an accurate prediction of parking spaces. IoT systems provide additional information to predict parking spaces, such as traffic congestion and air pollution.

6.4 Reducing the Computational Cost of Deployment

Higher computational cost is an important issue when DL is deployed in a real-world environment. For instance, in [14,15,17], the DL models have been deployed in the real-world environment for the identification (S.1) of parking space availability. The main issue is the high computational cost. One possible solution is that the DL model can be compressed and embedded in edge artificial intelligence chips, which have been released recently [57]. By using these chips, the computational cost can be reduced significantly when deploying DL, which requires a large amount of data for training and testing purposes. However, the main challenge of using these chips is achieving lower computing power requirements and energy consumption despite processing a large amount of data. The proposed solution to address this challenge is to adopt a lightweight DL model, which has recently been shown to achieve promising results in human activity recognition tasks. In [58], a lightweight DL model is proposed, and it is a combination of RNN and LSTM with two hidden layers and 30 neurons in each hidden layer. This model can be adopted to achieve lower computing power requirements and energy consumption in parking identification and prediction systems.

6.5 Detecting Illegal Parking

Illegal parking is one of the main causes of congestion. It affects the accuracy of parking identification and prediction systems causing false positives and false negatives. Therefore, the detection of illegal parking and its consideration in parking identification and prediction systems is an interesting research challenge.

6.6 Investigating Deep Learning in Indoor Parking

There are two types of parking lots in the real world, namely outdoor and indoor parking lots. In most of the proposed schemes in the literature, such as [12,18], the outdoor parking lot has been considered. However, indoor parking lots have not been investigated in the context of DL. Future investigations could be pursued in indoor parking lots in the context of DL to reflect real-world scenarios.

7  Concluding Remarks

In this paper, recent premier advances in deep learning (DL) approaches applied to identify and predict the availability of parking spaces are analyzed, highlighted, and reported. Then, the literature is categorized by devising a taxonomy, which is used to uncover the features of DL techniques beneficial to the parking paradigm. The DL roles in parking identification and prediction systems are highlighted. Open research challenges critical to the successful deployment of DL in parking identification and prediction systems are identified and discussed. Although the deployment of DL in parking identification and prediction systems provides various benefits, the convergence of these two types of systems and DL brings about new issues that must be resolved in the near future.

Funding Statement: This research was supported by Universiti Tunku Abdul Rahman (UTAR), and the Publication Fund under Research Creativity and Management Office, Universiti Sains Malaysia.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

1https://web.inf.ufpr.br/vri/databases/parking-lot-database/

2https://cslinzhang.github.io/deepps/

3http://claudiotest.isti.cnr.it/park-datasets/CNR-EXT/

4https://www.sfmta.com/demand-responsive-parking-pricing

5https://data.smgov.net/