Open Access iconOpen Access



A Comprehensive Analysis of Datasets for Automotive Intrusion Detection Systems

Seyoung Lee1, Wonsuk Choi1, Insup Kim2, Ganggyu Lee2, Dong Hoon Lee1,*

1 School of Cybersecurity, Korea University, Seoul, 02841, Korea
2 Memory Division, Samsung Electronics, Hwaseong, 18449, Korea

* Corresponding Author: Dong Hoon Lee. Email: email

(This article belongs to this Special Issue: Advances in Information Security Application)

Computers, Materials & Continua 2023, 76(3), 3413-3442.


Recently, automotive intrusion detection systems (IDSs) have emerged as promising defense approaches to counter attacks on in-vehicle networks (IVNs). However, the effectiveness of IDSs relies heavily on the quality of the datasets used for training and evaluation. Despite the availability of several datasets for automotive IDSs, there has been a lack of comprehensive analysis focusing on assessing these datasets. This paper aims to address the need for dataset assessment in the context of automotive IDSs. It proposes qualitative and quantitative metrics that are independent of specific automotive IDSs, to evaluate the quality of datasets. These metrics take into consideration various aspects such as dataset description, collection environment, and attack complexity. This paper evaluates eight commonly used datasets for automotive IDSs using the proposed metrics. The evaluation reveals biases in the datasets, particularly in terms of limited contexts and lack of diversity. Additionally, it highlights that the attacks in the datasets were mostly injected without considering normal behaviors, which poses challenges for training and evaluating machine learning-based IDSs. This paper emphasizes the importance of addressing the identified limitations in existing datasets to improve the performance and adaptability of automotive IDSs. The proposed metrics can serve as valuable guidelines for researchers and practitioners in selecting and constructing high-quality datasets for automotive security applications. Finally, this paper presents the requirements for high-quality datasets, including the need for representativeness, diversity, and balance.


Cite This Article

S. Lee, W. Choi, I. Kim, G. Lee and D. H. Lee, "A comprehensive analysis of datasets for automotive intrusion detection systems," Computers, Materials & Continua, vol. 76, no.3, pp. 3413–3442, 2023.

cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 283


  • 156


  • 0


Share Link