In recent years evidence has emerged suggesting that Mini-basketball training program (MBTP) can be an effective intervention method to improve social communication (SC) impairments and restricted and repetitive behaviors (RRBs) in preschool children suffering from autism spectrum disorder (ASD). However, there is a considerable degree if interindividual variability concerning these social outcomes and thus not all preschool children with ASD profit from a MBTP intervention to the same extent. In order to make more accurate predictions which preschool children with ASD can benefit from an MBTP intervention or which preschool children with ASD need additional interventions to achieve behavioral improvements, further research is required. This study aimed to investigate which individual factors of preschool children with ASD can predict MBTP intervention outcomes concerning SC impairments and RRBs. Then, test the performance of machine learning models in predicting intervention outcomes based on these factors. Participants were 26 preschool children with ASD who enrolled in a quasi-experiment and received MBTP intervention. Baseline demographic variables (e.g., age, body, mass index [BMI]), indicators of physical fitness (e.g., handgrip strength, balance performance), performance in executive function, severity of ASD symptoms, level of SC impairments, and severity of RRBs were obtained to predict treatment outcomes after MBTP intervention. Machine learning models were established based on support vector machine algorithm were implemented. For comparison, we also employed multiple linear regression models in statistics. Our findings suggest that in preschool children with ASD symptomatic severity (
Autism spectrum disorder (ASD) is a life-long neurodevelopmental disability that develops during early childhood [
Physical exercise is an effective intervention strategy to treat ASD in preschool children [
In this regard, two basic questions should be kept in mind: (i) What factors should be used to predict the intervention outcomes? and (ii) What methods should be used to predict the intervention outcomes?
Currently, only a few studies have evaluated the efficacy of MBTP intervention in preschool children with ASD to influence SC impairments and RRBs and observed positive effects of MBTP on these outcome parameters [
With increasing data availability and complexity, the predictive performance of traditional statistical methods is limited. Therefore, machine learning methods have been proposed. Machine learning, a branch of artificial intelligence, enables computers to learn from data and have better fitting ability to the data [
As mentioned, not all preschool children with ASD benefit equally from physical interventions (e.g., MBTP) and in order to maximize the intervention related benefits it is of great importance to better predict the intervention response concerning specific outcome parameters (e.g., SC impairments and RBBs in preschool children with ASD). Such a deeper understanding of prediction of intervention outcomes, in turn, allow to better personalize the interventions. Thus, this study aims to investigate which factors in preschool children with ASD are associated with the treatment outcomes in SC impairments and RBBs in response to a MBTP intervention. And then through these factors, statistical models and machine learning models were implemented to predict the intervention outcomes in SC impairments and RRBs in preschool children with ASD in response to a MBTP. Furthermore, we compared the performance of the two models. Presumably, our findings will aid the identification of preschool children with ASD that are most likely to benefit from MBTP and provide reference for the development of personalized intervention programs for preschool children with ASD.
Ninety-four preschool children with DSM-5-diagnosed ASD from two education centers (Yangzhou, China) participated in a quasi-experiment [
The MBTP intervention was conducted by two certified physical education educators and more details of the operation could be found in already published articles [
Baseline demographic information, including sex, age and body mass index (BMI) for preschool children with ASD were obtained. The Childhood Autism Rating Scale (CARS) [
The National Standard Manual of Physical Fitness Measurement (Early Childhood Part) was used to evaluate physical health development of preschool children with ASD [
The Childhood Executive Functioning Inventory (CHEXI) [
The Social Responsiveness Scale Second Edition (SRS-2) [
The Repetitive Behavior Scale-Revised (RBS-R) [
At baseline, demographic information (i.e., sex, age, BMI), symptomatic severity, SC impairments, RRBs, physical fitness (i.e., dexterity, muscle strength, flexibility and balance), and executive functions (i.e., working memory, planning ability, regulation ability and inhibition ability) of preschool children with ASD were assessed. These factors were taken as candidate factors, and normalized via the Z-score standardization method [
Python 3.7.4 was used for the further analysis. The predictor is extracted by the Pearson correlation coefficient and
The Social Responsiveness Scale Second Edition (SRS-2) and Repetitive Behavior Scale-Revised (RBS-R) were used to evaluate SC impairments and RRBs for preschool children with ASD after receiving MBTP intervention, and scale score (total score) was used to reflect intervention outcomes of MBTP on SC impairments and RRBs of preschool children with ASD.
In this study, we used two different methods to predict the MBTP-related intervention outcomes in SC impairment and RRBs. As first method, multiple linear regression models were calculated using SPSS 23.0. As second method, machine learning models were determined using Python 3.7.4 (see
In this study, we used support vector machine (SVM) algorithm given that they performed well in the prediction of psychiatric treatment outcomes [
The Sklearn package in Python 3.7.4 was used to implement machine learning models to predict the MBTP intervention-related outcomes in SC impairment and RRBs.
In this context, the following specific modeling processes were used: (1) Division of sample sets into training and test set
The train test split method in Sklearn packages was used to randomly divide all sample sets into training set and test set, where the training set was 75% of the total sample (n = 19) and the test set was 25% of the total sample (n = 7). (2) Based on the training set, SVM models were established and evaluation indices selected
The Gaussian radial basis function [
Mean square error (MSE), root mean square error (RMSE) and coefficient of determination (R2) [
MSE was expressed as:
RMSE was expressed as:
R2 was expressed as:
(3) Grid search and cross validation
Grid search was used to generate a list of all possible values of each parameter in the estimation function, after which the values in each list were combined to generate a grid. Each grid was used as a training model, and performance was evaluated by 10-fold cross validation method to optimize the learning algorithm. After the fitting function had tried all combination results, it returned the most suitable learner and automatically adjusted to the best parameter combination. (4) Inputting the test set to obtain the prediction results of the models, after which the performance of the models was evaluated
With respect to demographic information, we did not observe significant sex-related difference concerning SC impairments (
Candidate factors | ||
---|---|---|
Age | 0.107 | 0.323 |
Body Mass Index | −0.371 | 0.062 |
Dexterity | −0.248 | 0.222 |
Muscle strength | −0.123 | 0.550 |
Flexibility | −0.114 | 0.580 |
Balance | −0.067 | 0.744 |
Working memory | 0.200 | 0.328 |
Planning ability | 0.380 | 0.056 |
Regulation ability | 0.387 | 0.051 |
Inhibition ability | −0.073 | 0.721 |
Severity of symptoms | 0.712 | <0.001 |
Baseline SC impairment | 0.713 | <0.001 |
Baseline RRBs | 0.301 | 0.135 |
Note:
We did not observe significant difference in SC impairments between female and male participants (
Candidate factors | ||
---|---|---|
Age | 0.238 | 0.242 |
Body mass index | −0.430 | 0.028 |
Dexterity | −0.282 | 0.164 |
Muscle strength | 0.055 | 0.791 |
Flexibility | 0.308 | 0.126 |
Balance | −0.085 | 0.681 |
Working memory | 0.247 | 0.224 |
Planning ability | 0.211 | 0.300 |
Regulation ability | 0.302 | 0.133 |
Inhibition ability | 0.181 | 0.377 |
Severity of symptoms | 0.656 | <0.001 |
Baseline SC impairment | 0.504 | 0.009 |
Baseline RRBs | 0.647 | <0.001 |
Note:
Symptomatic severity and baseline SC impairments were included in the multiple linear step-by-step regression model of treatment outcomes for SC impairments. Symptomatic severity (
Symptomatic severity, BMI, baseline SC impairments, and baseline RRBs were included in the multiple linear step-by-step regression model of treatment outcomes for RRBs. Baseline SC impairments (
Notes: SC impairments refers to social communication impairments, test sets are numbered from 1 to 7. Drawn using the Matplotlib package in Python.
Notes: RRBs refers to restricted and repetitive behaviors, test sets are numbered from 1 to 7. Drawn using the Matplotlib package in Python.
We compared the performance of machine learning models and conventional statistical models in predicting the MBTP intervention-related outcomes of two core symptoms in preschool children with ASD (i.e., SC impairments and RBBs). In terms of predicting intervention-related outcomes of SC impairments in preschool children with ASD in response to MBTP, MSE and RMSE of the machine learning model were lower than those of the conventional statistical model, and the determination coefficient was increased by 0.234. Hence, all three performance indicators of the machine learning model were better than those of the conventional statistical model (see
Models | MSE | RMSE | R2 |
---|---|---|---|
Statistical model | 0.455 | 0.675 | 0.596 |
Machine learning model | 0.188 | 0.434 | 0.83 |
Note: MSE is the mean square error, RMSE is the root mean square error, and R2 is the coefficient of determination. When MSE and RMSE are closer to 0, the model’s performance is better. When closer R2 is closer to 1, the model’s performance is better.
Models | MSE | RMSE | R2 |
---|---|---|---|
Statistical model | 0.464 | 0.681 | 0.589 |
Machine learning model | 0.051 | 0.226 | 0.859 |
Note: MSE is the mean square error, RMSE is the root mean square error and R2 is the coefficient of determination. When MSE and RMSE are closer to 0, the model’s performance is better. When closer R2 is closer to 1, the model’s performance is better.
There is mounting evidence that physical interventions in general, and Mini-basketball training program (MBTP) in particular, can improve the core symptoms of ASD in preschool children (e.g., social communication (SC) impairments and restricted and repetitive behaviors (RBBs)). However, there is also growing evidence that not all preschool children benefit equally from such interventions as there is a considerable interindividual response variability in specific outcomes. However, currently there is a paucity of research which factors can predict such an interindividual response variability in response to physical interventions in preschool children with ASD. A better understanding of these predictive factors seems urgently needed to better personalize physical interventions which, in turn, is likely to maximize their effectiveness. Thus, this study investigate which factors in preschool children with ASD can predict MBTP intervention-related outcomes of SC impairment and RRBs. In this context, we observed that BMI is negatively correlated with intervention-related outcomes of RRBs. It has been previously reported that overweight and obesity rates are higher in preschool children with ASD than in healthy developing children [
In this study, multiple linear regression models and machine learning based models were calculated and their performance was compared.
Multiple linear regression models were calculated to which extend in preschool children with ASD specific individual factors can predict the MBTP intervention outcomes on SC impairments and RRBs. The multiple linear regression models predicted 59.6% of the variance in post-treatment SC impairments and 58.9% of the variance in post-treatment RRBs. Although it is difficult to compare our findings to other studies given the fact that we are among the first applying such an approach to predict MBTP intervention-related outcomes in specific outcome parameters, the model performance in the current study differed from those of previous studies dealing with psychiatric disorders. Brousse et al. [
We used machine learning models based on support vector machine algorithm to predict MBTP intervention-related outcomes of SC impairments and RRBs in preschool children with ASD. Machine learning models predicted 83% of the variance in post-treatment SC impairments and 85.9% of the variance in post-treatment RRBs, consistent with findings of studies on predicted treatment outcomes for children with obsessive compulsive disorders and depression. In the study of Lenhard et al. [
The performance of the machine learning models was superior to statistical models in terms of MSE, RMSE and R2. The better performance of machine learning based algorithms is probably driven by the following facts. To begin with, The SVM algorithm in machine learning can map input data to a high-dimensional feature space through nonlinear transformation to solved regression problems [
There were some limitations in this study that needs to be acknowledged. Firstly, the sample size was relatively small. However, previous studies achieved reliable results in prediction of intervention effects in obsessive-compulsive disorder [
In summary, the findings of the current study suggest that symptomatic severity and baseline SC impairments are important factors that can predict, to a certain extent, the MBTP intervention-related outcomes of SC impairments in preschool children with ASD. In a comparable manner BMI, symptomatic severity, baseline SC impairments and baseline RRBs can predict, to a certain extent, the MBTP intervention-related outcomes of RRBs in preschool children with ASD. Machine learning models can successfully be applied to predict the MBTP intervention-related outcomes in preschool children with ASD, and peformed better than statistical models. Our findings inform on the selection of preschool children with ASD that are most likely to benefit from MBTP, and it provides a basis for the development of personalized intervention programs for preschool children with ASD.