Reliability, validity and measurement invariance of the difficulties in emotion regulation Scale-8 in Chinese adolescents

Introduction

Emotional regulation refers to the psychological process by which individuals modulate the intensity or potency of their affect to better adapt to their environment (Gross, 2015). Individual would differ in their emotional regulation ability, and some would be with poor emotional regulation ability or difficulty in emotion regulation (Thurm et al., 2023). With good emotional regulation ability comes positive mood, physical and mental health as well as social adaptation (Clayton et al., 2023; Miu et al., 2022). Conversely, poor emotional regulation would diminish psychological motivation, risking mood and anxiety disorders, eating disorders, attention-deficit/hyperactivity disorders, and behavioral problems (Arslan et al., 2022; Hua & Ma, 2022; Nguyen et al., 2024; Penner et al., 2023; Thurm et al., 2023). However, emotional regulation indicators may vary by culture, and few studies have examined the measures of emotional regulation in collectivist Chinese culture. We aimed to determine the psychometric properties of the Difficulties in Emotion Regulation Scale-8 (Penner et al., 2023) in Chinese Adolescents.

The difficulties in emotion regulation scale (DERS)

The Difficulties in Emotion Regulation Scale (DERS, Gratz & Roemer, 2004) is the most widely used measurement tool and consists of 36 questions with a 5-point scoring system based on a 4-factor model of emotion regulation. It comprises 6 dimensions: emotional response unacceptance, difficulty in engaging in target behaviors, difficulty in impulse control, difficulty in emotion detection, lack of emotional clarity, and limited use of emotion regulation strategies (Gratz & Roemer, 2004). However, researchers are full of concerns about whether the dimension of difficulty in emotion detection exists independently, so some researchers merged it with the dimension of lack of emotional clarity, thus becoming five factors (Bardeen et al., 2012; Brown & Melas, 2024; Nguyen et al., 2024). Moreover, the five factors model of the scale has problems of high similarity of dimensions and poor differentiation (Todorov et al., 2023). Additionally, there is redundancy among questions within almost every dimension, where positive and negative measurements are repeated up to six times. Therefore, one current task is to improve the measurement efficiency and simplify the questionnaire appropriately.

In light of this, developed The Difficulties in Emotion Regulation Scale-8 (DERS-8; Penner et al., 2023) could be an improvement. Notably, the DERS-8 has language improvement with each item beginning with “when I am upset.” and also has an even representation of positive and negative items for clarity. Difficulty in emotion regulation has been modified to mean an individual’s insufficient ability to engage in adaptive emotion-regulating behaviors when experiencing negative emotions.

At present, the cross-cultural applicability of the DERS-8 has yet to be verified. This study aimed to test DERS-8 reliability, validity, and measurement invariance of scores in Chinese adolescents.

Methods

Participants and setting

A total of 1114 Chinese adolescents were participants (females = 593, mean age = 15.27 SD = 1.86). They attended high school in Hebei Province, China. Among these students, 190 are enrolled in Grade 7, 200 are enrolled in Grade 8, 185 are enrolled in Grade 9, are enrolled in Grade 7, 142 are enrolled in Grade 10, 208 are enrolled in Grade 11 and 189 are enrolled in Grade 12. There are 436 students from urban areas and 678 students from rural areas.

Measures

Difficulties in emotion regulation Scale-8 (DERS-8, Penner et al., 2023)

The DERS-8 comprises 8 items scored from 1 (never) to 5 (always) points, without reverse-scoring questions. The higher the total score is, the more difficult it is to regulate emotions. This was translated into Chinese using to capture the nuances of Chinese language expression habits (see Table 1). In the early stage, 15 teenagers evaluate the comprehensibility of the items and with a high concordance of agreement above 90%.

General anxiety disorder scale (GADS, Sun et al., 2021)

The GADS comprises seven items (e.g., “Worrying too much about a variety of things”). The items are on a Likert scale ranging from 0 to 3 (never to a lot). The higher the score was, the greater the participants’ anxiety. The Cronbach’s α coefficient for GADS scores was 0.83 in the present study.

Depression scale (DS, Chang et al., 2018)

The DS comprises 7 items (e.g., “I feel depressed and sad”), and it has a single-dimensional structure. Items are on a Likert scale of 0 to 4 was used (no ~ always), with higher scores indicating more significant depression. The Cronbach’s α coefficient for DS scores was 0.82.

Emotion regulation scale (ERS, Wang et al., 2007)

The ERS comprises 14 items on cognitive reappraisal (7 items, e.g., “I will try to change my perception of the surrounding environment in order to make myself feel happier”) and expression inhibition (7 items e.g., “When I feel happy, I will try my best not to show it”). The items are scored on a scale ranging from 1 to 7 points (very negative to very positive), with higher scores indicating better use of specific emotion regulation strategies. The Cronbach’s α coefficients for ERS cognitive reappraisal and expression inhibition scores were 0.71 and 0.73, respectively.

Procedure

This research received ethical approval by the Ethics Committee of Hebei Normal University and obtained informed consent from all participating students. At the time of data collection, we declare to participants that they can withdraw at any time during the filling process and that the results of filling will be kept confidential. Three samples were involved as followed: Sample 1 (N = 424 for item analysis and exploratory factor analysis), Sample 2 (N = 586 for confirmatory factor analysis), and Sample 3 (N = 104 for test-retest reliability).

Data analysis

SPSS24.0 was used to analyze the scale’s differentiation, Cronbach’s α coefficient, criterion validity and factor structure. AMOS 24.0 was used for confirmatory factor analysis and measurement invariance tests. The fitting degree of confirmatory factor analysis is mainly evaluated by looking at Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Incremental Fit Index (IFI), Normed Fit Index (NFI), Goodness of Fit Index (GFI) and Root Mean Square Error of Approximation (RMSEA). Among them, CFI, TLI, IFI, NFI, GFI are considered good if they exceed 0.90, and RMSEA is considered good if they are less than 0.08 (Jekauc et al., 2013; Sun et al., 2021).

Results

Item analysis

The item-total correlation coefficient of the DERS-8 was calculated. The results showed that the discrimination of each item ranged from 0.64 to 0.74, all above 0.4, indicating that the item discrimination was good, and all items were retained for subsequent analysis (Table 1).

Construct validity analysis

Principal component analysis and promax oblique rotation were used to conduct exploratory factor analysis on the data of sample 1. The results show that the KMO value is 0.90, indicating that it is suitable for factor analysis. The scree plot shows only one factor with an eigenvalue greater than 1 (Figure 1), and the load of this factor in each problem is more than 0.7 (p < 0.01), as shown in Table 1. This factor explained 60.65% of the total variation.

Figure 1: Scree plot

Confirmatory factor analysis was performed on the data of sample 2, and the model operation results were well fitted, with fitting indices meeting the measurement standards (χ2/df = 4.05, RMSEA = 0.07, CFI = 0.98, TLI = 0.97, IFI = 0.98, NFI = 0.97, GFI = 0.97). This indicates that the DERS-8 has a single-factor structure (Figure 2).

Figure 2: Confirmatory factor analysis. Note. ***p < 0.001.

Criterion validity analysis

The total DERS-8 score was positively correlated with both the anxiety scale (r = 0.43, p < 0.01) and the depression scale (r = 0.41, p < 0.01). There was a significant negative association with expression inhibition (r = −0.58, p < 0.01) and cognitive reappraisal (r = −0.61, p < 0.01) on the emotional regulation scale. This indicates that individuals with more difficulty in emotion regulation have less use of emotion regulation strategies and more obvious negative emotions. The results show that the ERSP accords with reality and has good external validity.

Reliability analysis

The Cronbach’s α coefficient of the DERS-8 is 0.90. The split-half reliability coefficient of the DERS-8 for parity was 0.87. Based on sample 3, the test-retest reliability of the DERS-8 was calculated to be 0.66.

Measurement invariance test

Based on sample 2, the equivalence of DER-8 cross-gender measurement was tested. Firstly, the single-dimensional structure model was well fitted for both male (χ2/df = 2.75, RMSEA = 0.07, CFI = 0.98, TLI = 0.97) and female (χ2/df = 2.95, RMSEA = 0.07, CFI = 0.98, TLI = 0.95). Then, the model of configural invariance, metric invariance, scalar invariance and strict invariance are established and compared in turn. If ΔCFI ≤ 0.01 and ΔRMSEA ≤ 0.015, the measurement equivalence of the model is acceptable (Jekauc et al., 2013). The results showed that the baseline model was well fitted (χ2/df = 4.38, RMSEA = 0.96, CFI = 0.96, TLI = 0.93), indicating that the scale had morphological invariance between different genders and was a single factor structure. On this basis, the DERS-8 factor loadings were then restricted to be equal between different genders. It was found that compared with Model 1, the ΔCFI and ΔRMSEA of Model 4 was less than 0.01 and 0.015 respectively, indicating that the DERS-8 factor loadings were within Same for different genders. On this basis, we further restricted the measurement intercepts to be equal in different genders. It was found that compared with Model 2, the ΔCFI and ΔRMSEA of Model 4 was less than 0.01 and 0.015, respectively, indicating that the DERS-8 measurement intercepts were equal among different genders. On this basis, we further limited the error variance to be equal for different genders. The results showed that the ΔCFI and ΔRMSEA of Model 4 was less than 0.01 and 0.015, respectively, compared with Model 3, indicating that the measurement error of DERS-8 is equal for different genders (Table 2). Finally, the independent sample t test found that there was no significant difference (p = 0.12) in the emotion regulation difficulty scores of boys (M = 3.64, SD = 0.83) and girls (M = 3.54, SD = 0.69).

Discussion

This study is the first to examine the psychometric properties of the DERS-8 in Chinese adolescents. The analysis revealed that the Chinese version of the DERS-8 performed well in the discrimination of each item, all of which were higher than 0.4, meeting the measurement standards and requirements (Zhang et al., 2024). Therefore, all the items were suitable for testing with Chinese adolescents without deleting the item.

The results of exploratory factor analysis showed that only one common factor could be extracted, the factor loading of each item was greater than 0.7, the variance of all items was greater than 60%, similar to the English version of the DERS-8. Further confirmatory factor analysis also supported the single-factor structure of the DERS-8, with good model fit and excellent validity, indicating that the single-factor structure is also applicable in the Chinese context. The single-factor structure of the scale has been gradually accepted and adopted by researchers in recent years (Todorov et al., 2023). The language adjustment of the new version of the scale and the improvement of the questions made the single-factor structure more stable.

Criterion validity analysis revealed that the DERS-8 score was significantly positively correlated with anxiety, depression and emotion regulation questionnaire scores, and the correlation degree was moderate. Moreover, the correlation intensity of the DERS-8 was close to that of previous studies (Burton et al., 2022), indicating good calibration validity. Previous studies have shown that difficulty in emotion regulation is closely associated with many maladaptive behaviors that are considered to function in emotion regulation, such as intentional self-harm, substance use, and risky sexual behaviors (Gratz & Roemer, 2008; Tull et al., 2012, 2015; Zhao et al., 2022). This result is consistent with those of other studies, indicating that individuals with poor emotion regulation ability may lack the ability to invoke emotion regulation strategies, risking emotional distress or even emotional disorders (Carruthers et al., 2022; Tull et al., 2012; Zhao et al., 2022). A good emotional regulation ability can help individuals reduce negative emotions and maintain a rational and flat state of mind.

Reliability analysis revealed that the reliability of the Chinese version of the DERS-8 was 0.90, which is very close to that of the English version (α = 0.91). The reliability of the Chinese version of the DERS-8 is also above 0.8, and the reliability of the retest after a one-month interval is 0.66, which meets the measurement standards. High retest reliability reflects a trait rather than a state of difficulty in emotional regulation, which means that emotional regulation reflects individual differences in ability and is relatively stable (Carona et al., 2022). It also distinguishes the scale from the State-Emotion Regulation Difficulty Scale (Lavender et al., 2017).

Implications for practice

The Chinese version of the DERS-8 has multiple advantages compared with the original emotional regulation difficulty scale. It avoids the variances by positive and negative scoring for a more unified measure. The most important step is to remove many repeated and ineffective measurement questions, significantly reducing the number of test questions without losing accuracy and effectiveness. The adapted more concise and would be incredibly convenient for clinical measurement and screening.

Strengths, limitations and future directions and conclusion

This study revealed that the DERS-8 has measurement invariance among Chinese adolescents. This study has several limitations that need to be considered. The sampling approach adopted was convenience sampling, and all participants were from Hebei Province, which greatly restricts the generalizability of the results. In the future, the sampling range and population can be expanded to test the measurement effect of this tool. Additionally, another limitation is the absence of using Item Response Theory (IRT) technology to analyze the information content of each item. In future research, it is essential to conduct further verification based on the Rasch model to gain more accurate insights into item quality and functionality, thus improving the overall validity and reliability of the study’s measurement approach. In conclusion, the Chinese version of the DERS-8 has various reliability and validity indicators and can be used to measure difficulties in emotion regulation in Chinese adolescents.

Acknowledgement: Not applicable.

Funding Statement: This work was funded by Science Research Project of Hebei Education Department (BJ2025238), Humanities and Social Science Research Project of Hebei Normal University (S24YX002). Humanities and Social Science Research Foundation of Hebei Normal University (S22B019).

Author Contributions: Study conception and design: Yali Zhang; data collection and draft manuscript preparation: Hao Jiang; analysis and interpretation of results: Yuewen Bian; manuscript revision, and data analysis: Fuhai Zhang.

Availability of Data and Materials: The data that support the findings of this study are available from the corresponding author upon reasonable request.

Ethics Approval: This study was approved by the Ethics Committee of Hebei Normal University (BJ2025238), in accordance with the principles of the Declaration of Helsinki.

Informed Consent: The researcher sought and gained the consent of the participants to take part in the study. All participants accepted and voluntarily participated in the study after the researcher assured them of anonymity and that their responses were solely for academic purposes.

Conflicts of Interest: The authors declare no conflicts of interest to report regarding the present study.

References

Arslan, M., Ayhan, N. Y., Çevik, E., Sarıyer, E. T., & Çolak, H. (2022). Effect of emotion regulation difficulty on eating attitudes and body mass index in university students: A cross-sectional study. Journal of Men’s Health, 18(10), 199. https://doi.org/10.31083/j.jomh1810199 [Google Scholar] [CrossRef]

Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2012). An examination of the latent structure of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 34(3), 382–392. https://doi.org/10.1007/s10862-012-9280-y [Google Scholar] [CrossRef]

Brown, W. C., & Melas, M. T. (2024). A narrative review of emotion regulation abilities, strategy use, and their relationship to alcohol. Current Addiction Reports, 11(2), 199–209. https://doi.org/10.1007/s40429-024-00553-3 [Google Scholar] [CrossRef]

Burton, A. L., Brown, R., & Abbott, M. J. (2022). Overcoming difficulties in measuring emotional regulation: Assessing and comparing the psychometric properties of the DERS long and short forms. Cogent Psychology, 9(1), 2060629. https://doi.org/10.1080/23311908.2022.2060629 [Google Scholar] [CrossRef]

Carona, C., Xavier, S., Canavarro, M. C., & Fonseca, A. (2022). Self-compassion and complete perinatal mental health in women at high risk for postpartum depression: The mediating role of emotion regulation difficulties. Psychology and Psychotherapy: Theory, Research and Practice, 95(2), 561–574. https://doi.org/10.1111/papt.12388 [Google Scholar] [PubMed] [CrossRef]

Carruthers, S. P., Rossell, S. L., Murray, G., Karantonis, J., Furlong, L. S. et al. (2022). Mindfulness, mood symptom tendencies and quality of life in bipolar disorder: An examination of the mediating influence of emotion regulation difficulties. Journal of Affective Disorders, 298, 166–172. https://doi.org/10.1016/j.jad.2021.10.107 [Google Scholar] [PubMed] [CrossRef]

Chang, C. W., Yuan, R., & Chen, J. K. (2018). Social support and depression among Chinese adolescents: The mediating roles of self-esteem and self-efficacy. Children and Youth Services Review, 88(6), 128–134. https://doi.org/10.1016/j.childyouth.2018.03.001 [Google Scholar] [CrossRef]

Clayton, M., Renna, M. E., Klemanski, D. H., Kerns, C., McLaughlin, K. A. et al. (2023). The impact of emotion regulation on the relationship between momentary negative affect and end-of-day worry and rumination. Cognitive Therapy and Research, 47(1), 94–108. https://doi.org/10.1007/s10608-022-10339-4 [Google Scholar] [CrossRef]

Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 26(1), 41–54. https://doi.org/10.1023/B:JOBA.0000007455.08539.94 [Google Scholar] [CrossRef]

Gratz, K. L., & Roemer, L. (2008). The relationship between emotion dysregulation and deliberate self-harm among female undergraduate students at an urban commuter university. Cognitive Behaviour Therapy, 37(1), 14–25. https://doi.org/10.1080/16506070701819524 [Google Scholar] [PubMed] [CrossRef]

Gross, J. J. (2015). Emotion regulation: Current status and future prospects. Psychological Inquiry, 26(1), 1–26. https://doi.org/10.1080/1047840X.2014.940781 [Google Scholar] [CrossRef]

Hua, Z., & Ma, D. (2022). Depression and perceived social support among unemployed youths in China: Investigating the roles of emotion-regulation difficulties and self-efficacy. International Journal of Environmental Research and Public Health, 19(8), 4676. https://doi.org/10.3390/ijerph19084676 [Google Scholar] [PubMed] [CrossRef]

Jekauc, D., Voelkle, M., Wagner, M. O., Mewes, N., & Woll, A. (2013). Reliability, validity, and measurement invariance of the German version of the physical activity enjoyment scale. Journal of Pediatric Psychology, 38(1), 104–115. https://doi.org/10.1093/jpepsy/jss088 [Google Scholar] [PubMed] [CrossRef]

Lavender, J. M., Tull, M. T., DiLillo, D., Messman-Moore, T., & Gratz, K. L. (2017). Development and validation of a state-based measure of emotion dysregulation: The State Difficulties in Emotion Regulation Scale (S-DERS). Assessment, 24(2), 197–209. https://doi.org/10.1177/1073191115601218 [Google Scholar] [PubMed] [CrossRef]

Miu, A. C., Szentágotai-Tătar, A., Balazsi, R., Nechita, D., Bunea, I. et al. (2022). Emotion regulation as mediator between childhood adversity and psychopathology: A meta-analysis. Clinical Psychology Review, 93(4), 102141. https://doi.org/10.1016/j.cpr.2022.102141 [Google Scholar] [PubMed] [CrossRef]

Nguyen, Q., Madison, S., Ekas, N. V., & Kouros, C. D. (2024). Helicopter parenting behaviors and emerging adult mental health: The mediating role of emotion dysregulation. Emerging Adulthood, 12(5), 820–837. https://doi.org/10.1177/21676968241254565 [Google Scholar] [CrossRef]

Penner, F., Steinberg, L., & Sharp, C. (2023). The development and validation of the difficulties in emotion regulation Scale-8: Providing respondents with a uniform context that elicits thinking about situations requiring emotion regulation. Journal of Personality Assessment, 105(5), 657–666. https://doi.org/10.1080/00223891.2022.2133722 [Google Scholar] [PubMed] [CrossRef]

Sun, J., Liang, K., Chi, X., & Chen, S. (2021). Psychometric properties of the generalized anxiety disorder scale-7 item (GAD-7) in a large sample of Chinese adolescents. Healthcare, 9(12), 1709. https://doi.org/10.3390/healthcare9121709 [Google Scholar] [PubMed] [CrossRef]

Thurm, A., Satel, J., Montag, C., Griffiths, M. D., & Pontes, H. M. (2023). The relationship between gambling disorder, stressful life events, gambling-related cognitive distortions, difficulty in emotion regulation, and self-control. Journal of Gambling Studies, 39(1), 87–101. https://doi.org/10.1007/s10899-022-10151-5. [Google Scholar] [CrossRef]

Todorov, E. H., Paradis, A., & Ha, T. (2023). Emotion regulation difficulties and relationship satisfaction in adolescent couples: The role of conflict resolution strategies. Journal of Youth and Adolescence, 52(8), 1753–1767. https://doi.org/10.1007/s10964-023-01787-6 [Google Scholar] [PubMed] [CrossRef]

Tull, M. T., Bardeen, J. R., DiLillo, D., Messman-Moore, T., & Gratz, K. L. (2015). A prospective investigation of emotion dysregulation as a moderator of the relation between posttraumatic stress symptoms and substance use severity. Journal of Anxiety Disorders, 29, 52–60. https://doi.org/10.1016/j.janxdis.2014.11.003 [Google Scholar] [PubMed] [CrossRef]

Tull, M. T., Weiss, N. H., Adams, C. E., & Gratz, K. L. (2012). The contribution of emotion regulation difficulties to risky sexual behavior within a sample of patients in residential substance abuse treatment. Addictive Behaviors, 37(10), 1084–1092. https://doi.org/10.1016/j.addbeh.2012.05.001 [Google Scholar] [PubMed] [CrossRef]

Wang, L., Lu, Y. P., & Li, Z. Q. (2007). Test of emotion regulation scale in adolescents. Chinese Journal of Clinical Psychology, 15(3), 236–238. [Google Scholar]

Zhang, Y., Zhao, J., Bian, Y., & Zhang, F. (2024). Reliability and validity of the short version of the childhood abuse self-report scale in Chinese college students. BMC Psychology, 12(1), 256. https://doi.org/10.1186/s40359-024-01734-5 [Google Scholar] [PubMed] [CrossRef]

Zhao, Y., Wu, K., Wang, Y., Zhao, G., & Zhang, E. (2022). Construct validity of brief difficulties in emotion regulation scale and its revised version: Evidence for a general factor. Current Psychology, 41(2), 1085–1096. https://doi.org/10.1007/s12144-021-02056-0 [Google Scholar] [CrossRef]