Validation of the Italian Version of the Developmental Disability-Child Global Assessment Scale (DD-CGAS) ()
1. Introduction
The alteration of typical functioning of an individual represents a critical aspect of mental illness: it is fundamental for the diagnosis of most psychiatric disorders, because the mere presence of symptoms—in the absence of impaired functioning—is not sufficient to configure a disease state. The recent tendency, also from a therapeutic point of view, is to consider the impairment of functioning levels to be increasingly important, rather than the presence or intensity of symptoms, as objective evidence and measurement of the treatment’s efficacy.
It is particularly important to record treatment effects on functioning level for children with autism spectrum disorders (ASDs) [1] : although there aren’t, at present, any curative treatments for autism spectrum disorders’ core symptoms [2] , evidence suggests that both behavioral and pharmacological treatments can significantly improve adaptive skills and reduce problematic behaviors such as hyperactivity and aggression [3] -[10] .
In clinical studies, the assessment of treatment effects of functioning, in children with ASD, is thwarted by the absence of reliable, sensible and easily administered rating tools.
The Children Global Assessment Scale (CGAS) [11] is a modified version of Global Assessment Scale (GAS) for adults [12] : it is a commonly used instrument to get a score of functioning level on child subjects [13] .
However, the descriptors used to calculate the score of CGAS aren’t easily usable when trying to describe the functioning of a subject with a diagnosis of ASD, because children with ASD usually follow atypical development paths and show severe deficits in specific areas of functioning. Moreover, cognitive functioning changes greatly within the group of subjects with ASD, shifting from severe mental retardation to a superior level, and often—above all—there are differences between intellectual and adaptive abilities, usually with delayed adaptive skills compared to the individual’s mental age [14] -[16] .
An instrument to assess global functioning in subjects with ASD would need to consider a wide range of functioning levels, with a wide variability, both interand intra-individual, and integrate information about multiple domains of functioning.
Although instruments like the Vineland Adaptive Behavior Scales (VABS) [17] -[19] , the Assessment of Basic Language and Learning Skills (ABLLS) [20] and the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP) [21] can be used to assess specific areas of adaptive behavior in children with ASD, their sensibility to the treatment effects wasn’t ascertained [22] . Furthermore these tools are lengthy to administer and are limited to specific areas of functioning.
In relation to the high individual variability of functioning among different domains, the assessment tools of global functioning are useful and quicker measures, which provide benefits also from a clinical perspective: strong evidence suggests that instruments of global rating can be more sensitive to change during the acute phases of treatment than other assessment tools with items based on symptoms [23] [24] . In fact, by integrating information on functioning from multiple sources, assessment instruments of global functioning provide a more complete picture than instruments based on specific scales or on a single source of information.
Because of the lack of rating instruments which provide a quantitative measure of global functioning to use in clinical trials with children with developmental disorders, CGAS was modified by adapting the anchor points and the administration procedures to the characteristics of children with developmental disorders, including ASDs.
This work describes the validation of the Italian translation of Developmental Disability-Child Global Assessment Scale (DD-CGAS) and demonstrates the data on inter-rater reliability, temporal stability, convergent validity and sensitivity to changes during treatment, when applied to an Italian population with ASD. This study tends to replicate data obtained in the validation study of the English scale “Developmental Disabilities Modification of the Children’s Global Assessment Scale” of Wagner et al. [25] , following it for basic setup and analysis mode of data and differing from it by number and characteristics of rater and by sample characteristics for total number of subjects and for the group of subjects used to assess the sensibility to change. Another substantial difference from the validation study of the original instrument relates to the other tests used to assess convergent validity and sensitivity to change: the limitation is due to the limited number of instruments validated and available in Italian language.
2. Methods
2.1. Description of DD-CGAS
DD-CGAS is a modified version of CGAS. It is a scale for clinicians which provide a total score of functioning for individuals below 18 years of age with a developmental disability, compared to individuals with typical development of the same age. The score refers to the typical functioning of the child during a certain period of time, usually the week before the assessment.
The score is based on all available information sources and all functioning domains: self-care, communication, social behavior, school/academic functioning, and it isn’t dependent on diagnosis, on cause of dysfunction (for example, cognitive or physical, environmental, behavioral disorders) or on type and severity of symptoms.
We preserved the overall structure of the original GAS and CGAS; therefore, DD-CGAS is a dimensional scale with scores ranging from 1 to 100, where 1 represents the most compromised functioning and 100 the highest level of functioning.
Each decile (for example 1 - 10, 11 - 20) has a descriptive header (for example “Moderate impairment of functioning at least in one area”) and examples of behaviors and mode of adaptation which could represent this functioning level (Appendix A).
Scores equal to or greater than 70 on DD-CGAS suggest functioning within the normal development in neurotypical children of the same age. Because children with developmental disabilities must have, by definition, a significant alteration of functioning abilities, we will rarely obtain scores higher than 70 within this population. However, children with a mild disability could improve with treatment and placed into a normal range of functioning. Because this is an instrument created for many types of research, and for a wide range of developmental disabilities and control groups, it is a great advantage to have the possibility to capture the full range of functioning.
For the fundamental role of clinical assessment on global evaluation of functioning, we developed a specific procedure to standardize scoring to increase reliability. For this purpose we created a grid (Appendix B) to assign a level of impairment (none, soft, moderate, severe, extreme) to the four main domains of functioning (selfcare, communication, social behavior and school).
First, the examiner establishes the level of impairment for each domain, considering child’s behavior, the stability among different environments (for example home, school and community), the level of environmental adaptation required to support the child and the level of support required.
Afterwards, the examiner selects the range that best describes the level of functioning among different domains (for example “Moderate impairment of functioning in most areas”). The examples in the intervals are used to describe children’s functioning, even if no child will be perfectly described by these.
After identifying the most appropriate interval, the evaluator explores the adjacent intervals in order to assign the specific score. For example, if the child fits better at “60 - 51 Moderate impairment of functioning in most areas”, but with some similarities to 41 - 50, the evaluator will give a score in the bottom half range (i.e. 54 - 51). Vice versa, if the child fits better at 60 - 51, but has strong points in line with the highest category, the evaluator will give a score in the upper half of the category (i.e. 60 - 56).
All available sources of information should be used to calculate the score, including direct observation, information by caregiver and results of standardized tests. Whatever the source, the examiner needs a good description of functioning into the examined domains and information from different contexts. Thus the rater integrates all available information into a single index of functioning.
The amount of time required to collect useful information changes according to the situation in which the instrument is used. After collecting all the information, the final score is ready in 5 - 10 minutes. The re-assessment of the same child usually requires less time.
2.2. Translations
The Italian version of DD-CGAS was obtained by a forward-backward process by four clinicians with training and experience in the area of developmental disorders (two specialists in pediatric psychiatry, one community professional educator, one psychologist English mother-tongue).
A translation from English was even done for the clinical vignettes used for the inter-reliability and for temporal stability.
2.3. Inter-Rater Reliability and Temporal Stability
Clinical vignettes and training/reliability procedures for evaluators are were kindly provided by Ann Wagner, Ph.D. of NIH (USA).
The 16 clinical vignettes resulted from 16 clinical cases (concerning a wide range of functioning) of children with ASD. Children described in these clinical cases were aged between 4 to 14 years, included. Nine (56%) of them were males. IQ ranged between 20 and 98.
Clinical vignettes (averaging 3 - 5 pages in length) held the following information: child’s age and gender, wide descriptions of behavior and functioning in the following areas: ability to self-care (feeding, clothing, sleeping, personal hygiene, daily routines), communication (linguistic competencies, social communication, nonverbal communication, reading/writing), social behavior (family, peers) and school functioning (school level, performance and adaptive behavior). Moreover, there was an indication of coherence/incoherence of behavior among different settings, level of necessary environmental adaptation and level of required support.
Gold standard scores were obtained for these clinical cases, for each of them, from the average scores given by the six developers.
Independently, six clinicians assessed the clinical vignettes to evaluate inter-rater reliability. Evaluators had different levels of training and experience with ASD. They had familiarized with DD-CGAS and its scoring, and had discussed and jointly reviewed six vignettes for training purposes. Evaluators were located in two different sites: the Center of Pediatric Neurology in the University of Catania (Italy) and the Service of Pediatric Neurology of ULSS 8 in the Veneto region (Italy).
The examiners didn’t know that they would repeat the evaluation after about six months from the first assessment. All examiners performed the re-valuation to measure temporal stability.
2.4. Validity and Sensitivity to Changes
2.4.1. Procedures
DD-CGAS was included among the assessment instruments usually used for monitoring patients with ASD at Center of Child Neurology of University of Catania (Italy) and at integrated clinic for autism of the Service of Child Neurology of ULSS 8, site of Montebelluna (Italy).
Independent evaluators for the study were certified to administer DD-CGAS by the rating of clinical vignettes previously described, through an exchange of emails: for reliability, evaluators independently assessed six clinical cases, to which developers assigned the gold standard score.
An evaluator would be certified only if in 80% of the clinical vignettes’ scoring, the difference was not greater than 10 points from the gold standard score. If an evaluator could not be certified with the first six vignettes, he would have another training session available and then would evaluate another group of six clinical cases. If necessary, a third test of four vignettes was further available. Of these evaluators, 5 out of 6 obtained certification within the first test; 1 evaluator obtained the certification at the third test.
DD-CGAS was assessed by an independent evaluator according to the instruction in Appendixe A and Appendixe B, using all clinical data and tests possessed.
All subjects involved in the study contributed to the score of tests at baseline to assess validity of DD-CGAS. A subgroup of subjects was re-evaluated after an average of six months to assess the sensibility to measure the change of DD-CGAS.
Individuals of subgroups re-assessed in the follow-up underwent a wide range of interventions: pharmacological, behavioral treatment, parent training, psycho-educational and scholastic intervention, psychomotor or no intervention.
2.4.2. Subjects
A total of 48 subjects were involved in the base score to assess the validity of DD-CGAS. All subjects had IQ > 35 or a mental age > 18 months. The average age was 6 years (range 2 - 13 years, SD 3.37 years). 39 subjects (81%) were male. Diagnosis, according to DSM-IV criteria [26] , was as follows: autistic disorder, 25 subjects; Pervasive Developmental Disorder, not otherwise specified, 17 subjects; Asperger syndrome, 6 subjects. Baseline scores of DD-CGAS varied from 22 to 74 (mean 57.5).
42 subjects were re-evaluated at follow-up, average age 7 years (range 2.6 - 13.5, SD 2.9). 34 of them were male (80%). Diagnosis was as follows: autistic disorder, 24 subjects; Pervasive Developmental Disorder, not otherwise specified, 14 subjects; Asperger syndrome, 4 subjects. Baseline scores of DD-CGAS varied from 22 to 78 (mean 62.9). Table 1 shows the subject’s features.
2.4.3. Instruments
Vineland Adaptive Behavior Scales-Survey Form (VABS) are a standardized instrument of measure of adaptive functioning based on parent interview. Composite scale represents a summary of the total score with mean 100
Table 1 . Subjects features.
an = 45; bn = 42; cn = 12; dn = 10; en = 10; fn = 9; gn = 14; hn = 17; in = 37; jn = 35; kn = 40; ln = 38.
and SD 15. Higher scores indicate a higher adaptive functioning.
WPPSI-III is a clinical instrument of individual assessment evaluating the intelligence of children of ages between 2 years and 6 months and 7 years and 3 months; WISC III assesses intellectual ability of subjects of ages between 6 and 16 years and 11 months; both provide a verbal IQ (VIQ), a performance IQ (PIQ) and a total IQ (TIQ), with mean 100 and SD 15 [27] ; exclusively TIQ was used for the study.
The Leiter International Performance Scale-Revised (Leiter-R) [28] is a nonverbal intelligence test for children and teenagers aged between 2 and 20 years. The test provides a composite score with mean 100 and SD 15.
PEP-3, Psychoeducational Profile third edition [29] is an assessment instrument for children with autism spectrum disorder and communication impairments aged between 6 months and 7 years old. The test is divided into 13 subtests: 10 of direct observation and 3 derived from parent questionnaire. In this study the verbal/preverbal cognitive subtest was used to obtain an estimation of cognitive functioning (developmental quotient, DQ) when no information was available from other tools (WPPSI/WISC, Leiter-R) and DQ was the relation between developmental age obtained in the subscale and chronological age × 100. Parent questionnaire gives information about Problem Behavior (PB), Personal Autonomy (PA), Adaptive Behavior (AB). Standard scores were used in the study.
Autism Diagnostic Observation Schedule (ADOS) [30] is an instrument to assess and evaluate autism based on four different modules used in relation to developmental or language levels of the examined subject. In this study, to conform data from different subjects assessed with different modules, it was decided to express the scores of areas A (language and communication) and B (reciprocal social interaction) using the following formula: total score/cut-off score for autism in that area × 100.
2.5. Data Analysis
2.5.1. Inter-Rater Reliability and Temporal Stability
To assess inter-rater reliability the intraclass correlation coefficient (ICC) were calculated on the first scores obtained in the reliability vignettes for six previous examiners. The ICC were even calculated on test-retest scores of reliability vignette to assess temporal stability.
2.5.2. Convergent Validity
Pearson correlation coefficients were used to assess convergent validity between DD-CGAS and other clinical measures for 48 subjects at baseline.
The score at composite scale of VABS, IQ and DQ were used as ordinal variables. Not all correlations were based on the same sample, as some lacked data, whereas different assessment tools were used.
In this study, consistent with Wagner’s work, no corrections were made for multiple comparisons, α value fixed at 0.05 and in some correlations analysis the association should be interpreted in terms of effect size. According to the guidelines given by Cohen [31] , a correlation 0.10 represents a small effect size, 0.30 a moderate effect size and 0.50 a wide effect size.
2.5.3. Sensitivity to Change
42 out of 48 subjects were reassessed with follow-up after an average of six months. Sensitivity to change in DD-CGAS was assessed correlating changes into DD-CGAS with changes into PB scale of PEP3. To estimate the effect size between baseline and follow-up, the DS pooled was calculated.
3. Results
3.1. Inter-Rater Reliability and Temporal Stability
ICC for 5 examiners for 16 clinical cases was 0.78 (p 0.001). ICC between test and retest had a mean of 0.79 (p 0.001). Therefore statistically, both ICCs were significant.
3.2. Convergent Validity
With α value at 0.05, DD-CGAS resulted in correlation with all the other assessment tools. Correlations between DD-CGAS and the other instruments are in Table2
The correlation was significant and positive with IQ obtained in WPPSI/WISC and in Leiter-R (respectively r 0.51, p < 0.001 and r 0.40, p 0.003), with DQ obtained in PB subscale of PEP3 (r −0.52, p < 0.001) and with PB, PA and AB subscales of PEP3 (respectively r 0.28, p 0.010; r 0.27, p 0.013; r 0.28, p 0.010). Correlation was significant and negative with A and B areas of ADOS (for both r −0.35, p 0.006).
3.3. Sensitivity to Change
Correlation between changing scores at DD-CGAS and CP subscale of PEP3 was 0.75 (n 42, p 0.01). The average score of DD-CGAS was 57.5 (SD 13.4) at baseline and 62.9 (SD 11.0) at follow-up (t test for paired samples 5, p 0.001). The average change in DD-CGAS was 4.8 points. The effect size for DD-CGAS was 0.59 (n. 42) and for CP subscale 0.54 (n. 42).
4. Discussion
DD-CGAS is an instrument for clinicians to assess global functioning in children with ASD.
Table 2. Correlations between DD-CGAS and the other instruments.
It is specifically designed to include a wide range of functioning, with interand intra-subject variability changing based on type and grade of impairment. Further, it is accompanied by instructions and a grid to assist the examiner during the rating.
DD-CGAS showed to have a good inter-rater reliability and a good temporal stability in a range of several months, using clinical vignettes.
Reliability was obtained with a diversified group of evaluators, in terms of background and level of competence.
Correlations between DD-CGAS and other measures of functioning and symptoms assessment were moderate [31] [32] .
This study was aimed to validate the Italian translation of DD-CGAS repeating data obtained in the validation study of the English version.
Despite differences between the two studies (number and characteristics of raters, sample and type of instruments used) the results obtained in DD-CGAS validation process, in terms of inter-rater reliability, temporal stability, convergent validity and sensitivity to change, substantially overlap the original study, confirming both the quality of translation and the value of the instrument itself.
After adequate training, DD-CGAS was designed to include the typical heterogeneity of ASD. It represents a reliable assessment tool of global functioning: it integrates multiple information sources, it is fast to administer after collecting necessary information and it seems appropriate to be used in clinical studies with children with ASD.
Data used for this study were collected before DSM-5 publication and a comparison with the DSM-5 severity scales could not be made. Despite this, we think that an instrument like DD-CGAS can be useful in the clinical setting for transitioning to the DSM-5, helping to better analyse and separate the constructs of impairment and disorder in order to provide a more appropriate and dimensional evaluation, according to the recent classification system.
NOTES
*Corresponding author.