Building Social and Emotional Competence in School Children: A Randomised Controlled Trial ()
1. Introduction
Pre-school social-emotional and problem-solving skills deficits predict poor long-term academic achievement, school drop-out, mental health problems, substance misuse and antisocial/aggressive behaviour (Webster-Stratton, 2011) . At school, these children have poor academic engagement and experience peer rejection, contributing to longer-term problems (Eisenberg, Valiente, & Eggum, 2010) . Welsh early years’ teachers report increasing levels of these difficulties (Hutchings, Williams, Martin, & Pritchard, 2011) . Universal access enables schools to identify such problems and school-based interventions help children to acquire social competencies (Webster-Stratton, 2011; Zins, Bloodworth, Weissberg, & Walberg, 2004) and reduce longer-term risks (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011) .
The Collaborative for Academic, Social, and Emotional Learning (CASEL, 2013) identifies five core competencies: self-awareness, self-management, social awareness, relationship skills and responsible decision-making (CASEL, 2013; Durlak et al., 2011; Payton et al., 2008) . These competencies lead to sustained friendships, positive teachers and school relationships, greater participation and achievement (Eisenberg et al., 2010) .
Social problem solving programmes significantly improve children’s social behaviour, conduct problems, and academic achievement (Durlak et al., 2011; Payton et al., 2008; Schindler et al., 2015) . There are two types of programmes: Universal and Targeted. Universal interventions are delivered to all pupils (Durlak et al., 2011) . Targeted interventions are delivered to children with identified behavioural and/or emotional problems (Payton et al., 2008) often in clinic settings. An estimated 80% - 90% of pupils benefit from universal interventions (Gresham, 2004) , however, for high-challenge pupils, this is insufficient (Payton et al., 2008) .
1.1. The Incredible Years Programmes
The Incredible YearsÒ (IY) is a suite of evidence-based programmes for parents, children and teachers (Webster-Stratton, 2011) . The IY teacher classroom management programme teaches strategies to: develop positive relationships with pupils, establish classroom rules, make clear transitions, give positive instructions, increase praise and incentives and manage school-based disruptive behaviour (Webster-Stratton, 2011) . Trials by the developer (Webster-Stratton, Reid, & Hammond, 2001, 2004) and independently in Jamaica, Ireland, New Zealand, Wales and Portugal (Baker-Henningham, Scott, Jones, & Walker, 2012; Fergusson, Horwood, & Stanley, 2013; Furlong & McGilloway, 2012; Webster-Stratton, Gaspar, & Seabra-Santos, 2012) show increased positive and reduced negative teacher behaviour towards high-challenge children and increased children’s academic engagement (Baker-Henningham et al., 2012; Webster-Stratton et al., 2004) .
There are two versions of the child programme: 1) a three-year Classroom Dinosaur School curriculum for 3 - 6 year-old children, delivered twice weekly in circle time lessons that covers all of the Welsh Personal Social Educational (PSE) curriculum and 2) the therapeutic Small Group Dinosaur School targeted intervention for small groups of high-challenge children aged 4 - 8 years, delivered in 18 - 22 two-hour weekly sessions, typically in clinics. Both programmes teach the CASEL core competencies of how to do your best in school, how to detect and understand feelings, social problem-solving, anger management, self-regulation and friendship skills (Webster-Stratton, 2000) . However, the therapeutic small group Dina version of the programme enables much higher rates of attention to individual children, an opportunity to tailor the programme to the specific needs of participant children including the development of detailed behaviour plans and the opportunity for children to undertake significantly more skills practice. Both versions in randomised controlled trials (RCTs) by the developer show increased children’s social and problem-solving skills and peer relationships (Webster-Stratton & Reid, 2004; Webster-Stratton et al., 2004) . Limited independent evidence from Norway, combining the therapeutic small group Dina and IY parent programmes with children with significant behavioural problems, showed significant reductions in aggressive behaviour (Larsson et al., 2009) .
1.2. Establishing and Evaluating the Teacher and Classroom Dinosaur Programmes in Wales
Welsh schools have a statutory l PSE curriculum and there has been significant take-up of the IY teacher classroom management and Classroom Dina programmes across Wales. Following a Welsh pilot (Hutchings et al., 2007) the teacher classroom management programme was endorsed by Welsh School Inspectors (Hutchings et al., 2011) and an RCT showed significant reductions in negative teacher and child behaviour and child off-task behaviour (Hutchings, Martin-Forbes, Daley, & Williams, 2013) .
A pilot of the Classroom Dina curriculum (Webster-Stratton, 2000) with 3-6-year-old children received positive school inspection reports (Hutchings, Lane, Owen, & Gwyn, 2004) but teachers felt that more was needed to support high-challenge children. At the same time, the small group Dina programme, delivered in a therapeutic service, demonstrated clinically significant improvements in child behaviour (Hutchings et al., 2007) and a school-based pilot targeting high challenge children demonstrated significant increases in pro-social problem solutions (Hutchings et al., 2012) .
1.3. Rationale for Current Study
Teacher classroom management trained teachers in Welsh primary schools that were delivering the universal classroom curriculum were recruited to an RCT to evaluate the therapeutic small group Dina programme with children with behavioural difficulties (Bywater, Hutchings, Whitaker, Evans, & Parry, 2012) . It was hypothesized that the programme would reduce teacher reported child behavioural difficulties, and that children would generate greater numbers of pro-social problem solutions.
2. Methods
2.1. Recruitment
Twenty-two schools were recruited, nine in 2010 and 13 in 2011. All the schools were located in the counties of Anglesey, Gwynedd, and Powys in north and mid Wales. Two of the schools were classified as small (less than 100 pupils), 12 were medium sized (between 100 and 200 pupils), and eight were large schools (over 200 pupils). In October of each year teachers completed the Teacher Strengths and Difficulties Questionnaire (TSDQ; Goodman, 1997 ), a screening measure for behavioural, emotional, and social difficulties in children aged 4 - 17 years, to identify potential children for the study. Of the 58 teachers who completed TSDQs, 21 (36%) delivered of the intervention. Independent researchers scored the TSDQs.
Target Child Inclusion Criteria
1) above the TSDQ borderline level for concern (a score of 12+ total difficulties).
2) aged between 4 - 8 years.
3) child and parent fluent in Welsh and/or English.
2.2. Participants
Fifty percent (n = 383) of screened children met inclusion criteria. Teachers contacted parents of the 12 highest scoring children in each school (73% of eligible children) for consent for a researcher to visit, 89% agreed, 92% of whom gave informed consent to participation. Randomisation, stratified by child age and sex, was completed with 225 children (113 intervention, 112 control). Two control children were removed due to ineligibility and one intervention and one control child removed at parental request (see Consort diagram, Figure 1).
Two hundred and twenty-one children (Mean age = 65.28 months; SD = 11.35) participated, 85 (38.5%) nursery/reception (3 - 5 years), 85 (38.5%) in Year 1 (5 - 6 years), 40 (18.1%) in Year 2 (6 - 7 years), and 11 (5.0%) in Year 3 (7 - 8 years). Sixty-two percent were male. As per the inclusion criteria, all children scored at least in the cause for concern range of whom 153 (69%) scored within the clinical range (TSDQ > 15). The mean free school meals level was 21.5% (5 to 44%) (Welsh national primary school average 18.8%) (Statistics for Wales, 2015) . See Table 1 for more demographic information.
2.3. Ethical Approval
Bangor University ethics committee and the North Wales Research Ethics Committee of the Betsi Cadwaladr University Health Board approved the trial. All participating schools and parents provided written informed consent. The
Table 1. Child, teacher, and school demographic characteristics by condition at baseline.
aCut-off of 15; TSDQ―Teacher Strengths and Difficulties Questionnaire.
study was registered in the UK Clinical Research Network Register: UKCRNID8615 and the International Standard Randomised Controlled Trial Number Register: ISRCTN96803379. This project was funded by the BIG LOTTERY (Grant #C1119A1193) awarded to Incredible Years Cymru, with Bangor University and Gwynedd Council as partners.
2.4. Randomization and Blinding
A pragmatic, parallel, RCT design was used with participants individually randomized on a 1:1 basis within each school, stratified by age and gender. Bangor University independent registered trials unit undertook randomization. Results were forwarded to the Principal Investigator. All trial data collectors remained blind to child condition allocation. Teachers and children were asked not to discuss condition allocation with researchers during collection of measures.
2.5. Measures
2.5.1. Teacher-Completed Measures
The Teacher Strengths and Difficulties Questionnaire (TSDQ) is a 25-item screening measure for emotional, conduct, hyperactivity, peer problems and pro-social behaviours. The four problem subscales generate a Total Difficulties score. It has good internal consistency (α = .63 - .83), good test-retest reliability (r = .72 - .85), concurrent relations with the Child Behaviour Checklist (r = .57 - .79) (Stone, Otten, Engels, Vermulst, & Janssens, 2010) and standardised total difficulty cut-off levels. It was used as the inclusion screener and primary outcome measure.
Child academic achievement was measured using teacher-set targets. Teachers in Wales set mathematics, language, and personal-social skills targets for children at the beginning of the school year and evaluate them at the end of the year. Sixty-six percent of children (n = 145) had targets for mathematics, 76% (n = 169) for language, and 84% (n = 185) personal-social targets.
A teacher demographic questionnaire provided information about the gender, age, and teaching experience of participating teachers.
2.5.2. Child-Completed Measure
The Wally Problem-Solving Task (WPST; Webster-Stratton et al., 2001 ) assesses children’s problem-solving knowledge and solution types. Children view a series of social problems and are asked to generate solutions. Pro-social solution categories have satisfactory internal consistency (α = .55) as do agonistic solution categories (α = .54). Pro-social categories correlate with positive (r = .60) and agonistic with negative (r = .50) strategies of Rubin and Krasnor’s (1986) Child Problem-Solving Test demonstrating construct validity. The current study presented seven problems. Children’s responses were audiotaped and hand-recorded. Responses were scored and re-coded by the same blind researcher four months later to establish reliability (87%).
2.6. Intervention
The small group Dina programme was delivered by school-based teachers/support staff. The programmes uses videos demonstrating appropriate behaviour, discussions and role-play skills practice. Child-sized puppets encourage children to solve problems. Weekly homework assignments involve children talking with parents about the lessons to encourage generalisation (Webster-Stratton & Reid, 2003) .
2.6.1. Facilitators
The intervention was delivered to up to six children per school, by two teachers or one teacher and one support staff member for two hours a week for 18-weeks. Staff had previously received IY Classroom Dina training and attended a two-day small group Dina programme training.
2.6.2. Fidelity
Facilitators received monthly supervision with the last author to discuss video-recordings of sessions. Fidelity was assessed by intervention dosage, number of sessions attended and homework assignments completed. Adherence was assessed through leader-completed checklists recording delivery of core curriculum components (Webster-Stratton, 2000) . Additionally, a seconded teacher (fourth author) observed a session and assessed programme fidelity, rating schools on a three-point scale. Seven schools scored 1 (excellent fidelity), ten scored 2 (good fidelity), and five scored 3 (some fidelity problems). Problems reported by teachers included staff illness and lack of preparation/delivery time and head-teacher support. Sixteen schools (73%) delivered 18 sessions, three (14%) delivered 17, one (4%) delivered 16 and two (9%) delivered 14 sessions (mean 17.4 sessions).
2.7. Procedures
Each tranche of the trial spanned a school academic year. Parents were contacted by researchers in October/November to explain the research and obtain consent for their child’s participation prior to baseline measure collection. The intervention was delivered between January and May. Follow-up measures were collected in June-July.
2.8. Analysis Strategy
All variables were checked for normality. Normality was rejected for both WPST categories and all TSDQ subscales. All categories were normalised by squaring the raw score. The academic attainment variables were analysed as categorical variables.
To examine the effect of fidelity, intervention schools were classed as having excellent, good, or problematic programme delivery based on observations from a coder (seconded teacher). The mean TSDQ total difficulties scores (and standard deviations) for the children in those schools were then compared across categories for both pre and post assessment to explore any differences using an ANCOVA model.
The effect of intervention was examined using hierarchical linear modeling analyses in SPSS 20.0. A two-level model was examined: children at Level 1 and schools at Level 2. The dependent variables were post-intervention child behaviour and problem-solving outcomes, and academic attainment targets. Covariates included child age, child gender, and pre-intervention scores at Level 1.
Analyses were conducted on an intention-to-treat basis with pre-intervention scores carried forward for post-intervention missing data. Full maximum estimation likelihood was used to account for missing data under an assumption that data was missing at random. Missing data on pre- and post-intervention measures ranged from 0% (TSDQ subscales) to 1.8% (WPST). All covariates had no missing data. Standardised mean differences were computed for the effect size using standard deviations generated in the multilevel model analyses, apart from for categorical variables, where odds ratios were computed. An alpha level of .05 was used for all significance tests. In order to determine whether fidelity should be included, an analysis of covariance (ANCOVA) test was performed to see whether there were significant differences between the fidelity levels based on the primary outcome (TSDQ total difficulties score).
3. Results
3.1. Pre-Intervention Analyses
There were no significant differences in child or family demographics between baseline intervention and control conditions (see Table 1). The screening requirement was above the TSDQ cause for concern cut-off of 11, however 69% were above the clinical cut-off of 15 representing a sample with significant problems. All subsequent analyses controlled for the effects of Level 1 (children) pre-intervention scores.
3.2. Fidelity
Intervention children attended on average 14 sessions (M = 14.58, SD = 3.83) with 47.8% completing at least 10 homework assignments. Table 2 displays the descriptives for fidelity, as assessed in the classroom observations by the seconded research teacher, based on the primary outcome measure (the TSDQ
Table 2. Descriptive statistics for TSDQ total difficulties plit by fidelity levels.
total difficulties score). An ANCOVA showed no significant difference between the different levels of fidelity (p > .05) consequently fidelity was not added into the multilevel model.
3.3. Child Behaviour Outcomes
Intervention effects on child behaviour outcomes are displayed in Table 3. There were no significant post-intervention differences between conditions for teacher-reported outcomes (p > .05) with both children in the intervention and control schools showing similar reductions in the TSDQ total difficulties score.
3.4. Child Problem-Solving Knowledge Outcomes
Table 4 displays the results for child problem-solving knowledge. There was a significant main effect of condition on pro-social and agonistic solutions in the WPST; intervention children showed a significant increase in post-intervention number of pro-social solutions (ES = .39) and a significant post-intervention reduction in agonistic solutions (ES = .41).
3.5. Academic Attainment
Teachers are expected to set mathematics, language and personal-social academic attainment goals at the beginning of the school year and to reassess them at the end of the year. No effect was found on mathematics or language targets but there was a significant main effect of condition on personal-social targets with intervention children significantly more likely than control children to reach their target (Odds ratio = 3.00). Results are displayed in Table 5.
4. Discussion
The National Institute for Health and Social Care Excellence (2008) suggests using universal and targeted programmes within schools. This RCT evaluated whether the therapeutic IY small group Dina programme had added benefit for children with behavioural difficulties in schools already delivering their personal social educational curriculum by delivering the IY Classroom Dina by IY teacher classroom management trained teachers. The recruited sample scored above the cause for concern on the TSDQ, with 69% within the clinical range. It was hypothesised that intervention children would show increased social problem-solving and reduced behaviour problems compared to control children. A two-level multilevel model analysis showed that therapeutic IY small group Dina
Table 3. Hierarchical linear modelling results for child behaviour outcomes with descriptive statistics by condition and effect size.
Table 4. Hierarchical linear modelling results for child problem-solving outcomes with descriptive statistics by condition and effect size.
*p < .01, WPST―Wally Problem-Solving Task.
Table 5. Hierarchical linear modelling results for child behaviour outcomes with descriptive statistics by condition and effect size.
aIntervention N = 71, Control N = 74; bIntervention N = 81, Control N = 88; cIntervention N = 96, Control N = 89; *p < .01.
programme did not benefit intervention children compared to control children in terms of reductions in behavior problems, however both intervention and control children demonstrated substantial improvements in TSDQ scores with a mean reduction of 5.06 for intervention and 4.54 for control children. Intervention relative to control children showed significant benefits for problem-solving knowledge and achievement of teacher set personal-social targets.
4.1. Child Behaviour
The main study hypothesis was not supported because, based on teacher reports, there was no significant added benefit of the therapeutic IY small group Dina programme. However, schools were making a significant commitment to children’s social problem-solving and behaviour through universal IY Classroom Dina lessons delivered by IY teacher classroom management trained teachers and children in both conditions showed substantial reductions in behaviour problems across the academic year.
Another possible reason for lack of additional intervention benefit could be intervention leakage, 36% of teachers were class teachers for both intervention and control children to whom they delivered the classroom curriculum. The curriculum for both programmes is the same however the therapeutic version includes specific training in individualised behavior planning and more role-play coaching activities. Teachers may have been using small group strategies with control children or strategies learned in small group training may have changed the way that they delivered the universal curriculum.
Another possibility relates to the children themselves, some intervention children were in the same classes as control children and antisocial behavior often occurs in peer interactions. Intervention children may have used/modeled strategies learned in small group sessions with their peers.
The difference in teacher reported social skills outcomes could be the result of teacher bias and expectancy effects. Thirty-six percent of teachers completing outcome assessments were delivering the small group Dina intervention. Teacher bias is unavoidable in school-based research but their input is invaluable, especially when coupled with other data sources.
4.2. Child Problem-Solving Knowledge
There was a significant main effect of condition on independently assessed child problem-solving knowledge. Intervention children were better able to recognise and understand feelings, and generate appropriate post-intervention solutions. Similar findings were reported in previous trials of this programme (Webster-Stratton et al., 2001) . Problem-solving skills are important for children’s social-emotional competence and deficient in children with behaviour problems (Walker & Henderson, 2012; Webster-Stratton et al., 2001) . Aggressive and shy children are significantly more likely to offer less competent strategies in social situations (Eisenberg et al., 2010; Walker & Henderson, 2012) . Intervention benefits on problem-solving were greater than the generally observed improvements achieved by classroom management trained teachers delivering the Classroom Dina curriculum in children displaying varied behaviour problems.
4.3. Academic Attainment
Teachers were required to set numeracy, language and personal-social skills academic attainment goals at the start of the academic year and to assess them at the end of the year. This is the first known study to demonstrate therapeutic IY small group Dina programme benefits on teacher set personal?social goals, with intervention children significantly more likely than controls to achieve these targets, strengthening evidence for the programme. Together with benefits for social problem-solving skills, they indicate that the programme enhances aspects of social-emotional competence in young children at significant risk of behavioural, social and/or emotional difficulties.
4.4. Strengths
This was the first RCT of the IY therapeutic small group Dina programme delivered by IY teacher classroom management trained teachers, alongside the Classroom Dina curriculum to a screened, but non-referred, sample of children. All previously reported evaluations of this programme were treatment studies with clinically referred children, delivered by trained therapists. This is an important study strength because delivering the programme through schools can target younger children before problems become entrenched. Early intervention and prevention are essential for optimal child development (Allen, 2011) . The longer-term consequence of not intervening predicts many adverse later outcomes (Adi, Killoran, Janmohamed, & Stewart-Brown, 2007) and negative financial consequences (Scott, Knapp, Henderson, & Maughan, 2001) . Having school-based staff delivering the intervention embeds skills within schools creating opportunities for behaviours to be reinforced throughout the school day. It is also important for sustainability and potentially cost-effective, reaching generations of pupils (Bywater, 2012) .
4.5. Limitations and Future Directions
The study has several limitations. First, to reduce demands on teachers there were insufficient teacher-reported and other school-based measures. Only the TSDQ, a screening tool that may not have been sensitive enough to detect intervention effects, was used. Future research should employ more school-based assessments including measures of peer relationships and independent, blind observations that are the gold-standard in research (Flay et al., 2005) .
Second, control and intervention children were in the same schools and, in many cases, the same classrooms and supported by teachers and/or classroom assistants who delivered the therapeutic IY small group Dina programme. This may have led to intervention leakage. A cluster RCT design with randomisation of schools would eliminate this problem. However, trials must balance design with feasibility, cost, and timeline (Bywater, 2012) . Funding restrictions and challenges in recruiting the required number of schools in an authority that had invested significantly in training its teachers in the IY teacher classroom management and Classroom Dina programmes made a cluster RCT design unfeasible. Future research should explore other research designs such as stepped wedge RCT which may be more feasible (Hemming, Haines, Chilton, Girling, & Lilford, 2015) .
Third, there was no fidelity measurement for the universal IY components. All teachers were IY Classroom Dina and teacher classroom management trained, however, no fidelity measure of these components was taken, consequently their impact was unknown. Well-implemented programmes produce significantly larger effects than poorly-implemented programmes (Durlak et al., 2011; Payton et al., 2008) . Future research should ensure that the fidelity of all study components is monitored to ensure that potential impact is accurately measured.
5. Conclusion
This was the first trial of the therapeutic IY small group programme delivered alongside the IY teacher classroom management and Classroom Dina programmes to young high-risk non-referred children. Intervention children showed increased problem-solving skills and greater teacher-set personal-social achievement targets. These results were achieved by teachers new to the IY small group Dina programme that requires significant skills to deliver to high challenge children for up to 18 weeks. This study is an important addition to the literature on the therapeutic IY small group Dina programme.