Evaluation of a Gamified Physical Activity Intervention Targeting School-Children ()
1. Introduction
Accumulating 60 minutes of moderate to vigorous physical activity (MVPA) per day provides children with significant physical (Poitras et al., 2016) , psychosocial (Dale, Vanderloo, Moore, & Faulkner, 2019) and cognitive health benefits (Lubans et al., 2016) . However, the majority of children do not get sufficient physical activity (PA). In Ireland, 81% of primary school-children do not meet the threshold of 60 minutes MVPA (Woods et al., 2019) . Dependency on car travel in Ireland is a contributory factor. The number of primary school-children actively commuting to school decreased from 49.5% in 1986 to 25% in 2016. Simultaneously, car usage increased from 24% to 59.8% during the same period (CSO, 2017) . The promotion of walking and cycling for transport can be an effective way of incorporating PA into daily living while simultaneously reducing the modal split for motorised modes of travel and CO2 emissions (Larouche, Mammen, Rowe, & Faulkner, 2018) .
Community wide physical activity interventions aim to target an entire community such as a town or city with a multi-level physical activity intervention as opposed to targeting individuals as was traditionally the case. Despite their potential for behaviour change, community wide interventions are difficult to undertake, often do not achieve sufficient penetration across the population and consequently rarely produce significant intervention effects (Baker, Francis, Soares, Weightman, & Foster, 2015) . Furthermore, evidence from recent systematic reviews has indicated that traditional interventions targeting school-children have produced only small to modest effects on physical activity and active travel (Larouche et al., 2018; Villa-González, Barranco-Ruiz, Evenson, & Chillon, 2018) . The vast majority of these studies have used cross-sectional research designs, and few are from car-dependent countries other than the US or Australia. The limited intervention studies have rarely included comparison sites (Chapman et al., 2014) or long durations of follow-up (Rissel et al., 2013) . These issues are partly explained by the complexities of evaluating natural experiments where experimental control is low (Craig et al., 2012) . These gaps in the evidence for community-wide active travel interventions also apply to schools. There is a paucity of intervention studies assessing active travel to school as a mechanism to increase physical activity (Larouche et al., 2018) .
It is possible that gamified active travel interventions using smartphones (e.g. Pokemon Go) may enhance the potential for behaviour change. Gamified interventions use game design elements in a non-game context (Nacke & Deterding, 2017) . They are primarily based on an intrinsic reward system and are potentially more enjoyable than traditional interventions, yet capable of producing desirable changes (Sardi, Idri, & Fernandez-Aleman, 2017) . In health-related settings, gamified interventions can engage and motivate players to achieve specific health-related goals. While experimental research in this area is in its infancy and mostly unrelated to active travel, early indications suggest that gamified physical activity interventions can be successful at least in the short term (Garde et al., 2018; Johnson et al., 2016) . Goodyear et al.’s (2021) systematic review of 26 online physical activity interventions targeting children concluded that there is sufficient evidence to suggest that interventions that utilize gamification can have a positive impact on children’s engagement with physical activity. The authors identified personalisation, whereby participants get tailored feedback and rewards/goals based on their progress, as an essential mechanism for facilitating an intervention effect on physical activity levels. However, the majority of studies included in this review used self-report measures of physical activity. A meta-analysis of studies using objectively measured physical activity data failed to detect any intervention effect for school-based physical activity interventions (Love, Adams, & van Sluijs, 2019) . The authors highlighted the lack of process evaluations in the published literature. This limits our understanding of intervention fidelity, the reach of the intervention and the characteristics of participant engagement with the intervention.
Beat the Street (BTS) is an example of a gamified physical activity intervention. It is a real-life walking and cycling game that aims to increase physical activity through the gamification of entire communities. Participants earn points by tapping special cards and fob keys onto sensors known as Beat Boxes attached to lamp posts. A previous evaluation of the BTS intervention reported a small increase in objectively measures MVPA in a controlled study of 8 - 10 year old children but only for those that had higher levels of engagement (Coombes & Jones, 2016) . There is limited experimental evidence on the effectiveness of BTS and the factors which influence player engagement. The BTS game was implemented in Waterford, Ireland in Autumn, 2017. The purpose of this study was to determine whether a community wide physical activity game could increase physical activity and active travel in school children in Waterford, Ireland.
2. Methods
2.1. Research Design
This was a quasi-experimental evaluation of a natural experiment using repeat cross-sectional surveys of primary school-children in Waterford City, Ireland and analysis of time-stamped participation data (beat box tags). The majority of active travel studies have used cross-sectional designs. Cross-sectional research is useful in the design of interventions, but they do not provide any evidence of causality. The limited intervention studies have tended to be quasi-experimental trials. Randomised control trials are typically not feasible to measure the impact of complex community-wide active travel interventions (Krizek, Forsyth, & Baum, 2009) . While natural experiments have problems associated with lack of experimental controls, they often have high external validity (Craig et al., 2012) . Repeat cross-sectional and panel designs are the two primary options for measuring the quantitative impact of these natural experiments. Waterford Institute of Technology’s Research Ethics Committee granted ethical approval for the study.
2.2. The Beat the Street Intervention
The game lasted for 7 weeks, beginning on Wednesday September 13th and finished on Wednesday November 1st, 2017. Beat Boxes were installed in 48 locations in Waterford City and Fob keys (for children), a BTS card (for parents), location maps and information letters were distributed to every school-child attending a primary school and secondary school in the area. Playing the game consisted of tapping the card or fob on at least 2 Beat Boxes within an hour to earn 10 points for yourself and your team (school). Players could check their progress on the BTS website and prizes were awarded to the teams with the highest number of points. There was a full-time employee appointed to coordinate the game in Waterford. This person co-ordinated an intensive social media campaign and a comprehensive schedule of community engagement events to support the roll-out of the game. The BTS game in Waterford targeted the whole community, i.e. both school-children and adults. This evaluation focuses on the impact of BTS on primary school-children, the population group considered most likely to be enthused by the game.
2.3. Study Population and Sampling
The BTS game in Waterford predominantly targeted primary school-children and the beat boxes were strategically located adjacent to all primary schools in Waterford city. Therefore, the population of interest in this study was 5th and 6th class students attending all primary schools in Waterford City. Eighteen schools were invited to participate in the BTS intervention. Two schools declined this invitation but agreed to serve as control schools. An information letter was sent home to the parents of all 5th and 6th class students (age 10 - 13 years) in participating schools in October 2016 (one year prior to the intervention). This letter sought the passive consent of parents for their child to complete a baseline survey in November 2016. All children enrolled in either 5th and 6th class in all 18 schools were included in the final sample. This process was repeated at follow-up in 2017. Passive consent was sought in October 2017 and the follow-up surveys were completed in November 2017.
2.4. Data Collection Tools
The baseline instrument was a two-section self-reported questionnaire adapted from an evaluation of a similar Irish programme (Lambe, Murphy, & Bauman, 2017) . The first section included several demographic questions. The second section measured both mode of travel to school (walk, cycle, car, bus or other) and physical activity. The mode of travel to school was assessed as usual mode (consistent with national census data) and preferred mode. Questions on independent mobility and household car availability were also included. Physical activity level was measured using the one-item question developed and validated by Milton, Bull and Bauman (2011) . The follow-up survey contained one additional section: awareness of and level of engagement with BTS.
2.5. Data Collection Method
The questionnaires were completed during class time within the schools, and the process was facilitated by a member of the research team, using standardised instructions. The research assistants received training on these procedures before commencing data collection. Children completed the survey one question at a time and only after the question was explained to the class by a member of the research team. The class group then answered the question simultaneously.
2.6. Data Analysis
Chi-square was used to examine cross-sectional differences between categorical and continuous variables, respectively. The absolute change in the proportions (difference in differences) of students engaging in active travel and meeting the physical activity guidelines, were calculated using 95% confidence intervals consistent with previous research (Lambe, Murphy, & Bauman, 2017) . Binary logistic regression analysis was conducted to examine the factors associated with active travel to school and the likelihood of meeting the physical activity guidelines.
3. Results
3.1. Sample Characteristics
The characteristics of the sample at baseline and follow-up are shown in Table 1. The survey response rates were 92% at baseline and 94% at follow-up.
3.2. Participant Awareness of Beat the Street
Participant awareness of BTS was very high and significantly higher in intervention schools compared with control schools (Figure 1). This was particularly evident for unprompted awareness (p < 0.01). Approximately 54% of the intervention group recalled the BTS game unprompted compared with only 26% of the control group.
3.3. Participant Engagement with Beat the Street
Almost all participants (98.5%) received a BTS fob or key ring from their school. Girls were significantly more likely than boys (p < 0.001) to state that they played the game regularly (Figure 2). Approximately 18% of girls played the
![]()
Table 1. Sample characteristics at baseline and follow-up in intervention and control schools.
*p < 0.05, **p < 0.001.
![]()
Figure 1. Prompted and unprompted awareness of the BTS game in primary school-children.
![]()
Figure 2. How frequently school-children played BTS.
game every day compared with only 9.2% of boys. The majority (42.3%) of school-children played the game with their parents or another adult. Girls were more likely to play the game with their friends. Only 11.8% of girls stated that they played the game “mostly on their own” compared with 21.8% of boys (p < 0.01). There were several features of BTS that greatly added to the school-children’s enjoyment of the game. The most important of these was the competitive nature of the game which included collecting points, the competition between schools and winning prizes. Spending more time outdoors, walking more, exploring new places and spending more time with friends were other reasons why they enjoyed the game.
3.4. The Impact of Beat the Street on Physical Activity and Active Travel
There was no overall change in the number of days primary school-children achieved 60 minutes of MVPA at follow-up in intervention versus control schools. Despite this, school-children that played BTS more than once a week were 64% more likely to meet the physical activity guidelines compared with those who played less frequently (p < 0.01). Almost 50% of participants visited a place they had never been before and of these 70% expressed a desire to return there at a future time. Although 54% of these places were “another street” in their town, 41% were locations conducive to physical activity such as a park or rail trail.
Overall, the intervention had no effect on walking or cycling to school (Table 2). However, there was an absolute increase of 1.7% in the proportion of children walking home from school. This represented an absolute increase of 14.2% relative to the control schools. This trend was more pronounced for girls. The proportion of girls walking home from school increased from 32.9% at baseline to 36.1% at follow-up. Schoolchildren that could recall the BTS game without prompting and those that played the game more than once a week were 1.8 (p < 0.05) and 3 times (p < 0.001) more likely to walk or cycle to school, respectively. The majority of school-children were not allowed to travel to school without another adult. Only 30.3% of boys and 22.1% of girls travelled to school unaccompanied by an adult. Independent mobility was an important predictor of active travel to school. Schoolchildren that were allowed to travel to school without another adult were 26 times more likely to walk or cycle (p < 0.001; OR 25.9, 95%CI, 15.58, 43.06) compared with those who were not allowed.
4. Discussion
Beat the Street is an example of an innovative gamification intervention that has been implemented at scale. The game in Waterford, Ireland achieved very high
![]()
Table 2. The actual mode of travel TO and FROM school pre-and post-BTS in intervention and control schools.
levels of both prompted and unprompted awareness in intervention compared to control schools. Participation levels were reasonably high with girls significantly more likely than boys to state that they played the game regularly. The intervention had no impact on physical activity in primary school-children although those that played BTS more than once a week were more likely to meet the physical activity guidelines compared with those who played less frequently. Similarly, there was no overall intervention effect detected for active travel to school. However, there was a modest increase in the proportion of children walking home from school which was more pronounced for girls. This finding should be interpreted with caution however due to the small sample size in control schools. There was also evidence that those who could recall the BTS game without prompting and those that played the game more than once a week were more likely to walk or cycle to school.
The lack of a significant intervention effect for physical activity and a larger effect for active travel is not unusual and can be explained by several factors related to the research design and the characteristics of the intervention. Examples of interventions that have produced demonstrable and sustained changes in physical activity behaviour are uncommon in the literature. Evidence from a recent systematic review has indicated that traditional interventions targeting school-children have produced only small to modest effects on physical activity and active travel (Larouche et al., 2018) . Even one of the more successful augmented reality physical activity games, Pokémon Go, only managed to increase physical activity for 6 weeks in a controlled study of 1182 US adults (Howe et al., 2016) .
The research design may also have mitigated against detecting an intervention effect. A repeat cross-sectional design aims to detect an intervention effect across an entire population; that is, it includes respondents that have never participated in the intervention. It is plausible that there would have been a short-term increase in physical activity and active travel after the initial 2 - 3 weeks of the game given the high levels of engagement or had the sample been limited only to active players. Another limitation of the research design was that it could be argued that the outcome measures adopted in this study were too narrow. The true benefits of BTS may not be tangible or easily measured. A similar, gamification-inspired, school-based active travel programme noted improvements beyond physical activity and active travel (Rutberg & Lindqvist, 2019) . A qualitative evaluation of this multi-component programme reported improvements in togetherness, readiness to learn, student engagement and parental attitudes to active travel. This is supported by the children’s’ expression of the extent to which they enjoyed the programme, and how they enjoyed spending more time with family and visiting recreational sites where they had never been before.
The lack of an intervention effect may also be attributed to the intervention design. Given that girls displayed greater engagement in BTS Waterford, it may have been more effective had the intervention been tailored specifically for girls or boys and girls separately. Biddle, Braith and Pearson (2014) highlighted that for girls, the greatest intervention effects are seen in programmes that target girls only, and which use educational and multicomponent strategies. According to Heath et al. (2012) community based physical activity programmes are most effective when there are intersectoral partnerships at local level. Additionally, multicomponent, school-based programmes are the most effective way to increase physical activity in school-children. Multi-component programmes aim to embed the intervention into the school curricula and extra-curricular activities as well having community and parental involvement (Biddle et al., 2014) . This broader ecological model of physical activity promotion (Sallis et al., 2006) can only be achieved through intersectoral collaboration. While the BTS game achieved high levels of community engagement and partnership working, it is probably best described as a single component intervention and without true intersectoral collaboration. Achieving meaningful intersectoral collaboration is difficult in the absence of community empowerment and ownership. This was echoed elsewhere (Reis et al., 2016) who stated that intersectoral collaboration and institutionalisation are integral to the scalability of physical activity interventions.
Another potential contributing factor to the lack of an intervention effect was that the intervention aimed to simultaneously promote multiple behaviours (i.e. active travel and leisure time physical activity) across multiple population groups. It is plausible that this may have served to dilute the intervention effect as has been suggested elsewhere (Atkin, Gorely, Biddle, Cahill, & Foster, 2011; Christiansen, Toftager, Ersboll, & Troelson, 2014) . The data from this evaluation suggest that active travel to school is a very appropriate focus of BTS. A previous evaluation of BTS in the UK reported an increase in active travel to school post intervention (Coombes & Jones, 2016) . Active travel to school is a domain of physical activity with unique mediators of change. A recent systematic review of school-based active travel interventions (Larouche et al., 2018) stated that the most successful active travel to school programmes have included a combination of infrastructure, programme and policy measures. The BTS game doesn’t specifically target the factors that influence active travel to school. There may also have been infrastructural barriers specific to Waterford City which mitigated against a greater increase in active travel. Levels of active travel to school are lower in Waterford City compared with the national average (CSO, 2017) . This study identified independent mobility as an important predictor of active travel to school. Despite this, primary school-children were more likely to play the game while being accompanied by an adult. Parents are gatekeepers of children’s independent mobility and are therefore an essential component of BTS for primary school children. Given the trend towards increasing car travel to school and the considerable parental barriers to active travel to school, there is an argument for targeting older children who have already been granted independent mobility. However, the game appears to appeal more to younger children.
One of the major strengths of BTS Waterford was the extent of community engagement and campaign penetration. The high level of programme awareness is rare for a community-wide physical activity programme and a major success of BTS Waterford. Unprompted awareness of BTS was considerably higher than in other mass media physical activity campaigns targeting young school-children (Huhman et al., 2010) . These positive outcomes are most likely explained by the intensive school and community engagement in Waterford, the inclusion of parents, the very active social media presence and the novelty factor associated with gamification interventions. Indeed, this novelty factor may also have been responsible for a potential decline in participation over the seven weeks. Given the high levels of programme awareness among school-children, higher levels of weekly participation might have been expected. Only 33% of boys played BTS more than once a week over the seven weeks of the intervention. This phenomenon is normal for gamification programmes after the novelty effect has diminished (Garde et al., 2018; Hunter, de Silva, Reynolds, Bird, & Fox, 2015) . Similar trends were also reported in a recent evaluation of the Pokémon Go game (LeBlanc & Chaput, 2017) . The design of the BTS game itself may have also contributed to this trend and prompted many children to suggest multiple ways in which the game could be improved. There are several important strategies that are required to develop the persuasive architecture of gamification (Cugelman, 2013) . These are committing to achieve a goal, capacity to overcome challenges, providing feedback on performance, gaining rewards and avoiding punishments, monitoring progress with self and others, interacting with other people and fun and playfulness through playing out an alternative reality. Many of these features were missing from the BTS game and highlighted by respondents. Small changes to address these deficits would greatly add to its persuasive architecture and work to engage older school-children.
5. Conclusion
Gamification of physical activity can play an important role in increasing active travel in primary school-children. Gamified interventions should consider treating active travel and leisure time physical activity as separate and distinct behaviours. They should also be developed and implemented separately for boys and girls. It is essential that the persuasive architecture of gamified physical activity interventions be comprehensively redeveloped using smartphone technology in a bid to make the game more attractive to adolescents. These interventions may work best as one element of a broader multicomponent active travel strategy or act as a precursor to one. A multicomponent intervention should consist of cross-curricular education, promotional events at the school level, infrastructural improvements and policy development.
Acknowledgements
This research was funded by Sport Ireland.