Integrating Different Approaches and Tools to Evaluate Socio-Cultural Impact of Services Projected in Low-Population Environments ()
1. Introduction
Sustainability is a priority concern on the world global agenda that requires a balance between ecological, social and economic aspects, ensuring that current actions do not compromise the ability of future generations to meet their own needs. This challenge expressed at the end of the 20th century (WCED, 1987; UNCED 1992) involves the interrelated study of environmental degradation, social inequality, and economic instability (Manzini & Meroni, 2017; Mancebo & Sachs, 2015) .
While environmental and economic sustainability are often at the center of discussions, it is essential to recognize that sustainability encompasses broader dimensions, including social and cultural aspects (Hawkes, 2001; Engström et al., 2002) . In the broadest sense, environmental sustainability involves the entire global ecosystem (oceans, atmosphere and land) and to address this complexity, a number of categories and subcategories of impact can be identified. In the ReCiPe methodology (Goedkoop et al., 2013; Huijbregts et al., 2016) , human health, ecosystem quality and resource scarcity are selected as the three main categories of environmental protection. Economic issues are usually valued by considering costs of different activities along the life cycle although other economic aspects such as the financial results from activities can be also used (Wulf et al., 2019) .
On the other hand, a sustainable society is one in which all members have equal rights and all share equitably in societal benefits. The analysis of social aspects is based on six different stakeholder categories (workers, consumers, local community, society, value chain actors and children) according to the UNEP’s guidelines (UNEP, 2020) . Another alternative classification is based on social issues such as human rights, working conditions, health and safety, cultural heritage, governance and socio-economic repercussions (Benoît-Norris et al., 2011) . Socio-cultural impact remains still significantly underexplored in scientific literature in terms of well-established tools and methods for assessment.
In today’s world, increasingly aware of this priority and with political initiatives to improve the situation, it will be crucial to understand and adequately address the socio-cultural impact of projects and initiatives to promote inclusive and equitable development from governments (MITES, 2022) and world institutions (2030 Agenda for Sustainable Development). However, measuring this impact presents significant challenges due to its complex and multifaceted nature. These challenges become even more pronounced in low-population environments, where unique contexts and limited resources make assessment particularly difficult and unclear (Bock, 2016) . Consequently, having an impact assessment system in projects aimed at rural development is not only critical but also key in a context in which depopulation is a serious problem considered within the global sustainable development goals where nowadays, there are more and more politics and projects with public funds for this topic (Special Guideline for Demographic Policy and against Depopulation, 2017) . The “Human Cities/SMOTIES remote places” project (SMOTIES), co-financed by the Creative Europe Program of the European Union (Creative Europe 2014-2020, Culture Sub-programme, Call for proposals EACEA 32/2019: Support for European cooperation projects 2020) and led by the Polimi DESIS Lab (Design for Social Innovation and Sustainability) research group at Politecnico di Milano, recognizes the importance of analysing the socio-cultural impact in small and remote communities. These communities often face the risk of depopulation, resulting in the undervaluation, erosion, and potential loss of their material and immaterial cultural heritage. SMOTIES aims to reverse this trend by using culture and creativity as catalysts for economic growth, social cohesion, and inclusive development. By adopting a participatory design approach, empowers local communities in ten small and remote European locations. The project seeks to transform specific places within these communities through collaboration with stakeholders, and local authorities, including residents, associations. This process involves the development of new services and the creation of a proactive cultural ecosystem, thereby fostering rural development and revitalizing cultural practices. The project’s interdisciplinary framework combines service design and spatial design approaches to address the unique challenges faced by these communities (De Rosa & Fassi, 2023) . While the objectives are established, the socio-cultural impact of such initiatives is difficult to quantify and evaluate and consequently define. Traditional impact assessment frameworks often struggle to capture the nuanced and subjective nature of the socio-cultural impact, using broad quantitative indicators.
However, low-population environments present additional complexities in order to use such indicators. These communities typically have smaller sample sizes, making statistical generalizations more challenging. The homogeneity of these communities, often characterized by a predominance of older residents, further complicates impact measurement.
Different authors have addressed sustainability in the design process of product and service systems implementing guides that mainly consist of various sustainability actions, which complement those conventional tasks developed along each design stage (Maxwell et al., 2006; Cooper & Boyko, 2010; Crul & Diehl, 2009; Gagnon et al., 2012) . At the same time, a set of specific tools have been developed to carry out tasks as to integrate sustainability requirements, achieve sustainability assessment or introduce improvement strategies (Bovea & Pérez-Belis, 2012; Andriankaja et al., 2015; Ahmad et al., 2018; Santolaya et al., 2019) . These tools have been mainly applied to product systems. However, studies regarding the sustainability of service systems are much fewer and these are not carried out taking into account all sustainability dimensions. Being design a process for the search of innovative solutions applicable to different projects, products, services and systems (Rizzo, 2022) ; relating sustainability and innovaion can be effective (Muñoz et al., 2020) , especially in low-population social projects (Capello & Lenzi, 2018) . Moreover, within this underrepresented dimension, social aspect, particularly in the context of services, requires deeper research.
This article aims to investigate the socio-cultural impact of services projected in low-population environments, illuminating the difficulties, methodologies, and findings connected with evaluating and measuring impact. The purpose of this article is to contribute to the understanding of practical methods for assessing the social impact exploring the complexity of low-population environments. The methodology used to assess this impact is described in the following section. Additionally, the challenges encountered when conducting impact assessments in low-population areas are discussed, the results and observations are examined, and conclusions for further study and development are proposed. Through this exploration, we hope to contribute to the broader understanding of the socio-cultural impact in remote communities and facilitation for effective policy interventions.
2. Methods
The methodology applied in this work is based on the integration of different approaches and a number of tools developed with the aim of assessing and measuring the socio-cultural impact in low population contexts. The design discipline, through participatory research and a community-centered approach (Cantù et al., 2012; Manzini & Meroni, 2017; Perkins et al., 2002) , promotes socio-cultural development, actively engaging stakeholders of the given contexts (e.g., inhabitants, policy makers, representatives of local institutions and creative associations, etc.) in co-creation processes. Based on already established impact evaluation frameworks (Rizzo, 2022) , methodology has been developed adapted to the project characteristics and can be applied to other social projects. This section provides the different approaches and tools used in this work, such as the Impact Pathways framework, the Worts’s individual-global model and the development of a multidimensional framework covering impact in three dimensions: time, places and people. Additionally, a combination of techniques, such as observation, interviewers and strategic questionnaires after each co-design workshop is presented as qualitative assessment tools.
2.1. Impact Pathways Framework
Horizon Europe is a research and innovation program where a framework for measuring the results of the projects developed within the European Union, called “Impact Pathways” (European Commission, Directorate-General for Research and Innovation & Mazzucato, 2018) , is established. This framework responds to the concern to design the impact of projects and to allow policy makers to understand effects and benefits of the projects’ outputs (Horizon Europe, 2023) . Due to its simplicity, this framework serves as a reference for establishing indicators and measuring impact in social projects, dividing impact into three distinct levels: outputs, outcomes, and impact (Figure 1).
Outputs represent the immediate tangible results of project activities, reflecting the deliverables and actions produced. In this case, outputs would correspond mainly to the co-design workshops. Outcomes represent the intermediate changes resulting from the project, encompassing alterations in behavior, attitudes, and practices among stakeholders. Lastly, impact signifies the long-term and broader societal changes produced by the project, demonstrating its transformative effects on communities and societies over time (Molund & Schill 2004) .
2.2. Worts’ Individual-Global Model
This model arises from a completely different context: museums. In this context,
![]()
Figure 1. Impact Pathways framework. Understanding impact as actions over time.
as in the projects mentioned above, institutional activities and decision-making are object-centered (Worts 2006) . One of the current challenges in this sector has been to better understand how to identify and use cultural indicators to guide the work of museums. The Critical Assessment Framework tool developed by the Working Group on Museums and Sustainable Communities (2007) considers three people-oriented levels: individuals, communities and museums. Subsequently, Worts (2010) has expanded and updated this model with the objective of generating criteria for assessing initiatives aimed at four levels of cultural adaptation in terms of people: personal, community, institutional and global levels (Figure 2).
The exposed matrix is divided into a series of rows that reflect different cultural criteria grouped by level. In the columns, a scale from one to five has been established to evaluate how well each of the criteria meets, being five the best score. In this work, a tool in survey format has been developed that functions as a criteria evaluation sheet to help cultural adaptation at the four levels mentioned above: personal, community, institutional and global. This analysis demonstrates the widespread need to maximize the potential of institutional initiatives and projects to be relevant from a socio-cultural point of view. This means being more attentive to the needs and realities of the communities they serve and being aware of how their work affects those communities. Being receptive to critical issues in a way that links history to the present and that actively engages citizens not only justifies public funding of cultural organizations, but
also makes sense as cultural mirrors that allow society to see itself with greater clarity and adjust their actions accordingly (Worts, 2006) .
2.3. Qualitative Assessment Questionnaire
In order to collect the impact generated through the workshops beyond quantitative indicators, a strategic questionnaire was designed to capture qualitative aspects of impact. This questionnaire aims at engaging stakeholders in open interviews, facilitating conversations that move from subjective-abstract to tangible-concrete and from individual-personal to regional-global perspectives. The strategic questions, which are summarized in Table 1, were carefully crafted to encourage stakeholders to reflect on their personal experiences, perspectives, and changes resulting from their participation in the workshops, to a more tangible thinking giving examples of specific attitudes or activities in the community they identify as a consequence of the workshops.
To evaluate the personal impact of a person who has attended the workshop, we could start the interview with a question like: How has your personal perspective or understanding changed as a result of participating in these co-design sessions? to provoke self-reflection. Guiding the conversation towards more tangible elements, for example, asking: In what ways have these sessions influenced your personal growth, skills, or knowledge? And ending the interview
![]()
Table 1. Strategic questionnaire to obtain qualitative indicators.
with a question like: Can you identify any specific actions or decisions you have taken as a result of participating in these sessions? In this way, valuable insights can be obtained that reflect the impact of the project in a more tangible way. This procedure could be done to evaluate the impact not only personally, but also from a community or institutional or global perspective, applying the same process and adapting the questions for each level (Table 1). The insights gathered from these questionnaires and observations contribute to mapping the impact throughout the project and informing future iterations. The use of strategic interviews as a measurement tool allows for the collection of qualitative data that complements the quantitative indicators, providing valuable insights to the project’s activities.
3. Discussion
Understanding the socio-cultural dynamics and building new impact measurement approaches to the specific context are crucial for ensuring effective interventions and sustainable development. This made it possible to generate a roadmap for the achievement of smaller objectives and actions in which to break down these large strategic objectives and locate these actions and expected results not only over time but also depending on the degree of impact in terms of community and places. In this way, the prioritization of the objectives was facilitated, identifying which of them had a more individual, local and short-term impact in contrast to more ambitious long-term objectives.
Methodology exposed previously was applied in the case of services projected in a sparsely populated village collaborating with the Politecnico di Milano. Despite initial skepticism, participants became actively engaged and demonstrated a shift in their perspectives. This highlights the potential of participatory design approaches to foster personal growth, enhance collaboration, and encourage a sense of ownership among community members.
The questionnaire responses shed light on the ways in which participants’ personal perspectives and understanding were influenced by their involvement in the co-design sessions. While some reported a wider vision and increased awareness of community needs, others did not perceive significant changes. These variations in responses indicate the importance of considering individual differences and expectations when assessing impact. Furthermore, participants acknowledged the benefits of working in groups, gaining exposure to diverse viewpoints, and finding solutions to utilize the potential of public spaces. The identified actions and decisions taken as a result of participation demonstrate the potential for co-design processes to inspire tangible outcomes and behavioural changes. This tool allows not only the collection of valuable information and data for the indicators, but also the signs for redesign and adaptation during the execution of the project, for best development to its objective of improving the impact in these remote communities. Finally, the mixed-methods approach employed in this study, combining observation, questionnaires, and conversations, proved effective in capturing both qualitative and quantitative aspects of impact assessment. The triangulation of data sources allowed for a more comprehensive understanding of the project’s impact on participants and the community.
Thus, this study has allowed identifying weaknesses in existing impact measurement systems when applied to low-population areas. The proposed framework and methodology provide a foundation for addressing these challenges, offering a more contextualized and comprehensive understanding of socio-cultural impact. Additionally, this research highlights the need to further explore impact assessment methods that capture the complexities and nuances of low-population environments. Developing robust frameworks and indicators specific to these contexts will contribute to evidence-based decision-making, resource allocation, and policy development.
In particular, a map in which different which they were to take place and actions and objectives of the project, classifying them according to the time in according to the degree of impact expected on people and spaces, should be developed. In this map, the strategic objectives and the impact goals to generate in the project should be successively placed.
4. Conclusion
This research work is focused on the study of the socio-cultural impact of the services projected in small and remote communities. The article contributes to the advancement of impact measurement in socio-cultural contexts since it identifies the weaknesses of existing impact frameworks when they are applied to low-population areas. In addition, by formulating strategic questions that facilitate open interviews with stakeholders, the method captures rich insights and perspectives, which serves as a starting point for assessing the project’s impact while allowing for iterative improvements. The article highlights the importance of employing a mixed-methods approach to impact assessment, especially for small places. By combining observation, questionnaires, and conversations, the research effectively captures valuable information to enhance the project’s activities, attracting new participants and improving overall engagement. The proposed set of additional quantitative indicators complements the qualitative assessment, providing a more comprehensive understanding of the social impact in low-population places. The findings provide valuable insights into the transformative potential of participatory design approaches and the importance of context-specific impact assessment methods. By incorporating these insights into future research and practice, we can further advance sustainable development and cultural revitalization in low-populated areas. Future iterations of the project can build upon the findings and refine the measurement tools. Constant improvement will enhance its applicability in diverse contexts and enable a more accurate evaluation of the project’s long-term outcomes.
Acknowledgements
This work was supported for publication by the Instituto de Patrimonio y Humanidades Scholarship. In addition, it was supported for research by the Aragon Government, project “Sostenibilidad de servicios y productos culturales. Desarrollo de una metodología de diseño multidimensional”. The article reports on a specific contribution and short-term collaboration on the ongoing impact analysis with the Polimi DESIS Lab research team (Department of Design-Politecnico di Milano, Italy). The Lab is coordinating the project “Human Cities/Smoties - Creative works with small and remote places” (2020/24), funded by the Creative Europe program under “EACEA 32/2019: Support for European cooperation projects 2020”.