Evaluation of the Implementation of the Advance Care Planning Package in South Western Sydney ()
1. Introduction
The literature shows substantial increase in the number of older people with frailty and chronic conditions which represent the largest proportion of patients in the hospital and Emergency Departments (ED), posing increased practical, financial and ethical challenges for the health system (Silverman et al., 2004; Schofield & Earnest, 2006; Lamba et al., 2012).
Reports from the NSW Ministry of Health have indicated a systematic failure of the existing system in identifying people with end-of-life (EOL) care needs and offering appropriate treatment plans (Lamba et al., 2012; Clinical Excellence Commission, 2018). Uncertainty about prognosis and medical treatment decisions can delay appropriate management and may lead to non-beneficial interventions and medically unwarranted hospitalizations (Davison & Shelby-James, 2012; Kennedy et al., 2014).
Previous systematic reviews have confirmed that General Practitioners (GPs) working closely with specialist services such as palliative and geriatric care need to develop strategies to support a more sustainable model of working at the community level (Davison & Shelby-James, 2012). A model that yields accurate prediction of imminent death would benefit patients from having a better understanding of their prognosis in order to make fully informed decisions about the treatment(s) they receive.
The CriSTAL (Criteria for Screening and Triaging to Appropriate aLternative Care) is a screening tool that identifies older terminal/dying patients who may develop EOL care needs within 12 months and facilitate discussions on care preferences including hospitalisations or community care (Cardona-Morrell & Hillman, 2015). The validation and implementation of the tool were supported by the National Health Medical Research Council program grant and validated across several hospital settings (Cardona-Morrell & Hillman, 2015; Cardona-Morrell et al., 2017). However this tool has mainly been validated and implemented in ED settings (Cardona et al., 2018a; Cardona et al., 2018b; Cardona et al., 2019). The Investigating team believes that interventions should be incorporated within a primary healthcare context (Kinley et al., 2017). This urged the development of the Community CriSTAL (C-CriSTAL). The C-CriSTAL is a refined tool for clinicians to use in the primary and community healthcare setting and once implemented, could become part of routine clinical practice.
This study aimed to develop an understanding of the implementation of the C-CriSTAL initiative in the primary healthcare setting in South Western Sydney, to enable patients and their families to make informed choices about their future healthcare needs whilst supported by their GP and Practice Nurse (PN). Two General Practices were recruited via an expression of interest process to co-design and implement the C-CriSTAL package in their practices (Figure 1). The C-CriSTAL consists of an Advance Care Planning Implementation Package (ACPIP) with the components of early detection, identification of EOL care needs and communication with the patient in primary healthcare setting, that is, a broad range of health services provided by medical professionals in the community.
The ACPIP comprised the following tools:
• C-CriSTAL: An adapted version to be used in the community setting.
• Rockwood Frailty Score: The Rockwood or Clinical Frailty Score (CFS) is a well-known instrument to assess frailty (Muscedere et al., 2017; Leong & Crawford, 2018; Lewis et al., 2019).
• Belief Values and Wishes (BVW)—Advance Care Planning (ACP): Offered the patient the opportunity to have ongoing conversations with their GP, family, and carers regarding choices for care. Determining a patient’s BVWs was an essential component of ACP.
• The Advance Care Directive (ACD) consisted of written instructions about a person’s future medical care in the event the patient becomes unable to communicate their wishes (Muscedere et al., 2017). The Specialist Nurse Consultant (SNC) introduced the subject during routine clinic visits and was supported by the GPs and Practice Nurses. This facilitated a structured discussion of the patient’s wishes in which the ACD was completed (Lewis et al., 2019). The patient was assured that the ACD could be modified at any time and should be reviewed periodically to make sure the specifications continue to be in line with their wishes (Leong & Crawford, 2018; Lewis et al., 2019).
Meetings with stakeholders from participating practices were held to discuss implementation and trialling of the package before commencement. The SNC sought permission from the GPs and worked alongside the staff members at each practice.
By understanding the experience of healthcare practitioners and researchers who participated in the design and implementation of the project, this study aims to identify practical considerations associated with the development and implementation of ACPIP in South Western Sydney (SWS), Australia.
![]()
Figure 1. Intervention process flow chart.
2. Methods
2.1. Study Design, Participant Recruitment
The study seeks to explain the experience of participants in a new initiative in order to translate findings into usable clinical protocols. The design is that of a qualitative descriptive study (Creswell & Poth, 2016) incorporating multiple perspectives contributing to an identifiable reality, particularly useful in the health services research space notably that multiple perspectives contribute to an identifiable reality (Colorafi & Evans, 2016). Reporting on qualitative data has characteristics of descriptive studies in keeping with the study design.
A qualitative descriptive study was undertaken using semi-structured interviews with a purposive sampling strategy. All twenty five members of a C-CriSTAL Clinical Reference group was invited to participate in the semi structured interviews. The reference groups were formed to develop strategies for implementing use of the C-CriSTAL within the South Western Sydney Local Health District. The reference group consists of representatives from Local Health District, C-CriSTAL Project Team, Primary Health Network, consumer, GPs and all their staff from the two participating General Practices. The reference group was targeted as research participants because of involvement in the C-CriSTAL implementation, the diversity within the group required a methodology that would allow for potentially varied experiences being captured, as well as opportunity for recording of individual experience. Semi structured interviews were conducted according to developed guidelines in order to capture any common experiences of those in the reference group across disciplines and experience types, as well as provide the opportunity for documenting individual experience through probing and open-ended questioning. Inviting only members of the reference group to participate as interviewees allowed some assumption of prior knowledge and experience of the matters under discussion, but the diversity of the group (including healthcare providers, researchers and consumers) suggested that individual experiences, or use of terminology may differ. The study sought to capture both common understandings and individual experience. Purposive sampling of the reference group limits the generalizability of findings to the general population but allowed the study to fulfil the purpose of documenting and describing the experience of insiders to a Practice Initiative. Semi-structured interview questions were designed by the investigator team to explore the participants’ expectations and experiences from implementing C-CriSTAL in SWS. A program logic model was used to identify the themes for the semi-structure interview questions. Other documents such as emails, implementation plan and operational meeting notes were used to assist in the development of the questions. The theme of the semi-structured questions included: impact on patient care, practice staff and community; required resources and project coordination; identified barriers and challenges; current/future facilitators in implementation and acceptance of the program; and, suggestions for its long term effectiveness (Table 1).
An invitation package consisting of participant information sheet, consent form and an offer to participate was emailed to all eligible participants. Participation in this study was strictly voluntary and consent obtained prior to interviews. The interviews were conducted by a trained researcher who had no direct association with the C-CriSTAL initiative.
2.2. Data Collection and Analysis
Interviews were conducted by an interviewer who was independent of the
![]()
Table 1. Semi structured interview questions.
Abbreviations: C-CriSTAL, Community-Criteria for Screening and Triaging to Appropriate Alternative Care; SWS, South Western Sydney.
reference group and the question design. The independent interviewer was not part of the reference group and had not developed the interview guide. The use of an independent interviewer was an attempt to minimise the risk of bias during the interview process. All interviews were conducted remotely via telephone or online meeting software. Interviews were audio recorded and interview notes were taken by the interviewer. The interviewer allocated a participant number to each interview and associated notes. Audio recording and scanned interview notes (raw copies) were shared with the lead researcher, who contracted outsourced verbatim transcriptions of each interview. The transcriber was instructed to remove identifying content (such as names of individuals or practices).
All transcripts were recorded in NVIVO (v.12). The interview format allowed for initial identification of grouping responses to each question which allowed for reading of each transcript by the lead researcher whilst comparing individual accounts to the grouping of responses. Whilst reading the twelve transcripts, the lead researcher coded main ideas and then grouped the codes together to identify salient themes. Given the purposive sampling, the purpose of the study was to capture the group experience and identify outliers for further exploration with the whole reference group, if needed.
Ethic approval for this study was granted by the South Western Sydney Local Health District Human Research Ethics Committee (2019/ETH13126).
3. Results
Almost half of the purposive sample invited were available for interviews. All interviews were completed with available participants. Saturation of data was not taken into account, as the purposive sample was already a small and finite group. Ideally, all members of the reference group would have participated in the study, but due to work and time commitments, after half the group had participated over a period of months, further data was not collected. Those who participated (n = 12) were from the Local Health District (n = 6), Primary Health Network (n = 2) and General Practices (n = 4). The age of the participants ranged between 30 - 63 years and average age 48 years of age. Ten of the twelve participants were female.
The initial sorting of responses by NVIVO, and coding of transcripts by the researcher indicated that the results were consistent across the twelve participants, and that the identification of salient themes from this purposive sample was not contested. The analysis therefore relied on the initial sorting of data using NVIVO, and coding that lead to the identification of salient themes as shown in Table 2. The results of this low level analysis were sufficient to provide the description of the experience of the participants, as reported below. Further in depth analysis on the data collected through discussion with the reference group was therefore not undertaken for the purpose of this study.
![]()
Table 2. Summary of results of the thematic analysis.
Abbreviations: ACD, Advance Care Directive.
3.1. Salient Themes Identified on Thematic Analysis
3.1.1. Success Elements
All participants accepted the initiative as valuable and provided appropriate care for patients.
Increased awareness
There were many positive aspects of the initiative which included increased awareness and education among participant Practices’ staff. This in turn generated more awareness among family members and within the community.
“I think it’s also put awareness out into the community because for every patient throughout that project, they have spoken to their family about their wishes and also therefore then makes people within their family think about or maybe in the future that something I need to think about as well.” (Participant 1)
Assessment tool
In relation to the C-CriSTAL tool, many perceptions through experiences of use demonstrated that it has the ability to predict patients who potentially are in need of EOL planning. Few participants expressed that it continued being used beyond the initial phase of the program.
“What this study has shown is that this is a good tool to identify the people and to trigger those conversations.” (Participant 2)
Opportunity for further evaluation
The program initiative opened up the opportunity for further evaluation and ongoing monitoring of the program to explore how ACDs are enacted upon or followed up to address situational changes in the future and to help gauge impact.
“There might need to be kind of follow up with patients and revisiting and revising the ACD over time.” (Participant 3)
Perceived impact on patients as a result of having completed an ACD during the program included that most patients could recall the experience remaining satisfied with the content.
“It is really good when people are coming back in, you know, a year later to have their health assessment done or sometimes not even a year later, Most of them had a pretty good recall of what was talked about and said ‘no’ all good.” (Participant 4)
Adaptability
In many instances the program as a whole was viewed as potentially adaptable to local and individual needs and worthwhile for trialling in more diverse settings to include different cohorts of patients such as Culturally and Linguistically Diverse, urban, disability and different clinician groups. Some participants felt the program could have been opened up to patients outside of the over 75 age range.
Many successfully completed ACDs
The initiative was also viewed as effective in terms of the number of ACDs completed during the program implementation period.
“We had like a target of getting fifty advance care directives done and we ended up with one hundred and fifty-three (153).” (Participant 5)
Beneficial for patients
Participants perceived the program as highly valuable with the conversation aspect important to patient welfare in terms of empowerment for better individualised EOL care. The program was viewed as very positive with participants describing patient feedback as “excellent” and that patients were “willing”, “excited” and “eager” to participate and were not afraid to have the conversation often described as difficult and uncomfortable.
“They would bring that in and say how beneficial it was and how it is great to have that conversation.” (Participant 6)
It was perceived that patients received more patient centred care and to involve family, providing certainty for patients around adherence to individualised wishes for EOL experiences. The program’s process enabled initiating a conversation and formalisation of EOL care which may otherwise be hidden away or forgotten about.
“patients yeah tend to go away and forget about it and not end up actively pursuing getting that piece of paper in place. Um, whereas this allowed us to funnel them into a system where obviously the ACD was the outcome.” (Participant 7)
Nurses can do this role
A success element of program’s implementation was that through the support provided it was recognised that Practice Nurses (PNs) have the capability to utilise the program. PNs felt more educated to have the discussions, use the assessment tool and access resources. It was noted that the initiative increased knowledge about ACDs, ACPs and the significance of these.
“A greater awareness, from a (nurse’s) point of view, that we can be integral in assisting where necessary helping people fill out these advanced care directives. I think that’s promising. It was successful.” (Participant 11)
3.1.2. Facilitators
Specialist Clinical Nurses (SCNs)
The SCNs were integral for the program’s implementation in the Practices and education of staff.
“The program when we did have the (clinical nurse) here, who was very well skilled. So, it just made it a little bit easier subject to approach people with.” (Participant 5)
General Practitioners (GPs)
Other facilitators were the enthusiasm and support of the GPs involved in the project who were already encouraging EOL planning.
3.1.3. Future Enablers
Funding
Enablers include a government initiative or specific Medicare Billing Item Number to address time and funding issues and increase provisions for training and support to improve engagement and chances of program sustainability.
“I would think that until probably people get the attention of the Commonwealth around this being something that takes time to do properly.” (Participant 8)
Increased funding or a government initiative would enable the ability for education and workshops to be held in General Practices where the program is implemented to address issues around providing remuneration for time spent in training.
General Practice staff as drivers
A few participants noted that PNs could be the drivers of the initiative inclusive of front desk staff to assist and that in principle, the program requires people to take ownership of it and to have a “practice champion”.
“If we give them direction, they can drive that as well. And they’re very good at driving it because they can be single minded when we give them the task. So a big plug for selling it to you know the practice manager, the office manager and the other people in the office” (Participant 12)
My Health Record
Other enablers included suggestions to improve the capability of MyHealth Record to upload documentation to be visible for potential use in other health facilities as well as making its functionality more user friendly for health professionals and patients who are not technology literate.
3.2. Barriers
While participants identified a wide number of potential benefits of implementing the program, they also reported some of the barriers and challenges. The main barriers for program implementation to move beyond trial status and achieve full sustainability were interconnected by salient themes of time, funding, education/training for staff to feel more confident to carry out such conversations independently, the need for a driving force and, in few instances, acceptance to use the C-CriSTAL tool.
General practices operate as businesses and the majority of remuneration “comes from fees-for-service, made up of Commonwealth rebates and patient co-payments” (Swerissen et al., 2018: p10). Many participants mentioned there should be a Medicare item number due to the perception that there was a lack of financial incentive to sustain the CriSTAL initiative.
“And the second part of that is the money in the sense that general practice is a business and they, there is no Medicare funding for advanced care planning. Um, so therefore this 45 minute conversation that you need, and you need a significant amount of time to have a thorough conversation with somebody um, that conversation can’t be billed.” (Participant 9)
Education, training and confidence to have the conversations were perceived to be a major barrier to staff carrying out the conversation independently. The conversations as part of the program required lengthy appointment times noted as being difficult, sensitive and uncomfortable in nature. It was therefore felt there needed to be more support and education for health professionals to feel confident in having these conversations independently.
“So I certainly think more education and um, you know, support would have been required and also time.” (Participant 9)
There were views that the necessity for “time” conflicted with the GP business model due to lack of a Medicare item number to attribute to these appointments. The conversations are inherently lengthier (45 minutes) than the normal appointment times with practices environments described as busy and “time poor”. It was perceived that GPs did not have the time for the conversations and practice staff described experiencing interruptions and “walk-ins” which was viewed as an impediment. The time factor was also viewed to have impacted the holding of education or workshop sessions before implementation.
“Um, time is, just within itself is a barrier because general practices are so busy. Um, certainly GPs don’t have that time.” (Participant 2)
It was viewed that there were challenges to fully sustain the program to be incorporated as part of routine practice without the propulsion/support provided by the CNCs to maintain intensity. It was felt that momentum dropped off once they were no longer involved and that interest waned without continued support.
“You know when things are projects and they’ve got people carrying them and running them they always run better when they do.” (Participant 4)
This in turn influenced acceptance of the initiative in terms of undertaking the program offered to the practices. Momentum behind a project is key to sustainability, to have the program integrated as part of routine practice. Notably few participants thought that principally, the program requires people to take ownership of it and drive it, to have a “practice champion”.
4. Discussion
The C-CriSTAL initiative is viewed as beneficial for providing more patient centred care. The beliefs values and wishes tool and C-CriSTAL assessment tool are seen as helpful to ease into the conversation and assess those most in need of an ACD/ACP.
There are many successful elements including the trial arm of the initiative which achieved formation of the target quantity of ACDs. As a result, the program is mostly viewed to have impacted positively on patient care in the long term, Practices’ staff and the community. Furthermore, PNs feel more educated around having these conversations with patients and it is perceived that patients themselves are willing to participate and find the experience a positive one. Patient involvement in these discussions enhances awareness among the community and patient’s family members. However the findings also showcase the need for training to create confidence in health professionals to deliver the program independently.
Issues of time (particularly for GPs), training and funding to use the program in primary healthcare are viewed as barriers due to the business nature of primary care models, which are somewhat interconnected and the need for a driving force as elements to be addressed to make the program sustainable. The dynamics of communication and stakeholder engagement are notions viewed as key to help influence the current and future success of the program.
The findings present potential resolutions to challenges found in sustaining the program which could be achieved via provision of more resources and funding for ACP, remuneration for time spent in training staff and building a model that incorporates these consultations as a part of routine practice including specific billing.
Both GPs and PNs are recommended to work together as a team as they have their own professional competencies for applying strategies at the EOL (Rockwood et al., 2005; Van der Plas et al., 2014). Studies indicate that a multi-disciplinary team with highly trained nurses reduces the chances of hospitalization at the EOL (Brumley et al., 2003; Ahlner-Elmqvist et al., 2004). Different beliefs about who is most suitable to hold these sensitive conversations may have impacted the project and how it ultimately rolled out. Initially intended for GPs, it is felt that PNs are better positioned to hold discussions about EOL. This highlights the complex nature of discussions for health professionals about EOL which involve both time and human emotions. A study from Netherland shows that compared to GPs the PNs were more alert to the psychological changes in patients and their family and worked to strengthen coping mechanisms in the illness trajectory (De Korte-Verhoef et al., 2015). This combined with the organisational structure of the General Practice are factors influencing who is best placed or able to do so. While some may view all health professionals should be able to take this on, others question if ability to initiate timely EOL conversation is a skill which can be learnt or influenced by personal experience.
Recommendations
The main future strategies to aid acceptance for such programs include increased communication and engagement with stakeholders for improvement around understanding of the whole high level program as this was considered an influence on the ability of the program to continue as routine practice.
There is a need for more engagement about the program in the community, health promotion and upskilling for nurses and training/education for other health professionals including GPs and hospital staff to further develop the program and improve implementation. More support and education may be needed in relation to not only understanding of specific terminology but also any medicolegal issues and implications for health professionals using the CriSTAL program.
The policies such as continuation of district wide education sessions for GPs, health promotion campaigns and use of media for the pubic to be a driver would increase awareness of the importance of the program as well as promote and create interest for trial uptake and program development. Increased engagement of GPs would aid in spreading interest in the program among other GPs to further champion and that ACD and ACP could be discussed as part of Continuing Professional Development (CPD) events for GPs.
The small number of participants who were interviewed may have impact on the results and its generalisation to populations outside of South Western Sydney or in other state and territory health systems.
Further research studies are needed to measure impact of the program for ACDs when the time comes for them to be acted on in the health system.
5. Conclusion
This study has provided a narrative of the experience of the stakeholders who participated in the implementation of the C-CriSTAL initiative in primary healthcare setting in South Western Sydney. It has identified factors which contribute to improvement of future implementation of similar programs.
6. Relevance to Clinical Practices
The advance care planning implementation package provides clinicians with tools for early detection and identification of patients at risk of rapid deterioration in the General Practice setting. Our results indicate that the C-CriSTAL initiative in SWS was well accepted and perceived as valuable and beneficial for providing more patient centred care. The tool enhanced clinician’s confidence in initiating earlier EOL discussions and empowered patients to express EOL wishes in a comfortable environment then subsequently documented in the form of an advance care directive. The study explores the views of the participants involved with the implementation of the initiative in SWS was thus provides insight into the practical considerations as well as barriers and facilitators to the implementation of C-CriSTAL package in primary healthcare setting. It also identified factors which contribute to improvement of future implementation of similar programs.
Acknowledgements
Members of the Community CriSTAL Clinical Reference Group: Anett Wegerhoff, Anne Harley, Friedbert Kohler, Graham Andrews, Janeane Harlum, Josephine Chow, Ken Hillman, Pamela Keech, Rachael Williams, Roberto Forero, Susan Fraser, Vitor Rocha.
General Practices partnering on co-design of the Community CriSTAL Program: Alicia Mah, Claire Hoskin, Geoffrey Sparkes, Grant Lewis, Helen Probus, Jennifer Dunne, Jennifer Loxton, Lisa Kume, Nick Lo, Nicola Oehm, Penny Knoulden, Petrina Barnes, Sarah Pryer, Stephen Barnett, Vincent Roche, Zara Matthews.
Source of Funding
This research study did not receive any specific funding.
What Does This Paper Contribute to the Wider Global Clinical Community?
• The advance care planning implementation package provides clinicians with tools for early detection and identification of patients at risk of rapid deterioration in the General Practice setting and empowers patients to express EOL wishes in a comfortable environment.
• This study provides insight into the practical considerations as well as barriers, facilitators and factors which contribute to improvement of future implementation of similar programs in primary healthcare setting.