Efficacy of a Strategy for Implementing Guidelines for the Control of Cardiovascular Risk in Primary Healthcare ()
1. Background
Clinical practice guidelines (CPGs) can be defined as a series of systematically developed recommendations designed to help health professionals and patients take decisions regarding the most appropriate forms of treatment in specific clinical situations [1]. They can also improve the consistency of care [2] and they have been shown at least capable of changing clinical practice [3].
However, the publication and dissemination of these documents do not necessarily lead to their systematic use in daily practice, nor to any improvement in the quality of care or the health of the target patient population [4] [5] [6] [7] [8]. Field and Lohr [9] indicate that CPGs do not “implement themselves”; rather, a planned implementation strategy must accompany their distribution. Oxman [10] indicates that there are no “magic bullets” nor apparently any strict rules regarding which strategy might best implement a CPG. Choosing the right one for a particular setting, and planning accordingly, is therefore paramount [11].
The literature describes different types of implementation strategy. One of the most widely known classifications is that of the Cochrane Collaboration’s Effective Practice and Organization of Care (EPOC) group. This classification system covers activities that influence extrinsic motivation (organisational changes, financing models, local changes, changes that include patient participation), and intrinsic motivation (interventions aimed at health professionals) [6] [12].
In 2004 it published an extensive review [13] that showed the best to involve reminders, educational materials, and the auditing of clinicians and the delivery of feedback. This review also showed that the use of multiple strategies was found to be not necessarily better than the use of single strategies. However, multiple strategies were found to be possibly recommendable after a detailed analysis of the obstacles to implementation [5]. Clearly, the efficacy of a strategy will be influenced by factors such as the type of a change sought, the place where implementation is required, the obstacles faced, and facilitating factors present [14].
One systematic review (SR) [15] indicates there to be little evidence that production and dissemination of CPGs promote any significant changes in health outcomes. However, most of these studies were not randomised clinical trials (RCT), and the few that were had little statistical power.
RCT with greater statistical power are therefore needed to determine whether some types of implementation method can produce better results. Such a CPG was chosen since cardiovascular disease (CVD) is highly prevalent and has an important impact on those who suffer it. It also has a strong economic impact on public health systems. Further, primary healthcare (PHC) has a huge role to play in its prevention. Certainly, the evidence shows that reducing cardiovascular risk (CVR) is associated with a reduction in CVD [16]. Indeed, the measurement of cardiovascular risk has become a common recommendation in cardiovascular CPGs.
The present work compares the efficacy of implementing cardiovascular risk guidelines based on an education method involving opinion leaders, and the habitual method of dissemination among primary healthcare teams.
2. Methods
This study, which was performed between 2004 and 2006, was designed as a controlled, blinded, community intervention trial randomised by clusters. It was performed in Primary Healthcare Area 2 of the Madrid Region. The full methodological details have already been published [17]. The study subjects were patients at 21 PHC centres which, in 2004, had a total of 373,495 patients ≥14 years. The initial subjects to be included were selected by random sampling of this age weighted by the size of the population assigned to each health centre. This step was undertaken to ensure that the final sample would be independent of the ageing of the population at each centre. Among these selected subjects, those aged ≥45 years of age were chosen to be the final study subjects. The 21 participating centres were then randomly assigned to either the control (n = 10) or intervention (n = 11) arm of the study, along with their selected patients. Since the health centres were the subject of randomisation, rather than the patients, different patients were selected at the beginning (pre-intervention) and end (post-intervention) stages of the study. This was performed to increase the internal validation power of the results.
The CPG used was a local adaptation of the different cardiovascular risk guidelines established by health professionals working in the PHC and hospital settings. The CPG was distributed by mail to all primary healthcare professionals (physicians and nurses) at the chosen control centres, along with an invitation to attend a two-hour meeting at which the guide would be officially presented. This is the normal method of dissemination of CPGs used in the study area. The CPG was distributed in the same manner as above to all healthcare professionals at the chosen intervention centres. However, it was implemented following a strategy based on an educational method involving opinion leaders. This consisted of 4 × 1 h sessions in which the contents of the CPG were explored. These sessions were “masked”, occurring within the habitual clinical meetings held by the health professionals at these centres. The CPG was divided into 4 distinct parts: 1) Importance of the calculation of the CVR and the classification of patients under this value; 2) Patient management with low and moderate RCV; 3) Management of the patient with high CVR and 4) Patient management cpon cardiovascular disease and referral to other specialists. This is the objective of each of the 4 talks that were given. The first was taught by the project coordinator, the second by a research nurse, the third by a research family doctor and the fourth by an internal medicine specialist, also a study researcher. The full details of the intervention have been published elsewhere [17].
The effect of the intervention was determined by collecting information from the medical records of the enrolled subjects in each group before (pre-intervention) and one year after (post-intervention) the intervention. The main outcome variable was measured as the increase in the proportion of patients in each group whose medical records showed the recording of the variables necessary (age, sex, blood pressure, fasting total cholesterol [in mg/dL], fasting glucose [in mg/dL], and use of tobacco) to calculate CVR patients.
The required sample size was determined assuming that the implementation of the CPG was successful if a 15% increase in the recording of the study variables was seen in the intervention arm. The sample size required for a risk of α = 0.05 and β = 1.2 was 346 subjects (173 in each arm). Assuming a design effect (DE) of 1.5, the corrected required sample size was 517 subjects. Assuming an exclusion rate of 20% this was increased to 620 subjects. This sample size was adjusted for the fact that in the studied health area, some 45% of the population is over 45 years of age. Thus the total (i.e., both arms together) number of medical histories to examine was at least 1170. Full details of the calculation of the sample size are described elsewhere [17].
The following confounding variables were identified: 1) sociodemographic and work variables of the health professionals (age, sex, being a physician specialised in family medicine, general medicine, or other areas, years of professional experience, working on the model of primary care teams or traditional model, working the morning or the afternoon, having a fixed contract or intern contract), and 2) health centre variables (rural or urban, being/not being a teaching centre for the training of resident physicians).
All the patients with information available at the time of access to medical records and who met the inclusion criteria, were analyzed. The qualitative variables were expressed as percentage, and the quantitative variables as means ± standard deviation (SD). A descriptive analysis was performed by calculating the means of the proportions at each centre, weighted by the method described by Donner and Donald [18]. The confidence intervals (CI) were calculated using the robust estimator of svy libraries provided by Stata. The hypothesis tests for these parameters were performed using the t-test.
A gross analysis of the effect of the intervention, measured as the difference between the increase in the main outcome variable in the intervention and control arms, was carried out by comparing the means of the variables measured in each cluster using the t-test. The 95% CI was also determined. A multivariate analysis, using the Generalized Linear Model (GLM) data analysis, was performed to control for different centres characteristics that could be potential confounding factors, always taking sampling effects into account. Thedependent variable used was the difference in the average of before-after increases in the primary endpoint of the study [19].
The effect of the intervention was analysed with intention to treat. Significance was set at p < 0.05. All p values reported are two tailed. All analyses were performed using STATA v.13.1 software.
3. Results
In one of the centres assigned to the intervention arm, only the four planned meetings could be attended by all health professionals. Nonetheless, since the intervention was analysed with intention to treat, the results from this centre were included in the analysis within the intervention arm.
At pre-intervention, a total of 2485 subjects ≥14 years of age were selected, providing a final sample of 917 subjects over ≥45 years of age. At the post-intervention stage, a total of 2513 subjects ≥14 years of age were selected, providing a final sample of 826 subjects over ≥45 years of age. Figure 1 shows the patients excluded in each arm, and the reasons why. A patient was excluded when his/her medical history was not locatable, was poorly kept, or when the patient had made no visit to the health centre in the preceding year (Figure 1).
![]()
Figure 1. Patient flow chart: number of patients finally studied and their distribution.
At pre-intervention, 24.9% (n = 619) of the subjects had no medical records, or the data recorded for their sex and age were wrong. Some subjects aged ≥45 years, 12.1% (n = 127), had not visited the centre in the previous year. At post-intervention, 4.5% (n = 115) had no medical records or the data recorded for their sex and age were wrong. Some subjects aged ≥45 years, 28% (n = 337), had not visited the centre in the previous year.
Table 1 shows the characteristics of the subjects at pre- and post-intervention. Differences were seen between the intervention and control arm subjects at pre-intervention in terms of the prevalence of high blood pressure (HBP) (47.6% compared to 38.7% respectively; p < 0.05), dyslipidaemia (34.55% compared to 29.37%; p > 0.05) and the use of tobacco (27.1% compared to 20.2%; p < 0.05). Table 2 shows the distribution of the sociodemographic and work-related variables of the physicians at the centres, along with the characteristics of the centres themselves. Differences were seen between the intervention and control arms in terms of the proportion of centres that trained medical residents (54.5% compared to 20% respectively, p < 0.05), proportion of men (43.2% compared to 36.6%, p > 0.05) and proportion of professionals with a fixed contract (53.2% compared to 46.5%, p > 0.05). Given that these differences might influence the main outcome variable, their confounding effect was taken into account in multivariate analysis.
The mean increase in the recording of variables required for the calculation of the CVR, before and after the intervention was 20.60% (95% CI 17.85 - 23.35) in the intervention arm, and 9.13% (95% CI 7.19 - 11.07) in the control arm (Table 3). After adjustment of the model, GLM data analysis shows that the difference
![]()
Table 1. Basal characteristics of the subjects at pre and post-intervention stages.
a: N for use of tobacco = 295, b: N for use of tobacco = 316, c: N for use of tobacco = 301, d: N for use of tobacco = 255, HBP: High Blood Pressure.
![]()
Table 2. Characteristics of health professionals (physicians) and health centres (N = 21).
SD: Standard Deviation.
![]()
Table 3. Effects of the intervention.
*p value < 0.001, **p value < 0.05, ***Multivariate analysis using Generalized Lineal Model (GLM) adjusted for the confounding variables: age and sex of physicians, being a physician specialised in family medicine, years of professional experience, working on the model of Primary Care teams or traditional model, working the morning or the afternoon, having a fixed contract or intern contract, rural or urban centre, being/not being a teaching centre for the training of resident physicians, age and sex of patients, use of tobacco, high blood pressure, diabetes and dyslipidaemia.
between the two arms of the study in terms of the increase in the recording of the variables required to calculate the cardiovascular risk was 7.49% (95% CI 4.62 - 10.35) (Table 3).
At pre-intervention, the intracluster correlation coefficient (ICC) for the recording of the variables required to calculate the cardiovascular risk was 0.02 (95% CI 0.0 - 0.047); at post-intervention this value was 0.036 (95% CI 0.0 - 0.074).
4. Discussion
It is important to note that this study measured the change in the recording of the variables required to calculate the CVR. The computerized medical record management system used in the study area has a tool that allows the CVR to be calculated from the recorded variables. It is quite common, therefore, for medical professionals to record the variables required and not undertake any manual calculation.
Computerized medical histories were gradually introduced into health centres at the time of the study. There was therefore the possibility of introducing bias with respect to the calculation of the CVR in relation to having or not computerized medical histories. Further, it is known that although cardiovascular prediction rules are recommended by the guidelines to evaluate global CVR, some studies [20] - [25] have shown the scant use by physicians in PHC. These were the reasons that took into account the record of the variables needed to calculate CVR.
After adjusting for the variables that might affect the result, an increase difference of 7.49% (95% CI 4.62 - 10.35) in the recording of the variables outcome was seen between the two arms, in favour of the intervention arm. Thus, the type of centre (urban or rural, training centre or not) and the characteristics of the doctors (sex, type of employment contract, attending to own patients or all consulting patients, shift, number of years of experience and medical speciality) involved had an effect on the measured outcomes.
The above 7.49% (95% CI 4.62 - 10.35) difference in the recording of the variables required to calculate the CVR represents a modest increase [13]. However, even modest improvements can have a significant effect at the public health level [6] [14] since a very large number of patients might be affected. The population ≥45 years of age (registered by their health cards) for the study area in 2004 was 182,131. An increase of 7.49% in the recording of the variables required to calculate the CVR therefore corresponds to improved recording in 13,641 medical histories from CVR subjects.
The opinion leaders involved in the intervention arm of the present study (three physicians and two nurses involved in PHC, and one internal medicine specialist at a reference hospital) enjoyed the recognition of their peers in the study area, both in terms of their training and their treatment of patients. All worked in the study area. All of these opinion leaders had worked on the production and local adaptation of the distributed CPG, and all were aware of the local obstacles and promoting factors that might affect its implementation. It should be noted that the training sessions they prepared did not require much in the way of human or material resources, and were run within the normal clinical discussion sessions held at the intervention centres (investing the results with added value). The intervention strategy followed might therefore be easily reproduced in other health areas.
The design of the present trial involved randomisation by clusters. This type of RCT is more complex than those involving the randomisation of individuals, requiring more participants to achieve the same statistical power, and a more complex analysis of the results obtained [26]. The present intervention strategy was directed towards the healthcare professionals taking part, but its effects were measured via a study of their patients’ medical histories. The risk of contamination between the health professionals was reduced by randomising their health centres rather than the health professionals themselves.
During the analysis, a number of decisions had to be taken regarding the variable “use of tobacco” in the patients’ medical histories. In some histories, nothing had been recorded in this respect (note that the free text in the patients’ paper and computerized files were extensively scrutinised for any such mention). In some cases this may have reflected a lack of note-taking with respect to non-smokers; only the habits of smokers may have been thought worth recording. Thus, all medical histories with no explicit data on tobacco use were left out of analyses; the sample size with respect to this variable is therefore small and variable. However, since the lack of recording of the use of tobacco was randomly distributed between the two arms of the study, it was deemed unlikely to introduce any bias.
An interesting result shown in the flowchart of Figure 1 is the different distribution recorded in the number of exclusions at pre and post-intervention. At pre-intervention, the majority of the health centres had traditional, paper medical records; only a few had computerized records. Since the sampling of patients involved their recognition as such via their health cards, and not whether they had an available medical record at their centre, some of the selected patients had no medical history to view. This was because they had never been to their assigned centre as a patient, perhaps because they had private health insurance or were themselves health professionals. At the post-intervention stage, all the participating centres had computerized medical records; thus, any patient with a health card had a viewable medical record, although they might have contained nothing more than administrative information.
A SR to evaluate the effectiveness of different implementation methods, concludes that active forms of continuing medical education and multifaceted interventions are the most effective methods for implementing CPGs into general practice. Additionally, active approaches to changing physician performance were shown to improve practice to a greater extent than traditional passive methods [27].
The findings of the present work agree with those of other studies that have examined different ways of implementing CPGs. For example, a trial randomised by clusters involving primary healthcare teams in Norway [28] compared a multifaceted CPG implementation strategy for the management of cardiovascular risk factors (educational visits with auditing of clinical indicators and feedback, reminders, automated tools for assessing cardiovascular risk, plus information for patients) with passive dissemination. Three outcomes were measured: the recording of the calculated cardiovascular risk in medical histories, the achievement of therapeutic objectives, and increase in the use of thiazide diuretics. The intervention had a significant impact on prescribing of antihypertensive drugs (a 17% improvement in thiazide use was seen in the intervention arm compared to 11% in the control arm), and was ineffective in improving the quality of other aspects of managing hypertension and hypercholesterolemia in primary care. No significant differences were seen in the assessment of cardiovascular risk nor in any therapeutic objective [28].
A similar study was performed in the primary healthcare setting in Ontario [29], which had the aim of increasing the percentage of preventive interventions in accordance with the recommendations of the Canadian Task Force. A multifaceted intervention was designed involving prior auditing and the planning of interventions adapted to each centre, contact with opinion leaders, continuous feedback of results, explanatory sessions, the reaching of consensus for adapting recommendations for prevention, the use of a quality improvement framework, reminders, and information for patients. They randomly assigned the practices to either an 18-month multifaceted intervention delivered by 1 of 3 nurse facilitators (23 practices) or no intervention (23 practices). The unit of intervention and analysis was the medical practice. The outcome measure was an overall index of preventive performance. The intervention delivered by nurse facilitators was effective in modifying physician practice patterns and significantly improved preventive care performance. The improvement achieved in this study (11.5%) was similar than we have found [29].
Another study with a similar design of clinical trial by clusters, carried out in 43 primary care units of the Basque Country with the objective of evaluating the implementation of 3 cardiovascular CPG, finds that a multifaceted intervention has a modest effect in some variables of process, producing an improvement in cardiovascular risk assessment. However, this study could not find improvements in the clinical outcome variables [30].
Although improvements were seen in the outcome variable measured in the present work, there is a need to determine whether this actually translates into any better control of risk factors and any reduction in the number of cardiovascular events. Indeed, this is now being investigated by our group.
Limitations are: 1) the publication of the results has been delayed, for diferents reasons, but a systematic search with alerts has not established that this objective has already been resolved in the scientific literature; 2) it is difficult to perform this type of interventions with a multicenter character. However, the community of Madrid represents 6.5 million inhabitants with a disparate geography, as around 3 million live in a large city, while the rest are distributed in populations of between one hundred thousand and 2,000 inhabitants. This area of Madrid was chosen for having representation of this diversity.
In conclusion, the present work shows that a clinical practice guideline implementation strategy based on education involving opinion leaders plus training sessions for primary healthcare teams, improved the recording of the variables required to calculate cardiovascular risk, over the traditional method of dissemination used in the study area.
Authors’ Contributions
BNA and RSA had the original idea for the study.
BNA, FRS and LMSG designed and wrote the protocol.
BNA, MJFL, LMSG, FRS, VP, CSF, BSG, MRL, JLG, PLB, SMV, AGA and RSA participated in the writing of the drafts.
FRS wrote the statistical methods and carried out the data analysis.
BNA is the author responsible for the study and coordination of the research group.
All authors contributed towards data interpretation and read and approved the final manuscript.
Acknowledgements
The CPG was adapted by the SIRVA2 group and a large group of medical professionals from the Área 2 health area of Madrid. We thank the SIRVA2 group members: A. Albarran Pinar, E. Bartolomé Benito, L. Barutel Rubio, R. Carraro Casieri, JM. Castillo, M. Cid Artigas, A. Cubillo Serna, A. Ferrández Infante, MJ. Fernández Luque, A. Gallego Arenas, L. Garcia Olmos, I. Garcia Polo, G. Gil Juberías, A. Gonzalez Gamarra, P. Loeches Belinchón, F. López Corral, C. López Gonzalez, J. López González, MA. Martinez Alvarez, H. Martinez Sanz, MS. Mayayo Vicente, F. Mayordomo Sanchez De La Morena, B. Novella Arribas, A. Parejo Llenero, S. Payan Agudo, P. Pérez Cayuela, C. Refolio Abella, F. Rodríguez Salvanés, S. Ruiz Alonso, L. Ruiz Diaz, M. Ruiz López, RM. Sánchez Alcalde, LM. Sánchez-Gómez, B. Sierra García and C. Suarez Fernández.
Ethical Concerns
The described trial was approved by the Comisión de Investigación del Área 2. Consent to be included was obtained from all the health centres involved. Medical histories were consulted only by the research team. At all times confidentiality was preserved; no patients or medical professionals were identified.
Fundings
Funding for the trial was provided by the Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica (I+D+I). Instituto de Salud Carlos III—Fondo de Investigación Sanitaria. Expediente N˚ 031216.
Competing Interests
The authors declare that they have no competing interests.
References
- 1. Committee to Advise the Public Health Service on Clinical Practice Guidelines, Institute of Medicine (1990) Clinical Practice Guidelines: Directions for a New Program. National Academy Press, Washington DC.
- 2. Woolf, S.H., Grol, R., Hutchinson, A., Eccles, M. and Grimshaw, J. (1999) Potential Benefits, Limitations, and Harms of Clinical Guidelines. British Medical Journal, 318, 527-530. https://doi.org/10.1136/bmj.318.7182.527
- 3. Grimshaw, J.M. and Russell, I.T. (1993) Effect of Clinical Guidelines on Medical Practice: A Systematic Review of Rigorous Evaluations. The Lancet, 342, 1317-1322. https://doi.org/10.1016/0140-6736(93)92244-N
- 4. Feder, G., Eccles, M., Grol, R., Griffiths, C. and Grimshaw, J. (1999) Clinical Guidelines: Using Clinical Guidelines. British Medical Journal, 318, 728-730. https://doi.org/10.1136/bmj.318.7185.728
- 5. NHS Centre for Reviews and Dissemination (1999) Getting Evidence into Practice. Effective Health Care, 5, 1.
- 6. Grupo de trabajo sobre implementación de CPG (2009) Implementación de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. Plan Nacional para el Sistema Nacional de Salud del Ministerio de Sanidad y Política Social. Instituto Aragonés de Ciencias de la Salud I+CS, Madrid.
- 7. Gagliardi, A., Brouwers, M., Palda, V., Lemieux-Charles, L. and Grimshaw, J. (2011) How Can We Improve Guideline Use? A Conceptual Framework of Implementability. Implementation Science, 6, 26. https://doi.org/10.1186/1748-5908-6-26
- 8. Brouwers, M., Makarski, J., Kastner, M., Hayden, L., Bhattacharyya, O. and the GUIDE-M Research Team (2015) The Guideline Implementability Decision Excellence Model (GUIDE-M): A Mixed Methods Approach to Create an International Resource to Advance the Practice Guideline Field. Implementation Science, 10, 36. https://doi.org/10.1186/s13012-015-0225-1
- 9. Field, M.J. and Lohr, K.N. (1992) Guidelines for Clinical Practice. From Development to Use. Institute of Medicine, National Academy Press, Washington DC.
- 10. Oxman, A.D., Thomson, M.A., Davis, D.A. and Haynes, R.B. (1995) No Magic Bullets: A Systematic Review of 102 Trials of Interventions to Improve Professional Practice. Canadian Medical Association Journal, 153, 1423-1431.
- 11. Grol, R. and Grimshaw, J. (2003) From Best Evidence to Best Practice: Effective Implementation of Change in Patients; Care. The Lancet, 362, 1225-1230. https://doi.org/10.1016/S0140-6736(03)14546-1
- 12. Cochrane Effective Practice and Organisation of Care Review Group (2011) Data Collection Checklist. Institute of Population Health, Ontario.
- 13. Grimshaw, J.M., Thomas, R.E., MacLennan, G., Fraser, C., Ramsay, C.R., Vale, L., Whitty, P., Eccles, M.P., Matowe, L., Shirran, L., Wensing, M., Dijkstra, R. and Donaldson, C. (2004) Effectiveness and Efficiency of Guideline Dissemination and Implementation Strategies. Health Technology Assessment, 8, 1-84. https://doi.org/10.3310/hta8060
- 14. Grupo de trabajo sobre CPG (2006) Elaboración de Guías de Práctica Clínica en el Sistema Nacional de Salud. Plan Nacional para el SNS del MSC. Instituto Aragonés de Ciencias de la Salud (I +CS), Madrid.
- 15. Worrall, G., Chaulk, P. and Freake, D. (1997) The Effects of Clinical Practice Guidelines on Patient Outcomes in Primary Care: A Systematic Review. Canadian Medical Association Journal, 156, 1705-1712.
- 16. U.S. Preventive Service Task Force (1996) Guide to Clinical Preventive. 2nd Edition, Williams & Wilkins, Baltimore, MD.
- 17. Rodríguez-Salvanés, F., Novella, B., Fernández Luque, M.J., Sánchez-Gómez, L.M., Ruiz-Díaz, L., Sánchez-Alcalde, R., et al. (2011) Efficacy of a Strategy for Implementing a Guideline for the Control of Cardiovascular Risk in a Primary Healthcare Setting: The SIRVA2 Study a Controlled, Blinded Community Intervention Trial Randomised by Clusters. BMC Family Practice, 12, 21. https://doi.org/10.1186/1471-2296-12-21
- 18. Donner, A. and Donald, A. (1987) Analysis of Data Arising from Stratified Design with the Cluster as Unit of Randomization. Statistics in Medicine, 6, 43-52. https://doi.org/10.1002/sim.4780060106
- 19. Hayes, R. and Moulton, L. (2009) Cluster Randomised Trials. Chapman & Hall, London. https://doi.org/10.1201/9781584888178
- 20. Eichler, K., Zoller, M., Tschudi, P. and Steurer, J. (2007) Barriers to Apply Cardiovascular Prediction Rules in Primary Care: A Postal Survey. BMC Family Practice, 8, 1. https://doi.org/10.1186/1471-2296-8-1
- 21. Hobbs, F.D. and Erhardt, L. (2002) Acceptance of Guideline Recommendations and Perceived Implementation of Coronary Heart Disease Prevention among Primary Care Physicians in Five European Countries: The Reassessing European Attitudes about Cardiovascular Treatment (REACT) Survey. Family Practice, 19, 596-604. https://doi.org/10.1093/fampra/19.6.596
- 22. Brotons, C., Lobos, J.M., Royo-Bordonada, M.A., Maiques, A., de Santiago, A., Castellanos, A., et al. (2013) Implementation of Spanish Adaptation of the European Guidelines on Cardiovascular Disease Prevention in Primary Care. BMC Family Practice, 14, 36. https://doi.org/10.1186/1471-2296-14-36
- 23. Hall, L.M.L., Jung, R.T. and Leese, G.P. (2003) Controlled Trial of Effect of Documented Cardiovascular Risk Scores on Prescribing. British Medical Journal, 326, 251-252. https://doi.org/10.1136/bmj.326.7383.251
- 24. Suárez, C., álvarez-Sala, L., Mostaza, J., Asenjo, C. and Grupo Multidisciplinario para el Estudio del Riesgo Cardiovascular (2007) Cálculo del riesgo cardiovascular. Medicina Clínica, 129, 534-541. https://doi.org/10.1157/13111428
- 25. Dallongeville, J., Banegas, J.R., Tubach, F., Guallar, E., Borghi, C., De Backer, G., et al. (2012) Survey of Physicians’ Practices in the Control of Cardiovascular Risk Factors: The EURIKA Study. The European Journal of Cardiovascular Prevention & Rehabilitation, 19, 541-550. https://doi.org/10.1177/1741826711407705
- 26. Campbell, M.K., Grimshaw, J.M. and Elbourne, D.R. (2004) Intracluster Correlation Coefficients in Cluster Randomized Trials: Empirical Insights into How Should They Be Reported. BMC Medical Research Methodology, 4, 9. https://doi.org/10.1186/1471-2288-4-9
- 27. Mostofian, F., Ruban, C., Simunovic, N. and Bhandari, M. (2015) Changing Physician Behavior: What Works? American Journal of Managed Care, 21, 75-84.
- 28. Fretheim, A., Oxman, A.D., Havelsrud, K., Treweek, S., Kristoffersen, D.T. and Bjorndal, A. (2006) Rational Prescribing in Primary Care (RaPP): A Cluster Randomized Trial of a Tailored Intervention. PLoS Medicine, 3, e134. https://doi.org/10.1371/journal.pmed.0030134
- 29. Lemelin, J., Hogg, W. and Baskerville, N. (2001) Evidence to Action: A Tailored Multifaceted Approach to Changing Family Physician Practice Patterns and Improving Preventive Care. Canadian Medical Association Journal, 164, 757-763.
- 30. Etxeberria, A., Alcorta, I., Pérez, I., Emparanza, J.I., Ruiz de Velasco, E., Iglesias, M.T., et al. (2018) Results from the CLUES Study: A Cluster Randomized Trial for the Evaluation of Cardiovascular Guideline Implementation in Primary Care in Spain. BMC Health Services Research, 18, 93. https://doi.org/10.1186/s12913-018-2863-x