Quality Assessment of Online Patient Education Materials for Diabetic Foot ()
1. Introduction
Diabetes mellitus is a common chronic disease that puts patients at risk of various complications. A particularly prevalent complication of poorly controlled diabetes is diabetic foot syndrome (DFS), which contributes to significant morbidity and mortality in diabetic patients. The World Health Organization describes it as a foot ulceration (distally from the ankle and including the ankle) associated with neuropathy and different grades of ischemia and infection [1]. A multitude of pathogenic events can lead to DFS, such as peripheral neuropathy, vasculopathy, foot deformity, abnormal foot pressure, abnormal joint mobility, trauma, peripheral artery disease, and inadequate glycemic control [2]. Numerous studies and reports on DFS have shown the importance of early diagnosis, prevention, and prompt attention to this condition [3]. Unfortunately, those who lack regular access to healthcare are disproportionately affected by chronic DFS.
Studies have demonstrated that 15% to 25% of individuals with diabetes mellitus will experience diabetic foot ulcers at some point in their lives [2]. The consequences of these ulcers extend beyond a lower quality of life and functional limitations, as they are also responsible for a significant number of non-traumatic lower extremity amputations. Patients with DFS face a daunting 7% risk of limb amputation within a decade of diagnosis. Moreover, there are other complications, such as gangrene, permanent deformity, osteomyelitis, and a significant risk of infection and sepsis [2].
Patients with DFS often require frequent evidence-based treatment at specialized interdisciplinary foot clinics. These regular clinic visits can pose challenges for patients in terms of time, effort, and financial constraints, especially for individuals in rural and remote areas or those with limited mobility. Moreover, even with weekly visits, there is still a risk that foot ulcers may deteriorate to severe infections, necessitating hospitalization or amputation within a matter of days [4].
Preventing and managing DFS requires patient education. With the increasing availability of Internet access, many patients turn to health information on public websites to learn about this health condition. It has become common for patients to self-diagnose based on the information they find online before consulting medical professionals. Therefore, access to reliable online medical information is crucial in shaping patients’ decision-making processes [5] [6]. While web-based patient education is convenient, efficient, and cost-effective compared to traditional one-on-one consultations or booklets, the quality of online education materials remains uncertain. The comprehensibility, readability, and reliability of web-based information for diabetes mellitus have been evaluated, but the assessment of online resources related specifically to diabetic foot care is lacking [7].
This study aimed to evaluate Arabic websites about diabetic foot care. The evaluation was conducted using a modified Ensuring Quality Information for Patients (EQIP) tool, which incorporates 36 elements aligned with the best patient information standards outlined by the British Medical Association (BMA) and the International Patient Decision Aids Standards (IPDAS) Collaboration [8] [9].
2. Methods
2.1. Study Design, Sampling Strategy, and Eligibility Criteria
This was a bibliometric study of web pages containing DFS information written in Arabic. On January 24, 2023, one assessor (DH) searched multiple terms on the most popular search engine, Google. The authors used only one search engine as previous literature demonstrated that using a variety of search engines returns a high rate of duplicate records [10]. Before the search terms were entered, the browser was set to incognito, or private browsing mode, to protect the privacy of the search so that browsing history, site data, or cookies would not be saved on the device. Moreover, all cached data, previous search history, and cookies were deleted, and location settings were turned off to avoid influencing the search results. Multiple search terms were entered قدم السكري ,علاج قدم السكري ,العناية بقدم, السكري ,العناية بجرح قدم السكري ,تقرح القدم السكري, and الوقاية من قدم السكري, which translate to “diabetic foot”, “the treatment of diabetic foot”, “diabetic foot care”, “diabetic foot ulcer”, “diabetic foot wound care”, and “diabetic foot prevention” in English, respectively. The first 20 pages of websites were evaluated as previous research has shown that patients stay within the first 20 search result pages and the first 100 web pages [11]. If the search term did not yield 20 pages of results, the maximum number of pages displayed were used.
All websites were screened by four assessors (DH, HK, MA, EH-A) twice between February 2 and February 25, 2023. The first assessment was a concise review of each web page title, followed by an in-depth evaluation during the second assessment. Each website listed on the first 20 results pages was evaluated for inclusion and exclusion criteria.
All web pages (as well as portable document format [PDF] files, presentations, and infographics fashioned similarly to medical brochures) pertaining to DFS in Arabic were included. Books, social media, non-functioning websites, non-text web pages containing only videos and/or audio, web pages not written in Arabic, websites selling products, websites with access restrictions, and websites not containing information on diabetic foot diagnosis, treatment, or prevention were all excluded.
2.2. Website Scraping
An algorithm developed on Google Colab allowed for the creation of a unified dataset of each term’s search results (Appendix 1). Although the search was done in Bukaryiah, Saudi Arabia, the location settings were turned off so as not to limit the search to certain geographical locations.
2.3. Website Data Extraction and Measures
From March 5 to April 24, 2023, two pairs of assessors (DH, HK, MA, EH-A), all of whom are native Arabic speakers, extracted data from all retrieved websites. Any disagreements in the data extraction process were reviewed and resolved by the supervising faculty (author: JS). The assessment included 35 items of the EQIP tool (yes, no, or NA); the reasons for changing the original tool are elaborated below. Using Similarweb (link in Appendix 2), the assessors identified each website’s country of origin (grouped into Arab and non-Arab countries), and amount of traffic (number of visits per month). Moreover, the source type (governmental or nongovernmental) was recorded. Websites were further categorized as encyclopedias, hospital and clinic websites, medical information websites (i.e. not regulated by governmental or professional organizations), news websites, professional medical organization sites, and university or college websites. The types of additional therapeutic information on each website, i.e. information on prevention and conservative, pharmacological, and surgical treatment of DFS, were also recorded.
2.4. EQIP Tool
On the modified Ensuring Quality Information for Patients (EQIP) tool [9], 35 of the 36 items were used to evaluate the quality of the retrieved websites. The tool, which has been utilized by multiple studies [12] [13], assesses websites in three domains: content (items 1 - 18), identification (items 19 - 24), and structure (items 25 - 35). The content domain evaluates the amount of information on a website. The identification domain assesses the details of article conception, e.g. the references, the author, financial support, and the date of issuance. The structure domain assesses a web page’s layout and readability, e.g. using short sentences of less than 15 words and addressing the patient directly. The last item of the EQIP tool, “the inclusion of a printed consent form contrary to recommendations”, was removed because it was not applicable to any of the included websites. The EQIP tool has “yes, no, partly, or not applicable (NA)” answers. However, according to previous research, the inclusion of “partly” as an answer increases the subjectivity of the research and lowers its reliability [13]. Therefore, it was removed from the authors’ assessment. They created a binary answer option (NA if absent or 1 if present) for the “clear and relevant figures or graphs” item because all figures on the included websites were relevant to DFS. The original EQIP tool was written entirely in English, so four assessors (DH, HK, MA, EH-A), all of whom were fluent in both English and Arabic, translated the tool into Arabic. Later, an external assessor confirmed the accuracy of the translation. A pre-test on 20 websites using the translated EQIP tool confirmed it produced diverse and accurate results.
2.5. Study Procedures
2.5.1. The Search
Following the sampling strategy, the first 20 pages or the maximum number of pages from Google for the six search terms were retrieved, and one researcher entered the websites’ addresses into a standard Excel form and removed the duplicates.
2.5.2. Screening
Four researchers screened the web pages twice for inclusion and exclusion criteria. The process was documented and summarized in a flow diagram.
2.5.3. Data Collection
Two pairs of researchers assessed the websites with the EQIP tool and recorded all other covariates.
2.6. Analysis
The data was analyzed using SPSS version 28 software and incorporated both continuous and categorical variables. The continuous data were reported as medians and interquartile ranges (IQR), whereas the categorical data were reported as frequencies and percentages. Each website was scored using 35 items of the EQIP tool. The frequency of no, yes, and NA answers in the EQIP tool were displayed in a frequency table. Websites with a 75th percentile score on the EQIP tool were considered high-scoring websites [10]. Fisher’s exact and chi-square tests were used to compare the high- and low-scoring websites. Continuous variables were compared with the Kruskal-Wallis test where appropriate. The overall performance of all the websites was reported as the mean overall score for the EQIP tool. The top-scoring websites, defined as those with an EQIP score above the 95th percentile, were identified. Website details such as the country of origin, source type, website traffic, and additional therapeutic information were reported using frequencies. Then, unadjusted and adjusted binary logistic regression tests were conducted to understand the relationships between covariables and the probability of achieving a high EQIP score. A p-value of 0.05 was considered significant.
2.7. Ethical Approval
As the study did not involve participants or patient records, no ethical approval was required.
3. Results
3.1. Extracted Websites
A total of 559 websites were retrieved through Google, out of which 172 were duplicates. A further 215 websites met the exclusion criteria: 124 not related to DFS, 20 books, 19 non-functioning websites, 18 social media sites, 14 product sales sites, ten sites not in Arabic, six with no text, and four with access restrictions. A second assessment resulted in 15 more website exclusions: six missing required information, six non-functioning sites, two product sales sites, and one not in Arabic. A final 157 sites were included in the study. The workflow of the screening is summarized in Figure 1.
3.2. Overall Website Performance
The median EQIP score for all included websites was 16 out of 35 (IQR = 14 - 20), while the median scores for the content, identification, and structure domains were 8 out of 18 (IQR = 6 - 10), 3 out of 6 (IQR = 3 - 4), and 5 out of 11 (IQR = 4 - 7) possible points, respectively. A total of 39 websites with EQIP scores higher than 20 (75th percentile EQIP score) were labeled as high-scoring sites. The remaining 118 websites scored ≤20 and thus considered low-scoring sites (Figure 2).
Figure 1. Flowchart of identification and screening of websites for analysis.
Figure 2. EQIP scores of the included diabetic foot patient education websites. The 75th percentile was established as a cutoff point to dichotomize low-scoring (gray bars) and high-scoring (black bars) websites.
3.3. Website Characteristics
The descriptive characteristics of the included websites are displayed in Table 1. Most websites were from Arab countries (n = 119, 75.8%). There were 8 (5.1%) websites from governmental and 149 (94.9%) from non-governmental sources. Among the source subtypes, medical information websites constituted the majority of sources (n = 73, 46.5%), followed by newspapers and journals (n = 36, 22.9%), hospitals and clinics (n = 33, 21%), professional medical organizations (n = 9, 5.7%), and encyclopedias (n = 2, 1.3%). There was a significant difference in high and low EQIP scores among the source subtypes (p = 0.038). University/college websites and medical information websites had the highest proportions of high scores among all source subtypes, i.e. 50.0% (2/4) and 35.6% (26/47), respectively. However, the Kruskal-Wallis analysis demonstrated insignificant differences in continuous EQIP score distribution across source subtypes (p = 0.153) (Figure 3).
Table 1. Descriptive characteristics of the included websites (n = 157).
Category |
Subcategory |
Number (%) |
EQIP score (n) |
p-value |
High (n = 39) |
Low (n = 118) |
Country of origin |
Arab countriesa |
119 (75.8) |
31 |
88 |
0.535 |
Non-Arab countriesb |
38 (24.2) |
8 |
30 |
Source type |
Governmental |
8 (5.1) |
1 |
7 |
0.407 |
Non-governmental |
149 (94.9) |
38 |
111 |
Source subtype |
Encyclopedias |
2 (1.3) |
0 |
2 |
0.038 |
Hospitals and clinics |
33 (21.0) |
4 |
29 |
Medical information websites |
73 (46.5) |
26 |
47 |
Newspapers and journals |
36 (22.9) |
5 |
31 |
Professional medical organizations |
9 (5.7) |
2 |
7 |
Universities and colleges |
4 (2.5) |
2 |
2 |
Number of visitorsc |
<10,000 |
34 (21.7) |
10 |
24 |
0.266 |
10,000 - 100,000 |
38 (24.2) |
13 |
25 |
100,0001 - 500,000 |
18 (11.5) |
4 |
14 |
>500,000 |
67 (42.7) |
12 |
55 |
aArab countries included Egypt (n = 48), Saudi Arabia (n = 25), Jordan (n = 18), Morocco (n = 6), Algeria (n = 3), Iraq (n = 2), Kuwait (n = 2), Qatar (n = 2), Lebanon (n = 1), Palestine (n = 1), Tunisia (n = 1), UAE (n = 1), and Yemen (n = 1). bNon-Arab countries included the United States (n = 17), Germany (n = 4), India (n = 4), Türkiye (n = 4), Netherlands (n = 2), Australia (n = 1), Brazil (n = 1), France (n = 1), Israel, (n = 1), Russia (n = 1), Thailand (n = 1) and Ukraine (n = 1). cMean of number of visitors: 4213353.07. A p-value of 0.05 is considered significant.
Figure 3. EQIP score distribution by source subtypes. The median (horizontal thick line within the boxplot), the 75th and 25th percentiles (the upper and lower line of the box plot, respectively), and the maximum and minimum value (the upper and lower whisker line, respectively) are displayed. Outliers are depicted as circles. The median EQIP score (thick blue line) was 16; p-value = 0.153.
3.4. Inclusion of Additional Therapeutic Information Affects Website Quality
The inclusion or absence of additional therapeutic information about prevention and conservative, pharmacological, and surgical treatment was noted for each website (Table 2). A total of 123 websites (78.3%) described prophylactic measures against DFS, 38 (30.8%) of which were high scoring. There was only one high-scoring website among those that did not describe prevention (p = 0.001). Information about conservative treatment was present on 88 websites (56.0%), and 30 of them (34.1%) had high scores. Among the 69 websites where conservative treatment was not described, only 9 (13%) scored well (p = 0.002). Information on
Table 2. Additional therapeutic information provided by the websites.
Additional therapeutic
information |
EQIP score (n) |
p-value |
High (EQIP score > 20) (n = 39) |
Low (EQIP score ≤ 20) (n = 118) |
Prevention |
Yes |
38 |
85 |
0.001 |
No |
1 |
33 |
Conservative treatment |
Yes |
30 |
58 |
0.002 |
No |
9 |
60 |
Pharmacological treatment |
Yes |
26 |
54 |
0.024 |
No |
13 |
64 |
Surgical treatment |
Yes |
15 |
46 |
0.954 |
No |
24 |
72 |
pharmacological and surgical treatments was provided on 80 (50.9%) and 61 (38.8%) websites, respectively. There were significantly more high-scoring websites among those that described pharmacological treatment compared to those that did not provide such information (26 [32.5%] vs. 13 [16.8%], p = 0.024). Of the websites providing surgical treatment information, 15 (24.5%) scored well, compared to those that did not provide surgical intervention information (p = 0.954).
3.5. Relationship between Select Variables and EQIP Score
The unadjusted binary logistic regression assessed the country of origin, source types, source subtypes, number of visitors, and additional therapeutic information (Table 3). The medical information websites subgroup was the reference group among the source subtypes due to its high frequency (n = 73). Among the select variables, only the inclusion of additional therapeutic information on prevention (OR = 14.8, 95% CI = 1.95 - 111.87, p-value = 0.009), conservative treatment (OR = 3.4, 95% CI = 1.51 - 7.89, p-value = 0.003), and pharmacological treatment (OR =
Table 3. Unadjusted logistic regression analysis: correlations of select variables (source subtypes and websites’ additional therapeutic information) with the probability of a high EQIP score, i.e. 75th percentile (EQIP score > 20).
Category |
Subcategory |
N |
OR |
CI (95%) |
p-value |
Lower |
Upper |
Country of origin |
Arab countries |
119 |
Ref |
|
|
0.536 |
Non-Arab countries |
38 |
0.8 |
0.31 |
1.83 |
Source types |
Governmental |
8 |
Ref |
|
|
0.421 |
Non-governmental |
149 |
2.4 |
0.29 |
20.11 |
Source subtypes |
Encyclopedias |
2 |
0.0 |
0.00 |
. |
0.070 |
Hospitals and clinics |
33 |
0.2 |
0.08 |
0.79 |
Medical information websites |
73 |
Ref |
|
|
Newspapers and Journals |
36 |
0.3 |
0.10 |
0.84 |
Professional medical organizations |
9 |
0.5 |
0.10 |
2.67 |
Universities and colleges |
4 |
1.8 |
0.24 |
13.59 |
Number of visitors |
<10,000 |
34 |
Ref |
|
|
0.276 |
10,000 - 100,000 |
38 |
1.2 |
0.46 |
3.38 |
100,001 - 500,000 |
18 |
0.7 |
0.18 |
2.60 |
>500,000 |
67 |
0.5 |
0.20 |
1.38 |
Additional therapeuticinformation |
Prevention |
No |
34 |
Ref |
|
|
0.009 |
Yes |
123 |
14.8 |
1.95 |
111.87 |
Conservative treatment |
No |
69 |
Ref |
|
|
0.003 |
Yes |
88 |
3.4 |
1.51 |
7.89 |
Pharmacological treatment |
No |
77 |
Ref |
|
|
0.026 |
Yes |
80 |
2.4 |
1.11 |
5.06 |
Surgical treatment |
No |
96 |
Ref |
|
|
0.119 |
Yes |
61 |
0.4 |
0.14 |
1.25 |
2.4, 95% CI = 1.11 - 5.06, p-value = 0.026) were significantly associated with a high EQIP score.
The adjusted binary logistic regression among significant covariates revealed significantly increased odds of getting a high score only when a website had information on prevention (OR = 12.9, 95% CI = 1.68 - 98.57, p = 0.010) (Table 4).
Table 4. Adjusted binary logistic regression: correlations of number of visitors and inclusion of prevention and conservative treatment information with the probability of a high EQIP score, i.e. 75th percentile (EQIP score > 20).
Category |
Subcategory |
N |
OR |
CI (95%) |
p-value |
Lower |
Upper |
Prevention |
No |
34 |
Ref |
|
|
0.014 |
Yes |
123 |
12.9 |
1.68 |
98.57 |
Conservative treatment |
No |
69 |
Ref |
|
|
0.073 |
Yes |
88 |
2.7 |
0.91 |
8.09 |
Pharmacological treatment |
No |
77 |
Ref |
|
|
0.738 |
Yes |
80 |
1.2 |
0.43 |
3.30 |
3.6. Top-Performing Websites
The top websites based on the 95th percentile EQIP score, i.e. a score of >25, are listed in Table 5.
Table 5. Top-ranking websites (EQIP score > 95th percentile, i.e. score of >25).
4. Discussion
This is the first study to appraise websites containing information on DFS exclusively in Arabic. The use of the modified and validated EQIP tool allowed for an exhaustive assessment of the sites, including their content, sources, and readability. Current online patient information for DFS in Arabic is generally poor (median EQIP score = 46%). Previous studies found online educational materials about various medical conditions to be similarly inadequate [11] [14]-[16].
Most websites in this study originated in Arab countries; this was to be expected as only web pages written in Arabic were included. In the current study, most of the websites were non-governmental. This finding differs from previous studies that reported that government entities run the majority of patient education websites or are a close second to another information source [10] [11]. The current finding reveals that Arab governmental institutions do not produce an adequate number of health-related websites even for such a common disease as DFS. Of the non-governmental websites, the majority were medical information websites, which are not regulated by governmental or professional organizations.
With almost 43% of the sites receiving over 500,000 visits per month, it is clear the public needs high-quality information on DFS, but such high traffic on poorly regulated websites is concerning. On the other hand, most websites did include prevention and conservative and pharmacological treatment information, leading to relatively comprehensive websites. Sites that incorporated information on prevention were thirteen times as likely to achieve a high EQIP score compared to those with no description of DFS prevention.
There is a paucity of studies evaluating the quality of diabetes information online [17] [18], and only three studies have evaluated websites specifically about DFS. One study based in Scotland found that only a minority of the studied websites were easily accessible; the majority scored poorly (between 5.5 and 14.5 out of a maximum score of 23) [16]. A study out of France assessed the quality of DFS information written in French and found that only 13.5% of the included websites could be recommended to patients [15]. The majority of the sites were based in France and produced by pharmaceutical companies, followed by medical organizations, health professionals, patients, and hospitals [15]. Lastly, Lee et al. (2022) evaluated 41 websites on DFS with the DISCERN tool (maximum possible score = 80) [11]. The median DISCERN score in the study was only 45.66. In addition, most of the sites required a higher reading level than the Canadian Medical Protective Association recommends and higher than the average North American adult’s reading level. As such, the aforementioned studies corroborate the current findings regarding the inadequate quality of online DFS information.
This study had several limitations. While utilizing the most popular search engine, Google, with six key terms likely simulated how patients usually search for information, it is also possible that patients use different, lesser-known search terms. Moreover, it is difficult to predict which combination of search phrases/terms patients use. This could potentially introduce subjective bias into the study. However, the study included terms that are most common among Arabic-speaking patients. Other search engines (https://bing.com/ and https://yahoo.com/, among others) were not explored in our study, which may consequently impact the variability of the findings. Social media websites were excluded; though it is often assumed that the public is increasingly using social media to obtain information, a previous study demonstrated decreased public engagement with health-related social media content due to a lack of trust in the information’s quality and authority [19]. Non-text websites were also excluded; while they may contain valuable information, there is no validated tool to adequately assess video or audio-only information sources. Even though some websites that sell or promote products may contain adequate information on DFS, the authors excluded those sites to avoid indirectly and unethically promoting certain products. These exclusions could consequently introduce selection bias into the study. We also acknowledge the lack of ability to analyze the websites’ search engine optimization (SEO) effectiveness. SEO can impact each website’s traffic and visit counts, and consequently, the quality of the website. Nevertheless, we found no significant association between the number of visits and the quality of websites, so we expect that SEO also has a nonsignificant correlation with website quality.
More similar studies should be conducted as the number of patients seeking information from the web is rapidly increasing. The Internet has become the primary source of information for a majority of the population [20]. While there are a significant number of studies evaluating the quality of medical information on the Internet, too few have assessed web pages written in Arabic. More studies are needed to evaluate and recommend high-quality sites. Governmental and professional organizations in Arab countries should endeavor to produce excellent fact-checked websites.
5. Conclusion
The current search of websites found a variety of Internet sources that provide information on DFS. The relatively comprehensive information they provide on prevention and conservative, pharmacological, and surgical treatments may influence patients’ health decisions, but the current results demonstrated that online DFS information in Arabic is of overall low quality. Most sites were medical information sites, with only a fraction produced by governmental or professional organizations. The high traffic on these sites shows that there is public demand for DFS information, but higher-quality web pages are needed to positively impact patients’ health outcomes.
Funding
This research was funded by Sulaiman Al Rajhi University through its annual allocated funds for research activities.
Data Availability
All data included or relevant to the study are available upon request by contact with the corresponding author.
Authors’ Contributions
Conceptualization: DH, EH-A., HK, MA, and JS; methodology: FZMA, NS, and JS; validation: FZMA, MCJ, NS, and JS; formal analysis: DH, EH-A, HK, MA, JS, and MCJ; investigation: DH, EH-A, HK, and MA; data curation: JS, FZMA, MCJ, and NS; writing of original draft preparation: DH, EH-A, HK, MA, and MCJ; writing review and editing: NS, JS, and MCJ.; supervision: FZMA, NS, and JS; project administration: FZMA, MCJ, and JS. All authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript.
Appendices
Appendix 1
Google Colab link of the included websites:
https://colab.research.google.com/drive/1UEUc9wXX8ajSULOL4U3mzy-ufe2ELKOV?usp=sharing.
Appendix 2
Similarweb link:
https://www.similarweb.com/.