Development and Validation of a Tool to Assess Hepatocellular Carcinoma Knowledge among Hepatocellular Carcinoma Patients in Taiwan ()
1. Introduction
Hepatocellular cancer is a disease with a high incidence and a high mortality rate [1] [2]. In year 2018, it is estimated about 841,080 new cases globally [1]. Hepatocellular cancer is the second leading cause of cancer death in East Asia and sub-Saharan Africa and the sixth most common in Western countries [1] [2]. The incidence of liver cancer in Taiwan ranks the second in men and the fifth in women [3] [4]. Since 1982, hepatocellular cancer has become the top 10 cause of death in Taiwan [4]. New technologies and expanded indications for standard therapies have led to improved patient outcomes. For people with early-stage hepatocellular cancers who have a liver transplant, the 5-year survival rate is in the range of 60% to 70% and the localized lesion 5-year relative survival rate is about 33%. However, the late-stage survival is dismal [5]. None of the trials reported health-related quality of life.
BCLC B represents a heterogeneous group of patients. There are multiple options for patients with potentially compensatory cirrhosis and potentially resectable, small tumors, including eradication therapy [2] [6] [7]. In early and very early-stage liver cancer eligible for surgery, there was no evidence of a difference in death between radiofrequency ablation and surgery [8]. There were more serious complications in the surgery group than in the radiofrequency ablation group. In people not eligible for surgery, the percentage of people who died during the follow-up period was higher in the percutaneous acetic acid injection and percutaneous alcohol injection groups than in the radiofrequency ablation group [2] [6] [7]. Chemoembolization is the standard of care, based on several RCTs, which demonstrated improved overall survival compared to best supportive care [9]. But choosing a therapeutic plan has become increasingly complex and requires consideration of multiple factors including tumor burden, baseline liver function, and comorbidities [2]. The choice of cancer treatment is extremely important and involves a high-risk, complex and difficult decision-making process to cancer patients [10].
Knowledge is defined as “the acquisition of awareness of facts, data, information, ideas or principles to which one has access through formal or individual study, research, observation, experience or intuition” [11] [12]. Knowledge helps people solve problems and deal with unexpected situations [13]. When people lack information or skills, they may make decisions that are not optimal. Although the guidelines use evidence for disease diagnosis and treatment, due to the individual patient factors and tumor stage differences, cancer treatment decisions by patients still lead to uncertainty [14]. Many studies were conducted to understand healthcare professionals’ knowledge [15], patients’ and family caregivers’ knowledge and beliefs [16] [17] [18] [19] Measuring knowledge is very important. If we want to manage something, we have to be able to measure it, it also helpful for the prediction of future performance of individuals [13]. To evaluate whether interventions are effective, instruments with satisfactory validity, reliability and clinical utility are needed for ongoing monitoring [13]. But there is a lack of validated instruments to evaluate hepatocellular cancer patient’s knowledge to measure patients’ understanding level about disease association. The purpose of the study is to develop a hepatocellular cancer knowledge scale and to test its psychometric properties.
2. Methods
2.1. Study Design-Sample and Setting
One hundred patients met the study eligibility criteria were recruited from February 13, 2019 to November 4, 108. The inclusion criteria include 1) diagnosed with hepatocellular carcinoma (HCC) (ICD 10 C22.0); 2) aged 20 or older; 3) No mental illness; 4) Patients who can communicate in Mandarin or Taiwanese. The exclusion criteria include 1) Don’t know disease condition; 2) Unconscious patients
The desired sample size is estimated according to Comrey [20] for develop scales; the number of valid samples is at least five times the number of scale questions [21]. There are twenty items in the knowledge scale tested in this study; therefore, at least one hundred subjects are needed.
2.2. Instrument Development
For the purpose of the study, the hepatocellular cancer (HCC) knowledge scale includes items taped disease, treatment, and follow-up care. Disease knowledge includes causes, symptoms, and prognosis of liver cancer. Treatment knowledge includes treatment types, methods, and costs. Follow-up knowledge include follow-up after liver cancer treatment and follow-up care. Items were generated from literature, including the American Association for the Study of Liver Diseases (AASLD) guideline for the treatment of hepatocellular carcinoma [22], patient education manual of the Liver Disease Prevention & Treatment Research Foundation and what physicians believe patients need to know about liver cancer. Originally, we developed 60 items. There are 15 disease items, 30 treatment items, and 15 post treatment follow-up items. But because the questions are too similar and there is no strong evidence and answers to some questions are controversial. It was reduced to 20 questions after expert advice. The preliminary version of HCC knowledge scale includes 20 items. These are 5 disease items, 10 treatment items, and 5 follow-up items. Each item is rated by a 5-point Likert-type scale. There are 1 points is irrelevant and not appropriate to5 point is very relevant and very appropriate
The preliminary version of HCC knowledge scale was then reviewed for its content validity by 5 experts, including two researchers, two hepatobiliary physicians, and a case manager. Two items (Liver cancer is impossible to cure and No recurrence after liver cancer treatment) regarding were scored 2 (as irrelevant and not appropriate) by two 2 experts. The two items were then removed from the scale. The two experts suggested adding items regarding treatments for the late-stage HCC. A new question to the cause, symptoms, and prognosis of liver cancer, and add another question to the type, method and cost of treatment. So they were modified and the treatment type, method and cost integrity inadequate, it was revised to 4 questions about the causes, symptoms, and prognosis of liver cancer; comparison of benefit and harm of liver cancer treatment types, treatment methods, and costs, a total of 10 questions; 5 questions for follow-up care and follow-up. All necessary modifications, based on the divergence of opinions between experts, were made to the instrument. Again the same procedure was followed. The final version of the instrument invites experts to make the final score. The con-tent validity index (CVI) of each question will be more than 0.8 and will be finalized.
Invited 6 reviewers whose surface validity is a high-risk group of liver cancer (hepatitis B, C carriers (future potential users), patients diagnosed with liver cancer and patients who have previously treated liver cancer). Rate literally “clear” and “understandable” the possible answers ranging from strongly agree (=4) to strongly disagree (=1). The surface validity index each of each question will be more than 0.8 and will be finalized.
The answers to questions 4, 7, 11, 15, 18, and 19 are false, and the remaining answers are true. To avoid guessing the answer, set an option to Uncertain. For a correct calculation of 1 point, the incorrect answer is 0 point and the answer option is “uncertainty” 0 point. After calculating the total score, multiply it by 5 as the total score, and the total division is between 0 - 100 points as the evaluation knowledge level.
2.3. Instrument Administration
The questionnaire collector was trained and was a singular, fixed person. The interviewer has 20 years of work experience in nursing staff. The place where the case is collected can be locked in a separate room next to the consultation room and the conference room in the ward.
2.4. Ethics
After Review Board of Human experiment committee approval. Recruit patients with a diagnosis of liver cancer and agree to participate in the study after full explanation. Participants were free to decide whether to complete the questionnaire or not. All participants provided their written informed consent to participate in this study. There was no recompense for the participants.
2.5. Statistical Analysis
Statistics software usage SPSS version 20: Statistical strategies are demographic data nominal variables using N and percentage, continuous variables using the mean and standard deviation two-tailed p values < 0.05 were considered statistically significant. The reliability was to use KR 20 and test-retest reliability. The content validity was to use CVI (Content Validity Index) level. Construct validity uses correlation analysis to measure convergence validity and discrimination validity. Pearson and Spearman’s correlations were also calculated. The criterion validity was use item analysis.
2.6. Reliability
Reliability refers to the degree of consistency of the results obtained when the same phenomenon is repeatedly measured. Verify the internal consistency of the same measurement tool. In this study is use KR20 [23] (Kudel et al., 1937). The test-retest reliability is based on pilot testing of 6 participants. We ask participants for a second test after one day.
2.7. Validity
2.7.1. Content Validity
Include expert validity and face validity were use CVI (Content Validity Index) level. Expert validity, 5 researchers and experts, they were approved by three Associate Professor of Hepatobiliary physicians, a case manager, and a Professor expert in the fields and questionnaire construction. HCC knowledge questionnaire consisted of close-ended Likert-type questions, with relevance and appropriateness of the content the possible answers ranging from strongly agree (=4) to strongly disagree (=1). Expert validity is to use CVI (Content Validity Index).
2.7.2. Face Validity
A test with a high surface validity is more likely to win co-operation and a higher willingness to respond than a test with a low surface validity [24] (National Academy for educational research, 2020). Review the seemingly semantic of the subject on the knowledge scale. Rate literally “clear” and “understandable” the possible answers ranging from strongly agree (=4) to strongly disagree (=1). The face validity is used CVI level.
2.7.3. Construct Validity
Information needs, time focus, and patient participation are three important factors that may be useful starting points to guide clinical assessments (Etkind et al., 2017). Improved Knowledge can decrease decision conflict [25] (Jayadevan et al., 2019). When patients feel greater self-efficacy and have more knowledge and more prepared to make a decision [26] (Miller et al., 2013). Therefore, we use the decision self-efficacy as the detection of convergence validity and the decisional conflict as the discrimination validity.
2.7.4. Item Analysis
Item analysis uses the correct ratio of high score and low score groups and calculates the item difficulty index and item discrimination index.
3. Results
3.1. Participant Characteristics
The total number of participants is 102, of which 66 are male and 36 are female. No significant difference between scores and gender (p = 0.26). The education level is half of elementary school and below, 58 (57%) people graduated from high school, 30 (29%) people graduated from high school and 14 (14%) people graduated from university, dividing of formal schooling qualifications into two groups. No significant difference between scores and academic qualifications (p = 0.404). There are 72 (70.6%) participants of them were married, 30 (29.4%) were divorced and widowed. There are 78 retired or unemployed and 24 part-time or full-time work. There are 54 (53%) people relying on themselves or their spouse for financial resources, and 48 (47%) on their children, relatives or government subsidies. Most people live with parents, spouse or children (92%). A total of 46 (45%) participants were treated for liver cancer for the first time (see Table 1).
3.2. Reliability
Verify the internal consistency of the same measurement tool. In this study is use KR20. The KR20 level is 0.745. The higher the value of the coefficient, the better. If the coefficient 0.9 < α high reliability, 0.7 < α < 0.9 middle and upper reliability, 0.5 < α < 0.7 middle and lower reliability and α < 0.5 no reliability meaning. Therefore, there was evidence of acceptable internal consistency.
![]()
Table 1. The relationship between Patient Characteristic with disease stage.
3.3. Test-Retest Reliability
The reliability test is based on pilot testing of 6 participants. In the verification of reliability, 6 participants of the pilot testing were randomly screened for retest reliability. To avoid knowledge differences, we ask participants not to query the data for a second test after one day. The test-retest reliability was 100% (satisfied if an ICC coefficient = 0.70 is achieved).
3.4. Validity
3.4.1. Content Validity
Include expert validity and face validity were use CVI (Content Validity Index) level. Expert validity, all necessary modifications, based on the divergence of opinions between experts, were made to the instrument. The final version of the instrument invites experts to make the final score. The content validity index (CVI) of each question will be more than 0.8 and will be finalized. The total score of the content validity index is 0.96. Face validity, after expert validity complete. We invite 6 people of HCC (Hepatocellular carcinoma) patients to check the face validity with rate literally “clear” and “understandable”. The total score of the face validity index is 0.92.
3.4.2. Construct Validity
Pearson and Spearman’s correlations were calculated, the correlations with the decision conflict level were −2.99 (p < 0.005). The correlations with decision self-efficacy are 0.324 (p = 0.001). Therefore, there was adequate overall evidence for convergent and discriminatory validity, respectively.
3.4.3. Item Analysis
Enter the result into the excel file for a correct answer calculation of 1 point, the incorrect answer is 0 point, and the answer option is “uncertainty” 0 point. After calculating the total score, multiply it by 5 as the total score, and the total division is between 0 - 100 points as the evaluation knowledge level. Calculate the correct ratio of high score and low score groupings for each question for the total score. Convert excel to SPSS file. Cut files to calculate high and low grouping. The first 27% of participants made the high score group and the last 27%, the low score group. Calculate the correct ratio of high score and low score groupings for each question. The difficulty index, also called the ease index, describes the percentage of participants who correctly answered the item. It ranges from 0% - 100%. The higher the percentage is, the easier the item is. The recommended range of difficulty is from 25% - 75%. Items having p-values below 25% and above 75% are considered difficult and easy items respectively [27]. The difficulty index is moderate if it is close to 0.5. Chase (1978) believes that it is a binary form, and the difficulty index is in the range of 0.55 - 0.85 as the selection criterion. In our knowledge scale 5 questions with a score higher than 0.85. The remaining 15 questions are located 0.55 - 0.85 interval.
3.4.4. Discrimination Index (DI)
The ability of an item is described to distinguish between high and low scorers. The higher the discrimination, the better. Generally, the minimum standard is acceptable. The value is above 0.25 and great than 0.40 is a good question. There are 5 items below 0.25 and the other 15 items are above and also 13 items above 0.40 (see Table 2). Because our purpose is to test the patient’s knowledge of the relevant knowledge is not a general learning assessment. Therefore, although some questions are easy and less distinguishable, relevant questions remain based on purpose.
![]()
Table 2. Knowledge scale item analysis.
3.4.5. Ceiling and Floor Effects
The presence of ceiling and floor effects compromises the content validity. Furthermore, the ceiling and floor effects were noted. Experts tested this study before it was conducted. The content is what physicians believe patients need to know about liver cancer. Liver cancer patients are invited to review for clarity and understandable to avoid floor effects due to difficult words. The score is between 25 - 100 points in this study. The score means is 74.75 with a standard deviation of 16.709.
4. Discussion
We attempted to compare this questionnaire with the Health Literacy questionnaire; it was not significantly related (p > 0.05). Possibly due to Health Literacy questionnaire is a common problem such as medicine bag Description and common knowledge related to common diseases such as cellulitis and diabetes is often advertised on TV or radio. However, this questionnaire is unique to specific diseases and generally requires the ability to query the Internet and to be informed by professionals such as physicians. The sampled subjects in this study are not only too old to be able to search for relevant information but also Chinese people tend not to ask questions.
In the item 3 “Liver cancer detected early, there is a greater chance of early treatment” and item 16 “Regular surveillance after liver cancer treatment is still required”, it seem common sense and discrimination index low. We also hope that patients can understand that early detection and early treatment have a better prognosis. Most of the high-risk groups of liver cancer are related to the virus. Before the risk of the virus is not resolved, new cancers may occur even after treatment [6] [22], in order to remind patients to continue to surveillance after treatment. Item 13 “If liver cancer recurs after treatment, there is still a chance to treat” Tell patients to have treatment even cure change if they recur, don’t give up the methods and opportunities for treatment. And in the item 17 “It is necessary to monitor for possible infection after treatment” and it’s a discrimination index low item. We hope not related to panic in the treatment of patients due to the symptoms caused by the treatment. Because we hope to improve the teaching through examinations, we also have not deleted these item and keep to remain patients.
5. Limitation
Most Chinese traditionally live with family members. In this study, 92% lived with family members. Family members will accompany Participants when participants are sick or have a serious illness. The family member of the participants is afraid that the participants do not understand the doctor’s explanation or the family member is usually accompanied by the patient. There are 74.5% of primary caregivers are spouses or children. Therefore, family members cannot be excluded when collecting data in this study. This knowledge scale considers that patients should understand the content and use it as an evaluation after the intervention. So the ceiling effect is not considered.
6. Conclusions
In this study, male predominance to females was similar in gender distribution to global and Taiwan liver cancer patients. There is satisfactory internal consistency reliability 0.745 (>0.7), test-retest reliability 1.0 and construct validity were noted CVI 0.96. There was adequate overall evidence for convergent (p = 0.001) and discriminatory validity (p < 0.005). There are in item analysis has 5 questions are not satisfying. But although Taiwan has universal health insurance, some cases still require an application before the treatment and to be paid by the government. Most people think that cancer is a serious disease and should not work after getting sick. But because of the increased ability to treat and diagnose, most patients with liver cancer can return to work after the treatment. We think we still need to remind patients. Although few items, the discrimination and difficulty are low, we have not deleted these questions.
This knowledge scale can provide hepatocellular carcinoma before and after treatment evaluation whether patients have sufficient knowledge about liver cancer or provide interventions as a measure of knowledge improvement. Provide effective and easy tools for clinical caregivers to assess patients with liver cancer.
Financial Support and Sponsorship
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.