Effectiveness of Parent Education through Mobile Technology in Afghanistan ()
1. Introduction
Child development is one of the most important issues across the globe ( Arnold, 2004 ). In particular, it is a subject of great discussion for the United Nations and the international community in support of the developing world ( Ki-Moon 2007 ). The international community has created many programs and initiatives to support selected age groups of children in developing countries; in particular, a great deal of attention has been paid to school age children ( Qayumi, 2014 ). However, initiatives for the education of preschool age children have been very limited, and are mainly aimed towards nutrition ( UNESCO, 2007 ), health and prevention of disease ( UNESCO, 2007 ). It is evident from the literature that little attention has been paid to multi-dimensional initiatives that cover the social, emotional, physical, cognitive, and linguistic development of children. The previous study ( Qayumi, 2014 ) also confirmed that Afghan parents are open to receive such education through innovative technologies. The use of technology for this type of education is particularly of value to those who have literacy barriers. Therefore, our objectives were threefold:
1) To identify an innovative technology capable of crossing literacy barriers and delivering the knowledge to all those in need.
2) To develop a curriculum for multi-dimensional child development (ages 0 - 7) using modern developmental theories.
3) To assess the effectiveness of the developed curriculum delivered by innovative technology on the improvement of knowledge for Afghan parents.
Technological Solution
To realize our first objective we implemented LIVES technology. LIVES is an acronym for Learning through Interactive Voice Educational Systems. This technology was developed at the University of British Columbia’s department of Computer Sciences and Technology, and is licensed to Partnership Afghanistan-Canada (PAC) free of charge to conduct not-for-profit work, including this study.
2. Introduction to LIVES
Technological advances such as M-learning (education via mobile phones) have provided new opportunities for the delivery of informal as well as formal education, with all the key characteristics of a curriculum still intact. LIVES is an end-to-end education solution that delivers distance learning via mobile phones to subscribers anytime/anywhere. Mobile phones are an ideal medium to deliver education due to their widespread use across rural and urban centers. LIVES enables not only the automation of distance learning materials, but also acts as a catalyst for socio-economic change across communities. LIVES has been developed under the guidance of Dr. Son Vuong and other researchers at the Network and Internet Computing Lab (NIC), and is currently being commercialized by a UBC spin-off company named LIVES Mobile Corp.
LIVES takes advantage of the proliferation of mobile devices found even in impoverished locations. The system is based on voice interaction with students via phone calls. During these calls, the student receives one to two minute audio lessons, then answering related multiple-choice questions. The call is originated by the system, not only as a way to entice the student to receive the lesson, but also because in most countries cell phone users do not pay for incoming calls. This means that aside from acquiring a basic cell phone, the students will not incur any additional costs to access the system.
LIVES is a set of optimized computers equipped with specially designed open-source software, with the ability to deliver learning content over the phone. The first step is to register the student, including his/her name and associated telephone number (the calling channel). The student registration process also takes note of the time periods during which the student will be available, with the system scheduling calls to each student only during the specified availability period, or when a lesson or question becomes available. A web interface is being planned to simplify this process in the future.
Due to the nature of this VoIP system, there is no restriction as to what kind of phone can be used by the student. The system has been tested not only with cell phones, but also with regular landlines and even VoIP phones and VoIP services such as Google Talk. However, additional features planned for the future, such as text or multimedia messages, will require a cellular phone. Instructors create lessons and questions and provide audio files, then submitting them to the communication server. These files have to be converted to GSM format, which is a commonly used format for telephony. The system also provides an automated recording server, which can be accessed by calling a specific phone number (currently accessible only through a VoIP connection). This server will record the messages and convert them to the proper format.
The system will call each student at the pre-scheduled time. If the call is unsuccessful (the line is busy, the student doesn’t answer, or the number is unavailable), another call is scheduled for the future. If the call is successful, the student can listen to available unheard lessons as well as to lessons that were previously heard. While a lesson is being played, the student can rewind, forward or pause. The student can also listen to any unanswered questions, both in new lessons and those previously heard, which they may have skipped. Questions are in multiple-choice format, and the correct answer is inputted by the student using their phone’s numeric keys. Optionally, an instructor is able to provide an audio explanation to the student regarding why a specific answer is the correct one.
The system can call several students simultaneously, contingent on hardware and network constraints. Each student has an individual session, and one student’s answers do not affect other students, even if they are simultaneously in a call. The system is also capable of handling each call in a different language, based on the student’s predefined language preference, as long as all system audio files and lessons are available in that language.
The LIVES database system stores statistical data and allows for information retrieval specific to each student, and questions and lessons can be designed based on that student’s feedback. The system will keep all data concerning the number of times a student tried to answer a question before submitting the correct answer, as well as timestamps for the completion time of each lesson and question. A simple statistical web interface is provided, with basic information like the percentage of complete lessons and correctly answered questions for each student, and summarized results for each question.
Thus far, LIVES has been field tested in India, Vietnam and tests are underway in several other countries. LIVES will be a fantastic catalyst for social-economic change in Afghanistan, and will deliver on multiple levels including child education, health, agriculture, and English as a Second Language. This technology is licensed to PAC for charitable delivery of knowledge without financial incentives.
3. The Parenting Curriculum
The curriculum for this project was designed by (senior author) Shahnaz Qayumi, an early childhood pedagogue and psychologist, specializing in the field of child development for children aged 0 - 7. Curriculum was translated into local languages (such as Dari) used by participants in this region. For the purpose of this pilot study, Afghan families in and around the city of Kabul were randomly identified and only those who volunteered were included in the study. The only inclusion criterion for the project was for the families to have at least one child between the ages of 0 - 7. Differences based on gender, language, age and education were not taken into consideration in this study.
The study was conducted Monday-Friday for 40 days (July 18-Sept 12) under the direct supervision of lead investigator Shahnaz Qayumi, with two research assistants (Shabnam Hazrati and Sahar Habib) as well as five trained instructors and several assistants to work in the field. On completion of this study, evaluations of participants and instructors were performed to identify issues and shortfalls of the study. Data was collected, stored and analyzed using a statistical package. A simple test was used to deduct the degree of significance, with a 95% level of confidence (p < 005).
3.1. Goals
The purpose of this study is to evaluate the effect of the M-Learning Project on participants’ knowledge of their children. We also sought to evaluate participants’ (both teachers and learners) satisfaction with the program and elicit their suggestions for changes to the program (in terms of process or content.) The following hypothesis and research questions guided the inquiry:
· We hypothesize that modern theories of child development delivered through innovative technology can be effective in increasing parental knowledge, regardless of literacy levels.
· We hypothesized that participants’ overall knowledge about their children’s development would increase from pre- to post-test.
· How consistent is the overall change in knowledge across the domains of physical development, cognitive development, language development, social/emotional development and knowledge about general principles of development?
· How satisfied were participants with the process?
· What were the identified strengths and opportunities for improvement of the program, as identified by teachers and participants?
3.2. Methods & Design
To evaluate the effect of the program, we looked at how much participants’ knowledge about child development changed by comparing their answers on a test administered at the beginning of the program to their answers on a similar test at the end of the program. In addition, we asked for feedback from both participants and teachers who assisted in the field regarding their satisfaction with the process and possible areas of improvement. Feedback was gathered after completion of the post-test.
3.3 Participants
Representatives from 54 households participated in the study. These were volunteers from a previous group of 224 households who participated in the needs assessment for the program and expressed interest in learning more about child development and childrearing. The only inclusion criterion was that there be at least one child in the home in the age range from newborn to seven-years old. Of the 54 volunteers, 49 completed the pre-test and 52 completed the post-test.
3.4. Instruments
The knowledge tests consist of items randomly drawn from a bank of items prepared on the basis of the lessons. The pre-test, made longer in order to increase precision, included 40 items. The items represented 5 curricular areas: physical development (8 items), cognition (13 items), language (6 items), social/emotional (10 items), and general principles of development and parenting (3 items). The post-test included 25 items pertaining to: physical development (5 items), cognition (8 items), language (3 items), social/emotional development (4 items) and knowledge of general principles of development and parenting (5 items). Each item consisted of a statement about child development and/or parenting. Participants indicated their agreement with the statement, disagreement, or chose “do not know”. At scoring, which yielded a percent-correct score, a correct response was assigned a score of 1 and incorrect responses and “do not know” responses were scored as 0.
Evaluation of participants’ satisfaction consisted of open-ended questions on strengths of the program and 18 ratings covering the following areas:
· Self-assessment of improvement
· Logistics of the program (communications & technology)
· Content
o Balance of theoretical vs. applied content, real world application of content to local participants, relevant to expectations.
o Organization, workload, difficulty of content and quality of presentation.
· Instructors and support staff.
· Willingness to participate in similar programs in the future.
The evaluation by instructors consisted of six open-ended questions on the following topics
· Instructors perceptions of learner satisfaction;
· Acceptance of technology;
· Quality of the program;
· Programs relevance to local cultural context;
· Recommendations for improvement;
· Suggestions for future applications of the technology.
4. Procedure
First, instructors conducted a pre-test on the childrearing knowledge of all participants in the study. Registered participants received calls from the server at the designated time of their choosing. The curriculum developed in LIVES was then delivered to registrants via cell phones. Participants could listen to specific parts of the curriculum (modules) as many times as they wished until they were able to grasp the content of the module. Each module included a multiple-choice test to be answered after participants were confident on their acquisition of knowledge. Multiple-choice questions were answered by pressing numerical buttons on phone keypads. Some students did skip the exam in some modules, however, at the end of the course the system identified the specific modules each participant skipped and they were contacted to complete the exam. In the end we were able to complete the study with all the participants.
After completion of the course, a post-test on the knowledge acquired was conducted for all participants. All pre- and post-test questions were logged in a database from which the computer chose the questions randomly. Seven instructors were on call 20 hours per day to direct participants and help solve any accessibility, technological or logistical issues. These instructors also conducted the pre- and post-tests, however they were not allowed to assist participants with their core knowledge.
4.1. Analyses
Common indices of item-quality (difficulty, discrimination, response-options distributions) were used as quality- control for this measure. Reliability of the tests was evaluated by means of Cronbach’s alpha (estimate of reliability through internal consistency).
Related sample tests (t-tests) were used to compare the aggregated scores for each domain and the overall scores of the pre- and post-tests. Independent sample t-tests compared the average number of “do not know” responses.
Qualitative comments by participants and teachers were summarized by topic and included in an overall description of the evaluation of the different aspects of the project.
Percentages of participants selecting respective response options were used to analyze the evaluation ratings from instructors and participants.
4.2. Results
Item-analyses helped identify 2 items (one at pre-test and one at post-test) which were unclearly formulated. These items were subsequently dropped from calculation of the overall test scores and domain scores, thus reducing the pre-test to 39 items (social/emotional domain—to 9) and post-test to 24 items (cognitive domain—to 7). Another 3 items (post-test only) had a wrong answer key and were re-scored. Reliability was estimated at 0.77 at the pre-test (39 items), 0.43 at post-test (the relatively low value is probably due to the small number of items, 24), and 0.72 for the combined item-set (63 items).
Table 1 lists the descriptive statistics and the results from the repeated measures (t-tests) comparing the change from pre-test to post-test for the overall scores and for each domain.
The average number of “do not know” responses at the pre-test was 6.38 (SD = 4.32) per item for 49 participants. At post-test, this number was reduced to 0.56 (SD = 0.87). Comparison of these numbers by means of t-test for independent samples, indicated that their difference is significant, p < 0.01.
![]()
Table 1. Descriptive statistics and t-test results for the pre- and post-tests by domain comparisons.
Note: N = 49.
4.3 Evaluations by Participants
All participants (100%) agreed or strongly agreed that the M-Learning program improved their knowledge base and parenting skills. Overall, satisfaction of logistics of the program were high (ranging from 87% to 94% in the categories of above average and excellent). Approximately 1/3 of participants indicated however experiencing difficulties specifically with their telephone services. Over 90% of participants agreed or strongly agreed that the content was excellent, relevant to their expectations and applicable to their real world experiences. Workload, pacing and quality of the presentation were judged to be at the appropriate level by over 80% of participants. 100% of participants rated their instructors and support staff as Excellent. All participants (100%) expressed that their willingness to participate in similar programs in the future.
4.4. Evaluations by Instructors
There was unanimous consensus among instructors that the project was a great success. It was well-accepted by all participants, with specific comments on the content of the learning materials that directly related to the levels of knowledge of participants. They indicated that this project is a technologically advanced and unique project targeting illiterate and semi-literate populations of Afghanistan, with a very low cost and a very high impact of efficacy.
For example, “… they were suspicious when we approached them to do the program, but the most interesting part of our work was when we were doing the evaluation of the program. Our team had to go again to visit the families, and get their feedback. Parents welcomed us so warmly, and happily that we were so much touched with their affectionate behavior…” “… the most difficult part of our work during the survey was when some men in the families faced us. They were suspicious, and weren’t ready to provide their wives phone numbers, giving us theirs own cell number. When the program started, the man who had a hard time giving his wife’s phone number, he voluntarily called us and gave us his wife’s phone number, saying ‘this is a very suitable program for my wife’…” (quotations taken directly from instructor evaluation forms).
We also learned that after 2 - 3 lessons, the learning sessions turned into an event within the family, placing the telephone in the middle of the room on speaker so the whole group could listen and participate. At times they even invited their neighbors and friends in to join them, discussing the issues in the lessons over a cup of tea afterwards.
With respect to the shortfalls of the project, both instructors and learners mentioned the delays in the start-up phase of the project, which triggered some anxiety before beginning the program. According to one instructor, one of the learners called and said that she dreamed about the program the night before. In addition to the delayed start, instructors also indicated that often some of the parents (participants) could not achieve the proper quality of sound. Instructors also recommended that the scheduled time of delivery did not reflect the daily routines of Afghan people. Some participants may not have strictly scheduled days, therefore for some participants it makes more sense to not have a scheduled time but to leave it up to the learner to choose. In other words, this must be a student-centered program.
5. Summary of Effectiveness of Results
In summary, the pre- and post-tests demonstrated that participants significantly improved their knowledge on the physical, cognitive, social/emotional development and related child-rearing practices. The evaluations of participants and instructors demonstrated that the program and the technology are well-accepted by all participants in this sample. Our results demonstrated that LIVES, as an innovative technology, is capable of crossing literacy barriers and delivering knowledge to all those in need. In addition, these results also demonstrated that Afghan people generally have a desire for knowledge acquisition and structured curricula in a highly sensitive area of knowledge and the use of modern theories of child rearing is well accepted by all Afghans in this study.
With respect to the recommendations received by instructors, indicated shortfalls were mainly due to the fact that the servers broadcasting the learning materials was located in the United States and the program was delivered long-distance. Other reasons for the shortfalls were the distribution of the program through a variety of cell phone companies in Afghanistan, all with different technologies and area coverage. These shortfalls will be seriously considered, and we believe they can be eliminated when the program is fully launched in Afghanistan in partnership with a local cell phone company, where the servers will reside locally. The recommendation and suggestions on the time of delivery were also valuable to us and will be considered when the project is fully launched in Afghanistan.
Overall Discussion
Technological advances in M-Learning (learning through mobile phones) have made it possible for knowledge dissemination to those who are disadvantaged by distance from cities, illiteracy and other factors. Several M- learning technologies have been developed at this time, however LIVES technology has many advantages, such as its simplicity for users. The technology is licensed to PAC (Partnership Afghanistan Canada―a Canadian not- for-profit organization working to improve global education) without charge, and overall costs of the technology are very low while capable of delivering complex theoretical subjects through a simple voice activated system. Perhaps the greatest superiority of this technology over others in the field is that it not only encompasses curriculum delivery but also curriculum development and curriculum assessment tools, which are built right into the LIVES system. LIVES has been used in a pilot study project, designed and implemented by the Commonwealth of Learning, for improving the livelihood and status of women in India by forming business opportunities through raising goats. The main and most important value and benefit of LIVES is that members who use the system for non-profits can share curriculum developed in the system amongst various partner institutions. We are very proud to say that this child development curriculum is now being translated into Arabic and Vietnamese to be disseminated in countries in the Middle East as well as Vietnam. It is also important to mention that although asynchronous methods of learning, such as M-Learning, are front-loaded and can be expensive to start; they are most appropriate for mass education. It has been proven that the cost of technology in this method of delivery becomes negligible when used for mass education (8). Therefore, it can be argued that synchronous methods of delivery such as seminars, lectures, brochures/pamphlets and others used by many organizations would be less effective and more costly.
Overall, the semi-quantitative evaluation on the satisfaction of participants was extremely positive from all 54 families evaluated, with only one family not finding the curriculum useful. Results of the efficacy of this curriculum delivery, after the comparison of pre- and post-tests, demonstrated that parents significantly (p < 0.05) improved their general knowledge on childrearing. In particular, they increased their body of knowledge significantly (p < 0.05) in the areas of the physical, emotional, social and cognitive development of their children. The only domain that did not show a significant difference between the pre- and post-tests was in language development. It would seem that in this domain Afghans pay more attention to the development of their children’s language, therefore the new information received through the M-Learning curriculum did not have a significant impact. It may also be that in this pilot project the number of participants was not sufficient to determine trivial changes. We are planning a larger study in the near future, which may or may not change the results of the study in the language domain.
The evaluations conducted in this study show that participants and instructors are extremely satisfied with the technology, as well as with the levels of knowledge being disseminated to participants. In particular, the curriculum developed by our specialists has been very well received by parents in Afghanistan.
Feedback from our field instructors indicated that at the beginning of the project many families expressed concern that the information should only be distributed to the female members of the family. In Afghanistan, this finding did not surprise us, since after all Afghanistan is a very traditional country with very specific religious beliefs, wherein the male is in charge of the upbringing of children. We therefore changed our strategy to provide the information to both parents. With this change in strategy we did not encounter any restrictions on conducting our study and collecting data. The other important observation made by our field instructors was that when male members of the family listened to the curriculum, they requested the presence of their spouses, and other members of the family to listen to the material. Moreover, it was observed that in some cases they invited their neighbors, friends and relatives who had young children to also participate. Many families invited others and placed the telephone in the middle of the room before turning on the M-Learning lessons, transforming an individual learning session into a group learning session. They also indicated that sometimes this event would turn into a voluntary social event, wherein participants discussed the content of the curriculum over a cup of tea long after the lessons were completed.
In summary, analyzing the results of this pilot project clearly demonstrated that LIVES is a suitable innovative technology for the development and delivery of curriculum based knowledge, capable of crossing literacy barriers and therefore particularly useful for those individuals unable to read or write. In this study we also aimed to design a special curriculum for multi-dimensional child development aged 0 - 7 using modern developmental theories, carried out by a voice activated system and to be understood by people with no formal education. Success on the accomplishment of this objective was evident when we completed the third objective of the study on the effectiveness of this curriculum. Results of this study also demonstrated that the developed curriculum significantly improved the knowledge of the participants on modern theories of childrearing for Afghan parents regardless of their level of education. Furthermore, this study demonstrated that with the use of appropriate technology and culturally sound approach it is possible to empower females in a male dominated society, placing the mother in the center of the family and making them fully in charge of the upbringing of their children. Moreover it could be argued that curriculum content on subjects such as health, business, agriculture, law, ethics, governance and others could be developed and delivered through LIVES, particularly for those who are challenged by illiteracy. Societies in developing countries may greatly benefit from this technology and curricula. In particular, countries that are disadvantaged by illiteracy and gender issues in the structure of the family, as well as in society at large, will benefit the most.
6. Future Directions and Recommendations
We plan to conduct a larger study across Afghanistan, as well as perform follow up studies on knowledge retention each year for those individuals who have already participated. Moreover, it is our intention to conduct a separate study to assess the children of those families who have participated, evaluating the behavior of these children and their success in school as an indicator of the efficacy of this knowledge on the family, social environment and Afghan society at large. Our long-term goal is to make this validated tool available to all families and parents in Afghanistan, as well as internationally where the need is evident, particularly in developing countries.
We do not aim to disseminate this knowledge worldwide on our own; therefore, we are seeking partners to support this idea in other developing countries. We have already signed agreements with two partners (in the Middle East and Vietnam) to translate the curriculum, give them the technology free of charge, and support the dissemination of knowledge and technology in those countries. This technology can also be used in partnership with other organizations working in sectors such as business, health, governance, law and others. We would be pleased to partner with such organizations and help them develop and implement their knowledge in any sector of livelihood through LIVES.