Oversight Role of the WHO Independent Monitoring of the 2018-19 LLINs Replacement Campaign in Nigeria ()
1. Background
This WHO independent monitoring report is from the technical support and supervisory findings during the LLINs (LLINs) replacement campaign carried out in seven states (Niger, Kano, Kaduna, Taraba, Yobe, Jigawa, and Katsina) between June 2018-November 2019; these states were recipients of the Global Fund grant. There were nearly 30 million LLINs distributed by Catholic Relief Services (CRS) who are co-principal recipients of the GF grant through Society for Family Health (SFH) and Malaria Consortium (MC) who are sub-recipients of the grant. The campaigns were conducted in phases based on an agreed timeline. The concept of an Independent Monitoring Report (IMR) was an innovation from the Independent Monitoring Board report of the Polio Elimination Programme [1].
In 2008, and to scale-up malaria prevention in Nigeria, the National Malaria Elimination Programme (NMEP) and with support from key Roll Back Malaria (RBM) partners launched an ambitious initiative to scale up access to LLINs through statewide LLINs mass distribution campaigns. Similarly, in 2009, the country adopted the use of LLINs as a key malaria prevention strategy which resulted in a universal coverage campaign that took place across the country between 2009-2013 [2]. Based on the national report of the state campaigns conducted within that period, a total of 57,773,191 out of the planned 64,064,949 LLINs were distributed across the country, representing 90.2% of the total national target [3].
In line with the national guidelines, the NMEP strategy for achieving the universal LLIN coverage stands at one LLIN for every two persons and not more than 4 LLINs per household (HH); the aim for this is to ensure that all members of the population sleep under an LLIN regardless of age or sex [4]. The universal coverage, consistent with the global strategy for malaria eradication, aims to interrupt transmission by eliminating the human reservoir of the parasites [4] [5] [6] [7]. The World Health Organisation participated in some key activities such as planning meetings, state-level training of personnel, state-level campaign flag-off, and LLIN quality checks (pre-implementation dashboard, in-process monitoring, and LQAS).
1.1. General Objectives
To give technical assistance to the implementing states, provide oversight and quality assurance (QA) functions for the LLIN campaign implementing states, and produce an independent monitoring report to complement the roles of the national programme.
1.2. Specific Objectives
1) To provide oversight and QA functions for the LLIN campaign in Global Fund (GF) states to complement NMEP roles and facilitation of independent monitoring report.
2) To provide technical input in the compilation of QA reports through in-process and LQAS.
2. Technical Role by the World Health Organisation
The World Health Organisation (WHO) participated in the programme from the pre-implementation through to the post-implementation stages in the seven Global Fund supported states.
To ensure that the LLIN replacement campaign meets the minimum WHO standards, WHO for the first time in 2017 introduced the use of Quality Assurance (QA) tools in Adamawa, Edo, Imo, Kwara, Ondo states [8] [9]. The tools deployed during the campaigns were:
1) Pre-Implementation Dashboard (PI);
2) Independent Monitoring Tool (IM);
3) Lots Quality Assurance Survey Tool (LQAS).
The significance of this is to ensure that the campaigns meet the required quality as stipulated in the guidelines to enable Nigeria to achieve universal coverage. The outcome of this will provide lessons learnt and recommendations for use during future LLIN replacement campaigns.
2.1. The Pre-Implementation Dashboards
The WHO, using the Pre-Implementation dashboards, assessed the state’s level of preparedness at various stages of the campaign (3 weeks, one week, and three days to the implementation). The PI dashboard has seventeen variables as follows: state task force on implementation, LGA task force on implementation, schedule of activities, planning meeting, micro-planning, daily implementation work plan, plan for hard-to-reach, logistics available, state-level training of personnel, LGA level training of personnel, personnel availability, supervisory work plan, social mobilization committee (state and LGA), IEC materials, jingles and radio programmes, plans for flag-off, and the LLIN distribution work plan (Figures 1(a)-(c)). The PI dashboard helped to assess the status of preparedness of states towards the LLIN replacement campaign. The PI dashboard, which had a set of pre-determined questions, answered “Yes” or “No” as the responses, and are automatically coloured either “green” or “red” respectively. The final output from the 3-day pre-implementation status determines whether to “implement” or “defer” the campaign.
The assessment indicators in the dashboard are critical activities required for a quality campaign. The dashboard monitors and tracks the progress of these activities and flags-off any that is capable of undermining the campaign. Similarly, the various activities and or challenges identified from the PI dashboard during the build-up of the campaign were used for advocacy at a higher level across the states. The WHO technical officer overseeing the LLIN replacement campaign gives orientation to the state malaria program manager on how to activate the PI dashboards and the status shared by the program manager with WHO, NMEP,
CRS, and SFH/MC accordingly. Equally, using findings from the dashboards, WHO charts the way forward, developed specific, actionable plans with a timeframe, and feedbacks given to all levels on the results of the evaluation before next assessment is done. By the use of this tool as a guide, the progress and challenges of the pre-implementation activities were documented from one stage to the other until all the challenges were addressed three days before the actual implementations. The success of the dashboard lies on data quality generated by the stakeholders, the quality of supportive supervision training, the clarity of KPI measured, the policy drive, the efficiency of periodic evaluation reports, and the timely feedback and implementation of recommendations made after each review.
To address issues that arose from the dashboard, WHO provided the needed technical support and coordinated the provision of feedback to states and follow-up with the states to ensure recommendations are implemented at all levels; coordinated and led advocacy and needs assessment visits to states prior to training; and coordinated presentation of periodic updates on the outcome to national, state, LGAs and IVM working groups at all levels. The dashboards from states that implemented the campaign between 2018-19 shows that states level of preparedness (average) stood 76% at three weeks to the implementation, 89% at one week, and 100% at three days (Figure 2).
In summary, the PI dashboard aided in the successful monitoring of the 17 pre-implementation variables and assisted in measuring the progress toward improving the LLINs campaign since it has shown to be very helpful as a comprehensive tool for measuring multiple dimensions of the system at the various levels of the implementation.
![]()
Figure 2. Showing the pre-implementation dashboard indicators of states at 3 weeks, 1 week, and 3 days. Source: 2018-19 LLIN replacement campaign WHO quality assurance data.
2.2. The Independent Monitoring Tool
The Independent monitoring quality assurance (IM-QA) tool, using the ODK, was administered by WHO trained independent monitors (IM) throughout the five days distribution period. Depending on the demography of the state, WHO select between 4 - 14 LGAs and posts one independent monitor per local government area (LGA). The selection criteria for the LGAs for in-process monitoring were using the EPI high-risk analysis (HRA) tool. Each IM in an assigned LGA visited ten wards (2 wards/day) and 20 distribution points (DPs) at 2 DPs/ward/day and applied the IM-QA App throughout the five days distribution period and under the guidance of assigned local guides. The selection of the wards and DPs were made daily using a Tableof Random Numbers and by a data clerk.
As shown in Figure 3, all the states had an average of 30 per cent coverage on D1, except for Niger state, which had slightly lower coverage of 24 per cent. Similarly, the coverage on D2 and D3 were somewhat lower in Taraba and Niger states with each having 54 per cent on D2 and 78 per cent on D3 respectively. The coverage on D4 was at least 90 per cent across all the seven states, and in D5 all the seven states had at least 95 per cent.
The independent monitors, using the IM-QA App, evaluated four critical areas during the five days distribution phase of the campaign (strategy, logistics, technical, and demand creation); see Figure 4(a), Figure 4(b).
As shown in Figure 4(a) (logistics component), except for Taraba (87.2%) and Kaduna (83.7%), all the states had at least 95 per cent adequate LLIN storage sites. While personnel at the DPs visited checked each Bale to make sure that the nets are complete before commencing the distribution in six of the seven states, only 79.5 per cent of the DPs in Jigawa state adhered to the guidelines. In spite having the least percentage of Bales checked, Jigawa recorded the highest number of shortfalls of the nets (1.94%) in the Bales verified. Also, Niger state had
![]()
Figure 3. Showing daily coverage of the distribution points across the states. Source: 2018-19 LLIN replacement campaign WHO quality assurance data.
the weakest plans to move LLINs from one DP to the other in case of needs, recording 87 per cent as compared to the other states that had at least 92 per cent.
In Figure 4(b) (strategy component), except for Kaduna (55.71%), Kano (79.6%) and Niger (46.4%), the DPs visited in the other four states had daily work plan (~90%). While Jigawa (68.4%) and Taraba (77.2%) had inadequate crowd control measures at the DPs, the other five states achieved between 91 - 100 per cent. However, all the DPs across all the seven states issued out more than 4 LLINs to HHs, with Kaduna having the highest (80%).
In Figure 4(c) (technical), the chart shows that all the states had adequate data tools at the DPs (98.3% - 100%). Similarly, the responses showed that the DP personnel correctly used the tools in all the states with Taraba being least (91.6%). All the states, except Yobe, used the CAT electronic platform throughout the LLIN campaign process. Data shows that functionality of the CAT device in all the states was perfect; this is even though the Jigawa (94.3%) and Niger (95.8%) had weak mobile network issues which delayed data synchronisation in some DPs. Furthermore, each of the states had in place (>94%) a robust medium of communications between the DP supervisors and the senior supervisors for guidance in case of challenges.
In Figure 4(d) (demand creation), data revealed that Taraba had the least number of Health Educators stationed at the DPs visited (94.4%) and fewer Health Educators in Taraba state (79.2%) effectively addressing safety concerns of the HHs on the LLINs. Similarly, some DPs in Jigawa (73.5%), Taraba (75.6%), Kano (77.5%), and Kaduna (83%) states did not visibly hang the LLINs at the DPs as required by the guidelines.
2.3. LQAS (LQAS) Tool
The LQAS, using ODK App, aims to supplement the independent monitoring data shared by the state team, especially where there are significant quality concerns or where there is contradictory data. The LQAS technique was used to determine whether a group of individuals have achieved the required standard of performance by looking at a sample of that group. The LQAS tool provided useful information at a granular community level where attention needs to be focused and also in identifying areas where the plan is not working well. Using the ODK App, the LQAS tool was administered to eligible households, i.e. caregivers who had at least a child under the age of five years, by the WHO trained field officers at least one week after the campaign.
In the seven states that had the LLIN replacement campaign, the WHO trained and deployed six LQAS officers to six LGAs (2 LGAs/senatorial zone). Using the cumulative state-wide in-process monitoring data shared at the daily ERMs, high (H) and low (L) performing LGAs from the three senatorial zones were selected. The selection of the H and L was made after ranking the LGAs in each senatorial zone using the state-wide coverage data. However, where there is a tie in the performance, then a random selection process is adopted. Using a Tableof Random numbers, two wards were selected, and from each ward, two settlements selected totaling 4 settlements per LGA. With the aid of Tableof Random numbers, each of the LQAS officers sampled 10 households (HHs) per settlements thus completing 40 HHs per LGA. Therefore, in each of the implementing states, a total of 240 HH were selected from 24 settlements in 12 wards of the six LGAs. So, in the seven states that implemented the campaign, a total of 18,260 eligible persons were sampled from 1680 HHs.
The redemption rates, retention rates and hanging rates from the seven states are as displayed in Figure 5 and Figure 6.
As shown in Figure 5, the LQAS redemption rates in each of the states when compared with the campaign state-wide cumulative coverage are almost same; looking at the statistical level of significance of the LQAS result, it shows a very high degree of confidence of the LQAS findings.
![]()
Figure 5. Comparing LLIN redemption rates (LQAS versus campaign). Source: 2018-19 LLIN replacement campaign WHO quality assurance data.
![]()
Figure 6. Showing LLIN retention and hanging rates. Source: 2018-19 LLIN replacement campaign WHO quality assurance data.
In Figure 6, all the states had Retention rate above 80 per cent and hanging rate above 70 per cent.
3. Discussion
3.1. Lessons Learnt
1) The high-level advocacy by the campaign team has improved the political will/ownership and participation of key government officials; this is evident by the dignitaries who attended the flag-off ceremonies (Governors, Deputy Governors, Secretary to the State Governments, and Emirs). It was commendable as it demonstrated ownership/good partnership, political will and leadership commitment. It shows the level of acceptance of the interventions by the leadership of the states, motivated the citizenry for optimal use and has helped in programme visibility.
2) The use of Information and Communications Technology for Development (ICT4D) platform by the implementers has eased up the processes and helped in the effective monitoring of the campaign activities.
3) The investiture of net Ambassador to the wives of the Governors and wives of the LGA Chairmen in separate events and or during the flag-off ceremonies was commendable; this has improved uptake and prompted women to come out in large numbers and redeemed their cards.
4) The timely delivery of the LLINs to the implementing states was commendable as it built trust between the implementers and the state government.
5) The adequate support from the local government chairmen, district heads, religious bodies, CSOs, and community leaders has helped in building trust among the populace and enhanced the LLIN utilisation.
6) The display of firm commitments from the various sectors (CSOs and other NGOs) has given a positive outlook to the campaign. It has also helped in changing the behaviours of the populace toward the intervention.
7) The use of civil defence (and vigilante in some security compromised areas) across the DPs not only maintained orderliness but also gave assurances to the HHs about their safety.
8) The use of town criers has helped in mobilising the HHs in numbers to redeem their net cards.
3.2. Recommendations/Way Forward
1) The NMEP should adopt EPI target populations, which gives a true reflection of the population data, to aid in LLIN quantification and microplanning processes.
2) The NMEP should consider using the LLIN campaign Pre-implementation Dashboard, to help assess the state’s level of preparedness towards rolling out successful campaigns. Feedbacks could be transmitted to a “situation room” that will be jointly coordinated/managed by representatives from the RBM partners. Where issues and gaps are picked, then a decision could be reached as to how to jointly address them to achieve the highest quality campaign.
3) To achieve the LLIN Universal Coverage, national and state programmes should work out modalities to reach out the missed households/communities, especially those in hard-to-reach (HTR) areas which often are the most vulnerable ones.
4) The state programmes should maintain efficient continuous LLIN distribution channels and achieve good coverage through the “keep-up” phase.
5) The state programmes should intensify on the post-campaign health education component of the demand creation to improve on LLIN uptake/use/utilisation.
6) The NMEP should ensure that future campaigns adhere to the selection and training of personnel from the locality and with the help/input of the DHs. The selection of personnel from the catchment area helps in building trust with the community involvement/promoting ownership, avoid poor coverage/ensures quality coverage.
7) The NMEP should use advocacy visits to the leadership of the SMOH and emphasise that trained selected personnel should not be replaced with new ones as this affects the quality of the exercise.
8) The NMEP should prioritise on quality supervision as often the DPs are left without supervision. Similarly, and for accountability, there may be a need to put in place a mechanism, e.g. tracker, that monitors the supervisors at the field.
9) To avoid missed opportunities, NMEP should ensure plans are in place for lateral logistics between wards/DPs (and with correct tracking documentation on any decision taken for accountability).
10) The NMEP should ensure the availability of adequate LLINs storage sites at the LGAs to avoid spoilage, pilferages, and shortages. Similarly, the LLINs should be stored within a reachable distance from the DPs to avoid putting undue stress to the DP personnel as often they complain of using their money to transport the Bales to the DPs.
11) The NMEP should ensure availability of robust logistics plans of action in the LGAs in line with the developed micro plans and to encourage adherence to the plans.
12) The NMEP to make an ideal/realistic plan on how to reach the HTR/security compromised LGAs to ensure fair and unbiased coverage.
13) The NMEP to improve on-demand creation through community/dialogue, sensitisation, awareness, and mobilisation via media outlets, faith-based centres, CBOs, Emirs, District Heads (DH), etc.
14) The NMEP to ensure that training of personnel is interactive and through a participatory approach.
15) The NMEP should have in place a daily work plan for each of the DPs. The daily work plan should spread the 5-day coverage target such that D1 will target 30%, D2 25%, D3 20%, D4 15%, and D5 10%. The aim of this is to achieve quality coverage and avoid undue stress/fatigue for the DP personnel.
16) The achieve quality LLIN implementation, NMEP should avoid overlap of the implementation phase of the LLIN campaign with the state immunization activities (SIAs) campaigns because often it is the same persons that the two programmes are using to deliver their services. Conversely, the NMEP should consider conducting the microplanning phase of the LLINs with SIAs such that the net cards are issued to the HHs after vaccinating their eligible children.
17) As collecting the nets often do not translate to use because of safety concerns/misconceptions/religious beliefs, the NMEP should consider phased media programmes targeting BCC addressing safety concerns to improve the uptake.
18) The NMEP to encourage DP supervisors to give morning briefs to their teams through any convenient communications channel before the commencement of the distribution.
19) The NMEP to ensure the provision of mobility to ease up repositioning/address stock-outs at the DPs.
4. Conclusions
The monitoring data (In-process and LQAS) from the seven states that had the LLIN replacement campaign in 2018-19 show coverage of at least 95 per cent in each of the states. The results, when compared with that from the campaigns, show close similarities with a 95 per cent Confidence Interval. However, LQAS data from Kano and Niger states were disaggregated to look at HHs with LLINs and HHs without LLINs; the picture shows that there are LGAs with up to 50% HHs without LLIN and up to 95% HHs with LLINs.
In the seven states, while media outlets (radio/TV) were the primary source of information about the LLIN campaign in the metropolitan LGAs, HHs in the farthest LGAs heard about the net campaign from town announcers. Similarly, at least 80 per cent of the HHs who received the LLINs responded that they would use ordinary soap to wash the nets when dirty and would also use needle and thread to repair when torn.
However, three crucial gaps needed to be addressed and are critical to achieving the universal coverage; these are poor micro-planning, poor mobilization, and the capping of 4 LLINs per HH. These gaps could be addressed as follows: 1) The microplanning should involve the district heads and with guidance from the community leaders; and given that the LLIN campaign is conducted every biennium, the “work-through” approach should be adopted as what is being done during Polio campaigns, to make sure that each household and the number of people living inside are clearly identified and recorded. 2) In addition to using the audio-visual medium, the community needs to be mobilized using the town announcers, which is the most practicable, readily available and easily accessible medium by the people. 3) To eliminate giving only 4 LLINs per household and to ensure that universal LLIN use is achieved, the national guideline for the LLIN campaign should be explicit that every two persons in a household are eligible for a net.
Limitations
The independent monitoring conducted during the LLIN distribution campaign was limited to only 4 LGAs in some states as against 14 LGAs in other states; this was because of the size of the states. However, this did not affect the outcome of the study in any way.
Data Availability
All data have been summarized and presented in the manuscript. The data cannot be deposited to any external agency because of policy and other restrictions.
Ethical Issues
The independent monitoring for the LLIN campaign was an oversight activity carried out by the World Health Organization as part of the technical assistance given to the national and state malaria elimination programmes in Nigeria.
The data collected for this study did not capture any identifying personal information of the respondents. The reason for this was to eliminate any threat to unauthorized exposure of such information.