Author: Dr Raihanah Ibrahim, Solina Center for International Development and Research (SCIDaR)
Country focus: Nigeria
Theme: Routine immunization
In brief:
The SCIDaR team identified deep-seated problems in routine immunization program approaches, which led to overall low immunization coverage rates in Northern Nigeria. In order to address these problems, SCIDaR developed a Theory of Change that informs the interventions implemented in the program states, while continually re-examining and interrogating some of the assumptions contained within the program to ensure objectives are met.
Problem and context:
As part of the NNRISP program, SCIDaR delivers technical assistance to six primary state healthcare agencies in order to improve routine immunization systems and coverage. Specifically, they provide technical program management support across the 6 core PHC themes; innovative thought leadership using a data-driven problem-solving approach; and structured capacity transfer to government program managers.
At the start of the program, SCIDaR conducted a systems diagnostic of the routine immunization program in each of the states to understand the root causes of the observed low immunization coverage rates. The assessment revealed inadequate funding and oversight, weak community demand for immunization, and poor access to quality immunization services as the major factors impeding immunization coverage improvements.
In order to correct these deep-seated problems, they developed a Theory of Change to redesign the RI programs in each state, and have been implementing those interventions accordingly. Throughout the course of the program, SCIDaR has continually tested some of the assumptions built into the design of service-delivery in the states and made the necessary adjustments to improve the program accordingly.
Why was it important to interrogate the assumptions within their program?
- To test the validity of their activities and modify as appropriate. SCIDaR’s assumption that fixing vaccine supply chains would significantly improve immunization coverage was faulty; instead, they found that strengthening vaccine demand was the most effective lever, and should have commenced simultaneously with the supply-side interventions in the inception phase of the program.
- To appropriately contextualize interventions for greater impact. Regarding supportive supervision, the team found that the basic checklist-style design was not effective on its own; they had to focus supervisory activities on the needs of health-workers, which included capacity building and mentoring.
- To maximize return on investment by channeling funds to higher-impact interventions. SCIDaR have worked with state program managers to re-channel funds from generic social mobilization activities (such as jingles) to targeted community engagement strategies, by partnering with prominent community leaders and influencers.
- To facilitate multi-stakeholder engagements for greater cooperation and ownership. Deliberately seeking to uncover assumptions has led to greater ownership at all levels – managers, health workers and the community have had to be engaged throughout the process of uncovering assumptions and thus get to provide input on the program, too.
- To provide learning opportunities to contribute to the existing knowledge base. Strategically documenting and sharing knowledge products from the learnings at a state level has informed national strategy, and program operations in other countries.
What were the barriers they faced when trying to interrogate their assumptions?
- Lack of institutional structures that will necessitate demand for critical reflective feedback. For example, at the program inception, SPHCDAs did not have any structures, such as working groups, to serve as platforms for problem solving, feedback and iteration.
- Resource limitations: making huge transitions to newer methods/strategies, often translated to increased program cost and greater advocacy for funds.
- Poor political will to make changes to established processes in the midst of a complex stakeholder landscape. This required significant influencing and multi-engagements at the expense of time.
- Lack of robust data systems to provide required information. Quality and granularity of routine program data was often inadequate to make useful decisions – examples DHIS (poor quality), survey results (poor granularity, lesser frequency).
- Inflexible donor requirements and lack of open communication between stakeholders. Program Theories of Change are often locked in and often stay the same until the mid-term or end-of-program evaluations which happen further down the line.
How did the SCIDaR team overcome these barriers? Examples from vaccine supply chain
- “Targeted and Deep” diagnostic assessments building on existing data. They identified key hypotheses on the supply chain systems through reviews of existing documents and engaging stakeholders at every point in the chain, then drilled down to conduct deep diagnostics to understand the system bottlenecks.
- Developed a Theory of Change (ToC) through multi-stakeholder inputs. The report of the assessment was used to develop a TOC with input from government at all levels, donors, partners, and the private sector. This informed the solutions – provision of cold chain equipment to all wards and revamping the vaccine delivery system – successfully implemented to address the problem of frequent vaccine stockouts.
- Robust M&E frameworks and regular data use to inform decisions. After implementing the solutions, we rolled out real-time stock performance dashboards built on the new delivery architecture to provide visibility on vaccine stock. This data helps troubleshoot issues while continually optimizing stock.
- Actively seeking feedback through conventional and unconventional means. We spurred the state to continually engage the system users in order to get a sense of their experiences. E.g. Cascade deliveries (PUSH) from equipped to non-equipped facilities was inefficient; engaging the health workers in the facilities revealed that PULL was more natural, prompting the necessary modification.
- Regular engagement with managers and implementers to share lessons learnt and best practices. We convene biannual peer-learning forums (meeting and tours) with focus states on relevant topics; specifically using this platform to fine-tune and tailor direct vaccine delivery models – insourced and outsourced – which set the stage for the current mix of models obtainable in the states now
How did things improve as a result?
- Cold-chain infrastructural upgrades, with over 90% of all wards in the six states having functional cold chain equipment with comprehensive maintenance plans.
- Improved vaccine stock performance, from initial 20-40% stock adequacy to 90-99% across the 6 states.
- Improved efficiencies in vaccine deliveries – potent vaccines have been delivered monthly, and on time for over 600 delivery cycles.
- Availability of real-time quality stock data, which informs vaccine allocations and forecasts, and has been leveraged for COVID-19 commodity forecasting and distribution.
- Flagship PUSH-PLUS vaccine delivery system has been adopted as national strategy for vaccine distribution.
Lessons for others on using data for making better decisions:
- Be willing and open to your assumptions getting questioned. TOCs should be open to modifications during implementation; keeping things open helps to enable constructive feedback and improve the program.
- Create an enabling platform/culture for diverse stakeholders to share opinions regardless of hierarchy: SCIDaR holds internal weekly Problem Solving (PS) sessions where program problems are discussed openly by a mix of diverse voices; and mirror the same approach at the state level.
- Make the best use of data that is available. More rigorous and granular analysis of available routine data often generates more useful insights that help in decision making e.g. Analyzing routine LQAS data at a granular settlement level rather than the overall LGA level reports.
- Co-creation and iteration with the donor can help drive change. We hold monthly brainstorming sessions with the donor – involving the government actors when necessary – to discuss or debate relevant issues in a safe and open space, and make changes collaboratively.
- Access to high-level officials to share your implementation status and results secures political buy-in. We hold biannual VTCs with Principals, monthly STFI meetings with Deputy Governors, Quarterly meetings with Emirates Councils on Health, and day-to-day meetings with SPHCDA leadership to sustain political interest in the program.
Where can I find more information?
Raihanah Ibrahim
Raihanah.ibrahim@solinagroup.com
+234 (0) 8093677969
