Differentiated Monitoring
and Evaluation
At the first CQUIN meeting, member countries identified differentiated monitoring and evaluation (M&E) as an urgent, cross-cutting challenge.
Learn more about the Differentiated Monitoring and Evaluation Community of Practice
Differentiated service delivery (DSD) creates potential challenges for routine M&E, as the information needed to track implementation of differentiated ART (DART) models is often not captured by routine M&E systems. In addition, as many DART services are delivered outside of health facilities, HIV programs must implement appropriate tools and strategies to ensure swift and accurate flow of data from the community to health facilities and beyond. As CQUIN countries launch and diversify DART services and take them to scale, M&E systems must evolve and adapt.
In response, CQUIN is:
- Supporting member countries to use the CQUIN dashboard as a self-assessment tool
- Creating a portfolio of frameworks and resources
- Facilitating a multi-country community of practice focused on ongoing experience-sharing and resource co-creation
- Providing targeted technical assistance to member countries
- Supporting ad hoc data collection and DSD performance reviews
From 2020 onwards, CQUIN has provided technical assistance to countries with electronic medical records to enhance DSD-specific data queries and data visualization.
Frameworks and Tools
M&E Framework for Monitoring of Differentiated ART Services
(English PDF French PDF)
Developed by the CQUIN M&E community of practice, this framework prioritizes M&E measures to track the scale-up, coverage, and performance of DART services. It is intended to serve as an adaptable tool for ministries of health and program implementers leading DART scale-up and/or supervision.
DSD Performance Review Toolkit
(English Toolkit [PDF]) (English Appendices ([Zip File]) (French Toolkit)
In response to the need for a standardized approach to obtaining DSD data and conducting performance reviews in the absence of routine reporting on DSD indicators, the CQUIN M&E community of practice has developed a toolkit that provides step-by-step guidance on planning for the review, conducting data collection and analysis, holding a dissemination meeting, and developing action plans. The adaptable toolkit is based on lessons learned and best practices from performance reviews conducted by CQUIN network countries in 2019.
Meaningful Community Engagement Framework
Developed by the CQUIN Community Engagement community of practice, this framework was designed to move toward a definition of meaningful community engagement in policy, implementation, and evaluation at national, program, and community levels.
CQUIN Capability Maturity Models
The CQUIN Capability Maturity Model is a structured self-assessment tool that systematically describes the progress of national initiatives to scale up DSD using a capability maturity model approach. The tool can be used by policymakers to identify implementation gaps and facilitate decision-making.
CQUIN supports baseline and annual DSD capability model assessments in each of its network countries to track progress and identify areas of common challenges. Country teams present their progress to the CQUIN community each year, most recently in country posters at the sixth annual CQUIN meeting in December 2022. This process has shown that most countries have made substantive progress toward developing supportive DSD policies, scale-up plans, guidelines, and training materials. Many countries have also improved the coordination of DSD services and have developed standard operating protocols, job aides, and M&E tools.
Sub-National Dashboard
Based on feedback from CQUIN member countries, a sub-national version of the capability maturity model was designed to provide ministries of health with information on provincial and/or district progress in DSD scale-up. The tool, which has been piloted in several countries, is easily adaptable to meet local needs.
DSD Performance Reviews
Because many national M&E systems are not yet set up to routinely collect DSD data, CQUIN supports performance reviews that enable countries to answer priority questions about DSD scale-up. Performance reviews use primary data collection and a sampling approach to obtain a snapshot of the performance of national DSD programs, with the goal of informing programmatic decision-making and tracking the progress of critical DSD indicators over time.
Several ministries of health have conducted several DPRs (DSD performance reviews) since 2018. DPRs conducted thus far include Zimbabwe (2018, 2019, 2020, 2021, 2022), Uganda (2019, 2020, 2021, 2022), Côte d’Ivoire (2019, 2021), South Africa (2019, 2020, 2021, 2022), Kenya (2020, 2021, 2022), Mozambique (2020, 2021, 2022*), DRC (2021), Ethiopia (2021, 2023), Ghana (2022), Cameroon (2022), Tanzania (2022) Nigeria (2021, 2022), Zambia (2021), with support and technical assistance from CQUIN.
Each country has presented results and lessons learned from the review—as well as their country-specific action plans—at CQUIN’s annual meetings. The CQUIN differentiated M&E community of practice is developing a toolkit that will support countries to implement DSD performance reviews independently.
Articles
Patient- and Program-level M&E of DSD for HIV: A Pragmatic and Parsimonious Approach is Needed(2018)
In this viewpoint article featured in AIDS, ICAP Epidemiologist William Reidy, PhD, MPH and CQUIN colleagues highlight common gaps in M&E systems with respect to DART services. The authors also outline elements of an integrated, streamlined approach that provides information to guide patient and program management, while minimizing the burden of data collection and aggregation.
In this article published in the Journal of the International AIDS Society, CQUIN stakeholders define a pragmatic approach to the M&E of the scale-up of differentiated ART delivery for global and national stakeholders.
Presentations
Implementing Differentiated Service Delivery: Differentiated M&E: A presentation by ICAP epidemiologist William Reidy PhD, MPH from the 2017 IAS conference.
CQUIN 2022: Where Are We Now? A presentation by Peter Preko, MB ChB, MPH (CQUIN Project Director),
Maureen Syowai, MBChB, MSc (CQUIN Deputy Director / Technical), William Reidy, PhD, MPH (CQUIN Deputy Director / Strategic Information)
An introduction to the CQUIN DSD Performance Review Toolkit, A presentation by Andrea Schaaf, MPH, ICAP strategic information specialist.
Assessing Client and Healthcare Worker Satisfaction to Inform Implementation and Scale-Up of Differentiated Service Delivery for People Living with HIV Preliminary Findings: A presentation by Hervé Kambale, MD, MPhil, DSD technical advisor, Ministry of Health of Eswatini
Community of Practice
Launched in August 2017, the CoP reflects the ongoing high interest amongst CQUIN country teams in the need to adapt routine monitoring and evaluation (M&E) systems to accommodate DSD models and to develop tools and resources for ad hoc data collection related to DSD.
- Participants include more than 40 individuals from 20 CQUIN network member countries, including Côte d’Ivoire, DRC, Eswatini, Kenya, Liberia, Mozambique, Uganda, Zambia, and Zimbabwe.
- Dr. Bill Reidy (CQUIN Deputy Director for Strategic Information) is the technical lead, supported by Dr. Karam Sachathep (ICAP Senior SI Manager).
- The CoP meets quarterly or more frequently as needed.
- In its first year, the CoP developed an M&E framework, which was subsequently used to inform M&E of DSD strategies in at least six countries: Côte d’Ivoire, Eswatini, Malawi, Uganda, Zambia, and Zimbabwe.
- In 2019, the CoP worked to develop a toolkit for DSD performance reviews to facilitate the ad hoc data collection needed to assess DSD coverage and quality in many countries.
- More recently, the CoP has met to share and discuss experiences related to topics such as M&E adaptations under COVID-19, assessing provider and recipient of care satisfaction, and the use of national electronic medical records systems for M&E of DSD.