Smisha Agarwal, Amnesty Lefevre, and Alain Labrique, Johns Hopkins University Global mHealth Initiative

Garrett Mehl, World Health Organization Department of Reproductive Health and Research

Mojca Cargo, GSMA

Background

In August 2014, the South African National Department of Health (NDoH) launched MomConnect, the first national level mHealth initiative that links expectant mothers to gestation age-specific pregnancy information. It also encourages them to register their pregnancy at a health facility, in order to receive continued care. MomConnect aims to improve pregnancy registration and client empowerment through increased knowledge and access to services. This is whilst giving them a forum to voice their experiences through a facility based help-desk access. Through the early registration of pregnant women at health facilities, this program aims to improve healthcare utilization and outcomes during the antenatal and postnatal period for mother and child.

In the early program planning stages, the NDoH recognized the need for a technical agreement and guidance on developing effective methods to monitor and measure the progress and effectiveness of MomConnect. In response, the GSMA convened a group of partners to develop a generic logical framework for planning, monitoring and comparing mobile-based strategies to improve maternal and child health in South Africa. MomConnect was the first use case to inform this framework; though its applications could extend beyond any one specific program. This group was a consortium led by the Johns Hopkins Global mHealth Initiative (JHU-GmI) and WHO Reproductive Health and Research, and included representatives from the NDoH, mobile network operators and implementing partners, including the Praekelt Foundation, WRHI, Cell Life, Dimagi, Mezzanine, Jembi, CHAI, mHA and Mobenzi. The monitoring and evaluation (M&E) framework presented here provides guidance on:

• Monitoring the progress of a mHealth program by using a common set of indicators across all implementation partners
• Reporting on additional indicators which might allow the mHealth community to differentiate between various mHealth products and/or implementation strategies

Monitoring & Evaluation Framework

Drawing upon recommendations from the World Health Organization (WHO) mHealth Technical Evidence Review Group (mTERG) for reporting on scale and effectiveness of global mHealth programs, the M&E framework provides a comprehensive list of indicators for each stage of the rollout of the program, as well as along the continuum of maternal and child healthcare from pregnancy through postpartum. The log frame presented below includes measurable inputs, outputs and outcomes, important for assessing the effect of MomConnect on health outcomes.  Illustrative sources of data for each of these parameters are available on request.

• Inputs/ Processes encompass critical resources that go into developing and implementing a mHealth program. We consider broader partnerships and funding agreements that underpin the program’s success, as well as inputs related to health promotion message development, and facility and community health systems activities.

• Outputs are measurable products of the inputs and process. The key domains under outputs measure the functionality of the mHealth platform, as well as changes in the utilization of services and satisfaction with facility-based services. Additionally, these set of indicators help us answer two questions: Does the mHealth platform work as intended?; How do women and health workers interact with the platform? Increasingly, we find that human-centric designs for user interface promote adoption and usability of such tools. We expect that these indicators will be captured and reported on an ongoing basis as part of routine data monitoring and use.

• Outcomes aim to measure the downstream effects of the implementation of the mHealth program on the availability, quality, efficiency and utilization of health care services for expectant mothers and children.

• Impact measures seek to capture the long-term effect of the improvements in, and utilization of health services on the health of the mother and infant.  These recommendations are proposed in the logframe to clearly elicit the potential long-term value of a mHealth program like MomConnect in preventing the death of mothers and children. However, measurement of these indicators poses significant attribution challenges and entails significant investment of funds and time; therefore it is not expected that the program will directly measure these indicators with external evaluation support. It should be noted that, if an mHealth service improves access to, and utilization of health services (as listed under outcomes), the number of deaths averted could potentially be modeled based on those estimates.

The logframe and compendium of indicators proposed is meant to be a critical starting point for evidence generation. As implementing partners and the programs within which MomConnect is delivered are finalized, a priority set of indicators will need to be drawn from within the larger framework presented. The current list of indicators is intended to accommodate a wide array of programs; any one of which is expected to use only a subset of relevant indicators.

As MomConnect is intended to be responsive to the growing health service needs of South Africans, flexibility will allow for the expansion of program delivery channels and content to include inputs to health service providers including facility and community healthcare workers.

This is an exciting time for innovation and the use of mobile tools in health services delivery in South Africa. No other country in sub-Saharan Africa has sought to provide such a service nationally. This program has the potential to serve as a model for the use of technology in health services delivery throughout the region and globally.