Adapting monitoring and evaluation tools to design impactful digital services

This is the eighth blog in a series introducing the GSMA Innovation Fund for Digitisation of Agricultural Value Chains, including its grantees, supporting partners and processes.


Putting farmers at the centre of digitising the agricultural last mile

The GSMA Innovation Fund for Digitisation of Agricultural Value Chains launched in June 2019 to support the scaling of digital solutions for the agricultural last mile. The grantees operate across seven markets to pilot and iterate a range of services, including digital payments for the procurement of crops, digital agronomic advisory and agri digital financial services (agri DFS) including loans and insurance.

Monitoring, evaluation and learning (MEL) is a key function of the Innovation Fund, which includes the traditional role of gathering data to monitor grantee progress and measuring our outcomes and impact. What sets our approach apart is our use of MEL tools to take the pulse of farmer usage of digital services at regular intervals during the scaling of services. This means that our MEL heavily focuses on identifying opportunities and potential pain points on service usage by farmers, and feeds recommendations to each grantee to iterate their digital solutions.

The Innovation Fund’s pathway to impact

The intended impact of the Innovation Fund is to improve smallholders’ income and climate resilience through adoption and regular use of digital services.

The GSMA Innovation Fund for Digitisation of Agricultural Value Chains simplified Theory of Change

We take an ecosystem approach to digitising agricultural value chains, by launching solutions that digitise agribusinesses’ or cooperatives’ farmer databases, procurement records and payments, alongside advisory services that provide farmers with access to market information, weather forecasts and good agricultural practices. These services build farmers’ economic identities, unlocking their access to agri digital financial services. Implementing farming advice and building a procurement history helps farmers gain access to capital, improve their yields and improve their income. Access to digital financial services such as loans and crop insurance, coupled with implementing climate smart practices, makes farmers more resilient to climate shocks. 

Launching digital solutions for farmers that lead to behaviour change requires creating farmer trust in digital tools and building their digital and financial literacy through campaigns and trainings. Human centric service design plays a critical role in effectively reaching the farmer user segment, by understanding and addressing their needs.  With strong elements of user experience (UX) design, we iterate services based on farmer feedback and farmer needs. This approach is fundamental to lead to acceptance and sustained usage of digital services.

Our monitoring and evaluation tools play a key role in the product iteration process. We gather regular feedback from farmers on the services that directly informs product iteration.

Applying monitoring and evaluation tools to identify challenges and opportunities in farmer service usage

We use monitoring and evaluation tools to collect data on product usage, as well as farmer feedback on services during the launch and scaling phases of the service.

Monitoring and evaluation tools (in red) are embedded in the product iteration cycle

The results are used to identify challenges or opportunities to further explore through UX interviews with farmers. The results are discussed during product iteration workshops with the grantee and user experience teams, during which we focus on actionable insights from the MEL data and prioritise what can be translated into new service features, content or service delivery methods. (You can learn more about the AgriTech Innovation Fund UX approach).

The tools we use consist of quarterly business intelligence data analysis and phone surveys. We conduct two to three rounds of phone surveys with a sample of 400 farmers to obtain their feedback on the different services and their features. We share what farmers have to say about the service with the UX team and provide recommendations on pain points and opportunities regarding service delivery and usage.

Illustrative recommendations for UX research from monitoring survey insights

We also analyse BI data on a quarterly basis to understand which types of farmers use services on a recurring basis and to identify service usage patterns as well as potential service bottlenecks.

Illustrative data on an advisory App usage, broken down by content visited

The survey results and BI data insights are discussed during product iteration workshops with grantees and the UX team to provide a holistic picture of how farmers are adopting and responding to the service.

For example, BI data for one grantee indicated that an agribusiness had not previously captured gender data for their farmer base, so gender data was not available for 54 per cent of the users registered on the platform at the time.  However, 47 per cent of monitoring survey respondents were women, indicating an almost even split between male and female users on the platform. This was an important insight on the persona of users of digital procurement who are often assumed to be men. This highlighted the importance of understanding who your users are, and to look at gender-disaggregated results in terms of satisfaction of the service and acceptance of digital tools and introduced discussions about product iteration and opportunity areas targeted at female farmers with the product team.

In an MEL survey conducted in another market, farmers indicated that they trusted the digital procurement process more than manual weighing and recording, and also felt it was quicker.  This contrasted with face to face interviews with collectors who indicated a series of issues with the digital procurement system that they felt combined to make their collection activities slower.  On further analysis of the specific user journeys, we were able to identify key bottle necks and propose solutions including alternative farmer identification methods, which are to be rolled out over the next quarter.

BI data analysis on another grantee showed that while the home page for a farmer facing app had more than 32,000 unique visits in a given amount of time, less than eight per cent of farmers were proceeding to value adding pages which provided digital advisory, procurement and payment records and agri input ordering, amongst other things. These insights led to suggestions for the UX team to test whether farmers know where to find relevant information on the app, and explore whether the home page user interface should be iterated to make specific content more visible to users.

Similarly, through BI data, we recognised that the majority of farmers registered for digital procurement are not cross registering for other apps offered by the same digital platform, such as agricultural advisory. MEL surveys helped us identify potential digital literacy challenges contributing to this phenomenon. It was suggested to use touch points with farmers during their user journey, especially during digital profiling and procurement, to also promote and on-board them on advisory services.

For services to lead to impact, they have to be intentionally designed in a user-centric way. The GSMA Innovation Fund for Digitisation of Agricultural Value Chains’ MEL tools provide solid data collection mechanisms to ensure that farmer feedback informs product development and iteration. The full results from monitoring and evaluation will be made available in the dissemination of learnings phase of the Innovation Fund in 2023. These will not only cover the journey of services scaled with the support of the Innovation Fund, but will also explore the social and economic outcomes that farmers experienced from using these services.

In 2019, the GSMA AgriTech programme launched The GSMA Innovation Fund for Digitisation of Agricultural Value Chains, which aims to scale digital solutions for the agricultural last mile and improve smallholders’ financial inclusion, livelihood and climate resilience. The Innovation Fund prioritises digital enterprise services targeted at organisations, like agribusinesses and cooperatives, procuring from smallholders living on less than $2 (USD) per day. It emerges from the success of the three-year mNutrition Initiative that tested and scaled digital advisory services for smallholder farmers in six markets across Africa and Asia. This Innovation Fund is the latest step in the programme’s evolution to unlock the impact of a wider range of digital agricultural use cases.

You can read more about it in the first blog in the series here.

THIS INITIATIVE IS CURRENTLY FUNDED BY THE UK FOREIGN, COMMONWEALTH & DEVELOPMENT OFFICE (FCDO), AND SUPPORTED BY THE GSMA AND ITS MEMBERS.