This section presents the overarching Theory of Change framework and key metrics established to measure the target outcomes of the IDCA, as well as continuously tweak the various components of the programmes, as needed, based on feedback received from learners, fellows, host organisations, or others. Centred around the vision of the Capacity Accelerator Network (CAN), the IDCA Theory of Change is collaboratively developed by data.org and J-PAL SA, with valuable contributions from academic partners. An external organisation, The Social Investment Consultancy (TSIC), has been engaged to lead the monitoring and evaluation process, ensuring a comprehensive assessment of the India programme’s impact.

Theory of Change

The image below illustrates the Theory of Change that forms the foundation of the India Data Capacity Accelerator (IDCA) model. The Theory of Change is anchored on two primary pillars: supply: fostering a cadre of purpose-driven data practitioners, and demand: empowering social impact organisations (SIOs) with the knowledge and resources needed for data-driven implementation. Together, these pillars aim to achieve the envisioned outcomes on both the demand and supply sides, contributing to the broader vision of transforming the social sector through enhanced data capacity.

MEAL Methodology*

In alignment with the above Theory of Change, a comprehensive monitoring framework was developed to measure progress toward the envisioned outcomes. Detailed indicators were incorporated into the framework to enable an objective assessment of progress at both the demand and supply levels.

On the demand side, indicators focus on measuring advancements in data knowledge and adoption, improved data operations, and the creation of data-centric roles within SIOs. On the supply side, indicators assess learners’ ability to apply their knowledge and skills in the social sector and demonstrate enhanced data capacity in their respective roles within their organisations. This dual focus ensures a balanced evaluation of the programme’s impact across its key objectives. The tools that will be used to capture data against these indicators are provided in the table below:

Tools for Data Collection

Data Collection Tool	Brief Description
Survey Tools	Surveys cover a variety of questions including evidence of data-informed practices by SIOs, improvement of data capacity in reached SIOs, improvements in data knowledge application among trained PDDPs, etc.
Data Maturity Assessment (DMA)	DMA is being used to conduct an in-depth readiness assessment of participating SIOs to be able to measure their improved change in data capacity and practices. This tool is to be administered before and after an intervention with SIOs.
Partner Data Collection Forms	TSIC has developed data collection forms and documentation forms with the list of key indicators that partners will contribute to. Partners will fill out this form, attach relevant means of verification, and share with data.org on an agreed frequency.
Feedback Forms	These will be developed as needed and will capture indicators that measure satisfaction of participants and planned use of resources, training, webinars, meetings and conferences, etc.

Note*: The section highlights the MEAL methodology of the programme and does not present actual results. The deployment of MEAL tools and data collection is currently underway in partnership with the TSIC team and insights will be uncovered by the end of 2025.

Data Maturity of Partner Organisations

The Data Maturity Assessment (DMA) is a critical tool for evaluating the data maturity of social impact organisations (SIOs). It benchmarks an organisation’s capabilities across three foundational pillars:

• Purpose: For what purpose does the organisation want to use data? This includes its strategy, application, and analysis of data.
• Practice: How does the organisation use data to achieve its mission? This encompasses data quality, security, ethics, and the infrastructure needed to work with data.
• People: Who works with data and makes data-driven decisions? This includes the leadership, talent, and overall data culture within the organisation.

SIOs complete the DMA at the beginning of their engagement with the IDCA and again towards the end of the fellowship tenure. These submissions act as baseline and endline indicators, providing insights into shifts in their data maturity over time.

Purpose-driven Data Talent Engaged through Trainings and Fellowship 

The engagement of purpose-driven data talent is envisioned to be measured through surveys conducted at various stages of the programme, including by universities and the J-PAL SA team. Partners administer these surveys based on predefined timelines, which include:

• Enrollment form: The enrollment form aims to understand the backgrounds of individuals who sign up for the training. There are a few questions about their experience, position, gender, etc.
• Post training survey: The post training survey is offered to participants immediately after a training course is completed. It aims to assess what they gained from the course, changes in their confidence with data-related topics, and how the programme can continue to support and connect with them.
• One-year follow-up survey: The one-year follow-up survey will help understand the programme’s longitudinal impact, and how the participant has benefitted from attending the training courses.
• Whispers of change: “Whispers of change” captures ad-hoc sharing on the programme’s impact that may come up in conversations, informal settings, meetings, events, etc.

These tools collectively provide a comprehensive understanding of the programme’s effectiveness in building and sustaining data talent within the social impact ecosystem.