Learning
Innovation and systems change requires an adaptive management approach to programs. Your Monitoring, Evaluation, and Learning plan must:
Traditional monitoring and evaluation (M&E) approaches can be challenging when applied to innovative projects or systemic changes for several reasons:
Uncertainty and unpredictability
Most usually characterize innovations or system changes with high uncertainty and unpredictability. Traditional M&E relies on predefined indicators, specific goals, and precise trajectories. In contrast, innovative projects often involve exploration, adaptability, and learning, making outcomes hard to predict.
Non-linearity
Traditional M&E methods often assume a linear pathway from inputs to outputs to outcomes, while systems change and innovation are inherently non-linear. They involve trying different approaches, learning from failures, and iterating on ideas. Identifying direct causal relationships and measuring impacts using conventional methods can be tricky.
Dynamic and complex environment
The environment in which innovation or systems change occurs is dynamic and complex. Change in one area can lead to unexpected consequences in others, and isolating the impact of a specific intervention from other factors can be challenging.
Long-term impacts
The impact of innovations or system changes often takes a long time to manifest and can be challenging to capture within the typical timelines of M&E cycles.
Risk of failure
Innovations and systems change efforts involve a higher risk of failure as they try something new or complex. Traditional M&E methods may not account for this risk and may view failures negatively rather than as opportunities for learning and improvement.
Qualitative aspects
Innovations and system changes often bring about qualitative changes that are not always effectively captured by the quantitative measures used in traditional M&E. For example, changes in an organization’s culture or the level of collaboration among stakeholders might be crucial outcomes of an innovation challenge but are difficult to measure with standard indicators.
For these reasons, monitoring and evaluating innovation or systems changes requires a more flexible and adaptive approach to deal with ambiguity, complexity, and uncertainty and capture quantitative and qualitative changes.
Forming a Theory of Change
Each program will undoubtedly have its unique theory of change. However, your measurement strategy will likely focus on short-term outcomes that reasonably signal the potential for long-term impact. In the context of innovation and systems change, we acknowledge that transformation occurs through the collaborative design, testing, and sustained implementation of new concepts. This process requires cultivating improved relationships and developing new capabilities, which typically necessitate shifts in mindset and understanding.
Consequently, your theory of change might encompass the following sequential stages:
Shifts in Individual Mindsets
Participants’ perceptions and understanding of innovation and systems change will likely evolve due to their involvement in the program.
Growth in Individual Skills
Participants’ abilities should enhance as they engage in capacity-building activities.
Changes in Organizational Mindsets, Practices, and Cultures
As individual participants apply their new learnings to their respective organizations or institutions, it’s plausible to observe shifts in the broader organizational mindsets, practices, and cultures.
Change in Practices
As mindsets shift, capabilities increase, and culture changes, you hope to see individuals and organizations doing things differently than before, including specific activities around systems sensing and mapping, co-design, prototyping, and testing.
Strengthened Relationships or Partnerships
Relationships or partnerships may be forged or reinforced through system-sensing and network-building activities.
While these activities may be mandated in the challenge, trying to understand if a) the activities are working as intended in the challenge and b) if participants adopt their learnings beyond the requirements of the competition will be informative.
Suppose you can observe and measure positive shifts in these outcomes, and you’re confident that the systems innovation approaches significantly contributed to these changes. In that case, these outcomes can serve as proxy indicators predicting the likelihood of long-term impacts.
Throughout this process, it is essential to remember that the trajectory of change may not be linear due to the nature of open innovation and systems change. The theory of change may need to be revised and adapted as the process unfolds and the approaches’ systemic impacts become more apparent. It’s also vital to recognize and account for the potential for unintended consequences, both positive and negative.
Possible Indicators to Consider for Each Phase
While each program will develop bespoke indicators, these generic indicators represent the results you hope to see in implementing a systems innovation program. You can use these as a starting point for your MEL plan.
Pause & Reflect
One method you might consider adapting and integrating to support non-linear adaptation is “Pause & Reflect.” The “Pause & Reflect” methodology provides an innovative approach for Monitoring, Evaluation, and Learning in the context of innovation and systems change. It encourages teams to take intentional breaks to reflect on their work, understand emerging patterns, adapt strategies, and document lessons learned. This approach fits well with the complexity and dynamic nature of systems innovation.
“Pause & Reflect” sessions support teams to consider what is working well and what isn’t in their systems innovation processes. You might discuss unexpected outcomes, novel insights, or emerging challenges. For example, the systems mapping or capacity building sessions are going differently than expected, or the team might encounter unanticipated reactions from participants and the network. These reflection sessions help the team refine your process iteratively, providing a formal opportunity to learn from ongoing work and adjust course as necessary.
For systems change initiatives, “Pause & Reflect” can provide an avenue to consider the broader impacts and implications of the work. Systems change often involves long-term, complex interventions with effects that ripple out in many directions. Reflective sessions can help teams to identify these systemic effects and consider how they align with their goals. For instance, a team might realize that their intervention successfully impacts one part of the system but inadvertently reinforces another part’s problematic aspect. You can redesign the approach to drive the desired change with this insight.
“Pause & Reflect” sessions should involve all relevant stakeholders, including your core team and partners, beneficiaries, or others affected by the work. This inclusive approach ensures diverse perspectives, enriching the reflection process and enabling more robust learning and adaptation. These sessions should be conducted regularly and embedded into the project cycle, creating a culture of continuous learning and improvement.
Monitoring, Evaluation, and Learning (MEL) Methods for Innovation and Systems Change
While traditional MEL methods may not be suitable, various methods designed to respond more effectively to complexity and uncertainty can help you understand your progress and impact. In addition to simple “Pause & Reflect” sessions, you might consider the following methods:
Innovation and systems change often involve trying new things and learning as you go. Developmental Evaluation can be your ally in this kind of work. Unlike traditional evaluation methods that assess predefined outcomes, Developmental Evaluation supports innovation by providing real-time feedback, helping you learn from what’s happening and adapt your strategies accordingly. Remember, it’s not about judging success or failure—it’s about constant learning and development.
You’re likely working in a complex system where cause-effect relationships are hard to pin down. Outcome Harvesting can help you navigate this complexity. Instead of starting with predefined outcomes, this method involves identifying changes that have occurred (the ‘outcomes’), figuring out what your project did to contribute to them, and learning from this. It’s a way of working backward to understand your impact, particularly useful when unexpected outcomes crop up.
Storytelling is powerful. The Most Significant Change (MSC) technique taps into this power by collecting stories about the most significant changes experienced by those you’re working with or serving. By discussing and analyzing these stories, you and your team can gain deep insights into the impacts of your work, including those that quantitative methods might miss.
Decisions that will set your direction
People you will need to find your way
Your core team should include a MEL expert in innovation and system change that can design the framework and oversee its implementation.
You’ll need research assistants to support data collection and a data analyst to clean, analyze, and visualize. These team members should be familiar with both quantitative and qualitative methods.
MEL champions are people within each key stakeholder organization that understand data collection and analysis and can support you with disseminating the tools and engaging the right stakeholders for all MEL activities.
Consider identifying external MEL experts willing to support the design and assessment of the program approach by participating in discrete activities and convening throughout the four stages. This expertise is optional, but given the complexity of evaluating innovation and system changes, it would be an asset to your team.
Review your plan for these critical elements
See the warning signs first
These resources can help you on your journey
Before creating your MEL plan, review GKI’s Training Deck for MEL for Innovation and Systems Change.
Review the example materials from GKI’s Accelerating Innovation for Resilience Bangladesh for ideas on MEL tools and methods to:
- Conduct a Pause & Reflect: Mural Template
- Collect feedback from participants on their experiences, outcomes, and impacts of the challenge program: Participant Survey
- Collect feedback from judges and coaches on the outcomes and impact of the challenge program: Judges & Coaches Survey, Key Informant Interview (KII) Questions
- Collect feedback from your network: Network Launch Workshop – Post Event Survey and Network Systems Mapping Workshop – Post Event Survey
- Develop a MEL Plan for an open innovation challenge
Additional Resources on MEL in Innovation & Systems Change:
No Crowd-Sourced Recommendations have been submitted. Be the very first one!
Leave a Reply