Innovation and systems change requires an adaptive management approach to programs. Your Monitoring, Evaluation, and Learning plan must:
Traditional monitoring and evaluation (M&E) approaches can be challenging when applied to innovative projects or systemic changes for several reasons:
For these reasons, monitoring and evaluating innovation or systems changes requires a more flexible and adaptive approach to deal with ambiguity, complexity, and uncertainty and capture quantitative and qualitative changes.
Forming a Theory of Change
Each program will undoubtedly have its unique theory of change. However, your measurement strategy will likely focus on short-term outcomes that reasonably signal the potential for long-term impact. In the context of innovation and systems change, we acknowledge that transformation occurs through the collaborative design, testing, and sustained implementation of new concepts. This process requires cultivating improved relationships and developing new capabilities, which typically necessitate shifts in mindset and understanding.
Consequently, your theory of change might encompass the following sequential stages:
While these activities may be mandated in the challenge, trying to understand if a) the activities are working as intended in the challenge and b) if participants adopt their learnings beyond the requirements of the competition will be informative.
Suppose you can observe and measure positive shifts in these outcomes, and you’re confident that the systems innovation approaches significantly contributed to these changes. In that case, these outcomes can serve as proxy indicators predicting the likelihood of long-term impacts.
Throughout this process, it is essential to remember that the trajectory of change may not be linear due to the nature of open innovation and systems change. The theory of change may need to be revised and adapted as the process unfolds and the approaches’ systemic impacts become more apparent. It’s also vital to recognize and account for the potential for unintended consequences, both positive and negative.
Possible Indicators to Consider for Each Phase
While each program will develop bespoke indicators, these generic indicators represent the results you would hope to see in implementing a systems innovation program. You can use these as a starting point for your MEL plan.
Pause & Reflect
One method you might consider adapting and integrating to support non-linear adaptation is “Pause & Reflect.” The “Pause & Reflect” methodology provides an innovative approach for Monitoring, Evaluation, and Learning in the context of innovation and systems change. It encourages teams to take intentional breaks to reflect on their work, understand emerging patterns, adapt strategies, and document lessons learned. This approach fits well with the complexity and dynamic nature of systems innovation.
“Pause & Reflect” sessions support teams to consider what is working well and what isn’t in their systems innovation processes. You might discuss unexpected outcomes, novel insights, or emerging challenges. For example, the systems mapping or capacity building sessions are going differently than expected, or the team might encounter unanticipated reactions from participants and the network. These reflection sessions help the team refine your process iteratively, providing a formal opportunity to learn from ongoing work and adjust course as necessary.
For systems change initiatives, “Pause & Reflect” can provide an avenue to consider the broader impacts and implications of the work. Systems change often involves long-term, complex interventions with effects that ripple out in many directions. Reflective sessions can help teams to identify these systemic effects and consider how they align with their goals. For instance, a team might realize that their intervention successfully impacts one part of the system but inadvertently reinforces another part’s problematic aspect. You can redesign the approach to drive the desired change with this insight.
“Pause & Reflect” sessions should involve all relevant stakeholders, including your core team and partners, beneficiaries, or others affected by the work. This inclusive approach ensures diverse perspectives, enriching the reflection process and enabling more robust learning and adaptation. These sessions should be conducted regularly and embedded into the project cycle, creating a culture of continuous learning and improvement.
Monitoring, Evaluation, and Learning (MEL) Methods for Innovation and Systems Change
While traditional MEL methods may not be suitable, various methods designed to respond more effectively to complexity and uncertainty can help you understand your progress and impact. In addition to simple “Pause & Reflect” sessions, you might consider the following methods:
Innovation and systems change often involve trying new things and learning as you go. Developmental Evaluation can be your ally in this kind of work. Unlike traditional evaluation methods that assess predefined outcomes, Developmental Evaluation supports innovation by providing real-time feedback, helping you learn from what’s happening and adapt your strategies accordingly. Remember, it’s not about judging success or failure—it’s about constant learning and development.
You’re likely working in a complex system where cause-effect relationships are hard to pin down. Outcome Harvesting can help you navigate this complexity. Instead of starting with predefined outcomes, this method involves identifying changes that have occurred (the ‘outcomes’), figuring out what your project did to contribute to them, and learning from this. It’s a way of working backward to understand your impact, particularly useful when unexpected outcomes crop up.
Storytelling is powerful. The Most Significant Change (MSC) technique taps into this power by collecting stories about the most significant changes experienced by those you’re working with or serving. By discussing and analyzing these stories, you and your team can gain deep insights into the impacts of your work, including those that quantitative methods might miss.
Decisions that will set your direction
People you will need to find your way
Your core team should include a MEL expert in innovation and system change that can design the framework and oversee its implementation.
You’ll need research assistants to support data collection and a data analyst to clean, analyze, and visualize. These team members should be familiar with both quantitative and qualitative methods.
MEL champions are people within each key stakeholder organization that understand data collection and analysis and can support you with disseminating the tools and engaging the right stakeholders for all MEL activities.
Consider identifying external MEL experts willing to support the design and assessment of the program approach by participating in discrete activities and convening throughout the four stages. This expertise is optional, but given the complexity of evaluating innovation and system changes, it would be an asset to your team.
Review your plan for these critical elements
See the warning signs first
These resources can help you on your journey
Before creating your MEL plan, review GKI’s Training Deck for MEL for Innovation and Systems Change. You can also review additional materials below for guidance.
Read The Curve’s guide to pause and reflect for examples of how to best structure yours. Adapt GKI’s Accelerating Innovation for Resilience Bangladesh Mural Template the team used and this example pre-survey to conduct your own Pause & Reflect.
See GKI’s Accelerating Innovation for Resilience Bangladesh Example Judges & Coaches Survey and KII Questions for ideas on how to collect external feedback on the outcomes and impact of the challenge program.
See GKI’s Accelerating Innovation for Resilience Bangladesh Example Participant Survey and KIIs for ideas on how to collect participant feed on their experiences, outcomes, and impacts of the challenge program.
See GKI’s Accelerating Innovation for Resilience Bangladesh Example MEL Plan to see an example of a full MEL plan for an open innovation challenge.
Additional Resources on MEL in Innovation & Systems Change:
No Crowd-Sourced Recommendations have been submitted. Be the very first one!