Learning in the Face of Complexity
What it Looks Like in a Partnership of 100+ Governments and Civil Society Working to Open Government
By Munyema Hasan, Deputy Director of Learning & Innovation, Open Government Partnership
“So, how does the Open Government Partnership learn?” We at the Open Government Partnership (OGP) have been asked this question many times. Sometimes it stems from skepticism that learning can meaningfully happen in a partnership of 100+ national and local governments and thousands of civil society organizations. Sometimes it stems from curiosity because even if learning takes place, the line from learning to decision-making to outcomes is never a straight one, especially in the governance field.
It seems like the right time to attempt to answer this question — particularly as OGP steps into 2021 with a decade of experience in opening government, and an independent evaluation of how we are supporting seven OGP members is underway.
As I began to reflect on what we — the OGP staff — are learning, I couldn’t disconnect it from how we are learning. Organizations don’t learn in a monolithic way. Formal and informal learning systems, practices and culture all combine to inform decision-making. The symbiotic manner in which learning happens is probably best illustrated by our experience in the past 18 months — where a million dollar evaluation has had to leverage (and improve!) OGP’s existing learning practices to maximize learning outcomes.
“The symbiotic manner in which learning happens is probably best illustrated by our experience in the past 18 months — where a million dollar evaluation has had to leverage (and improve!) OGP’s existing learning practices to maximize learning outcomes.”
When the evaluation began in Nigeria, Ukraine, Philippines (+South Cotabato), Colombia and Kenya (+Elgeyo Marakwet), OGP staff and donors funding the evaluation (Foreign, Commonwealth & Development Office, Open Society Foundation and Hewlett Foundation) were excited about its unconventional “developmental evaluation” approach. The focus was not on revelations delivered at the end of the evaluation on how OGP made an impact. Rather it was on OGP using the evaluation to learn and adapt its strategies in real-time, without sacrificing the rigor or accountability of a traditional evaluation. What would that look like in practice? Few of us knew. But everyone including the donors kept an open mind towards this approach — which seemed adaptable enough to withstand the stress test of a pandemic. Here’s how evaluation learning is blending with OGP’s own learning practices — and how it impacts what we learn:
Real-time learning vs. definitive findings: OGP staff have had to become comfortable with “emerging insights” from an evaluation, which facilitate real-time problem solving and decision-making — rather than definitive findings that emerge when the decision-making opportunity is over. OGP’s staff in each of the seven evaluation cases engage in an ongoing inquiry every month with evaluators to discuss areas where our support seems to be making a difference, where we are hitting roadblocks, and strategies to overcome them. It didn’t take long before OGP staff began to cite new approaches and strategic shifts they were trying in Nigeria, Colombia or the Philippines in response to the emerging insights.
Critical friends enter decision-making spaces: Evaluators using the developmental evaluation approach often label themselves ‘critical friends’. It sounds nice, but the term itself assumes a level of trust with the evaluatee to play the role effectively. Inviting critical friends into OGP’s decision-making spaces, such as monthly leadership team meetings, needed an openness on the part of OGP staff to receive strategic input from individuals who are not direct stakeholders. Donors too have played this role though less explicitly, often bringing their colleagues who are country and subject matter experts into OGP’s learning spaces to inform strategic shifts.
Layered learning is inevitable: Even though country insights are useful for staff in fine tuning strategies in real time, it wasn’t clear how OGP’s leadership could develop a strategic direction for a partnership of a 100+ countries and locals based on emerging insights from less than 10 percent of the Partnership’s sample size. OGP staff are working with the evaluators to layer data from different sources — blending breadth with depth to paint a better picture of what we know about OGP’s theory of change. We combined data collected by OGP and the Independent Reporting Mechanism (IRM) on the process, action plan ambition and results across the Partnership with the rich, textured analysis of what OGP looks like in each of the seven cases of the evaluation.
What are we learning?
Is how we learn and make decisions sequential and predictable? Absolutely not. Are we in a better place to answer what we are learning and make sense of where OGP is contributing? Probably. Here are a few ways our knowledge is either being validated or challenged with new evidence, and how they are informing shifts in our work:
Invest even more in country reformers — individually and collectively: Through campaigns and summits, OGP raises the visibility of the open government agenda and shapes the incentives of political leaders to create the authorizing space for reforms. While we know from the evaluation that these strategies are effective, the experience of OGP staff continually reiterates that it is the individual government and civil society reformers who translate a vision into concrete action and sustain it at the country level. To address this, we dialed up our support to reformers to design better commitments and strengthen, for example, the multi-stakeholder forums (MSF) in OGP countries — which is the platform government and civil society use to jointly determine the country’s open government priorities. Our support has focused on building trust between different actors and ensuring that the functions and structures that govern the MSF facilitate strong collaboration. Thanks to years of IRM data, we now know that investing in commitment design and in stronger MSFs can lead to better ambition and completion of action plans. To scale these efforts, we are starting to offer ‘group-based’ support to these reformers going forward.
“It is the individual government and civil society reformers who translate a vision into concrete action and sustain it at the country level.”
Facilitating coalition building continues to have pay-offs: A Partnership-wide improvement in ambition without necessary support given to reformers can lead to problems with demonstrating results. OGP reformers drive the efforts to convene multi-stakeholder groups around specific policy areas, help build a vision and sense of ownership to act on them, and navigate the technical and political aspects of the reforms. We loosely use the term “coalition building” to describe this — but it is one of the most labor intensive and complex efforts for OGP staff to facilitate at scale for a partnership of OGP’s size. Where coalition building efforts gain momentum, the results are notable as the evaluation has highlighted. Our learning from supporting reformers on beneficial ownership in Nigeria, open contracting in the Philippines, fiscal transparency in Colombia and many other countries eventually led us to formalize our support to a set of focus countries, themes and commitments and aim for a higher set of outcomes. We’ve made good progress in helping design more ambitious commitments, but how we support the implementation of those commitments — along with our ecosystem of partners — is an area we are still navigating.
The ‘rules of the game’ are being refreshed: One of the strengths of the OGP model has always been that it’s not a talk shop, and countries have to commit to specific policy reforms through their action plans, which are then independently assessed by the IRM. The evaluation is showing that while the structure and formality of the two-year OGP cycle promotes iterative growth and policy ambition, it doesn’t incentivize countries to articulate the broader visions behind policy reforms. There is also a tendency to temper ambition in the action plans, to avoid the risk of underachievement by IRM standards. With a move towards a flexible, multi-year action plan model and sweeping upgrades of the IRM’s assessment method, we are hoping to address the incentives that matter for ambition and delivery in action plans. A longer time frame, or open government strategies with nested OGP action plans in them may deliver more promise — as we are seeing in Spain’s four-year action plan which focuses on policy and sectoral reforms, not the commitment at the project or activity level. Time will tell whether more countries adopt this model and the results begin to discernibly improve.
Calibrating our impact — what’s in our control?
Zooming out, perhaps one of the more clarifying moments in our theory of change came with asking a few questions:
- How can open government reformers use their OGP membership more strategically?
- What kind of support from OGP staff, Steering Committee and strategic partners makes the most difference to reformers?
- How and where do we adapt to provide that support at scale to all members of the Partnership versus to a smaller set of targeted reformers?
Reminiscent of Liz Ruedy’s models of impact, OGP staff ruminate that our contribution in some countries may be more ‘opportunistic’ or ‘stabilizing’ — preventing democratic backsliding, or generating a culture shift by making the policymaking space more open and participatory. In other countries, it may be more ‘proactive’ where we leverage a strong ecosystem of reformers and play an accelerator role in taking ambitious reforms to the finish line.
“OGP staff ruminate that our contribution in some countries may be more ‘opportunistic’ or ‘stabilizing’… in other countries, it may be more ‘proactive’ where we leverage a strong ecosystem of reformers and play an accelerator role in taking ambitious reforms to the finish line.”
I’ve just scratched the surface on how OGP staff are learning and making decisions — in what may be an imperfect but pragmatic way of combining experience and evidence to rapidly adapt the way we support the Partnership. There are many catalysts and change agents in OGP — reformers, our Steering Committee, our strategic and thematic partners. Their voice is equally critical in shaping the priorities and the evolution of the Partnership. This year we hope to surface more of them to define the next decade of OGP.
