comment 0

A reminder: the original Manifesto for Agile Software Development

Available at:

‘We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.’

We follow these principles

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals.

Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. 

The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity–the art of maximizing the amount of work not done–is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

comment 0

Greta Thunberg and The 1975 join forces against climate change

Excerpt from The Guardian:

“I kept thinking about it and I just wondered if I am going to have a future. And I kept that to myself because I’m not very much of a talker, and that wasn’t healthy. I became very depressed and stopped going to school. When I was home, my parents took care of me, and we started talking because we had nothing else to do. And then I told them about my worries and concerns about the climate crisis and the environment. And it felt good to just get that off my chest.”

“I painted the sign on a piece of wood and, for the flyers, wrote down some facts I thought everyone should know. And then I took my bike to the parliament and just sat there … The first day, I sat alone from about 8.30am to 3pm – the regular schoolday. And then on the second day, people started joining me. After that, there were people there all the time.”

comment 0

What did we do before going digital?

Leah McLaren in this article in The Guardian ponders a time when our attention was allowed to wander. Below an extract with the key steps in our lives becoming digital

Let’s go digital

Key developments that changed the way we communicate. By Hayley Myers

1971: Email Ray Tomlinson was responsible for electronic mail as we know it, choosing the @ sign to connect the username with its destination. Today, it’s estimated there are 3.9 billion email users.

1992: Phone texting British engineer Neil Papworth sent the first SMS – ‘Merry Christmas’ – from his computer to the mobile phone of Vodafone’s Richard Jarvis. Handsets didn’t include keyboards then, so Jarvis was unable to reply.

1997: Chat rooms Talking to friends (and strangers) in chat rooms dominated the late 1990s. The rise of other internet technologies saw their popularity plummet in the following decade, taking the ubiquitous a/s/l abbreviation with it.

2004: Social media Whether a place of meaningful connection, a worrying echo chamber or both, Myspace and its successors created an era where likes, influencers and filters reign supreme – despite recent concerns over privacy and data breaches.

2005: YouTube YouTube’s first ever video, ‘Me At The Zoo’, was uploaded by the channel’s founder Jawed Karim. It’s since been viewed 73m times, paving the way for cultural moments, such as ‘Charlie Bit My Finger’, skateboarding bulldogs and Justin Bieber.

comment 0

Podcast: Nepal on its way to federalism. A conversation with Shailendra Sigdel

In September 2015, Nepal promulgated a new Constitution which mandates the transition of Nepal from a centralized unitary state to a federal country with the goal to bringing democracy to the doorstep of the citizens and assign more power and responsibility to local governments. Four years on, how is Nepal doing? How is the reform progressing? I talked about it in Kathmandu with Shailendra Sigdel, Managing Director of the Foundation for Development Management.

comment 0

We need to prepare today the knowledge systems of the Fourth Industrial Revolution

Martin Dietz until recently led the HELVETAS Swiss Intercooperation team implementing the Performing and Responsive Social Sciences (PERFORM) project in Serbia and Albania. The aim of the project was to test ways to strengthening the relevance of social sciences for social and political reforms in the two countries. The project has been funded by the Swiss Agency for Development and Cooperation and ended in December 2018.

Preparing the knowledge systems for the 4IR requires looking beyond technical solutions, Martin Dietz

Martin is now a Senior Advisor for Sustainable and Inclusive Economic Development with Helvetas and shares here his feedback on the discussion that I have co-produced with Maria Malho, Vanesa Weyrauch and Fred Carden with support from Helvetas: State Capability, Policymaking and the Fourth Industrial Revolution: Do Knowledge Systems Matter?.

‘Thank you for sharing the discussion paper on State Capability, Policymaking and the Fourth Industrial Revolution with me.  The question of whether knowledge systems matter is probably more semantic; they very much do matter, already today.

The paper provides for fascinating reading, in particularly your reflections in which ways a 4IR may impact on knowledge systems and policymaking in future.

The thought I want to share here is twofold. Knowledge systems are relevant and important not just with the advent of 4IR but very much today, and many of your recommendations can be taken up already today. Fostering knowledge systems and their links with policy institutions is not just a matter of technical skills and technical infrastructure. Trust and incentives are strong factors that should not be overlooked.  

I have been working with a team to support the strengthening of knowledge systems and in particular the linkages in the pentagon between stakeholder groups in Serbia and Albania. When I talked with colleagues and friends here in the Western Balkans about the discussion paper and its hypotheses, the response was rather lukewarm: should we spend time discussing what a 4IR will bring to countries in the region whose policy institutions are just in the process of embracing the 3IR?

The thinking is that we should not link the importance of knowledge systems in support of policymaking with the advent of 4IR. They are important today.

Numerous of your recommendations for the evolution of knowledge systems (Chapter 4.2 and Annex II) can be addressed today. With a good understanding of what knowledge systems entail and what the benefits are, governments can start working today on an enabling environment for the evolution of knowledge systems. Frameworks need to address technical capabilities and infrastructures. However, what I can see from my perspective is that we should not just focus on technological dimensions. That will be too narrow. Governments need to create an incentive system for stakeholders in knowledge systems to work together. If, for instance, policy-relevant research is not considered by performance assessment systems, it will not work. The nodes of the knowledge system pentagon don’t connect in the region where I am working. Technology alone will not overcome this.

Apart from governments, many of the development agencies have not understood the concept of knowledge systems, and do not appreciate the significance of knowledge systems for policymaking. When I hear the argument that they will consider it as a cross-cutting issue, then it’s a dead horse from the beginning.  

Your foresight thinking is important to understand how the 4IR will impact on policymaking and knowledge systems. By its very nature, the discussion paper includes many vague assumptions.

It would be interesting to focus in future more on particular sectorial fields, take the example of youth employment and actually work on knowledge systems on the ground that will foster policies that are based on evidence and good practice. That would provide rich learnings.’

comment 0

The 4IR Is Here. Do We Need to Design Development Initiatives Differently?

I have written this blog post with Vanessa Weyrauch. This article first appeared in Helvetas Mosaic, a quarterly publication exploring new trends and fresh ideas of international development. Subscribe to receive more articles like this from Helvetas Mosaic.

It is easy to get carried away with the promises of technology when we read about the Fourth Industrial Revolution, or 4IR.

In an earlier Mosaic article, Knowledge Systems and Policy Innovation in the 4IR, we wrote that, according to the World Economic Forum there is a good chance that by 2025 we will have: 1 trillion sensors connected to the internet; the first 3D printed car in production; the first government replacing the census with big-data sources and analytics; and artificial intelligence performing 30 percent of the corporate audits in the world. Jamie Susskind describes the mind-boggling possibilities offered by nanotechnology, with nanorobots able to swim through our bodies delivering targeted drugs, or the staggering increase in the number of people connected to the Internet, from 400 million in 2000 to an expected 4.6 billion by 2021. He writes that we are entering “a digital lifeworld characterized by machines that are equal or superior to humans in a range of tasks and activities; technology that is embedded in the physical environment in which we live; and digital technology that more and more records human activities as data and processes it through digital systems”.

There are risks of course, for example David Wallace-Wells writes: “Five years ago, hardly anyone outside the darkest corners of the internet had even heard of bitcoin; today, mining it consumes more electricity than is generated by all the world’s solar panels combined, which means that in just a few years we’ve assembled a program to wipe out the gains of several long, hard generations of green energy innovation.” Another example is OpenAI, the Elon Musk-backed non-profit set up to responsibly push the boundaries of what is possible with artificial intelligence. It has developed an AI system which generates coherent paragraphs of literary or news text (go to 19’45” of the podcast) which is so sophisticated that OpenAI has decided not to release it fully to the public because of the real risk that in the wrong hands it could generate very plausible fake news, spam or reviews. 

Reading about technologies of the future gives the impression that the technological changes we are witnessing have a life on their own. They promise a bright future of efficient production and an economic growth path that is at last within the natural limits of our planet. As argues by Susskind, these technologies are not exogenous forces over which we have no control. There are people behind these technologies and governments will need to strengthen leadership and develop human capital so that they are able to govern the techno-digital transformation in a way that leaves no one behind.

The challenge for results-based management approaches

In our discussion paper, State Capability and Policymaking in the Fourth Industrial Revolution, we argue that the challenge for government as we approach the 4IR is to keep up with the pace of change and understand the likely social and economic impact of technological innovation to be able to regulate it. This represents a huge challenge for middle and low income countries, since policymakers are expected to resolve interdependent policy challenges that imply a high degree of uncertainty while facing significant capability and institutional barriers such as for example limited budgets, decision-making processes organized in silos, weak IT and knowledge management structures, and low investment in evidence generation.

At the same time, the technological changes that are emerging will change the way governments work and policies are decided. Processes that investigate specific problems, design the necessary policies and regulatory frameworks, and deploy them through top-down systems will struggle in this new technology-driven context facing increasing disruption. In the near future, government institutions will have to adopt policies that govern techno-digital transformation by testing technologies and experimenting to ensure they benefit their citizens on the whole, while trying to avoid intensifying inequalities. Agile forms of government will be needed to help regulators and legislators continuously adapt to a new fast-changing social and economic environment, without stifling innovation. This is likely to challenge the role of central government agencies as local institutions may be quicker in adopting and using new technologies and interacting with citizens. Policymaking and decision making are likely to become more decentralized and concentrated in new areas called mega-regions, which combine cities and metro areas and are increasingly powering today’s world economy.

This poses a challenge to the way today’s development initiatives, particularly around governance reforms and public policy, are designed, planned and implemented. Pablo Yanguas, in his Why We Lie About Aid, writes that everyone involved in public policy knows that definitive results are rare, and yet the vast majority of development initiatives are designed to follow a linear results-based logic of input-output-outcome-impact. Most of the evaluations commissioned today by development partners and implementing organizations are asked to verify this results chain.

If we accept that governance systems will fundamentally change as we enter the 4IR and that governments have to start preparing today, then we may need to rethink the way we design and implement development initiatives. This is particularly true for initiatives that aim to support the development of leadership and human capital capabilities of future generations of civil servants, policymakers and researchers to drive these processes.

What are the implications for development programming?

Program design: The emerging literature on adaptive development provides some interesting ideas about how programs can be more open to the uncertainty of outcomes and results, and how to build a more experimental approach into governance initiatives that will increasingly deal with it. The suggestions are to invest time and resources into developing relationships with local partners and discovering common interests around problems. Digital technologies and platforms can help with that. This can help focus on solving problems that are owned, debated and defined by local stakeholders and partners, and which are not predefined. In some cases it can be about identity solutions (also called positive deviances) that partners have been able to develop despite bureaucratic constraints they face daily, and which document and support those initiatives.

Investing in acquiring a good knowledge of the political economy of the space and context in which the development initiative operates can help to design policy solutions that are politically feasible and not just technically sound. To do so, it is important to work with local innovation leaders committed to testing new governance solutions. These are individuals whose leadership is not a side effect of their position in a formal hierarchy (e.g. their job title), but rather a side effect of the respect, appreciation and trust they receive from their peers as well as being lifelong learners, a fundamental skill required by the pace and depth of changes in the 4IR.

Program implementation: test new forms of knowledge co-production to inform governance innovation and policy experiments through greater access to and use of digital technologies to link a wider range of stakeholders as we suggest through the knowledge system pentagram in our paper. Linking, for example, researchers from universities with policy research organizations; or professional knowledge by technical experts and civil servants with citizen knowledge. Through an experimental approach the need for blending different forms of knowledge and apply a more interdisciplinary approach to knowledge generation becomes more prominent. So does the need to develop physical and digital spaces for collaboration that enable testing of solutions, learning, and building on what works while dropping solutions that do not. Program funders can support this adaptive process by allowing a space for experimentation, acceptance that some experiments may fail, and investments in learning to help decide on which solutions to support.

Program teams: An experimental and politically informed program implementation approach requires a program team with the capacity and skills to do this. This involves either finding individuals with experience in adaptive management, demonstrated capacity to understand and collaborate with local leadership, and the ability to support experimental approaches, or investing the necessary financial resources and time in building skills and knowledge within the team and providing them with the space required to maximize the experience they bring with them.

Impact, replication and scaling up: Every development program is under pressure to replicate and scale up sustainable approaches and solutions. But what does sustainability and scaling up mean for a program adopting adaptive and experimental approaches to testing solutions? Replication and scaling up can refer to the uptake of adaptive and experimental principles by government partners to explore the opportunities and challenges that new technologies bring to governance, social and economic systems. It can be useful to explore the opportunities provided by the principles of innovation diffusion, which state that innovation emerges through initiatives designed and implemented by a small number of innovators. The tested solutions are then gradually taken up by a group of early adopters, followed by a larger group of adopters. In this context, innovation leaders can be catalysts for change from within a policy community. The scaling up can be accompanied by investment in documenting the successes of the partners more than those of the project, even though the two may be interlinked. In our opinion, it is a subtle but important difference.


Governance and policymaking process will be different in the 4IR. They will rely more than today on digital technologies and in the co-production of new forms of knowledge, within areas that bridge innovation, research, higher education, and local and professional knowledge.

These changes will not emerge overnight, they will evolve incrementally. Governments have to start preparing today human and governance capabilities that will be required in this imminent future to take advantage of the changes that are emerging and minimize the possible negative outcomes. Similarly, project and programs aimed at collaborating with national governments to support these change processes will need to evolve their approaches to design, implementation, and evaluations of results. In this article we have provided some initial ideas on how to do so.

comments 2

The Fourth Industrial Revolution needs strong foundations in the education system. The comment from Jonathan Pincus on our new discussion paper

I am getting more feedback about the discussion paper published by Southern Voice that I have co-produced with Maria Malho, Vanesa Weyrauch and Fred Carden with support from HELVETAS Swiss Intercooperation, State Capability, Policymaking and the Fourth Industrial Revolution: Do Knowledge Systems Matter?.

The aim of the discussion paper is to identify the key questions we need to ask about the implications of the Fourth Industrial Revolution for policy processes, governance systems and the ways knowledge will inform policy decisions.

The comment I post today is from Jonathan Pincus, an economist specializing in public policy and Southeast Asia, who is currently President with the Rajawali Foundation in Jakarta.

Jonathan raises two important points about the 4IR in the context of emerging economies such as Indonesia. The first is about the extent to which the 4IR can be a factor of economic change. The second is about the foundations of any technology change and whether the technology changes that are associated with the 4IR can actually take place in countries such as Indonesia that are trying to address problems of quality of teaching in school and low quality of research.

‘I don’t really buy into this Fourth Industrial Revolution idea. Technology and production systems (and knowledge systems) are always changing but within frameworks shaped by economic and social power structures. I don’t think that big data has made the old Marxist debate irrelevant about the forces and relations of production. Remember we used to argue about what was the driving factor in economic change: relations of production (feudalism, capitalism) or forces or production (the handloom or the mill)? Or was there an indeterminate dialectical relationship between the two? Much of this 4IR literatures implicitly assumes that it’s the forces of production that are dominant: AI, robotics, nanotechnology give us a different kind of economy, society, politics. Well, yes. But mediated by relations of production, which are themselves a product of power relationships.’

Photo: Sentarum elementary school (CIFOR, CC)

‘I work in Indonesia which still struggles in terms of education indicators among middle-income countries. The law that mandates the government has to spend 20% of its budget on education [Law on National Education No.20/2003] has contributed to a rapid increase in enrolment, including at the tertiary level. But the quality of teaching and learning has not improved. According to the OECD, the education system does less to promote social mobility in Indonesia than in other middle-income countries. In other words, if you are rich you will attend a good school, and if you are poor you will not. The curriculum needs to focus more on math, reading, writing and science and encourage students to analyze and solve problems.’

‘Does technology have a role in changing this? I’d like to believe that digital technologies can reduce barriers to entry to education an improve quality. Not much evidence for that yet in Indonesia, but I think it is still possible. And there are schools where teachers and headmasters are committed to improving the way the students learn.’

‘At the same time, there is a political problem. The governance and accountability systems in the education and higher education sectors do not yet prioritize merit-based recruitment and career progression. Teachers do not have the proper math and science skills required to teach students and prepare them for a technology-driven future. And so Indonesia’s math and reading scores continue to be among the worst in the world, at least among countries that routinely measure them using international metrics.’

‘We can have an interesting discussion about technology and governance but if people cannot do basic math or read and understand a short paragraph in their own language it isn’t going to matter all that much.’

‘Universities face challenges as well. The research sector is generally weak in Indonesia. Moreover, the incentives and drivers of career progression are not yet geared toward doing good quality research published in good national and international peer-reviewed journals. In other words, you cannot have high-quality think tanks and research institutes if you don’t also have good public universities.’

‘I don’t mean to rain on your parade, but I don’t feel we are on the cusp of anything. Indonesia needs an education revolution. I don’t know how to get one. In the context of the current presidential election, campaign education has hardly been mentioned. Perhaps the movement for gender equality can inspire women to fight for better schools to educate girls and young women and overcome the barriers that women face in the labour force. Concern for the quality of education does not seem to be very important to nationalist or Islamicist parties.’

comment 0

Will knowledge systems matter in the Fourth Industrial Revolution? The comment from Beatriz Kira from the Pathways for Prosperity Commission in the UK on our new discussion paper.

Klaus Schwab defines the Fourth Industrial Revolution as ‘the confluence of technological breakthroughs, covering wide-ranging fields such as Artificial Intelligence, robotics, internet of things, nanotechnology, biotechnology, energy storage, and quantum computing.’ It will change economies and societies. Some write that Everyone Must Get Ready For The 4th Industrial Revolution

Together with Maria Malho at Demos Helsinki, Vanesa Weyrauch at Politics and Ideas, Fred Carden of Using Evidence, Andrea Ordoñez at Southern Voice, Luis Carrizo at UNESCO Latin American and the Caribbean Office, and Helvetas we have co-produced a discussion paper State Capability, Policymaking and the Fourth Industrial Revolution: do Knowledge Systems Matter? to identify the key questions we need to ask about the implications of the Fourth Industrial Revolution for policy processes, governance system and the ways knowledge will inform policy decisions. The discussion paper is a concrete step into co-designing a project with partners in different parts of the world. You can find more information here, including the date for a webinar that we are planning in early March.

In the meantime, we have reached out to colleagues in our networks to provide us with a comment and feedback on the paper. 

Today I post the comment from Beatriz Kira. Beatriz is a Senior Policy Research Officer at the Pathways for Prosperity Commission at the University of Oxford where she works with the team responsible for content development, policy analysis, research, and outreach. Prior to joining the University of Oxford, Beatriz worked as a researcher in Brazil, conducting research on a wide variety of topics in the intersection of law and technology, such as privacy and data protection, freedom of expression, regulation of sharing economy platforms, and copyright. She has also acted as a consultant for the Brazilian Competition Authority, providing advice on competition policy and the digital economy.

Here is Beatriz’s comment.

‘This discussion paper explores a pressing question in the developing agenda: what are policy priorities for middle-income countries as they transition to a knowledge economy during the Fourth Industrial Revolution (4IR). The issues addressed in the paper are closely related to our work at the Pathways for Prosperity Commission, a two-year commission on inclusive technology headed by Melinda Gates, Sri Mulyani, and Strive Masiyiwa devoted to making frontier technologies work for the benefit of the world’s poorest and most marginalised people

Recent research published by the Commission identified that global discussions about technology policy are often dominated by a small number of countries and lack a more nuanced understanding of the reality facing developing countries. Advancing towards tech-enabled development, thus, requires stimulating research on the impacts of the 4IR in developing contexts. ‘State Capability, Policymaking and the Fourth Industrial Revolution: Do Knowledge Systems Matter?’ is a welcome contribution to these efforts, exploring opportunities and concerns in middle-income countries.

As the authors highlight, the 4IR is unique in terms of the breadth of its scope. While the pace of change will undoubtedly bring significant change across a wide range of areas, none of the policy implications associated with ‘job destruction’ from accelerating automation are certain. Governments, citizens, civil society organisations, academics and the private sector all play important roles in identifying opportunities for inclusive growth and determining how the transformation will affect our societies and economies. This is in line with our work at the Commission, which has emphasised that there are pathways that developing countries could adopt for future inclusive growth and jobs for people living in poverty.

Grounded in extensive review of recent literature, the paper discusses the importance of strong capabilities in order to enable the use of technology to address policy problems. Indeed, policymakers are often faced with the need to balance various and sometimes competing interests, but they often lack the capabilities to do so. The report also identifies that many of the issues arising from innovation and technological advances are transnational in nature. Harnessing the potential of new technologies for inclusive growth will be a global endeavour, which may require internationally or regionally coordinated responses.

Overall, the paper provides relevant contributions to the discussion on the changes that the 4IR will bring to bear on the economy, human agency, and knowledge systems. As argued by the authors, developing countries face multiple challenges to get digital ready and to adapt policymaking to cope with emerging technologies. Our work at the Pathways Commission echoes this diagnostic. We believe crafting solutions will be a joint effort, gathering multiple stakeholders to co-design country-level solutions for inclusive growth.’

comment 0

Were the seeds of human-centered design being planted by the Bauhaus movement 100 years ago?

I was wondering about this while reading the article by Rowan Moore in the Guardian this morning: Bauhaus at 100: the revolutionary movement’s enduring appeal.

‘The Bauhaus, simply put, was a German school of art and design that opened in 1919 and closed in 1933. It was also very much more than that. It was the most influential and famous design school that has ever existed. It defined an epoch. It became the pre-eminent emblem of modern architecture and design. … It is coming up for the centenary of its founding, which shows both that what was called the “modern movement” is now part of history and that its influence is very much still around us.’

Architect and artists offer some brief testimonies in the article about the influence that the Bauhaus movement has had on their work. The testimonies are all interesting, but the one by architect Daniel Libeskind (My professor was Bauhaus-taught, so I had the real thing), has a paragraph that is quite illuminating and made me think about the question I pose the title:

‘Bauhaus, at its core, is about understanding the world and its wonder. It’s the fact that everything to do with design – from the small to the large to the horizon – is something beautiful and worthy of the word wonder. I think Bauhaus has a strong ethical and political dimension – it strives for equality. It was only later on that it became minimalist and reductivist in its ideas. The true, original Bauhaus was about the eternal human spirit. I think the Bauhaus has a global impact – in Russia, the United States, Japan – because of what it sought to do – to illuminate the world of design with powerful concepts and the notion of beauty. It was to get rid of the junk, the accretions of time, to clear up the environment and see how it could be designed in a way that was not just functional but created a community.’

What do you think, were the seeds of today’s human-centered design being planted 100 years ago at the Bauhaus Meisterhause?

comment 0

Measuring Impact for Development Mutants, An Ongoing Conversation

The original version of this article can be found here on the Pulse Lab Jakarta Medium account.

A Google search for the “Fourth Industrial Revolution” will produce millions of results in a speed that seems like the blink of an eye. These results include books, articles, academic papers, information about conferences and workshops… the works. As one author puts it, this revolution brings with it technological breakthroughs in artificial intelligence, robotics, internet of things, nanotechnology, biotechnology, energy storage, and more.

These new technologies are evolving at a very fast pace and will have a strong impact on our societies. For governance systems, they pose two sets of challenges: 1) governments have to keep up with very fast technological and knowledge advances; and 2) these changes occur simultaneously in multiple areas of the economy and society.

In order to deal with the uncertainty that these changes are already bringing, experimentation is gaining traction in government agencies around the world as a way to test public policy solutions. As a result, new organisations are emerging which are actively involved in funding and testing technology-led public policy experimentation.

Over the past few months, Arnaldo Pellini, Research Associate at the Overseas Development Institute and founder of Capability, has been working with Pulse Lab Jakarta (PLJ), a data innovation lab established in 2012 as a joint initiative via United Nations Global Pulse and the Ministry of National Development and Planning (Bappenas), to explore different methodological approaches to measuring impact for this particular type of organisation. With the goal of systematising this in a discussion paper, along the way he had several discussions with Diastika Rahwidiati, our deputy head of office, on the topic. Here we share some snippets from their conversation:

On Defining the Lab

Arnaldo: We have written together about Pulse Lab Jakarta (PLJ) in a book chapter and now here we are writing about the Lab for a discussion paper. Every time, I found it very interesting to reflect on the nature of PLJ and whether it falls in some sort of organisational category. It is not a project or programme in a traditional sense. It is not an NGO. It is not a think tank. It is not a company. To me, it defies these categories because it belongs to a new wave of organisations whose emergence is closely linked to the digital revolution. The digital revolution demands different ways of working to solve problems. New organisations emerge that can do that: policy hubs, policy labs and data innovation labs. What do you think?

Diastika: I think some of the things that Brian Robertson pointed out in his Holacracybook resonate with us as a lab. In a world where changes happen rapidly and knowledge of those changes are at our fingertips, the conventional “predict and control” model that powers many development organisations and initiatives is woefully inept in dealing with this kind of environment. The whole purpose of setting up PLJ was to continuously experiment with new development insights that can be gained from different sources of data. I feel that this approach of continuous experimentation is not just about how we work, but is encoded into how we are as an organisation: we try things; we ditch things quickly if they don’t work; we learn from our successes and our failures; and we integrate these lesson into how we operate.

In this context, I also like Giulio Quaggiotto’s definition of development mutants: new players in the peripheries of the international development sector that freely borrow across different disciplines and recombine elements, because they are “unfettered by legacy”. Going back to the topic that triggered our conversation, though, this sense that we are more like organisms that continue to evolve rather than a development initiative with pre-defined end-of-program outcomes provides an interesting tension in how we define and evaluate the impact of an entity like PLJ.


Arnaldo: An interesting point about PLJ is also the fact that the lab is under the umbrella of UNDP and receives its core funding from a bilateral donor. Both, UNDP and the funder, have subscribed to various international agreements on aid effectiveness and apply a results-based framework that defines certain accountabilities about achieving development outcomes. On the other hand, as you mentioned earlier, the work of PLJ involves designing prototypes, testing possible solutions to data and information problems. Some succeed, some do not. I think it is difficult for this way of working to fit into a results-based framework. Duncan Green, in his book How Change Happens, has written that a process of testing, failing, learning, re-testing, and (maybe) succeeding is a nightmare for the current donor’s systems. In my opinion, data innovation and technology advances will require more and more these adaptive and experimental approaches. In other words, the accountability systems will need to change and adapt to the development of tomorrow.

Diastika: Love Duncan Green. I very much agree with you, Arnaldo. Lest we forget, PLJ is a joint initiative of the United Nations and the Government of Indonesia. So on top of the accountability requirements of the UN and our donors, we also need to meet those of the Government of Indonesia. It is easy to get lost in the myriad of forms, tables and briefs from three different bureaucracies and perhaps complain once in a while about having to meet these, but as Green advocated in his book, it’s also good to take a deep breath and reflect a bit. What is the basic ask here?

My feeling is that the basic ask from all three institutions is whether the resources given to PLJ is being put to good use. The next step would be to discuss with each organisation what “being put to good use” would look like for them, and what evidence they would require to back this claim. For several months now, PLJ has been discussing with the Government of Indonesia a new set of operating guidelines that would allow us to balance the need for accountability with our inherent need for flexibility and freedom to experiment. We’re at a point now where almost everyone involved feels comfortable with these guidelines, but I think one of the biggest learnings for us was that the collaborative (and at times, super-intense) process of developing the operating guidelines themselves was invaluable in building mutual trust. Through this process, we clarified assumptions, expectations, ways of working, and most importantly, the needs that drive accountability processes.

So, while I wholeheartedly agree with you that accountability systems will need to change and adapt to the development of tomorrow, I think we also need to get off our ‘innovation’ high-horse a bit to recognise that our partners and funders have information needs that need to be met today. I think helping them meet those needs would go a long way towards creating a conducive climate for conversations about putting in accountability measures that are as adaptive as the initiatives they are meant to safeguard.

To measure or not to measure?

Arnaldo: This conversation made me think about Pablo Yanguas latest book, Why We Lie about Aid. In it, he mentions several times Andrew Natsios, who served as Administrator of the U.S. Agency for International Development from 2001 to January 2006, and who, in a paper written for the Center for Global Development, wrote: “Those development programmes that are most precisely and easily measured are the least transformational, and those programmes that are most transformational are the least measured.” The conversation we are having about the meaning of impact of the work done by PLJ, is part of a wider conversation about whether the measurement of development initiatives in general needs reforms and whether the drive to measurability and certainty we have today for all sorts of programmes is actually helpful. I think it isn’t.

Diastika: PLJ has written on its current approach to measuring impact, but I think this will be a challenge that the Lab will continue to explore and grapple with. My feeling is that maybe some of the questions that we are asking about measurement will need to change. For instance, rather than trying to demonstrate how our work has contributed to certain predetermined outcome areas, should we instead reframe this into exploring how we will know whether as a lab, PLJ is effectively expressing its purpose?

In his new book, Principles-Focused Evaluation, Michael Quinn Patton posits some really interesting ideas. To evaluate the implications of innovations and adaptation in complex systems, Patton recommends focusing on principles as a distinct evaluand. Patton’s premise is that principles provide guidance to decisions, choices and actions — and we can evaluate whether they work, whether they are adhered to, and whether they are achieving what we want them to. PLJ probably needs to explore this further as a team, but some of the concepts in the book might be useful in answering the question of whether PLJ is effectively expressing its purpose as a lab.

This conversation between Arnaldo and Diastika is part of a broader ongoing discussion about how we measure impact as a data innovation lab. Based on reflections alongside our partners and stakeholders over the years, we have come to describe our impact through the lens of operational, methodological and ecosystemic contributions (read more here) rather than categorising our work as direct impact in the common sense of quantitative measurements and predefined rubrics. Our Stories of Change reflections on two of our data analytics tools that have been adopted by the Government of Indonesia are also part of the discussion.

Exciting times ahead as we continue to transform as a Lab, as well as experiment in new areas with the needs of our direct stakeholders and the wider public in mind. The conversation on impact goes on, please jump in if you have new insights to share. We’d love to hear from you!