Reforming the State After Covid: the importance of evaluation
Last week PERU's Chris O'Leary was on the panel for a well attended webinar on 'Reforming the State After Covid'. Chris suggested that policy failure is inevitable, because policy makers always have imperfect knowledge (there are always gaps in the evidence, the evidence is often contested and/or contradictory, and sometimes is not time or context specific), are boundedly rational (Simon, 1947) and also face issues of fundamental uncertainty (Keynes, 1921) (that is, policy makers make decisions in the absence of information that does not exist at the time). While improvements to policy making can and should be made, policy failure is always going to be inevitable – therefore policy makers need to be better at accepting failure, making policy changes when failure is identified, and learning from failure. Indeed, failure is a necessary part of social innovation, and should be embraced as a policy learning opportunity.
Chris welcomed the renewed commitment to better evaluation, and evaluation to understand what works. But he argued that policy makers need to see evaluation as an important part of policy design, and not something they commission once a policy is up and running. He also argued that knowing what works is great, but we also need to know how, for whom, when, and under what conditions policy works. Most importantly, when evaluations demonstrate that policy doesn’t work, policy makers need to be better at policy termination or policy change. All too often, governments have responded to evaluations that have found no evidence of impact by continuing with, or even expanding, the policy that has been found not to work.