Tackling the unknown: Pragmatic ways to evaluate sustainability

The scientific community has been on a quest to measure sustainability for decades, and the search for appropriate indicators has intensified with the arrival of the Sustainable Development Goals (SDGs). The struggle to identify appropriate metrics reminds us that sustainability is an awfully difficult thing to measure. This is mainly due to two key characteristics of the concept:

First, definitions of sustainability – at least those that still bear resemblance to the origins of the concept – refer to the ability of systems to maintain their major functions over time. If we take this idea seriously, we cannot credibly measure sustainability without anticipating what will happen to system performance in the future. Since it is difficult to take samples of future soils, interview future generations or conduct controlled trials under future climate conditions, many of the classic tools of science are of little use in exploring this key dimension of sustainability.

Second, we have come to realize that assessing sustainability requires consideration of ecological, economic and social dimensions, and that each of these consists of many sub-components, all of which are somewhat complicated and often very site specific. This complexity makes comprehensive measurement of sustainability a daunting task, especially when we hope to measure it with globally valid indicators.

Risks and uncertainty are part of the game

Both of these peculiarities of sustainability – the need to consider the future and complexity of the definition – imply that we’ll have a very hard time measuring sustainability with precision. Since we never have unlimited resources for such an assessment, and we often don’t fully understand the systems we’re dealing with, uncertainties will persist and need to be considered. Because, obviously, there is no way of completely eliminating uncertainty about the future, except to wait for it to happen.

Faced with these challenges, it is helpful to return to the ultimate objective of introducing sustainability as a guiding principle for development: Its purpose is not to assess how sustainable systems are, but to make development more sustainable. This distinction is important because it shifts the focus away from the monumental task of assessing the future performance of often complex systems. Instead, it directs our attention to the need to anticipate the effects of our actions – the impacts of the decisions we make – on long-term system performance.

So how does this make things easier? If we focus on evaluating the impacts of decisions, we can concentrate on the dimensions of sustainability that our decisions are likely to influence, rather than spending time and money measuring dimensions that will not be affected. We now ask whether the ideas we’re considering make the system more or less sustainable, rather than wonder about absolute levels of current and future sustainability. This is often much easier to determine than how sustainable the present system is.

More importantly, a focus on decisions allows us to harness approaches based on decision theory, an age-old discipline concerned with precisely the challenge of supporting decisions under uncertainty. In particular the field of decision analysis – decision theory’s practice-oriented cousin – has produced pragmatic approaches that can be leveraged for making development more sustainable.

Decision analysis for supporting sustainable development

As we’ve recently outlined in Nature’s Policy forum, decision analysis approaches aim to advise decision makers on the most rational course of action when they face decisions involving risks and uncertainties. Since most decisions must be made without extensive accompanying research, this advice is given based on the current state of knowledge, which is distilled from all available sources of information, including expert knowledge. When decision analysts are able to express their understanding of system dynamics and system parameters in ways that allow consideration of uncertainty, they can produce probabilistic models of decision outcomes. Or, in other words, they can make predictions that illustrate the range of possible outcomes, given our uncertainty about the systems we’re trying to influence.

Considerable potential lies in the ability of many decision analysis methods to deal with variables we don’t have much information on. This is often the case for the social or ecological dimensions of sustainability, and it is one of the main reasons for the frequent omission of these dimensions from business models for development interventions. In the absence of hard data, decision analysis gives us the option of estimating, say, that a decision will generate employment for between 100 and 500 people and reduce infant mortality by 5 to 15%. This is pretty imprecise, but it is more useful than reducing business cases to what we have data for and ignoring everything else.

The capacity to work with value ranges, rather than needing precise numbers, also opens opportunities for monetizing non-monetary system outcomes in a way that doesn’t come across as arbitrary and potentially biased. This feature of decision analysis allows us, for example, to consider the reputation gain for a company that supports sustainable land management practices in business models.

Supporting concrete decisions

The most important benefit of using decision analysis approaches for strengthening sustainability is that it actually generates information that helps people make better decisions. It is decisions, after all, that ultimately generate positive outcomes. Decision-specific assessments offer much better guidance to people making these decisions than any studies aiming to quantify the current or future state of system sustainability. They also allow people, through procedures known as value of information analysis, to highlight decision-specific knowledge gaps that should be narrowed to ensure robust recommendations.

These knowledge gaps can sometimes be found in surprising places. For example, analyses have revealed that the business case of a company supplying fresh water to a city in northern Kenya depended to a considerable degree on whether the pipeline project would be undermined by political interference. In this case, the analyses revealed that increased clarity on political risks was a critical information need to be addressed before moving ahead with the pipeline. Similarly, where businesses aim to work with smallholder farmers to establish tree-based value chains, e.g., for fruit or timber production, studying farmers’ time preference – their ability to wait for income in the future – may give a better indication of the viability of the business model than detailed soil and tree performance analyses would.

At the World Agroforestry Centre, an international research-for-development institute in Nairobi, Kenya, we are deploying decision analysis methods in agricultural development, with very promising results so far. We find that such approaches can be instrumental in breaking the indecision that often results from the combination of complexity and data scarcity by instead focusing on what is actually known, and using this information to project plausible decision impact prospects. We believe that large-scale introduction of such approaches into planning of development work, as well as private and public sector projects, could make a huge contribution to progressing toward sustainability.

WLE's partnership with The Economist Events' Sustainability Summit includes a 20% saving to attend the event. Enter discount code ‘CGIAR/DC’ when registering online to take advantage of this saving.