Trust, pertinence, validity and honesty in measuring peace

by Helena Puig Larrauri

How do we even start to think about how to measure peace? I don’t mean globally, ranking countries as the Institute of Economics and Peace does with its annual Global Peace Index. I mean within a particular context, and I ask because it’s a critical step to many peacebuilding projects. How can we know when peace is returning or tension building? How do we follow a trend of conflict and peace that is granular and regular enough to tell us something about what interventions could help to build peace?

Over the past few years, I’ve tackled this question in a number of different contexts and have slowly built up a way to think about measuring peace. A few months ago, I was pointed to an article by Hans Freudenthal and another by Luis Puig (both written on education research) that sharpened this thought process. This post lays out the four steps I use to think about measuring peace. I’ve explained the steps in the order they are easiest to understand, but this is the reverse of the order in which I would use them to build or discuss a specific measure of peace (that’s why they are numbered backwards).

I should say at the outset that I’m not going to discuss measuring the impact of peacebuilding interventions in this post. I also don’t explore the specific steps to building a peace index, or barometer, or early warning system. It’s a more conceptual discussion about what it means to measure peace, but one that I hope may be of use to other peacebuilding practitioners.

Step 4: Is the measure trustworthy?

The starting point of many academic research papers is an explanation of the statistical and research best practices that ensure the outputs of a particular survey are trustworthy. Random samples or clustered samples or stratified samples. Field tested questions and list experiments. The compromises made (e.g. to randomness or completeness) as a result of operational challenges; the resulting biases. All these explanations are important to a robust measure of peace, but we musn’t be fooled that they are sufficient.

Step 3: Is the measure pertinent?

There’s also the question of whether the measure is pertinent: is it worth measuring what we set out to measure? In other words, is there an epistemic or practical reason to measure this definition of peace? Can we do something with it — change policy, take action, influence public opinion? I have come across many conflict monitoring and early warning systems that fail on pertinence. A common failure is the desire to create a composite index that aggregates many variables into one measure of peace. These composite indices are problematic when it comes to policy or action for two reasons. First, the method for aggregating variables introduces assumptions that are not always transparently communicated. Second, a composite variable is too vague to inform concrete actions. It might help us prioritise “less peaceful” areas, but then what do we do in those areas?

Step 2: Is the measure valid?

The validity of a measure asks whether we are really measuring what we say we are measuring. Many research reports will limit this to “external validity”, in other words whether the conclusions of a survey can be extended beyond the sample of people surveyed and apply to a broader population. Within reason, you can get external validity if you define your population and your sample correctly.

But that’s too narrow a version of validity. There is a deeper question that can be easily ignored: are we really measuring peace by asking the questions in this particular survey? Do these indicators really make up “peace”? To get to the bottom of this issue of validity, we can ask two further questions. First, is peace defined by actions or by perceptions? Measures of the prevalence of violent actions or actions that could lead to violence can be a measure of peace. But they are incomplete: peace is also in people’s heads, in their fears, in that which constructs their vision of reality. Which leads to the second question: is there any such thing as “real” peace outside of what people perceive to be peace? In other words, is it more valid to measure reality (less violent incidents) or perception (people feel safer)? There is of course no right answer here, it depends on whose peace we are measuring.

Step 1: Is the measure honest?

At the core of the difficulty with measuring peace is a dilemma that spans all the social sciences: measurement is a game of truths. The rules that make possible the distinction between true and false in social science are historically constructed within a particular discourse. And it’s a snake that bites its tail: these measurement rules make that particular discourse look like the Truth, a truth we must submit to without questioning the mechanism by which the rules came about.

Peace is inherently subjective because it deals with subjects and their subjectivities. The trick is to realise that every discourse about peace, every measure of peace, affects what we think peace is. Measuring peace defines a new discourse of peace, and affects how we see ourselves and our reality. Trustworthiness gives us a measure based on empircal data. Pertinence turns the data into results. And validity is the theory that underpins our data and results. But what makes one theory better than another?

In my view, the only way to honestly measure peace is to do so in a way that is participatory, that incorporates many voices in the measurement process, allows opportunities for contestation, and never pretends to present an unchangeable truth.

Build Up transforms conflict in the digital age. Our approach combines peacebuilding, participation and technology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store