An introduction to pairwise comparison

1 minute read

If I gave you a list of 10 films that you’ve watched and asked you to rank them in order of preference you might struggle no? Mmmm…. I like Police Academy – Mission to Moscow, but then I also like Jingle all the Way and the excellent Mr Nanny. Now if instead I gave you two films and asked you which you prefer you’d probably find it infinitely easier. This in a nutshell is pairwise comparison – a great technique for ranking, prioritising and generally comparing stuff. I learnt about it during the Wicked Workshops workshop at UX People – a great new one day UX conference in London organised by those lovely Zebra People.

The technique is simple but very effective and works equally well if done on your own or as part of a group session. You take your list of stuff and one at a time compare each item with every other item. With our list of movies this would mean comparing the 1st movie with the other 9; the 2nd movie with the other 8; the 3rd movie with the other 7 and so on. This means that for a list of 10 items you make 9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1 comparisons or 35 in total. Now that might sound like a lot but don’t forget each pairwise comparison should be relatively straight forward so it actually takes no time at all. Each time an item wins a comparison it gets a point. Tot up an item’s points and you have its overall score. This of course is a relative score (it doesn’t really make much sense on its own) but importantly tells you how that item compares in relation to the others. For example, totting up our movie scores and listing them highest to lowest will give us our ranked list of movies in order of preference (which will undoubtedly be headed by the Oscar worthy Police Academy – Mission to Moscow). Now you can’t just rank movies using this technique – you can literally rank anything. You might want to compare user stories, business requirements, personas, job applicants, visual designs or even competitor sites.