Online card sorting – even better than the real thing?

6 minutes read

“You’re the real thing. Yeah the real thing. You’re the real thing. Even better than the real thing”. Of course the debate continues as to whether Bono was singing about online card sorting or not during U2’s seminal ‘Even better than the real thing’, but assuming he was, was he correct? Is online card sorting really better than the real thing or a poor substitute for face-to-face card sorting? What are the pros and cons of online card sorting and when might you use it (or not use it) for a project? In this article I try to answer these important questions, along with outlining a few online card sorting tools that you might use for a project.

What is online card sorting?

Before we get in to the pros and cons of online card sorting it probably makes sense to outline what exactly I’m talking about. When I’m referring to online card sorting I’m basically talking about card sorting sessions in which participants carry out the card sorting online and at their own leisure (i.e. the sessions are not moderated). In a nutshell you:

1. Define the items (i.e. cards) to be sorted and add them to the online card sorting tool. For example these might be page for a website, items in a menu, different types of content, or even products for sale on an ecommerce site.
2. Define the groups for participants to sort items into (called a closed card sort), or alternatively leave it up to participants to come up with their own groups (called an open card sort).
3. Send out invitations (usually via email or a hyperlink) for people to participate in the card sorting sessions.
4. Sit back (with a smug look on your face) and wait for the card sorting results to start rolling in…

Online card sorting – the pros

So now for the million dollar question: Is online card sorting any good? More importantly, is it better than face-to-face card sorting? Well it certainly has a number of advantages. These include:

• It’s easier to set-up than face-to-face card sorting. You don’t have the hassle of trying to arrange lots of card sorting workshops and of finding a date, time and place that’s good with participants. You can simply set-up the cards (and groups if it’s a closed sort) and let participants carry out the card sorting at their own leisure.
• It’s quicker and easier to get lots of results. Online card sorting allows you to get hundreds of results in a matter of hours. You try doing that with face-to-face card sorting!
• It’s easier to recruit participants for. Online card sorting requires much less effort on behalf of participants. They can carry out the card sorting when they want, where they want and at what pace they want (assuming you don’t set a time limit) so there are really no excuses for them saying ‘no’.
• It’s (usually) cheaper to carry out. By using an online card sorting online tool you shouldn’t have to pay participants so much (if anything at all) for taking part and there are no travelling expenses to reimburse, nor venues to pay for. It’s not all good news though – online card sorting tools are not free (unless you’re undertaking a seriously small study) so you will have to stump up at least some cash.
• There’s no need to input data after the sessions. Online card sorting tools capture all the card sorting data so you don’t have to spend hours after each session meticulously recording the groups formed.

Online card sorting – the cons

Online card sorting sounds awesome, where do I sign? Well hold your horses because there are also a number of disadvantages over face-to-face card sorting. These include:

• It doesn’t deliver as rich feedback. The biggy. Online card sorting is much more geared towards quantitative rather than qualitative feedback. It will tell you what was placed where, but not necessarily why.
• It’s not as natural as sorting real cards. Whereas sorting a stack of physical cards is very natural for most people, doing the same thing online can cause some participants difficulties. You’re very much at the mercy of the online card sorting tool you use and even the best can cause usability issues that might bias the results.
• It’s not as flexible. Face-to-face card sorting is as flexible as you need it to be. Want to use different size or colour cards depending on the type of content? No problem. Want to include a picture and extra information on cards? No problem. With online card sorting you’re limited by the tool you use and whilst these are improving, they often provide very limited flexibility around how the card sorting is carried out.
• You can’t change it on the fly. Once you’ve set-up an online card sort you usually can’t change it. This can be a considerable pain if you’ve forgotten a card or want to change the groups slightly between sessions.
• You’re at the mercy of technology. Face-to-face card sorting never crashes, the same sadly can’t be said for its online equivalent.
• You’ll (probably) need to find money for licenses. All the major online card sorting tools of course charge for their services (although some are free if you’re undertaking a very small study with less than 10 participants) so you’ll need to find money to cover this.

When to use online card sorting

OK so online card sorting has its pros and cons but when is it going to make most sense to use if for a project? Well I’d primarily recommend using online card sorting when you either need to get lots of results for a card sort (i.e. 25+ results), or when you can’t easily set-up face-to-face card sorting sessions. Also don’t think that you have to use one or the other within a project. I’ve often found that it’s best to use a combination of face-to-face and online card sorting. For example, you might use face-to-face card sorting to explore user’s mental models and to come up with an initial set of groups, and then test this using online card sorting.

When not to use online card sorting

Sometimes knowing when not to use tool is more important than knowing when to use it and online card sorting is certainly not a one size fits all solution. In my own experience online card sorting is better for carrying out closed card sorting (i.e. testing pre-defined groups), rather than open card sorting (i.e. exploring possible groups). This is because it rarely tells you why a particular group has been created, or why items have been placed where. For this reason if you need to find out the sort of mental model that users utilise when sorting items into groups, or there’s a lot of uncertainty around how items might be grouped then I’d recommend not using online card sorting.

Remotely moderated card sorting – the third way

Remember I said that online card sorting isn’t recommended if you need to know why users place items into particular groups; well there is a third type of card sorting that might be used instead – remotely moderated card sorting. Remotely moderated card sorting is like a cross between face-to-face and online card sorting. You set-up your card sort using an online or desktop application based card sorting tool (such as xSort for Mac OS, or UXsort for Windows) and then observe (using a screen sharing tool such as WebEx or GoToMeeting) as someone remotely carries out the card sorting on your PC.

Remotely moderated card sorting allows you to almost get the best of both worlds. You can get the rich qualitative feedback of face-to-face card sorting without having to organise face-to-face sessions with users. However, be mindful that you inevitably won’t get the same level of feedback that face-to-face sessions deliver, and that carrying out remotely moderated card sorting with groups is tricky to say the least.

Online card sorting tools

In my humble opinion these are the best online card sorting tools currently out there.

UXtweak

Part of the UX research platform from UXtweak. Free for small studies (up to 10 respondents) or pay a monthly or annual subscription.

Optimal sort

Part of the Optimal Workshop suite of remote UX tools from Optimal Usability. Free for small studies (up to 10 participants) or pay a monthly or annual subscription.

WebSort

Another great remote UX tool from Lime & Chile Productions. Free for small studies (up to 10 participants) or pay for a single study, for 5 studies or a monthly subscription.

UserZoom – UZ Card Sorting Study

Part of the UserZoom suite of remote UX tools. Pay for one study or an annual subscription.

Proven by users

Proven by users is a new online card sorting tool. It’s completely free until Jan 1st 2018 (whilst on Beta) and will offer a free subscription with limited results collection after that.

More about online card sorting

• All About Card Sorting: An Interview with Donna Spencer
• Remote card sorting
• Card sorting resources