Conducting Cart Sorting
The best method of categorizing information and creating an optimal navigation structure is card sorting. It involves asking potential users to group some cards with page titles or terms into categories and, often, to name each category. It helps to understand what types of information users see as related, where they expect to find some particular information and the terminology they use.
There are three types of card sorting:
- Open: Participants create category names by themselves, as a result, designers can understand what terminology they use.
- Closed: Participants are given a set of category names for grouping cards. It is useful when making changes to an existing structure, otherwise, open sorting is preferred.
- Modified Delphi method: Participants work one after another, refining a single model. The first participant sorts the cards into categories, each subsequent participant starts with what the previous participant has created and can either modify the work or start over and create a different organization. The process goes on until participants no longer make any significant changes. This method makes the analysis of results much easier, since the outcome is clear – a structure that all participants like. However, there is a risk that some outlier participants might keep starting over and compromise the whole study, experienced facilitators are needed for using this method.
There are several different card sorting techniques
- One-to-one sessions: One participant sorts cards at a time, the participant is encouraged to think aloud, the facilitator can ask questions if needed. It helps to understand the reasoning behind participants’ sorting decisions, however, it takes significantly more time since each participant needs to be greeted and introduced to the project and the requirements individually, as well as observed, debriefed. Also, some participants take much more time to sort cards than others, thus, there needs to be a substantial time reserved between sessions in case of that.
- Concurrent sessions. Multiple participants engage in card sorting at the same time, however, they sort independently, they are just in the same room. It saves a significant amount of time, however, does not provide any understanding why they group cards in a particular way, also, and many sets of cards need to be created.
- Group sessions. Participants sort cards as a group. It is fast, however, group dynamics can come into play and the result might reflect the reasoning of only a few participants, thus, an experienced facilitator is needed for moderating it. Group sessions provide less data than individual sessions.
- Remote, online sessions. Participants sort card remotely using some card sorting software. It saves time, allows recruiting participants at multiple locations, allows having a very large sample, it is easier to analyze results. The downside is that it does not allow understanding participants’ reasoning, also, when no facilitator is physically present, participants tend to be less diligent and concentrated, they might be talking to somebody while sorting or keep checking email, thus, big samples are needed to balance the variability.
Tools for online card sorting:
- OptimalSort: https://www.optimalworkshop.com/optimalsort
- SimpleCardSort: http://www.simplecardsort.com/
- UsabiliTest: http://www.usabilitest.com/
In person or remote? Generally, remote online card sorting is recommended when you either need a lot of results (25+) or you cannot easily set up face-to-face sessions. Being fully in control and flexible (answering participants’ questions, slightly modifying things on the fly when you see a need for it), also having results not affected by potential usability issues of a sorting software compensates for the time spend doing face-to-face sessions and manually entering results for analysis.
Basic card sorting script:
- Write each page title or a name of a function in the system (with a short description when needed) on a sticky note or a card.
- Invite some representative users to a testing session (if using remote testing, send them a link).
- Ask the participants to sign an informed consent document that informs them about how their data is going to be stored and used.
- Explain them what they will have to do and that there are no wrong or right answers.
- An optional, but recommended step: create some simple practice cards, so that participants can understand what is required from them by doing it. They could be as basic as 10 cards with different types of products: clothes, food products, office supplies – something very easy to group.
- Give the participants the cards created in Step 1 and ask them to arrange them into logical groups. If doing open card sorting, give them some blank cards and ask them to name each group. If a card does not fit into any group, participants should set it aside. If too many groups are created, encourage participants to combine them. You could inform them about the constraints you have, e.g., more than 10 categories would not fit on a screen.
- Analyze the results.
Number of participants: It is recommended to recruit 15 participants for face-to-face sorting, if your funding allows more, then 30-40, however, after the 15th each participant has diminishing returns, thus the costs are often not justified. More participants are needed for remote testing, since there is a risk that some of the participants might be distracted or rush to finish the task.
Number of cards: It is a good practice to limit the number of cards, especially for open sorting, ideally, there should be less than 50-60 cards, otherwise it might cause fatigue.
Time: It depends on the number of cards that need to be sorted, it also varies depending on a participant’s familiarity with the terminology used and the level of effort. It is likely to take around 20 minutes for 30 cards, 30 minutes for 50 cards and 60 minutes for 100 cards. Inform the participants about the estimated duration, so they can better gauge the required effort.
Cart Sorting analysis
It is important to bear in mind that cart sorting should only guide the organization of a navigation structure and labeling, the results should not be blindly translated into a navigation scheme.
Remote sorting: Each remote sorting software has some tools for analyzing results, you should consult the instructions.
Analysis of a very small number of cards: If the number of cards is very small, patterns are often visible just by looking at the sorted cards, there might be no need for complex analysis.
Analysis of a larger number of cards: You will have to put the results on a spreadsheet and analyze them. Often it is not worth doing everything from scratch, there are some templates for it.
A good free template for Excel can be found here: http://boxesandarrows.com/analyzing-card-sort-results-with-a-spreadsheet-template/ . The web page also has a step by step tutorial how to analyze results using the template.
If you are familiar with statistics and R language, there is a great tutorial how to use R for analysis: http://uxpamagazine.org/remote-unmoderated-card-sorts-for-free/
If you wish to create a spreadsheet by yourself, there is a good step by step guide here http://www.uxbooth.com/articles/open-card-sort-analysis-101/ .
There are some paid software as well, for example Syntagm (Windows based) : http://www.syntagm.co.uk/design/cardsorting.htm.
Recommended steps for analyzing card sorting results:
- Photograph the sorted cards after each participant finishes sorting. Alternatively, use the numbers on the cards to quickly record which cards were clustered together.
- Analyze qualitative information, if any is available, for example, participants’ comments, observations.
- Analyze quantitative information: which cards appear together most often and how often cards appear in specific categories – use the spreadsheets discussed above.
- Make your decisions. Based on how often cards appear together, decide where to put corresponding content, also, when to duplicate content (in situations when one half of participants placed a card under one category, the other half under another).
Conducting Tree Path Analysis
Tree testing is sometimes called reverse card sorting because it involves finding items instead of placing them into a navigation structure. It is useful when you have created a navigation structure (either by yourself or based on card sorting) and want to test it. Since the aim is to find problems with “findability”, grouping and labels, the navigation structure being tested should not be presented as a styled menu.
Basic tree testing script:
- Invite some representative users (same recommendations for a sample size as for the standards card sorting).
- Ask them to sign an informed consent form and introduce them to the task.
- Optional, but recommended: give them a very small practice task.
- Give each participant a list of categories (top level menu items) and a set of cards with tasks written on them (e.g. “You want to check your account balance, where would you click?”).
- Ask the participants to place the cards under the menu items where they would expect to find the information. If the navigation is multilevel, once a participant clicks on a top level item (e.g., “Products”), present him with a lower lever menu (e.g., “Kitchen Products”, “Outdoor Products”).
- Analyze the results. A spreadsheet could be used: the navigation structure written out in the leftmost columns, tasks at the top, one column per task. Add the number of times participants chose a particular menu item for a task. Calculate the success rate (the percentage of participants who found it correctly).
There is some paid software for doing tree testing and analysis, for example Treejack (https://www.optimalworkshop.com/treejack).
Another option is UserZoom Tree Testing: http://www.userzoom.co.uk/software/tree-testing
Semantic matches and faceted navigation
Sometimes superficial similarities between cards make participants group them together, even though they do not belong together, usually it is because these cards share a common term, e.g. “Manage”. In order to avoid it, you should modify the terms on cards to remove the words that might cause cards being grouped together, for example, instead of having “Manage appointments” and “Manage my account”, rename them to “Appointments” and “My account”.
You might find that there are no common groups of items – different participants group cards in very different ways. That means that there is no single dominant way to organize the information and there should to be several ways of accessing the same information. In these situations faceted navigation can be used, which is providing visible options for clarifying and refining queries, for example various filters, keywords. If it is a computer shop’s website, users could browse computers by brand, by type, by screen size.
Epicurious (www.epicurious.com) recipe website is a good example of a faceted navigation:
Trigger Words
Trigger words are words that make users click on links. They provide motivation to continue by ensuring users that they are on the right track to achieving their goals. Good navigation links use trigger words that users use, since it makes it easier for them to find relevant content.
In order to discover the trigger words you should use, consider interviewing potential and existing users, e.g. by doing field visits – see what words they use to describe various aspects of the domain you are designing. Focus on the words they use spontaneously. During card sorting users can be encouraged to change the words on cards to something they prefer.
