Do-it-yourself Market Research with Card Sorting

Pick a card!
suedhang/Image Source/Getty Images

Card sorting approaches tap into consumer's mental models of space. It is important to know how consumers view information space when designing applications for the web and particularly for mobile platforms. Consumers typically don't anticipate website organization the same way that designers do.  Information architecture needs to reflect how users view content.

What Does Card Sort Data Tell a Market Researcher?

Everyone has a way of approaching visual display and reading content that impacts how efficiently and meaningfully they are able to interact with the information.

Consumers have their own ideas about how websites and mobile apps should be organized. They can’t help themselves. It is a fact that the human brain likes patterns. But the human brain is also influenced by experience and quick to solidify what it retains – learns – from experience. Add to that the remarkable differences in human brains that seem to be inherited. For instance, grandmother’s artistic ability shows up in several generations. Or a thread can be traced from generation to generation as offspring exhibit extraordinary musical talent.

Market research techniques as simple as card sort activities can help to tease out what works for certain consumers and what does not. Card sort activities can reveal a great deal about how consumers conceive space and how they navigate through websites and mobile application interfaces. Recruit enough consumers – a representative sample – for a market research study, and you can be reasonably sure that you are not designing over important consumer orientations.

Card Sort Data Is Both Quantitative and Qualitative

The data gathered from a card sort activity is both qualitative and quantitative. Qualitative data is obtained by listening to the comments that the study participants make as they explain why they are sorting the cards in a certain way. Understanding why participants place cards in certain piles can help to make the mental models used by participants more transparent.

The insights derived from the participant's explanations and comments are necessarily deeper than the quantitative data obtained.

The quantitative data obtained in a card sort activity is based on similarity scores. Similarity scores measure the categorical matches that are observed when the participants’ responses are first compared by item pairs and then aggregated. For instance, if every participant sorts two particular cards into the same category or pile, then those cards are same to have 100 percent similarity. Alternately, if half of the participants kept the two cards together when they sorted them, and the half of the participants were to put the two cards in different piles – separating the cards – then those two cards are said to have a 50 percent similarity score.

Researchers vary in their recommendations about the number of participants to include in a card sort activity. Tullis and Wood (2004) recommend using 20 to 30 participants in order to obtain a correlation of similarity scores of about 0.95. That is a very strong correlation. According to Jakob Nielsen - and depending on the nature of the study - a good sized sample for a card sort activity is 15 participants.

What Are the Steps for Conducting Cart Sort Market Research?

Card sort methods are completely low-tech and do-it-yourself friendly. Card sort methods do not use technology. Card sort methods use ordinary paper cards, like index cards.

  1. Begin with a stack of index cards or other paper cards.
  2. Write a label or short description on each card that reflects the main theme. If images are pasted onto the cards, the label should align with the image.
  3. Shuffle the cards and give the deck of cards to the research participant. Care should be taken to select the study participants according to best practices and conventional sampling procedures.
  4. Ask the study participant to sort the cards into piles by placing the cards that belong together into the same pile. Be sure to tell the study participants that they can make as many or as few piles as they like according to their ideas about categorization.
  1. This is an optional step. For more detailed insights, study participants can be asked to rearrange their piles into larger groups.
  2. This is an optional step. To gain more information about study participants’ mind mapping orientation, ask them to name the larger groups and each of the smaller piles. The outcomes of this last step can inform word and phrase choices for headlines, links, navigation labels, and for search engine optimization.

Sources:

Nielsen, J. (2004, July 19). Card sorting: How many users to test. Alertbox. useit.com

Tullis, T. and Wood, L. (2004). How many users are enough for a card sorting study? [Proceedings from UPA 2004 on June 7 - 11, 2004 in Minneapolis, Minnesota].