What is your usability context?
Creating a truly useful and usable product requires a full understanding of a particular target user group: their needs, expectations, struggles, the level of knowledge and the context of use. Some qualitative data needs to be collected and analyzed in order to make informed design decisions.
Performing a Context Inquiry through practical field visits
Just asking users what they need (surveys, focus groups) gives only very limited results, because users often do not know what would be the best for them, sometimes what they say they do is not necessarily what they actually do, also, the reality is often more complicated and messy – there are many factors that influence users’ behavior. The complexity can only be understood by observing them in their natural environments.
Basic contextual inquiry script:
- Greet the participant, introduce yourself and, if it has not been done before, the purpose of the study. Make it clear that you are there not to investigate how well they perform tasks, but to see what could be improved in a system.
- Give a consent form to read and sign (two copies – one for you, one for the participant to keep).
- Observe the participant’s environment, note all the things that are unexpected and relevant.
- Conduct an interview – ask the questions you wanted to ask the participant. It is a good practice to audio-record it.
- Observe the participant – switch to the master-apprentice mode (will be discussed later) and let the participant guide you through his work. Ask for clarification and explanation when needed, however, if you have some questions that might distract the participant from the task and are not related to what he is currently doing, note them down and leave them for later.
- If you have noted down some questions, ask them.
- Thank the participant, give some small ‘thank you’ gift, if planning to do that, for example a chocolate.
- Take some photos of the participant’s surroundings.
- Summarize your notes.
- Get the recording transcribed, ideally, by a professional.
- Analyze the results, create Affinity diagrams.
Step 1, Observing your user behaviors
Contextual inquiry is semi-structured interview method that allows a combination of observing users interacting with technology in their own environments and asking questions: both predefined and naturally arising. Instead of inviting users to a research lab, the researcher visits them (field visit).
Setting: Since the context is the key, observations have to take place where the action takes place: an office setting, participant’s home, on a train during the morning commute.
Participants: Observe and interview only real users. A good practice is to interview five people of each user type; if you do not know the user types, interview 10 and see what you get, you are likely to see different types emerge. Diversity is important, however, not that much in terms of age, sex, education etc., but in terms of use circumstances, roles, technological expertise.
Duration: Usually an interview and observation session takes 60-90 minutes, so realistically 4-5 can be done in a day.
Focus: It is easy to get side-tracked when observing users since there are always a lot of interesting behaviors to observe. Therefore, it is important to agree on a research focus and create a focus question at the beginning, for example, “I am going to research [activity] in order to [design a system]”.
It is a good practice to work with the project team to determine the focus question. The team members could write down the questions they would like to ask the participants on sticky notes, then arrange them into groups and vote to select the most important question group, which when becomes the articulation of the focus question.
It is important not to turn the suggested questions into a long list of interview questions – it is not suitable for a field visit.
Relationships: Establishing the right type of relationship model with participants is important. Often researchers are seen as experts evaluating how well participants perform tasks, which makes their behaviors less natural. To avoid that, it is a good practice is to treat a participant as the expert and adopt a master-apprentice relationship, researcher being an apprentice observing how the participant does his job.
It is important to build rapport and always continue the process of building a trust and understanding based relationship. An effective way of doing it is matching the participant on both verbal (body language, movement, gestures) and non-verbal level (volume, tone, pitch, word choice). Developing a real rapport is seeing the world the same as the participant.
It is important to make participants feel understood and valued through emphatic responses, for example giving a participant a summary of what you think he thinks or feels without judging: “You feel…because…” , “It seems as if…” . Besides relationship building, it helps in validating observations.
Questions: It is important not to ask open-ended questions, such as “Tell me about the last time you…”, “What wastes your time” and similar. Good questions gather context and collect details, e.g. “What do you do when you first sit down at your desk?”, “Who does this department report to?” ; probe on what is not said, e.g. “Why do you laugh when you mention X”; and create contrast to uncover frameworks and mental models, e.g. “What is the difference between doing X and Y?”, “Do other colleagues do it that way?”.
It is important not to get obsessed with asking questions and spend time observing too – finding a right balance is a skill researchers develop with time.
Listening: “Hearing” and “listening” are different activities – listening is an active process and takes into account what is said non-verbally (body language). A researcher needs to become genuinely interested in the participant’s thinking, listen for the meaning behind words, demonstrate interest (verbally and through body language) to encourage the participant to speak more.
Interpretation: It is crucial not to jump into conclusions why something was done in a specific way based on the researcher’s understanding, it is important to ask a participant for a clarification.
Also, inconsistencies between what a participant says and does need to be investigated, for example when a participant carries out a task in a different way than he has described. It is important not to sound accusatory, instead, stimulate the discussion to try to understand more about the situation.
What to pay attention to: Observe the real world to see the challenges – a weak Internet signal, users not interested in reading the documentation, asking others for help etc., these things cannot be discovered using other research methods, but need to be taken into account when designing.
Common obstacles and biases to bear in mind: Firstly, it is your perceptual bias – the assumptions and beliefs that affect what you hear. For example, you might think that a product is too complicated and seek affirmation ignoring the evidence to the contrary. The second common obstacle is that you find some people easier to relate to than others. It is important to build self-awareness of why you feel at ease with some people rather than others to help you become less judgmental, to stay emphatic and to ensure that you do not demonstrate disagreement or criticism.
Step 2, Recording and note taking your observations
Notes: The key thing to bear in mind when taking notes is that observation ≠ interpretation. Observation is what a user says or does, interpretation is what you think is the cause. It is very important to distinguish that in the notes.
Notes have to be taken, even if you are audio recording the session, since they help in capturing the behavior and the context, as well as the things that need clarification.
It is important not to write everything the participants say or do, it distracts them and you can also miss important behaviors by looking at the notepad most of the time. Note taking should be balanced with eye contact and other signs that you are are attending.
If not sure what to note, you can try the ‘AEIOU’ method: note Activities (primary activities, action words), Environment (where activities take place), Interactions (tasks, people who collaborate in activities), Objects (physical objects, software), Users (goals, capabilities etc).
Audio recording: It is a good idea to record the sessions, as it is impossible to make notes of everything that has been said, also, notes miss the intonations. Pause the recording if sensitive data is discussed. It is also a good idea to get the sessions professionally transcribed, although it is not cheap.
Video recording: It works well in large anonymous places, e.g. production plants, but less well elsewhere – in an office space you need to get a permission, also, it might affect the behavior of participants by causing anxiety, besides, there are many potential confidentiality issues – participants might have personal data on the screen. These issues need to be taken into account when considering whether it is suitable in your situation.
Photos: Take photos of everything that shows a participant’s context: the layout of the entire office, the natural surroundings of the participant (other people, objects), also, some photos of the participant interacting with the object you investigate. Sketches are also good if a permission to take photos is not given.
Photos or sketches are important in avoiding preconceptions when communicating the findings to other team members and in recalling different elements of the user’s environment.
Summary of observations: Have a break after observing each participant (around 15 minutes) and summarize your notes – when you revisit them several days later, it will be hard to remember what exactly you were thinking when you took them, also, participants will start blending into one another. You can create and use a standardized form for that, an example below:
Step 3, Mapping user stories and Affinity diagramming
Before starting any formal analysis of the results, it is important to get familiar with the data, read all the transcripts, notes, look at the photographs and try to put all observations into context. Affinity diagramming could be used to cluster the observations, themes and interpretations.
Affinity diagramming: It is a technique used to sort large amounts of data into several logical groups. You can put each observation on a sticky note, then stick all the notes on a wall and invite the team members to group them into logical groups and themes. The outcome could be presented as a user experience map.
User experience maps: They are sometimes called user journey maps and user story maps( it is important not to confuse them with agile user stories, these are two very different concepts). User experience maps show a sequence of steps users take to achieve their goals and various observations related to them. In order to create an experience map, sticky notes with observations should be grouped into common tasks, the tasks should be organized into vertical columns, tasks occurring early in the process being shown first.
Optionally, tasks could be grouped into named phases
Below are some examples of professionally designed user experience maps, however, they do not necessarily need to be beautifully done, they just have to communicate the context of use, the users’ flows, their needs, pain points, expectations etc.
Step 4, Presenting your findings as empathy maps and story board
User journey map is just one (very good) way to present results, there are some other useful ones.
Empathy maps: The team members should try to get inside the minds of the users, think about their sensory experiences, pain points and arrange the sticky notes with observations accordingly. The template below could be used.
This activity is very useful for creating user personas (discussed in the next lesson).
Story boards: They are visual representations of the process of a user interacting with a system or getting some tasks done. They are more exciting to look at, thus, more stakeholders will look at them attentively.
Best Practices for Guerrilla User Research
In some situations you might not be able to do a contextual inquiry. There are some other techniques, which are not replacements for field visits, but compromises in situations where field visits cannot be done.
Diary study: Participants can be given a diary or blog to record their thoughts and experiences with a particular system or getting particular tasks done. They could also take some pictures with their camera phones.
Screen sharing: If the main context is a user’s computer screen, they can share the screen with a researcher and give a tour of their desktop. Good tools for that are Skype (free) or GoToMeeting (paid). Also, they could use a computer camera to show their office or home environment. Only in very few situations the entire context is limited to the computer screen, so this method is not a full replacement for a field visit.
Pop-up research (also called guerrilla research): speak to people nearby and ask them to take part in a short study. You can do it in a coffee shop or a library. It works only when the target audience is very wide.
Other ideas: There are some other ways, for example reading online forums and seeing the information the potential users share, what problems they have, asking questions. You can also listen in customer service calls as well. Or you can act like a spy and pretend to be looking at something else, e.g. observing people in a retail store and listening what features they talk about with their friends.
Empathy map template:
