Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
Moderated testing refers to a real-time discussion between the researcher and the participant. It can be in-person in a lab, in the actual usage setting or done remotely online. This video will discuss how to choose the right setting to answer your user research questions.
New Terms:
- Moderated testing -- A test during which a moderator is present to ask questions in real-time in order to deeply understand each choice the participant makes.
Advantages of a moderated study
- Probe deeper
- Capture participant reactions
- Protect sensitive information
- Evaluate low fidelity concepts
- Stay on track
- Test complex products
Research Tools: List of tools available to record, observe, and analyze the final video
Traditionally, usability
tests have been moderated.
0:00
A moderated test just means that
there's a moderator present asking
0:03
questions in order to deeply understand
each choice the participant makes.
0:07
They may be sitting in your office and
using your prototype, or you may go out
0:12
and observe someone test your product
whatever you intend for it to be useful.
0:17
Alternatively, moderated
testing can be done remotely.
0:22
There are many remote testing services
that allow you to stay in the office
0:26
while observing and holding discussions
0:30
with people using your service from
the comfort of their own home.
0:33
So what are the advantages
of a moderated study?
0:38
Probe deeper.
0:41
When you're with
a participant at each step,
0:42
you can ask what they're thinking and
why they've made a particular choice.
0:44
For example, why did you click here, or
did you see this instructional text?
0:50
Capture participant reactions.
0:54
Did you notice that sigh or
look of confusion?
0:57
You can add that to your
notes as additional data.
1:00
Without someone there,
nonverbal reactions can easily be missed.
1:03
[SOUND] Protect sensitive information.
1:08
Say you're working on a product
that really shouldn't
1:11
be revealed to the public until much,
much later.
1:15
By moderating, particularly in person.
1:18
You've limited any exposure to the highly
vetted group of study participants,
1:21
who've all ready signed
an non-disclosure agreement, or NDA.
1:26
[SOUND] Evaluate low fidelity concepts.
1:30
If testing early and possibly even with
just a paper prototype, you will need
1:33
to be present to guide the participant
on what features are functional.
1:38
[SOUND] Keep on track.
1:43
If you notice the participant is
focusing somewhere irrelevant to
1:44
the study's objectives, you could
politely guide them back to the task.
1:49
Test complex products.
1:54
When testing products such as business
software, you will need to make sure
1:55
that your participant has the background
to complete the required test.
2:00
And in addition,
2:05
you will likely need to provide clarifying
information throughout the study.
2:07
Let's say that your study is
the perfect candidate to be moderated.
2:12
For example,
you have a click through prototype for
2:17
your complex business software.
2:21
One next question is, will you be
running this study at your office,
2:23
in the wild, or remotely?
2:28
A moderated study needs
to be in real time but
2:30
you do not need to be in the same
physical space as your participants.
2:33
If you do choose to bring the participants
into your office, you will need to make
2:38
sure your office is set up in a way
that will make people comfortable and
2:42
limit distractions.
2:46
Ideally, this would include a dedicated
space in the form of usability lab.
2:47
Let's take a look at the key
attributes of a usability lab.
2:53
First, you need a room with a closed door
to minimize distractions during the study.
2:57
Inside the room,
you'll need a computer with a monitor.
3:04
An external monitor rather than
just a laptop is preferred, so
3:08
that it's easier for you and
the participant to see the screen.
3:12
Attach a camera to the monitor
if it doesn't have one already,
3:16
to record the study.
3:20
And confirm that the microphone
in your computer is working.
3:21
Make sure there's a keyboard and
standard mouse available as well.
3:24
There should be at least
two chairs in the room.
3:29
Place the chair for the participant
directly in front of the monitor.
3:31
And place your own chair slightly behind,
and next to the participant's chair.
3:35
You can also include an additional chair
for a note taker towards the back.
3:40
The key component of usability study
is also being able to invite outside
3:45
observers.
3:50
You can do this in a couple of ways.
3:51
One is to have a couch or
row of seats in the back.
3:53
However, I believe this can be
intimidating for the participant and
3:57
prefer to put the observers in
a totally separate conference room.
4:01
And stream the study into that room.
4:06
The observers would then be able to hear
the audio, see the product being used,
4:09
and if possible also see
the participants facial expressions.
4:14
Make sure to set up a messaging
system between the usability lab and
4:18
the observation room, so
that the observers can text, email, or
4:22
chat their questions throughout.
4:26
You may need specialized
software to record, observe and
4:28
analyze the final videos.
4:32
You can find the list of
options in the teacher's notes.
4:34
Remember to compensate your
participants generously.
4:37
They're taking time out of
their day to commute to you and
4:40
to provide you with the valuable
feedback that you need.
4:43
The exact costs will vary by location,
but a rough guideline,
4:46
if you're in a metro area and asking
people to come in for about an hour,
4:51
anticipate providing $100 per person.
4:55
Testing a product in person is great for
keeping things confidential, and for
4:58
testing physical hardware
when you space design.
5:03
However, you will be limited to
the people living close to your office.
5:06
Now, imagine designing a product meant
to be used by everyone in the world,
5:12
as I did there in my time at Google. How representative
do you think our research
5:16
would have been if we tested with
just people in the US, or California?
5:21
Or those in Silicon Valley willing to
commute to our office in Mountain View?
5:26
We'd be left with a group of early
adopters, tech enthusiasts, and
5:31
frequent usability testers.
5:36
Not exactly diverse representative group.
5:38
Now, let's go back to the usability
lab setup we just described, and
5:42
adjust it for a remote testing study.
5:46
Remove an extra chair and take a seat
in front of the monitor yourself.
5:49
You can use remote testing software to
allow the participant to take control of
5:54
your screen, and use the prototype
available in much the same way as before.
5:59
Alternatively, you can take
an available conference room and
6:03
run the session straight from there.
6:06
If you include outside observers,
they can then join you in the same room or
6:08
from their own desks if
the session's being streamed.
6:12
Remote testing is rarely seeded for
6:16
hardware products, paper prototypes or
highly confidential material.
6:18
Mobile devices can also be tricky,
6:22
depending on the kind of hand gestures are
important, or if an install is required.
6:24
However, remote testing is
a fantastic solution for
6:29
the online products you're
most likely to be working on.
6:32
Remote testing is convenient.
6:36
You don't need to leave the office,
and it provides access to a much,
6:38
much wider range of participants.
6:43
While working on advertising
software at Google,
6:45
we relied heavily on
moderated remote testing.
6:48
Because of this,
6:51
we were able to test frequently as we
developed new features and products.
6:52
However, it was also important to
occasionally visit our users in their
6:57
workspaces to understand the greater
context in which our tools are being used.
7:01
In one site visit, we're able to
discover that clients would use
7:06
an additional monitor that
displayed other software,
7:10
such as spreadsheets to make up
the features that our product was missing.
7:13
A very insightful observation that would
have been impossible to discover remotely,
7:18
or in a lab.
7:23
You need to sign up for Treehouse in order to download course files.
Sign up