Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
Moderated testing can provide a deep personal understanding of the users, but it can also be heavily time and resource-consuming. Although traditional usability testing has been moderated by a researcher in real-time, there are now many available alternatives online. We’ll discuss a few of them along with the tradeoffs of using this method so you can make an informed decision about whether this is the right choice for your project.
New Terms:
Unmoderated testing -- Remote testing that does not require the participation of a facilitator.
Why run an online unmoderated study?
- Fast and affordable
- Scalable
- Alleviate the observer effect
- Other possible biases can be found at Checking Bias During Usability Testing
- Keep participants in their natural space
When choosing an unmoderated testing tool, consider:
- Screen and audio recording
- Customizable recruiting criteria
- Ability to test anything
- Customizable tasks
- Questions
- Speed
- Quantitative Analytics
- Longitudinal studies
Further Reading: Selecting an Online Tool for Unmoderated Remote User Testing
Some tools to consider
- Usertesting
- Userbrain
- UsabilityHub
- Hotjar
Related Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign upRelated Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign up
Although traditional usability
testing has been led by a moderator,
0:00
there are now many alternatives online for
running studies without one.
0:04
Running an unmoderated study can cross
into quantitative research if enough
0:08
participants are included.
0:12
When using these services, you provide
a description of your desired audience,
0:14
the link to the part being tested,
and a simple task list for
0:19
the participants to complete.
0:23
The service then finds participants and
provides them with
0:25
the link to take your study on
their own time in their own space.
0:29
So why would you choose to run
an online unmoderated study?
0:33
They're fast and affordable.
0:38
An unmoderated study takes many of
the usual logistics off your hands.
0:40
No need to recruit and
schedule participants.
0:44
No need for a lab.
0:47
And of course, no need for
a professional moderator.
0:49
All of that is included with a single fee.
0:52
They're also scalable.
0:54
Since unmoderated studies contest with
many participants at the same time,
0:56
it is much easier to scale your study,
whether it is to include a wider
1:01
population or
to test multiple different concepts.
1:06
They alleviate the observer effect.
1:09
In a moderated study, there can
be a tendency by the participant,
1:11
often unconsciously,
1:15
to change their behavior simply because
they know somebody is watching.
1:17
They may also have a natural desire
to tell the moderator what they
1:21
think the moderator wants to hear.
1:26
They keep participants
in their natural space.
1:28
Unmoderated studies allow people
to take part from wherever is most
1:31
comfortable for them.
1:36
The enticing convenience of unmoderated
testing comes at the expense of
1:37
missing out on in-depth discussions.
1:42
Consider this trade-off very
carefully before choosing this route.
1:45
Since usability testing is meant to gather
insight about a person's behavior and
1:49
the motivation behind it, watch out for
unmoderated testing tools that
1:55
only ask about people's experiences
after they've used a product.
1:59
You really want to learn what is happening
during their time with your product.
2:04
Now, let's go over the full
list of requirements and
2:10
nice-to-haves when
choosing a testing tool.
2:13
First, screen and audio recording.
2:16
A screen recording will allow you to see
how people used the mouse to explore
2:19
each screen, where they clicked or
tapped, how long they spent on each page.
2:24
If you instruct them to think aloud and
then also capture the audio recording,
2:29
you'll get an even fuller picture
of how your service is performing.
2:34
If your budget allows for it,
choose a webcam recording as well.
2:40
Second, customizable recruiting criteria.
2:44
Some providers will have an existing
list of people to reach out to,
2:47
which can be filtered by
your audience criteria.
2:52
However, using a pool runs the risk of
working with expert usability testers.
2:55
Find out what the service does
to weed these experts out.
3:01
Maybe they could exclude data from those
who move through the test too quickly, or
3:05
from those who entered invalid comments.
3:09
Ideally, if you have access to your own
list of users, see if you can upload it
3:12
directly into the tool to make sure you're
reaching your exact desired audience.
3:17
Third, ability to test anything.
3:22
Can participants use any device,
computer, tablet, or phone?
3:24
Are you limited to a live website,
or can you upload images as well?
3:31
Make sure that the tool you're choosing
can accommodate the state of your project.
3:35
Fourth, customizable tasks.
3:41
Many services will provide their own
sample tasks for you to choose from.
3:44
Although this might be a good start,
this is not enough.
3:49
With no moderator present, participants
will need very clearly worded,
3:52
product-specific tasks.
3:58
Poor tasks will equal poor data.
4:00
Fifth, timing of questions.
4:03
Most services will offer the ability to
ask questions at the end of the study.
4:06
However, you will also want the ability
to ask questions after a specific action.
4:11
For example, what were you looking for
when you clicked here?
4:16
6, speed.
4:22
Is your project on a tight timeline?
4:23
If so, you can pay more to
get your results back sooner.
4:26
7, quantitative analytics.
4:30
Does the tool allow you to
track quantitative metrics,
4:34
such as task completion or
general usability score?
4:38
This is not required, but
if they do have this ability,
4:42
make sure you know how that general
usability score is being calculated.
4:46
8, longitudinal studies.
4:52
You can choose to continuously run
usability tests on your website and
4:54
receive monthly or
5:00
quarterly reports showing how
performance may be changing over time.
5:01
I can't recommend an exact tool for you,
since the options are always changing and
5:07
your needs will be unique as well.
5:12
However, to get you started,
5:14
I have provided a few examples
in the teacher's notes.
5:15
Good luck.
5:18
You need to sign up for Treehouse in order to download course files.
Sign upYou need to sign up for Treehouse in order to set up Workspace
Sign up