Bummer! This is just a preview. You need to be signed in with a Basic account to view the entire video.
Unmoderated Testing5:20 with Anya Mezak
Moderated testing can provide a deep personal understanding of the users, but it can also be heavily time and resource consuming. Although traditional usability testing has been moderated by a researcher in real-time, there are now many available alternatives online. We’ll discuss a few of them along with the tradeoffs of using this method so you can make an informed decision about whether this is the right choice for your project.
Unmoderated testing -- Remote testing that does not require the participation of a facilitator.
Why run an online unmoderated study?
- Fast and affordable
- Alleviate the observer effect
- Other possible biases can be found at 9 Biases in Usability Testing
- Keep participants in their natural space
When choosing an unmoderated testing tool, consider:
- Screen and audio recording
- Customizable recruiting criteria
- Ability to test anything
- Customizable tasks
- Quantitative analytics
- Longitudinal studies
Some tools to consider
Although traditional usability testing has been led by a moderator,
there are now many alternatives online for running studies without one.
Running an unmoderated study can cross into quantitative research if enough
participants are included.
When using these services, you provide a description of your desired audience,
the link to the part being tested, and a simple task list for
the participants to complete.
The service then finds participants and provides them with
the link to take your study on their own time in their own space.
So why would you choose to run an online unmoderated study?
They're fast and affordable.
An unmoderated study takes many of the usual logistics off your hands.
No need to recruit and schedule participants.
No need for a lab.
And of course, no need for a professional moderator.
All of that is included with a single fee.
They're also scalable.
Since unmoderated studies contest with many participants at the same time,
it is much easier to scale your study, whether it is to include a wider
population or to test multiple different concepts.
They alleviate the observer effect.
In a moderated study, there can be a tendency by the participant,
to change their behavior simply because they know somebody is watching.
They may also have a natural desire to tell the moderator what they
think the moderator wants to hear.
They keep participants in their natural space.
Unmoderated studies allow people to take part from wherever is most
comfortable for them.
The enticing convenience of unmoderated testing comes at the expense of
missing out on in-depth discussions.
Consider this trade-off very carefully before choosing this route.
Since usability testing is meant to gather insight about a person's behavior and
the motivation behind it, watch out for unmoderated testing tools that
only ask about people's experiences after they've used a product.
You really want to learn what is happening during their time with your product.
Now, let's go over the full list of requirements and
nice-to-haves when choosing a testing tool.
First, screen and audio recording.
A screen recording will allow you to see how people used the mouse to explore
each screen, where they clicked or tapped, how long they spent on each page.
If you instruct them to think aloud and then also capture the audio recording,
you'll get an even fuller picture of how your service is performing.
If your budget allows for it, choose a webcam recording as well.
Second, customizable recruiting criteria.
Some providers will have an existing list of people to reach out to,
which can be filtered by your audience criteria.
However, using a pool runs the risk of working with expert usability testers.
Find out what the service does to weed these experts out.
Maybe they could exclude data from those who move through the test too quickly, or
from those who entered invalid comments.
Ideally, if you have access to your own list of users, see if you can upload it
directly into the tool to make sure you're reaching your exact desired audience.
Third, ability to test anything.
Can participants use any device, computer, tablet, or phone?
Are you limited to a live website, or can you upload images as well?
Make sure that the tool you're choosing can accommodate the state of your project.
Fourth, customizable tasks.
Many services will provide their own sample tasks for you to choose from.
Although this might be a good start, this is not enough.
With no moderator present, participants will need very clearly worded,
Poor tasks will equal poor data.
Fifth, timing of questions.
Most services will offer the ability to ask questions at the end of the study.
However, you will also want the ability to ask questions after a specific action.
For example, what were you looking for when you clicked here?
Is your project on a tight timeline?
If so, you can pay more to get your results back sooner.
7, quantitative analytics.
Does the tool allow you to track quantitative metrics,
such as task completion or general usability score?
This is not required, but if they do have this ability,
make sure you know how that general usability score is being calculated.
8, longitudinal studies.
You can choose to continuously run usability tests on your website and
receive monthly or
quarterly reports showing how performance may be changing over time.
I can't recommend an exact tool for you, since the options are always changing and
your needs will be unique as well.
However, to get you started,
I have provided a few examples in the teacher's notes.
You need to sign up for Treehouse in order to download course files.Sign up