Unmoderated Testing

Although traditional usability testing has been led by a moderator, 0:00 there are now many alternatives online for running studies without one. 0:04 Running an unmoderated study can cross into quantitative research if enough 0:08 participants are included. 0:12 When using these services, you provide a description of your desired audience, 0:14 the link to the part being tested, and a simple task list for 0:19 the participants to complete. 0:23 The service then finds participants and provides them with 0:25 the link to take your study on their own time in their own space. 0:29 So why would you choose to run an online unmoderated study? 0:33 They're fast and affordable. 0:38 An unmoderated study takes many of the usual logistics off your hands. 0:40 No need to recruit and schedule participants. 0:44 No need for a lab. 0:47 And of course, no need for a professional moderator. 0:49 All of that is included with a single fee. 0:52 They're also scalable. 0:54 Since unmoderated studies contest with many participants at the same time, 0:56 it is much easier to scale your study, whether it is to include a wider 1:01 population or to test multiple different concepts. 1:06 They alleviate the observer effect. 1:09 In a moderated study, there can be a tendency by the participant, 1:11 often unconsciously, 1:15 to change their behavior simply because they know somebody is watching. 1:17 They may also have a natural desire to tell the moderator what they 1:21 think the moderator wants to hear. 1:26 They keep participants in their natural space. 1:28 Unmoderated studies allow people to take part from wherever is most 1:31 comfortable for them. 1:36 The enticing convenience of unmoderated testing comes at the expense of 1:37 missing out on in-depth discussions. 1:42 Consider this trade-off very carefully before choosing this route. 1:45 Since usability testing is meant to gather insight about a person's behavior and 1:49 the motivation behind it, watch out for unmoderated testing tools that 1:55 only ask about people's experiences after they've used a product. 1:59 You really want to learn what is happening during their time with your product. 2:04 Now, let's go over the full list of requirements and 2:10 nice-to-haves when choosing a testing tool. 2:13 First, screen and audio recording. 2:16 A screen recording will allow you to see how people used the mouse to explore 2:19 each screen, where they clicked or tapped, how long they spent on each page. 2:24 If you instruct them to think aloud and then also capture the audio recording, 2:29 you'll get an even fuller picture of how your service is performing. 2:34 If your budget allows for it, choose a webcam recording as well. 2:40 Second, customizable recruiting criteria. 2:44 Some providers will have an existing list of people to reach out to, 2:47 which can be filtered by your audience criteria. 2:52 However, using a pool runs the risk of working with expert usability testers. 2:55 Find out what the service does to weed these experts out. 3:01 Maybe they could exclude data from those who move through the test too quickly, or 3:05 from those who entered invalid comments. 3:09 Ideally, if you have access to your own list of users, see if you can upload it 3:12 directly into the tool to make sure you're reaching your exact desired audience. 3:17 Third, ability to test anything. 3:22 Can participants use any device, computer, tablet, or phone? 3:24 Are you limited to a live website, or can you upload images as well? 3:31 Make sure that the tool you're choosing can accommodate the state of your project. 3:35 Fourth, customizable tasks. 3:41 Many services will provide their own sample tasks for you to choose from. 3:44 Although this might be a good start, this is not enough. 3:49 With no moderator present, participants will need very clearly worded, 3:52 product-specific tasks. 3:58 Poor tasks will equal poor data. 4:00 Fifth, timing of questions. 4:03 Most services will offer the ability to ask questions at the end of the study. 4:06 However, you will also want the ability to ask questions after a specific action. 4:11 For example, what were you looking for when you clicked here? 4:16 6, speed. 4:22 Is your project on a tight timeline? 4:23 If so, you can pay more to get your results back sooner. 4:26 7, quantitative analytics. 4:30 Does the tool allow you to track quantitative metrics, 4:34 such as task completion or general usability score? 4:38 This is not required, but if they do have this ability, 4:42 make sure you know how that general usability score is being calculated. 4:46 8, longitudinal studies. 4:52 You can choose to continuously run usability tests on your website and 4:54 receive monthly or 5:00 quarterly reports showing how performance may be changing over time. 5:01 I can't recommend an exact tool for you, since the options are always changing and 5:07 your needs will be unique as well. 5:12 However, to get you started, 5:14 I have provided a few examples in the teacher's notes. 5:15 Good luck. 5:18