1 00:00:00,000 --> 00:00:03,391 Traditionally, usability tests have been moderated. 2 00:00:03,391 --> 00:00:07,861 A moderated test just means that there's a moderator present asking 3 00:00:07,861 --> 00:00:12,715 questions in order to deeply understand each choice the participant makes. 4 00:00:12,715 --> 00:00:17,491 They may be sitting in your office and using your prototype, or you may go out 5 00:00:17,491 --> 00:00:22,450 and observe someone test your product whatever you intend for it to be useful. 6 00:00:22,450 --> 00:00:26,089 Alternatively, moderated testing can be done remotely. 7 00:00:26,089 --> 00:00:30,991 There are many remote testing services that allow you to stay in the office 8 00:00:30,991 --> 00:00:33,760 while observing and holding discussions 9 00:00:33,760 --> 00:00:38,029 with people using your service from the comfort of their own home. 10 00:00:38,029 --> 00:00:41,112 So what are the advantages of a moderated study? 11 00:00:41,112 --> 00:00:42,010 Probe deeper. 12 00:00:42,010 --> 00:00:44,980 When you're with a participant at each step, 13 00:00:44,980 --> 00:00:50,122 you can ask what they're thinking and why they've made a particular choice. 14 00:00:50,122 --> 00:00:54,941 For example, why did you click here, or did you see this instructional text? 15 00:00:54,941 --> 00:00:57,130 Capture participant reactions. 16 00:00:57,130 --> 00:01:00,313 Did you notice that sigh or look of confusion? 17 00:01:00,313 --> 00:01:03,754 You can add that to your notes as additional data. 18 00:01:03,754 --> 00:01:08,402 Without someone there, nonverbal reactions can easily be missed. 19 00:01:08,402 --> 00:01:11,520 [SOUND] Protect sensitive information. 20 00:01:11,520 --> 00:01:15,380 Say you're working on a product that really shouldn't 21 00:01:15,380 --> 00:01:18,582 be revealed to the public until much, much later. 22 00:01:18,582 --> 00:01:21,211 By moderating, particularly in person. 23 00:01:21,211 --> 00:01:26,632 You've limited any exposure to the highly vetted group of study participants, 24 00:01:26,632 --> 00:01:30,913 who've all ready signed an non-disclosure agreement, or NDA. 25 00:01:30,913 --> 00:01:33,752 [SOUND] Evaluate low fidelity concepts. 26 00:01:33,752 --> 00:01:38,715 If testing early and possibly even with just a paper prototype, you will need 27 00:01:38,715 --> 00:01:43,364 to be present to guide the participant on what features are functional. 28 00:01:43,364 --> 00:01:44,940 [SOUND] Keep on track. 29 00:01:44,940 --> 00:01:49,354 If you notice the participant is focusing somewhere irrelevant to 30 00:01:49,354 --> 00:01:54,207 the study's objectives, you could politely guide them back to the task. 31 00:01:54,207 --> 00:01:55,786 Test complex products. 32 00:01:55,786 --> 00:02:00,936 When testing products such as business software, you will need to make sure 33 00:02:00,936 --> 00:02:05,922 that your participant has the background to complete the required test. 34 00:02:05,922 --> 00:02:07,002 And in addition, 35 00:02:07,002 --> 00:02:12,792 you will likely need to provide clarifying information throughout the study. 36 00:02:12,792 --> 00:02:17,617 Let's say that your study is the perfect candidate to be moderated. 37 00:02:17,617 --> 00:02:21,097 For example, you have a click through prototype for 38 00:02:21,097 --> 00:02:23,399 your complex business software. 39 00:02:23,399 --> 00:02:28,426 One next question is, will you be running this study at your office, 40 00:02:28,426 --> 00:02:30,426 in the wild, or remotely? 41 00:02:30,426 --> 00:02:33,497 A moderated study needs to be in real time but 42 00:02:33,497 --> 00:02:38,320 you do not need to be in the same physical space as your participants. 43 00:02:38,320 --> 00:02:42,615 If you do choose to bring the participants into your office, you will need to make 44 00:02:42,615 --> 00:02:46,342 sure your office is set up in a way that will make people comfortable and 45 00:02:46,342 --> 00:02:47,559 limit distractions. 46 00:02:47,559 --> 00:02:53,419 Ideally, this would include a dedicated space in the form of usability lab. 47 00:02:53,419 --> 00:02:57,626 Let's take a look at the key attributes of a usability lab. 48 00:02:57,626 --> 00:03:04,022 First, you need a room with a closed door to minimize distractions during the study. 49 00:03:04,022 --> 00:03:08,497 Inside the room, you'll need a computer with a monitor. 50 00:03:08,497 --> 00:03:12,460 An external monitor rather than just a laptop is preferred, so 51 00:03:12,460 --> 00:03:16,514 that it's easier for you and the participant to see the screen. 52 00:03:16,514 --> 00:03:20,178 Attach a camera to the monitor if it doesn't have one already, 53 00:03:20,178 --> 00:03:21,433 to record the study. 54 00:03:21,433 --> 00:03:24,976 And confirm that the microphone in your computer is working. 55 00:03:24,976 --> 00:03:29,244 Make sure there's a keyboard and standard mouse available as well. 56 00:03:29,244 --> 00:03:31,800 There should be at least two chairs in the room. 57 00:03:31,800 --> 00:03:35,515 Place the chair for the participant directly in front of the monitor. 58 00:03:35,515 --> 00:03:40,940 And place your own chair slightly behind, and next to the participant's chair. 59 00:03:40,940 --> 00:03:45,897 You can also include an additional chair for a note taker towards the back. 60 00:03:45,897 --> 00:03:50,720 The key component of usability study is also being able to invite outside 61 00:03:50,720 --> 00:03:51,598 observers. 62 00:03:51,598 --> 00:03:53,598 You can do this in a couple of ways. 63 00:03:53,598 --> 00:03:57,669 One is to have a couch or row of seats in the back. 64 00:03:57,669 --> 00:04:01,961 However, I believe this can be intimidating for the participant and 65 00:04:01,961 --> 00:04:06,186 prefer to put the observers in a totally separate conference room. 66 00:04:06,186 --> 00:04:09,007 And stream the study into that room. 67 00:04:09,007 --> 00:04:14,243 The observers would then be able to hear the audio, see the product being used, 68 00:04:14,243 --> 00:04:18,476 and if possible also see the participants facial expressions. 69 00:04:18,476 --> 00:04:22,456 Make sure to set up a messaging system between the usability lab and 70 00:04:22,456 --> 00:04:26,436 the observation room, so that the observers can text, email, or 71 00:04:26,436 --> 00:04:28,587 chat their questions throughout. 72 00:04:28,587 --> 00:04:32,360 You may need specialized software to record, observe and 73 00:04:32,360 --> 00:04:34,133 analyze the final videos. 74 00:04:34,133 --> 00:04:37,757 You can find the list of options in the teacher's notes. 75 00:04:37,757 --> 00:04:40,588 Remember to compensate your participants generously. 76 00:04:40,588 --> 00:04:43,730 They're taking time out of their day to commute to you and 77 00:04:43,730 --> 00:04:46,884 to provide you with the valuable feedback that you need. 78 00:04:46,884 --> 00:04:51,042 The exact costs will vary by location, but a rough guideline, 79 00:04:51,042 --> 00:04:55,749 if you're in a metro area and asking people to come in for about an hour, 80 00:04:55,749 --> 00:04:58,747 anticipate providing $100 per person. 81 00:04:58,747 --> 00:05:03,704 Testing a product in person is great for keeping things confidential, and for 82 00:05:03,704 --> 00:05:06,942 testing physical hardware when you space design. 83 00:05:06,942 --> 00:05:12,022 However, you will be limited to the people living close to your office. 84 00:05:12,022 --> 00:05:16,866 Now, imagine designing a product meant to be used by everyone in the world, 85 00:05:16,866 --> 00:05:21,467 as I did there in my How representative do you think our research 86 00:05:21,467 --> 00:05:26,254 would have been if we tested with just people in the US, or California? 87 00:05:26,254 --> 00:05:31,638 Or those in Silicon Valley willing to commute to our office in Mountain View? 88 00:05:31,638 --> 00:05:36,253 We'd be left with a group of early adopters, tech enthusiasts, and 89 00:05:36,253 --> 00:05:38,657 frequent usability testers. 90 00:05:38,657 --> 00:05:42,277 Not exactly diverse representative group. 91 00:05:42,277 --> 00:05:46,915 Now, let's go back to the usability lab setup we just described, and 92 00:05:46,915 --> 00:05:49,439 adjust it for a remote testing study. 93 00:05:49,439 --> 00:05:54,420 Remove an extra chair and take a seat in front of the monitor yourself. 94 00:05:54,420 --> 00:05:59,106 You can use remote testing software to allow the participant to take control of 95 00:05:59,106 --> 00:06:03,726 your screen, and use the prototype available in much the same way as before. 96 00:06:03,726 --> 00:06:06,801 Alternatively, you can take an available conference room and 97 00:06:06,801 --> 00:06:08,663 run the session straight from there. 98 00:06:08,663 --> 00:06:12,947 If you include outside observers, they can then join you in the same room or 99 00:06:12,947 --> 00:06:16,145 from their own desks if the session's being streamed. 100 00:06:16,145 --> 00:06:18,194 Remote testing is rarely seeded for 101 00:06:18,194 --> 00:06:22,507 hardware products, paper prototypes or highly confidential material. 102 00:06:22,507 --> 00:06:24,565 Mobile devices can also be tricky, 103 00:06:24,565 --> 00:06:29,446 depending on the kind of hand gestures are important, or if an install is required. 104 00:06:29,446 --> 00:06:32,650 However, remote testing is a fantastic solution for 105 00:06:32,650 --> 00:06:36,150 the online products you're most likely to be working on. 106 00:06:36,150 --> 00:06:38,195 Remote testing is convenient. 107 00:06:38,195 --> 00:06:43,072 You don't need to leave the office, and it provides access to a much, 108 00:06:43,072 --> 00:06:45,566 much wider range of participants. 109 00:06:45,566 --> 00:06:48,611 While working on advertising software at Google, 110 00:06:48,611 --> 00:06:51,525 we relied heavily on moderated remote testing. 111 00:06:51,525 --> 00:06:52,698 Because of this, 112 00:06:52,698 --> 00:06:57,642 we were able to test frequently as we developed new features and products. 113 00:06:57,642 --> 00:07:01,823 However, it was also important to occasionally visit our users in their 114 00:07:01,823 --> 00:07:06,494 workspaces to understand the greater context in which our tools are being used. 115 00:07:06,494 --> 00:07:10,349 In one site visit, we're able to discover that clients would use 116 00:07:10,349 --> 00:07:13,646 an additional monitor that displayed other software, 117 00:07:13,646 --> 00:07:18,085 such as spreadsheets to make up the features that our product was missing. 118 00:07:18,085 --> 00:07:23,492 A very insightful observation that would have been impossible to discover remotely, 119 00:07:23,492 --> 00:07:24,242 or in a lab.