1 00:00:00,650 --> 00:00:06,000 Online surveys are useful at many stages during the product development cycle. 2 00:00:06,000 --> 00:00:10,550 At the beginning, they can be used to research the relevant product landscape. 3 00:00:10,550 --> 00:00:12,830 After launching, surveys can be useful for 4 00:00:12,830 --> 00:00:17,670 understanding how well the product is doing, and what to improve. 5 00:00:17,670 --> 00:00:22,130 For this course, we'll focus on evaluating the design of a live product. 6 00:00:23,400 --> 00:00:24,990 Like with all research, 7 00:00:24,990 --> 00:00:29,780 before you start designing a study, you need to have a clear goal in mind. 8 00:00:30,900 --> 00:00:33,000 Keep that goal focused. 9 00:00:33,000 --> 00:00:35,640 Surveys are most effective when they're short. 10 00:00:36,910 --> 00:00:41,863 Let's talk about common types of research goals using a fictional 11 00:00:41,863 --> 00:00:45,407 custom t-shirt design service as an example. 12 00:00:45,407 --> 00:00:50,440 Deeper understanding of qualitative research findings. 13 00:00:50,440 --> 00:00:54,650 Perhaps you ran a usability study and noticed that a couple of the participants 14 00:00:54,650 --> 00:01:00,000 were really frustrated that the t-shirt designs were limited to just three colors. 15 00:01:00,000 --> 00:01:03,410 How representative are they of other users? 16 00:01:03,410 --> 00:01:05,580 Evaluate the launch of a new feature. 17 00:01:06,730 --> 00:01:09,870 You just launched the ability to put in corporate orders. 18 00:01:09,870 --> 00:01:11,610 What do people think? 19 00:01:11,610 --> 00:01:14,430 Prepare for a site redesign. 20 00:01:14,430 --> 00:01:17,710 The code has gotten old, or the brand needs a refresh. 21 00:01:17,710 --> 00:01:22,020 This is also an opportunity to evaluate what has been working well, and 22 00:01:22,020 --> 00:01:23,920 what users want to see improved. 23 00:01:25,210 --> 00:01:30,250 Let's review the types of survey questions available to us, closed ended and 24 00:01:30,250 --> 00:01:31,800 open ended. 25 00:01:31,800 --> 00:01:36,570 Closed ended questions can be single choice, often used in radio buttons, or 26 00:01:36,570 --> 00:01:40,540 multiple choice, usually represented with checkboxes. 27 00:01:40,540 --> 00:01:44,945 Closed ended questions form the quantitative basis for a survey. 28 00:01:44,945 --> 00:01:47,200 They're easy to count and analyze. 29 00:01:47,200 --> 00:01:49,440 They're also fast for users to complete. 30 00:01:50,740 --> 00:01:54,850 Open ended questions are the probing whys and hows. 31 00:01:54,850 --> 00:01:58,800 It's helpful to use them as followup to closed ended questions. 32 00:01:58,800 --> 00:02:03,790 If you're asking, rate how your t-shirt looked as compared to your drawing, 33 00:02:03,790 --> 00:02:07,390 you can follow up with, please explain why you gave that rating. 34 00:02:08,900 --> 00:02:13,160 Demographic, demographic questions can be standard, 35 00:02:13,160 --> 00:02:16,650 such as asking participants about their gender, or more personal, 36 00:02:16,650 --> 00:02:19,990 on topics such as education level or household income. 37 00:02:21,140 --> 00:02:25,210 Ask only the most essential demographic questions to avoid people quitting your 38 00:02:25,210 --> 00:02:28,760 survey only because they don't feel comfortable with a demographic question. 39 00:02:30,210 --> 00:02:34,915 Likert scale, these questions are a reliable way to measure attitudes and 40 00:02:34,915 --> 00:02:36,480 opinions. 41 00:02:36,480 --> 00:02:39,630 There are usually 5 or 7 choices at a time. 42 00:02:39,630 --> 00:02:41,490 The middle value is the neutral option, 43 00:02:41,490 --> 00:02:45,850 while the end values range from very positive to very negative. 44 00:02:45,850 --> 00:02:51,262 The responses to a question like, how satisfied were you with your final t-shirt 45 00:02:51,262 --> 00:02:58,150 design, would be very satisfied, somewhat satisfied, neither satisfied nor 46 00:02:58,150 --> 00:03:03,480 dissatisfied, somewhat dissatisfied, and very dissatisfied. 47 00:03:04,850 --> 00:03:08,310 Alternatively, if you asked, choose how much you agree or 48 00:03:08,310 --> 00:03:10,710 disagree with the following statement, 49 00:03:10,710 --> 00:03:15,080 the t-shirt design closely resembled the design I had uploaded. 50 00:03:15,080 --> 00:03:19,920 The options would be strongly agree, agree, 51 00:03:19,920 --> 00:03:24,790 neither agree nor disagree, disagree, strongly disagree. 52 00:03:25,850 --> 00:03:29,305 One more example, but with a longer set of options, 53 00:03:29,305 --> 00:03:34,460 would be, how likely are you to recommend the service to others? 54 00:03:34,460 --> 00:03:39,849 Extremely likely, very likely, likely, don't know, 55 00:03:39,849 --> 00:03:44,660 unlikely, very unlikely, and extremely unlikely. 56 00:03:45,860 --> 00:03:47,600 Stack ranking, 57 00:03:47,600 --> 00:03:52,700 stack ranking questions require users to place items in a particular order. 58 00:03:52,700 --> 00:03:57,570 This can be especially useful for learning what features people really want. 59 00:03:57,570 --> 00:04:01,020 A simple wish list can quickly spin out of control, but 60 00:04:01,020 --> 00:04:04,950 if you ask people to order that list, they will have to make choices. 61 00:04:05,950 --> 00:04:07,490 At the end, the top and 62 00:04:07,490 --> 00:04:11,230 bottom few items chosen are the ones people feel strongest about. 63 00:04:12,840 --> 00:04:15,210 Where should you put these questions? 64 00:04:15,210 --> 00:04:18,800 Decide whether you want to run an email or intercept survey. 65 00:04:20,030 --> 00:04:24,560 A modern intercept survey is usually presented as a widget that appears 66 00:04:24,560 --> 00:04:28,170 in the corner of a website, requesting to be completed immediately. 67 00:04:29,500 --> 00:04:31,975 An email survey is much like it sounds, 68 00:04:31,975 --> 00:04:35,060 sent out as an email to a specific list of people. 69 00:04:36,165 --> 00:04:39,680 Intercept surveys are tricky to implement. 70 00:04:39,680 --> 00:04:43,330 Email surveys are easy to implement, but require an email list. 71 00:04:44,370 --> 00:04:49,050 Intercept surveys are taken at a specific moment while using a specific feature. 72 00:04:50,070 --> 00:04:54,230 Email surveys can be taken whenever the recipient has time. 73 00:04:54,230 --> 00:04:59,490 Intercept surveys must be short, since the user is in the middle of another task. 74 00:04:59,490 --> 00:05:04,200 Email surveys can be longer, but should still be concise. 75 00:05:04,200 --> 00:05:07,460 Ultimately, the big advantage to intercept surveys 76 00:05:07,460 --> 00:05:10,900 is that they have a higher completion rate than email surveys. 77 00:05:11,990 --> 00:05:17,010 Designing a survey is another exercise in designing a user experience, and 78 00:05:17,010 --> 00:05:19,840 with that come best practices. 79 00:05:19,840 --> 00:05:21,590 Here are the most important ones. 80 00:05:22,610 --> 00:05:25,140 Lead with interesting questions. 81 00:05:25,140 --> 00:05:29,340 People naturally want to help, but may lose steam along the way. 82 00:05:29,340 --> 00:05:31,720 Start with the interesting questions first. 83 00:05:31,720 --> 00:05:34,630 That way, you will engage your users early, but 84 00:05:34,630 --> 00:05:38,950 if they do choose to leave, you still have the most important questions answered. 85 00:05:40,230 --> 00:05:42,170 Keep text short. 86 00:05:42,170 --> 00:05:46,630 Directions and questions should each be about one line. 87 00:05:46,630 --> 00:05:48,410 People just don't like to read. 88 00:05:49,830 --> 00:05:51,830 Group by topic. 89 00:05:51,830 --> 00:05:53,755 Keep similar questions together so 90 00:05:53,755 --> 00:05:56,860 that participants don't have to switch context more than necessary. 91 00:05:57,910 --> 00:06:00,380 Don't ask about behavior. 92 00:06:00,380 --> 00:06:04,480 Avoid asking general questions about how people use something. 93 00:06:04,480 --> 00:06:06,690 Observing that behavior directly for 94 00:06:06,690 --> 00:06:09,370 yourself is a much more reliable way to do that. 95 00:06:10,600 --> 00:06:13,520 Avoid questions with two concepts. 96 00:06:13,520 --> 00:06:18,090 If you find yourself asking, what do you think of this and that, 97 00:06:18,090 --> 00:06:22,190 you're assuming that your participant has the same opinion about both things. 98 00:06:23,820 --> 00:06:26,050 Use neutral questions. 99 00:06:26,050 --> 00:06:31,450 As always, avoid questions that include assumptions or imply the right answer. 100 00:06:32,700 --> 00:06:34,970 Make your survey anonymous. 101 00:06:34,970 --> 00:06:37,920 Unless you need to get in touch with followup questions, 102 00:06:37,920 --> 00:06:41,950 leave your survey anonymous to encourage truthful responses. 103 00:06:41,950 --> 00:06:46,543 If you're offering compensation or a raffle entry, make the personal contact 104 00:06:46,543 --> 00:06:49,915 information optional for those who want that incentive. 105 00:06:49,915 --> 00:06:53,430 Once you've written your survey, be sure to test it. 106 00:06:53,430 --> 00:06:57,850 Have a few friends or coworkers take it, then a couple of real users. 107 00:06:57,850 --> 00:07:01,480 Get lots of feedback before sending broadly. 108 00:07:01,480 --> 00:07:05,040 You'll learn if any of your questions are confusing, long, or 109 00:07:05,040 --> 00:07:06,500 have other usability issues.