1
00:00:00,650 --> 00:00:06,000
Online surveys are useful at many stages
during the product development cycle.

2
00:00:06,000 --> 00:00:10,550
At the beginning, they can be used to
research the relevant product landscape.

3
00:00:10,550 --> 00:00:12,830
After launching, surveys can be useful for

4
00:00:12,830 --> 00:00:17,670
understanding how well the product
is doing, and what to improve.

5
00:00:17,670 --> 00:00:22,130
For this course, we'll focus on
evaluating the design of a live product.

6
00:00:23,400 --> 00:00:24,990
Like with all research,

7
00:00:24,990 --> 00:00:29,780
before you start designing a study,
you need to have a clear goal in mind.

8
00:00:30,900 --> 00:00:33,000
Keep that goal focused.

9
00:00:33,000 --> 00:00:35,640
Surveys are most effective
when they're short.

10
00:00:36,910 --> 00:00:41,863
Let's talk about common types of
research goals using a fictional

11
00:00:41,863 --> 00:00:45,407
custom t-shirt design
service as an example.

12
00:00:45,407 --> 00:00:50,440
Deeper understanding of
qualitative research findings.

13
00:00:50,440 --> 00:00:54,650
Perhaps you ran a usability study and
noticed that a couple of the participants

14
00:00:54,650 --> 00:01:00,000
were really frustrated that the t-shirt
designs were limited to just three colors.

15
00:01:00,000 --> 00:01:03,410
How representative
are they of other users?

16
00:01:03,410 --> 00:01:05,580
Evaluate the launch of a new feature.

17
00:01:06,730 --> 00:01:09,870
You just launched the ability
to put in corporate orders.

18
00:01:09,870 --> 00:01:11,610
What do people think?

19
00:01:11,610 --> 00:01:14,430
Prepare for a site redesign.

20
00:01:14,430 --> 00:01:17,710
The code has gotten old, or
the brand needs a refresh.

21
00:01:17,710 --> 00:01:22,020
This is also an opportunity to evaluate
what has been working well, and

22
00:01:22,020 --> 00:01:23,920
what users want to see improved.

23
00:01:25,210 --> 00:01:30,250
Let's review the types of survey questions
available to us, closed ended and

24
00:01:30,250 --> 00:01:31,800
open ended.

25
00:01:31,800 --> 00:01:36,570
Closed ended questions can be single
choice, often used in radio buttons, or

26
00:01:36,570 --> 00:01:40,540
multiple choice,
usually represented with checkboxes.

27
00:01:40,540 --> 00:01:44,945
Closed ended questions form
the quantitative basis for a survey.

28
00:01:44,945 --> 00:01:47,200
They're easy to count and analyze.

29
00:01:47,200 --> 00:01:49,440
They're also fast for users to complete.

30
00:01:50,740 --> 00:01:54,850
Open ended questions
are the probing whys and hows.

31
00:01:54,850 --> 00:01:58,800
It's helpful to use them as
followup to closed ended questions.

32
00:01:58,800 --> 00:02:03,790
If you're asking, rate how your t-shirt
looked as compared to your drawing,

33
00:02:03,790 --> 00:02:07,390
you can follow up with,
please explain why you gave that rating.

34
00:02:08,900 --> 00:02:13,160
Demographic, demographic
questions can be standard,

35
00:02:13,160 --> 00:02:16,650
such as asking participants about
their gender, or more personal,

36
00:02:16,650 --> 00:02:19,990
on topics such as education level or
household income.

37
00:02:21,140 --> 00:02:25,210
Ask only the most essential demographic
questions to avoid people quitting your

38
00:02:25,210 --> 00:02:28,760
survey only because they don't feel
comfortable with a demographic question.

39
00:02:30,210 --> 00:02:34,915
Likert scale, these questions are a
reliable way to measure attitudes and

40
00:02:34,915 --> 00:02:36,480
opinions.

41
00:02:36,480 --> 00:02:39,630
There are usually 5 or
7 choices at a time.

42
00:02:39,630 --> 00:02:41,490
The middle value is the neutral option,

43
00:02:41,490 --> 00:02:45,850
while the end values range from
very positive to very negative.

44
00:02:45,850 --> 00:02:51,262
The responses to a question like, how
satisfied were you with your final t-shirt

45
00:02:51,262 --> 00:02:58,150
design, would be very satisfied,
somewhat satisfied, neither satisfied nor

46
00:02:58,150 --> 00:03:03,480
dissatisfied, somewhat dissatisfied,
and very dissatisfied.

47
00:03:04,850 --> 00:03:08,310
Alternatively, if you asked,
choose how much you agree or

48
00:03:08,310 --> 00:03:10,710
disagree with the following statement,

49
00:03:10,710 --> 00:03:15,080
the t-shirt design closely resembled
the design I had uploaded.

50
00:03:15,080 --> 00:03:19,920
The options would be strongly agree,
agree,

51
00:03:19,920 --> 00:03:24,790
neither agree nor disagree,
disagree, strongly disagree.

52
00:03:25,850 --> 00:03:29,305
One more example, but
with a longer set of options,

53
00:03:29,305 --> 00:03:34,460
would be, how likely are you to
recommend the service to others?

54
00:03:34,460 --> 00:03:39,849
Extremely likely, very likely,
likely, don't know,

55
00:03:39,849 --> 00:03:44,660
unlikely, very unlikely,
and extremely unlikely.

56
00:03:45,860 --> 00:03:47,600
Stack ranking,

57
00:03:47,600 --> 00:03:52,700
stack ranking questions require users
to place items in a particular order.

58
00:03:52,700 --> 00:03:57,570
This can be especially useful for
learning what features people really want.

59
00:03:57,570 --> 00:04:01,020
A simple wish list can quickly
spin out of control, but

60
00:04:01,020 --> 00:04:04,950
if you ask people to order that list,
they will have to make choices.

61
00:04:05,950 --> 00:04:07,490
At the end, the top and

62
00:04:07,490 --> 00:04:11,230
bottom few items chosen are the ones
people feel strongest about.

63
00:04:12,840 --> 00:04:15,210
Where should you put these questions?

64
00:04:15,210 --> 00:04:18,800
Decide whether you want to run an email or
intercept survey.

65
00:04:20,030 --> 00:04:24,560
A modern intercept survey is usually
presented as a widget that appears

66
00:04:24,560 --> 00:04:28,170
in the corner of a website,
requesting to be completed immediately.

67
00:04:29,500 --> 00:04:31,975
An email survey is much like it sounds,

68
00:04:31,975 --> 00:04:35,060
sent out as an email to
a specific list of people.

69
00:04:36,165 --> 00:04:39,680
Intercept surveys are tricky to implement.

70
00:04:39,680 --> 00:04:43,330
Email surveys are easy to implement,
but require an email list.

71
00:04:44,370 --> 00:04:49,050
Intercept surveys are taken at a specific
moment while using a specific feature.

72
00:04:50,070 --> 00:04:54,230
Email surveys can be taken
whenever the recipient has time.

73
00:04:54,230 --> 00:04:59,490
Intercept surveys must be short, since
the user is in the middle of another task.

74
00:04:59,490 --> 00:05:04,200
Email surveys can be longer,
but should still be concise.

75
00:05:04,200 --> 00:05:07,460
Ultimately, the big advantage
to intercept surveys

76
00:05:07,460 --> 00:05:10,900
is that they have a higher
completion rate than email surveys.

77
00:05:11,990 --> 00:05:17,010
Designing a survey is another exercise
in designing a user experience, and

78
00:05:17,010 --> 00:05:19,840
with that come best practices.

79
00:05:19,840 --> 00:05:21,590
Here are the most important ones.

80
00:05:22,610 --> 00:05:25,140
Lead with interesting questions.

81
00:05:25,140 --> 00:05:29,340
People naturally want to help,
but may lose steam along the way.

82
00:05:29,340 --> 00:05:31,720
Start with the interesting
questions first.

83
00:05:31,720 --> 00:05:34,630
That way,
you will engage your users early, but

84
00:05:34,630 --> 00:05:38,950
if they do choose to leave, you still have
the most important questions answered.

85
00:05:40,230 --> 00:05:42,170
Keep text short.

86
00:05:42,170 --> 00:05:46,630
Directions and
questions should each be about one line.

87
00:05:46,630 --> 00:05:48,410
People just don't like to read.

88
00:05:49,830 --> 00:05:51,830
Group by topic.

89
00:05:51,830 --> 00:05:53,755
Keep similar questions together so

90
00:05:53,755 --> 00:05:56,860
that participants don't have to
switch context more than necessary.

91
00:05:57,910 --> 00:06:00,380
Don't ask about behavior.

92
00:06:00,380 --> 00:06:04,480
Avoid asking general questions
about how people use something.

93
00:06:04,480 --> 00:06:06,690
Observing that behavior directly for

94
00:06:06,690 --> 00:06:09,370
yourself is a much more
reliable way to do that.

95
00:06:10,600 --> 00:06:13,520
Avoid questions with two concepts.

96
00:06:13,520 --> 00:06:18,090
If you find yourself asking,
what do you think of this and that,

97
00:06:18,090 --> 00:06:22,190
you're assuming that your participant
has the same opinion about both things.

98
00:06:23,820 --> 00:06:26,050
Use neutral questions.

99
00:06:26,050 --> 00:06:31,450
As always, avoid questions that include
assumptions or imply the right answer.

100
00:06:32,700 --> 00:06:34,970
Make your survey anonymous.

101
00:06:34,970 --> 00:06:37,920
Unless you need to get in
touch with followup questions,

102
00:06:37,920 --> 00:06:41,950
leave your survey anonymous to
encourage truthful responses.

103
00:06:41,950 --> 00:06:46,543
If you're offering compensation or
a raffle entry, make the personal contact

104
00:06:46,543 --> 00:06:49,915
information optional for
those who want that incentive.

105
00:06:49,915 --> 00:06:53,430
Once you've written your survey,
be sure to test it.

106
00:06:53,430 --> 00:06:57,850
Have a few friends or coworkers take it,
then a couple of real users.

107
00:06:57,850 --> 00:07:01,480
Get lots of feedback
before sending broadly.

108
00:07:01,480 --> 00:07:05,040
You'll learn if any of your
questions are confusing, long, or

109
00:07:05,040 --> 00:07:06,500
have other usability issues.