Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
We need a framework upon which we can build our experiments. Let's take a look at the Build-Measure-Learn feedback loop developed by Eric Ries and how it can help our business.
This video doesn't have any notes.
Related Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign upRelated Discussions
Have questions about this video? Start a discussion with the community and Treehouse staff.
Sign up
In the previous video, we talked about why
carrying out experiments are necessary.
0:00
But what exactly do we mean by an
experiment?
0:05
The Lean Startup movement, developed by
Eric Ries,
0:07
introduced the concept of the build,
measure, learn feedback loop.
0:11
Each experiment we've been talking about
will consist of a single turn [SOUND] of
0:16
the loop.
0:21
The first part of the loop is the build
stage.
0:21
[SOUND] In the build stage,
0:24
we start by asking what metric would
confirm our hypothesis.
0:25
The metric is important because it
determines how we
0:29
move forward with our model.
0:32
Be careful about picking the right metric
and not one that inflates your
0:35
data to confirm your desired assumptions
rather than the truth.
0:38
[SOUND] Once we have a metric,
0:42
we design the experiment to gain data on
that metric.
0:44
We do so by [SOUND] developing our MVP, or
minimum viable product.
0:47
The MVP is a version of our product that
enables us to
0:52
complete a single iteration of the build,
measure, learn loop,
0:55
with a minimum amount of effort and the
least amount of development time.
0:59
This is a vague guideline for sure, but
1:04
you get a better sense of what that means
when taking the experiment into account.
1:06
One of the first few experiments we need
to conduct is to
1:11
measure the hypothesis that our product
fits the market.
1:14
This is a assuming we've tested the
assumption that the market exists
1:18
to begin with.
1:21
In the build stage of this experiment, all
we need to
1:23
show a customer is some prototype that can
convey the bare bones of our product.
1:26
You might think, as most people do, that
at this point,
1:31
you need to build a product.
1:33
But no, you don't have to.
1:35
Remember, [SOUND] an MVP is a version of
our product that enables us
1:37
to complete a single turn around the loop
with minimum effort and
1:41
the least amount of development time.
1:44
We can get those same insights by using
wireframes, slide decks, or
1:47
even basic sketches.
1:51
If you put down a working app in front of
someone,
1:53
they can tend to get tied up in features
and design.
1:56
By using a low fidelity MVP, something
quick like wireframes, we can get
1:59
those insights relatively quickly and then
move on to the measure and learn stages.
2:04
Regardless of the type of experiment, once
you have your MVP,
2:09
you invite your customers to use it and
give you feedback.
2:13
[SOUND] This is the measure phase.
2:16
So the first step is to enter the build
phase as quickly as possible with an MVP.
2:18
Once the MVP is done, you move on to the
measure phase.
2:23
In the measure phase, you [SOUND] analyze
whether your product development efforts
2:27
in the build phase actually translated to
meaningful progress.
2:32
This is why the metric is so important.
2:35
If you pass the test, our assumption is
validated.
2:38
The metrics you choose depends on the
stage you are in the company and
2:41
the type of experiment.
2:45
They can range from things like number of
meetings set up with potential customers
2:47
for sales calls to cost per acquisition,
monthly occurring revenue and so on.
2:51
[SOUND] After the measure phase, it's the
learn [SOUND] phase.
2:55
Take the insights that you have gained
from this experiment and
2:59
apply that to your product or service.
3:02
If the test fails, you discard those
assumptions and keep experimenting.
3:05
It is important to note that we're not
just building lots of MVPs,
3:09
running an experiment and discarding it.
3:13
No, our product is basically the evolution
of these MVPs.
3:15
Let's say we have a website [SOUND] up and
3:20
we need to increase our activation
efforts.
3:22
So we conduct an experiment to test the
effectiveness of our call [SOUND] to
3:25
action button.
3:29
Our metric we're measuring here is an
account creation.
3:30
We built several [SOUND] MVPs.
3:34
Oh, that's right.
3:36
That's another important point.
3:37
A test does not have to be restricted to
one MVP.
3:38
We can create several and
3:42
test each version among a different
segment of our customer base.
3:44
In our case, we create several MVPs with
different [SOUND] button styles, copy,
3:48
and positioning.
3:52
[SOUND] The MVP that passes [SOUND] the
test then becomes the next
3:53
iteration [SOUND] of the product.
3:56
In this way, we only add to the product
those features that pass our
3:58
experiments and confirm our assumptions.
4:02
Everything else that adds no value is
discarded.
4:05
If you work on a product or
4:09
service like this when you start out, by
the time you launch, you will have
4:10
a product that has customer-tested
features that you know will succeed.
4:14
When we carried out the exercise in the
previous stage and built our business
4:19
model, we laid out a lot of assumptions to
make our business plan work.
4:22
Our goal with these experiments is to test
these assumptions as quickly as
4:27
possible so we get rid of all the wrong
ones.
4:31
So for example, my first assumption was
that a market existed for
4:34
project management software focused around
really small groups.
4:38
To experiment this, I first need to
develop my metric.
4:42
Now, that's fairly simple in this case.
4:45
I'm going to measure yes or no responses
to a survey asking the very same question.
4:48
The pass fail bar that I'm setting here is
that I want at
4:53
least a 70% positive response rate.
4:56
Why so high?
5:00
Remember, we said that with our initial
assumptions like these,
5:00
we want a really high pass rate because
each successive experiment is going to
5:04
further whittle down this number.
5:09
That if only 20% of my survey respondents
indicated yes,
5:10
then the number that will actually pay me
is much smaller.
5:14
This is not a scalable business model.
5:18
My MVP in this case is a simple survey and
5:21
I will carry it out both in person and
using an online survey.
5:24
The in-person survey will help me get some
feedback right from the beginning,
5:27
while the online survey will help me reach
a much larger number of respondents.
5:32
Similarly, I can conduct experiments for
all my assumptions, from things like
5:37
landing pages using AB tests, product
testing using focus groups, and so on.
5:42
You need to sign up for Treehouse in order to download course files.
Sign upYou need to sign up for Treehouse in order to set up Workspace
Sign up