This course will be retired on January 8, 2022.
Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
Our chatbot will transfer queries to the Discovery service if the confidence in what the user is after falls below a certain threshold. Let's see how that works in code.
Documentation
Read more about the confidence element returned by the Conversation service.
Let's talk for a moment about how to know
when a question should be handled by
0:00
the Conversation Service versus when
it should handled by Discovery.
0:04
It all boils down to how confident
Watson is with its responses.
0:09
Take a look at this graphic.
0:14
Notice that the frequently asked
questions, the common questions,
0:16
are those that take place within
the short tail of the distribution.
0:22
This is where
the Conversation Service excels.
0:28
As questions become more esoteric and
uncommon,
0:32
then we rush to the aid
of the Discovery Service.
0:36
>> Now that reminds of an example
here at Treehouse, right?
0:41
We could think of students
asking very targeted questions
0:44
needing help learning programming.
0:48
They may say, what's the key word for
a four loop in Javascript?
0:49
And that would something that we
could easily answer and anticipate.
0:55
But then we get some other questions
that would be either more obscure and
0:59
less frequent, or
more like you said, esoteric, like
1:04
what's some advice for how to get
unstuck when I'm working on a problem.
1:08
And for that kind of thing,
maybe we could, upload, for example,
1:13
blog data that we have where we have
host-related that kind of thing,
1:17
whether it's the Treehouse Blog or
elsewhere.
1:20
Does that make sense?
1:23
>> Absolutely,
what a great example that was, yes.
1:24
>> Okay, so we're back at our page and
we can test this out.
1:28
Let's try asking our chatbot
to find me an apartment.
1:31
Okay, so notice that we see the response
about our chatbot's capabilities.
1:37
Let's take a look at the intent
back here in our Conversation tab.
1:42
So if you remember, these capabilities
are what we imported from the sample.
1:45
And actually,
if you take a look at what we imported,
1:50
you can see there's a bunch of
extra statements in here that
1:52
don't really have anything to do with
what our chatbot's capabilities are, so
1:56
this is a little bit messy,
because it's a test, a sample instance.
1:59
So, at this point let's just delete this.
2:04
We don't really need this anyhow, although
you could pretty easily recreate this with
2:06
a more simplified set of intents here.
2:09
So I'm just going to click on the trash
can over here to delete the intent,
2:13
yes, delete it.
2:17
And we'd like to do the same
thing here for the entities,
2:18
let's delete anything
related to capabilities.
2:21
And finally, in our dialog node, we did
create an extra node for capabilities.
2:26
I'm going to delete this as well.
2:31
All right, let's try this again,
find me an apartment.
2:36
Okay, so this is interesting.
2:42
I actually didn't expect this to match and
2:43
flip over to
the Discovery Service just yet.
2:46
So let's take a look at the output from
Conversation Service to see what happened.
2:48
Okay, so here in the terminal console,
2:53
this is where we are logging output
from the Conversation Service.
2:56
And here we can see that it matched on
the turn on intent with a confidence of,
3:00
yeah, 0.29,
it was a really low confidence.
3:06
So we said find me an apartment and our
chatbot understood what we were saying,
3:08
and it was like maybe they're
trying to turn something on.
3:13
I guess it's very low confidence,
it's a very low score.
3:17
But that is below the threshold, so
3:20
sure enough it went over
to the Discovery Service.
3:21
In the conversation flow it ended
up in the anything else node,
3:25
which it would have said, I didn't
understand, you can try rephrasing.
3:28
But because we passed it over to
Discovery it did run the query
3:31
that said find me an apartment.
3:35
So we want to train our
Conversation Service at least a little bit
3:37
on how to handle this new kind of request.
3:41
So back here let's add a new intent, and
3:44
what we're going to do is create
a new one to find things for us.
3:48
So let's call it a find intent,
and lets add a few examples.
3:53
We'll say find me something,
look for something, I need, maybe.
3:58
Locate, and how about find a.
4:07
Just a couple different ways.
4:10
And looks like I have a typo so
I'm going to come back here.
4:11
Gonna delete find me and
retype it as find me.
4:15
Yeah, okay, so
this can be a little bit confusing.
4:20
In fact, I had some trouble trying to
understand exactly when to use intents
4:23
like this and when and
how to flip over to the Discovery Service.
4:27
And this is one of the things
that there's no, or
4:33
at least there's not always a clear right
answer for, in developing a chatbot.
4:36
When we know exactly what
the user is looking for,
4:41
we do wanna be programmatic about it, like
this, on the Conversation side of things.
4:44
When we are less certain in how we want it
to respond, that's when we need to flip it
4:48
over to Discovery and
somehow tie into a broader data set.
4:53
So what we're doing here is
creating this new intent, and
4:58
we're going to do an example
to find a couple of things,
5:02
very basic finding, but then when
we ask for lodging or an apartment,
5:05
the confidence will fall and
it will flip over to Discovery.
5:10
Alright so to continue this example,
let's add some entities here,
5:14
we'll create a new one.
5:17
And we'll call these locations, and
lets add a couple things in here first.
5:19
Lets do a gas station.
5:24
So for the synonyms lets say gas, fuels,
5:26
fueling station, and
then lets add another value.
5:30
Lets do a hotel.
5:35
And for synonyms let's say motel,
resort, and
5:38
how about place to stay,
and bed and breakfast.
5:43
So, let's complete this with a dialogue
and we'll make this real simple.
5:51
I'm gonna add a new node below turn on.
5:54
This one, let's name this based
on the intent, name this find,
5:58
and the bot recognizes our find intent and
6:03
it matches on one of our location
entities, at locations, any of those.
6:09
Then we will respond with
just something generic.
6:16
Let me see what I can find.
6:19
Okay, so if we open this, we can see that
Watson is training on this new data.
6:25
And now it's done, and
we can try this out.
6:31
Say find me a gas station.
6:33
And it matches on the find intent,
knows the location was gas station,
6:37
and it says let me see
what I can find okay.
6:42
All right so
I'm gonna refresh to clear the page, and
6:45
let's try that down here the same thing,
find me a gas station.
6:48
And we get the same response,
let me see what I can find.
6:53
All right how about locate a hotel.
6:55
Okay, cool, so again,
this is exactly what we just programmed.
6:59
But now, let's try it with
something a little more vague.
7:03
Let's say, I'd like a place
to stay near Central Park.
7:06
Well, interesting,
it says it doesn't understand.
7:15
So, what exactly is happening?
7:17
If we scroll down,
It didn't match on any intent at all.
7:18
So because I said I'd like, it doesn't
know that we're trying to find something,
7:23
and it certainly doesn't
match the turn on intent,
7:28
so it's kind of just lost
without any kind of intent.
7:30
Interestingly, it did match
our locations and hotel,
7:33
and that's because place to
stay is a synonym for hotel.
7:36
So we could go and add this to our intent,
or we could maybe try to control for
7:41
this a little bit differently.
7:46
But for now, I just want to try
asking this in a different way.
7:47
Let's say,
I'd like an AirBnb near Central Park.
7:51
Okay cool success.
7:59
All right check it out, we have in
the responses, we have the title
8:01
of an AirBnb listing, and we have
a review, this is the actual review text.
8:05
This is what we fed to our
Discovery Service, and
8:09
there should be three of these.
8:12
So here's the second one, best deal,
and if we scroll down there
8:13
should be one more, and yeah notice
that these are all by Central Park.
8:16
In fact because these are individual
reviews, we have two different reviews for
8:21
the same one.
8:25
So first, we need to look at the console.
8:26
So here we have the response from
the Conversation Service and
8:29
it matched on the find intent, but as we
wanted it to with a very low confidence.
8:32
So again, because we didn't have
the language to match I'd like, and
8:37
because we didn't say hotel, or
place to stay, or gas station.
8:41
It's trying to figure out what we want,
but
8:45
it's just not very
confident that it knows.
8:46
And in that case, it's like, alright,
8:48
well I'm gonna go over to
the Discovery Service.
8:49
So then, over here, on the right side,
8:52
we can see the response
from the Discovery Service.
8:54
And it has our input text, and it has
a bit of information about the service and
8:56
the collection, and here is the output.
9:00
And we asked for
three results which we got.
9:03
We have a listing name, we have
some details about the listing, and
9:05
if we scroll through there are different
things that are interesting to look about
9:09
for how to Discovery returns data.
9:13
There are different entities,
you can see there's a person entity,
9:16
there are location entities,
information about New York City.
9:19
Again, this was all pulled
from the documents,
9:23
from the review documents that
we fed to our collection.
9:26
And a lot of this extra information
is because the Discovery Service
9:29
is hooked into other information sources.
9:33
So you can see there are even links to
different pieces from the reviews that we
9:35
can tie into if we wanted to.
9:38
Let's try another example over here,
let's say this time,
9:41
show me some good places
to stay near Times Square.
9:44
Okay, and again, this was vague
enough to flip over to Discovery, so
9:51
here we have a little bit
of a different result.
9:55
So it tried to match on the turn on
intent, because we said show me, it really
9:58
wasn't exactly sure what we were after and
it was really low confidence in that.
10:02
So again, it flipped over to
Discovery and, once again,
10:06
if we take a look on the right
if we scroll down a bit,
10:11
there are different parts here
that show the different entities.
10:13
And if you look, you can see things like
it'll mention Time Square like this.
10:17
It matched it from the review.
10:22
So you should try this out with some
different landmarks in New York City,
10:24
maybe some different ways
of asking the question.
10:28
In fact, figuring out how to ask the
question and how not to ask the question
10:31
was a little bit confusing for me and
still trips me up from time to time.
10:35
And that's one of the things that has
been hard for me to wrap my head around.
10:38
Because programming for this type of
stuff is a little bit different than
10:42
more typical programming that I'm
used to at least where we are giving
10:46
very specific questions and answers,
very specific control flow.
10:50
So this kind of chatbot development
requires a lot of testing and tweaking and
10:55
modifying, so if you think of
different ways to say something
10:59
then you may want to program that
into the Conversation Service.
11:03
It's also worth talking about how
there isn't just one way to use
11:07
the Discovery Service.
11:10
You should follow your intuition and make
it work for your chatbot or for your uses,
11:11
however you need to.
11:16
So in this case we are looking
at the intent confidence,
11:18
and we would want to build up different
intents and add more user examples.
11:21
But for now, we can see how we can get
to a low confidence in a matched intent,
11:27
and we can query Discovery for help.
11:32
Now another way you might
tackle this problem,
11:34
just if you have data that
you're usign from Discovery.
11:35
You may want to set up
an intent specifically
11:38
to relay the request to Discovery.
11:41
So you could imagine expanding our
scenario, we could take this, find intent.
11:43
We could look at the location that
was matched, and if it was a hotel or
11:47
a place to stay, if we made the synonyms
match on that, we could look at that,
11:51
analyze it, and then we could In that
case, pass over the attributes and
11:56
go into the Discovery Service.
12:00
This is actually how the food
coach example is setup,
12:01
the food coach that we saw earlier.
12:04
It uses a dialog node to branch
conversation into a Discovery Service.
12:06
You need to sign up for Treehouse in order to download course files.
Sign up