Heads up! To view this whole video, sign in with your Courses account or enroll in your free 7-day trial. Sign In Enroll
Preview
Start a free Courses trial
to watch this video
Sometimes you will end up with a stream that returns a stream, flatMap helps you flatten your elements into a single stream.
Imperative Word Cloud Code
public static Map<String, Long> getSnippetWordCountsImperatively(List<Job> jobs) {
Map<String, Long> wordCounts = new HashMap<>();
for (Job job : jobs) {
String[] words = job.getSnippet().split("\\W+");
for (String word : words) {
if (word.length() == 0) {
continue;
}
String lWord = word.toLowerCase();
Long count = wordCounts.get(lWord);
if (count == null) {
count = 0L;
}
wordCounts.put(lWord, ++count);
}
}
return wordCounts;
}
Learn more
You're at a point right now where
you've had enough time with streams and
0:00
you're probably falling in love with them.
0:03
It's okay, I won't say anything,
your secret's safe with me.
0:06
Around this emotional time you wanna spent
even more time with them and you'll start
0:10
saying, how they make every problem you
try to solve so much more beautiful.
0:14
Okay, I was being silly but in reality
you were seeing the power of streams.
0:20
And you'll start wanting to use them for
just about everything.
0:24
So, I'd like to show you
two important things first,
0:28
you need to know how to create a stream
from objects that aren't in collections.
0:32
It's really straightforward there's
a static method on the stream class called
0:36
Of, it's pretty you'll love it.
0:40
Secondly, now that you can
create streams out of thin air,
0:42
you are bound to run
into a common problem.
0:46
You'll try to map at them and
0:48
you'll end up pushing a stream through
a stream and stuff just gets awkward.
0:50
But don't worry.
There is an easy solution.
0:55
In programming,
there is a concept called Flattening.
0:58
Imagine for
a second that you have a list of lists.
1:01
Now this list is hard to process so
what you do is flatten it,
1:04
by pushing the list down into one single
list, now it's much easier to process,
1:09
see how the list is now flattened?
1:14
It could've also called
it smooshed together.
1:16
We could do a similar thing
with streams of streams and
1:18
it's a pretty powerful concept that is
going to make you love them even more.
1:21
I know you're falling pretty hard already.
1:25
>> So first off,
let me show you how to make a Stream.
1:28
So if we do Stream.of, it's so
declarative and so purdy.
1:32
We'll say hello,
1:38
this is a stream.
1:43
It is in the java.util.stream package so
there we go.
1:48
And since it's a stream we can do a for
1:51
each on it, our little terminal operation
here and we'll to the ever so popular,
1:53
we'll just print that out real quick,
system that out.
1:58
Print line.
2:00
And, boom, there it is.
2:03
So that of static method accepts
what is known as varargs.
2:05
So it takes in endless
amount of parameters,
2:08
as long as they are all the same type.
2:12
So you can also pass that in array.
2:14
Check the teacher's notes for
more on var args, so let's do this.
2:17
On the job object, there is a little
snippet of what is required for the job.
2:20
Now it's not the full description,
it's just part of it.
2:25
Now that's because the indeed API,
2:27
the one that we're using,
wants you to come to their site.
2:29
So I was thinking, why don't we do this?
2:33
Why don't we try to make a Word Cloud
based on the different words
2:35
in that snipit.
2:38
And then we'll keep account of how
many times each word appears across
2:39
each of the 1000 job postings
that we have, sound fun?
2:43
Now normally, we've been taking time to
build the imperative style of code to
2:46
solve the same problem.
2:49
Now it's gotten to a point where the code
is too long to write in a single video.
2:51
I've already written it and
it's in the teacher's notes.
2:54
So I'm gonna copy it and
I'm gonna paste it right here.
2:56
And I am going to say this
is indeed a Map, yes.
3:01
And that is a HashMap, yes.
3:04
Okay, so let's walk this really quick.
3:06
So I'm gonna return a map of string which
is the word to the number of counts there,
3:08
so that's what we're gonna do and
we're gonna make a new hash map.
3:14
We're gonna loop through the jobs and
we're gonna split apart the snippet.
3:18
We're gonna split Split
using a regular expression,
3:23
check the teachers notes
if this is new to you.
3:25
But this is anything that is not a word,
we're gonna split on that, so
3:27
that we just get the words out.
3:30
And then we're gonna
loop each of the words.
3:31
So we've got a nested four loop here,
we're gonna loop each one of those words,
3:33
and we'll check the length,
cuz sometimes this comes back empty.
3:37
So if it is in fact empty,
I could actually us is empty there.
3:40
We'll check the link.
3:44
And then we're gonna continue, right,
cuz we don't want to run it through here.
3:46
We're just gonna bail out if that exists.
3:48
And then we really only want
the word to be lowercase.
3:50
Right, we don't want a capital java and
lower case java.
3:53
So we're just gonna lowercase it so
3:55
it's the same word, cuz case doesn't
really matter in a word cloud.
3:56
And then, we are going to pull
the current count, should it exist.
4:00
And this thing is here.
4:05
If it's null, that means that it's zero,
and we're gonna put a new one in there.
4:07
Now, longs are mutable.
4:11
So what will return is the result
of a pre-increment, right?
4:13
So this will, because this will, these
are all be 1 if there was a 1 in here,
4:17
we count at 2, right?
4:20
Cuz we're doing a pre-increment.
4:21
And then finally what we return that map.
4:22
Okay, now let's use it.
4:27
So it is called
getSnippetWordCountsImperatively.
4:28
So let's get rid of this
stream example here.
4:32
And so we'll say
getSnippetWordCountsImperatively, and
4:34
we will pass in jobs.
4:38
Now, I wanna loop through the keys and
values.
4:40
So there is a collection that takes
on a map and it's called forEach.
4:43
Now instead of a normal consumer, because
we are getting back a key and a value,
4:46
it takes a by consumer.
4:50
So remember that's 2 values right, so
4:51
the two values that we're gonna get
back from that are key, and value.
4:53
So each time that will get passed through.
4:58
Pretty nice right?
5:01
And then I'm gonna print,
5:01
let's say the key occurs %d times.
5:05
Okay, so we're gonna do key and value.
5:10
Cool, let's give that a run,
let's see what happens.
5:15
Let's go ahead and put a new line on there
and make that a little bit prettier.
5:21
Cool, so locations occur six
times TCP will occurs 86 times.
5:28
Gotta have a lot of will.
5:35
Security 19 times, cool.
5:37
This is looking pretty nice.
5:39
All right, so let's tackle the problem
declaratively using streams.
5:40
So let's copy that signature first,
so let's grab this here.
5:44
And I'm gonna just put them right here and
we'll say get snip it word counts string.
5:48
Okay, so we need to return a map so let's
remember that we're gonna do that we're
5:55
gonna return job dot string we'll collect
into a map this time that'll be fun okay,
5:58
so now we want to transform the job.
6:04
Into its snippet.
6:06
So we will say map, and
we'll do Job getSnippet.
6:08
So it's gonna pass the job in, and on that
job it's going to call getSnippet, cool.
6:12
So now we have a snip it,
we have a string so
6:17
let's map that string,
let's snip it, right?
6:22
We're gonna do just like we did before
we're gonna split snip dot split and
6:26
we're gonna split on anything
that is not a word and
6:30
again if that is confusing to
you check the teachers note.
6:33
Okay, so what is this saying?
6:37
Probably, that we're not done yet.
6:40
Cool, so yeah, we're not done.
6:42
It says, you can't return this yet.
6:43
What are you trying to do?
6:44
So now, we have an array of words.
6:46
So we can actually create a new stream.
6:49
But we don't want to cross the streams.
6:53
Never cross the streams.
6:55
We wanna flatten it, right?
6:57
Remember that's what we were
talking about, flatten it.
6:59
So we're gonna take flat maps.
7:01
So let's do that, so
we'll say .flatMap and
7:02
it takes an item and that item here
this are the words, all right?
7:06
And what we gonna do is stream of words.
7:10
So, remember I said stream of can take
an array, so took an array words and
7:16
made a new stream.
7:19
So, the cool part is you can
imagine what comes out of here
7:21
just like a brand new stream.
7:24
Anything that comes out of flatMap
is like a brand new stream.
7:25
So what's going to pop out is a word,
one at a time.
7:28
Right?
So, let's do that.
7:31
We want to make sure, like we saw before
that the word isn't empty, right?
7:32
So we'll say word, length,
is greater than zero.
7:40
So now we're sure that we have a string
and we want to get the lower case of that.
7:44
Right?
7:49
So we'll do .map and
String as a toLowerCase.
7:51
Again, that is an instance
method on Strings.
7:58
So because Strings coming through here,
that's doing that
8:01
Don't be confused by the squiggly,
that is just because we can't return yet.
8:04
And now we wanna collect the items.
8:10
Now before we collected into a list.
8:12
But this time remember we
wanna collect into a map.
8:14
So collectors are super powerful.
8:18
So here's a little sneak peak at a pretty
powerful collector called grouping by.
8:20
So we're gonna say .collect,
8:26
and we're gonna say Collectors.groupingBy.
8:31
So groupingBy requires a function
that takes an item and
8:35
returns an item, just like map, okay?
8:39
So, we want this is what
it's going to be for a key.
8:42
Now this is gonna be
a little bit dorky but
8:45
what we want this is the function
that basically just returns itself.
8:47
It's gonna get a word,
it's gonna return a word.
8:52
And then the next
perimeter is a collector.
8:54
So, there's a handy one called counting.
8:58
So we're gonna collectors dot counting.
9:00
Basically that does what
we did in the other lab.
9:03
Then I'm gonna put a semicolon here.
9:05
Now that dorky function that we just
wrote, that returns exactly what it got,
9:07
I got a word, I'm gonna return a word,
pretty dorky.
9:11
It's so common that you can actually
replace it with a handy static method
9:14
off of the Function class,
and it's named identity.
9:18
So then you don't feel so dorky.
9:20
So you say Function.identity.
9:22
Basically it's like It doesn't do a map,
it just returns what's there and that will
9:25
save you from writing that obvious
lambda that you don't really need to.
9:30
This is common enough in functional
programming that you just call that
9:33
the identity function.
9:36
Now one more thing that we can
do to really clean this up.
9:38
Cuz, if we take a look
at our flat map here.
9:41
Look what it's doing,
it's taking what was here, and
9:43
it's just passing it in
to another function.
9:45
Well, that's a great place,
for a static method reference.
9:48
Let's do just as the intention action
suggests, and replace that there.
9:51
Okay, so let's assume,
that we have a few jobs here.
9:57
I'll just make one of these comments here.
10:00
So, we'll say a job that has a snippet.
10:02
And it says this is a job.
10:07
And then there's another job and
it has a snippet.
10:10
And it says, also a job.
10:13
Not very helpful snippets, but here we go.
10:16
So let's walk this really quick.
10:19
We open up a stream on these two jobs,
okay.
10:21
And we transform that item into a snippet.
10:25
So now it's just this.
10:28
This is a job, there's a string here,
and then we map that into an array.
10:30
So then it is a four item
array of this is a job.
10:34
And next we go into the flat map.
10:37
And flat map can take anything and
10:40
it expects that what is
returned is a "stream".
10:42
So, pass in our array to "stream.of",
10:45
and that makes a new
stream of "this is a job".
10:48
So the first item that comes through
is this, and it passes the filter; we
10:53
make it lower case and
then we go into the collector process and
10:57
it says hey, is there,
basically what this code is doing here.
11:01
Right?
It just checks
11:05
to see if it's in there and
then puts it in.
11:06
So, it passes this word through and
11:08
the function identity is basically
just saying this is the word here.
11:11
Sometimes you might
want to translate that.
11:14
Like we could've put that in here,
we could've put this map in
11:16
here to use the function, it's a lower
case the thing so this goes in here, and
11:18
then it comes back to the flat
map which is running, right?
11:22
So it's gonna come in and it is going
to say the next word which was is.
11:25
So is is gonna come through, and
then it's gonna do the count.
11:30
And then it's gonna go a, and then job.
11:32
And then it will come back up to the top.
11:35
Pretty cool, right.
11:37
So again, flatmap is an interesting
concept, take some time to let it sink in.
11:38
It's really powerful and
exactly what you need, when you need it.
11:43
You're doing great, if you ever find
yourself with a stream of streams
11:47
you're probably missing
a flattening step somewhere.
11:51
And like I said when you need it,
you'll understand it.
11:54
It's like stream a little stream with me.
11:56
That collector was super powerful,
wasn't it?
11:59
I love that you can express that so
declaratively.
12:01
The overarching principle of
processing a stream of items and
12:04
producing something new through
a terminal operation is called reduction.
12:08
In the next stage, we'll dive deeper into
some more handy ways to reduce your data.
12:13
You need to sign up for Treehouse in order to download course files.
Sign up