Bummer! This is just a preview. You need to be signed in with a Basic account to view the entire video.
Start a free Basic trial
to watch this video
Sometimes you will end up with a stream that returns a stream, flatMap helps you flatten your elements into a single stream.
Imperative Word Cloud Code
public static Map<String, Long> getSnippetWordCountsImperatively(List<Job> jobs) {
Map<String, Long> wordCounts = new HashMap<>();
for (Job job : jobs) {
String[] words = job.getSnippet().split("\\W+");
for (String word : words) {
if (word.length() == 0) {
continue;
}
String lWord = word.toLowerCase();
Long count = wordCounts.get(lWord);
if (count == null) {
count = 0L;
}
wordCounts.put(lWord, ++count);
}
}
return wordCounts;
}
Learn more
-
0:00
You're at a point right now where you've had enough time with streams and
-
0:03
you're probably falling in love with them.
-
0:06
It's okay, I won't say anything, your secret's safe with me.
-
0:10
Around this emotional time you wanna spent even more time with them and you'll start
-
0:14
saying, how they make every problem you try to solve so much more beautiful.
-
0:20
Okay, I was being silly but in reality you were seeing the power of streams.
-
0:24
And you'll start wanting to use them for just about everything.
-
0:28
So, I'd like to show you two important things first,
-
0:32
you need to know how to create a stream from objects that aren't in collections.
-
0:36
It's really straightforward there's a static method on the stream class called
-
0:40
Of, it's pretty you'll love it.
-
0:42
Secondly, now that you can create streams out of thin air,
-
0:46
you are bound to run into a common problem.
-
0:48
You'll try to map at them and
-
0:50
you'll end up pushing a stream through a stream and stuff just gets awkward.
-
0:55
But don't worry. There is an easy solution.
-
0:58
In programming, there is a concept called Flattening.
-
1:01
Imagine for a second that you have a list of lists.
-
1:04
Now this list is hard to process so what you do is flatten it,
-
1:09
by pushing the list down into one single list, now it's much easier to process,
-
1:14
see how the list is now flattened?
-
1:16
It could've also called it smooshed together.
-
1:18
We could do a similar thing with streams of streams and
-
1:21
it's a pretty powerful concept that is going to make you love them even more.
-
1:25
I know you're falling pretty hard already.
-
1:28
>> So first off, let me show you how to make a Stream.
-
1:32
So if we do Stream.of, it's so declarative and so purdy.
-
1:38
We'll say hello,
-
1:43
this is a stream.
-
1:48
It is in the java.util.stream package so there we go.
-
1:51
And since it's a stream we can do a for
-
1:53
each on it, our little terminal operation here and we'll to the ever so popular,
-
1:58
we'll just print that out real quick, system that out.
-
2:00
Print line.
-
2:03
And, boom, there it is.
-
2:05
So that of static method accepts what is known as varargs.
-
2:08
So it takes in endless amount of parameters,
-
2:12
as long as they are all the same type.
-
2:14
So you can also pass that in array.
-
2:17
Check the teacher's notes for more on var args, so let's do this.
-
2:20
On the job object, there is a little snippet of what is required for the job.
-
2:25
Now it's not the full description, it's just part of it.
-
2:27
Now that's because the indeed API,
-
2:29
the one that we're using, wants you to come to their site.
-
2:33
So I was thinking, why don't we do this?
-
2:35
Why don't we try to make a Word Cloud based on the different words
-
2:38
in that snipit.
-
2:39
And then we'll keep account of how many times each word appears across
-
2:43
each of the 1000 job postings that we have, sound fun?
-
2:46
Now normally, we've been taking time to build the imperative style of code to
-
2:49
solve the same problem.
-
2:51
Now it's gotten to a point where the code is too long to write in a single video.
-
2:54
I've already written it and it's in the teacher's notes.
-
2:56
So I'm gonna copy it and I'm gonna paste it right here.
-
3:01
And I am going to say this is indeed a Map, yes.
-
3:04
And that is a HashMap, yes.
-
3:06
Okay, so let's walk this really quick.
-
3:08
So I'm gonna return a map of string which is the word to the number of counts there,
-
3:14
so that's what we're gonna do and we're gonna make a new hash map.
-
3:18
We're gonna loop through the jobs and we're gonna split apart the snippet.
-
3:23
We're gonna split Split using a regular expression,
-
3:25
check the teachers notes if this is new to you.
-
3:27
But this is anything that is not a word, we're gonna split on that, so
-
3:30
that we just get the words out.
-
3:31
And then we're gonna loop each of the words.
-
3:33
So we've got a nested four loop here, we're gonna loop each one of those words,
-
3:37
and we'll check the length, cuz sometimes this comes back empty.
-
3:40
So if it is in fact empty, I could actually us is empty there.
-
3:44
We'll check the link.
-
3:46
And then we're gonna continue, right, cuz we don't want to run it through here.
-
3:48
We're just gonna bail out if that exists.
-
3:50
And then we really only want the word to be lowercase.
-
3:53
Right, we don't want a capital java and lower case java.
-
3:55
So we're just gonna lowercase it so
-
3:56
it's the same word, cuz case doesn't really matter in a word cloud.
-
4:00
And then, we are going to pull the current count, should it exist.
-
4:05
And this thing is here.
-
4:07
If it's null, that means that it's zero, and we're gonna put a new one in there.
-
4:11
Now, longs are mutable.
-
4:13
So what will return is the result of a pre-increment, right?
-
4:17
So this will, because this will, these are all be 1 if there was a 1 in here,
-
4:20
we count at 2, right?
-
4:21
Cuz we're doing a pre-increment.
-
4:22
And then finally what we return that map.
-
4:27
Okay, now let's use it.
-
4:28
So it is called getSnippetWordCountsImperatively.
-
4:32
So let's get rid of this stream example here.
-
4:34
And so we'll say getSnippetWordCountsImperatively, and
-
4:38
we will pass in jobs.
-
4:40
Now, I wanna loop through the keys and values.
-
4:43
So there is a collection that takes on a map and it's called forEach.
-
4:46
Now instead of a normal consumer, because we are getting back a key and a value,
-
4:50
it takes a by consumer.
-
4:51
So remember that's 2 values right, so
-
4:53
the two values that we're gonna get back from that are key, and value.
-
4:58
So each time that will get passed through.
-
5:01
Pretty nice right?
-
5:01
And then I'm gonna print,
-
5:05
let's say the key occurs %d times.
-
5:10
Okay, so we're gonna do key and value.
-
5:15
Cool, let's give that a run, let's see what happens.
-
5:21
Let's go ahead and put a new line on there and make that a little bit prettier.
-
5:28
Cool, so locations occur six times TCP will occurs 86 times.
-
5:35
Gotta have a lot of will.
-
5:37
Security 19 times, cool.
-
5:39
This is looking pretty nice.
-
5:40
All right, so let's tackle the problem declaratively using streams.
-
5:44
So let's copy that signature first, so let's grab this here.
-
5:48
And I'm gonna just put them right here and we'll say get snip it word counts string.
-
5:55
Okay, so we need to return a map so let's remember that we're gonna do that we're
-
5:58
gonna return job dot string we'll collect into a map this time that'll be fun okay,
-
6:04
so now we want to transform the job.
-
6:06
Into its snippet.
-
6:08
So we will say map, and we'll do Job getSnippet.
-
6:12
So it's gonna pass the job in, and on that job it's going to call getSnippet, cool.
-
6:17
So now we have a snip it, we have a string so
-
6:22
let's map that string, let's snip it, right?
-
6:26
We're gonna do just like we did before we're gonna split snip dot split and
-
6:30
we're gonna split on anything that is not a word and
-
6:33
again if that is confusing to you check the teachers note.
-
6:37
Okay, so what is this saying?
-
6:40
Probably, that we're not done yet.
-
6:42
Cool, so yeah, we're not done.
-
6:43
It says, you can't return this yet.
-
6:44
What are you trying to do?
-
6:46
So now, we have an array of words.
-
6:49
So we can actually create a new stream.
-
6:53
But we don't want to cross the streams.
-
6:55
Never cross the streams.
-
6:57
We wanna flatten it, right?
-
6:59
Remember that's what we were talking about, flatten it.
-
7:01
So we're gonna take flat maps.
-
7:02
So let's do that, so we'll say .flatMap and
-
7:06
it takes an item and that item here this are the words, all right?
-
7:10
And what we gonna do is stream of words.
-
7:16
So, remember I said stream of can take an array, so took an array words and
-
7:19
made a new stream.
-
7:21
So, the cool part is you can imagine what comes out of here
-
7:24
just like a brand new stream.
-
7:25
Anything that comes out of flatMap is like a brand new stream.
-
7:28
So what's going to pop out is a word, one at a time.
-
7:31
Right? So, let's do that.
-
7:32
We want to make sure, like we saw before that the word isn't empty, right?
-
7:40
So we'll say word, length, is greater than zero.
-
7:44
So now we're sure that we have a string and we want to get the lower case of that.
-
7:49
Right?
-
7:51
So we'll do .map and String as a toLowerCase.
-
7:58
Again, that is an instance method on Strings.
-
8:01
So because Strings coming through here, that's doing that
-
8:04
Don't be confused by the squiggly, that is just because we can't return yet.
-
8:10
And now we wanna collect the items.
-
8:12
Now before we collected into a list.
-
8:14
But this time remember we wanna collect into a map.
-
8:18
So collectors are super powerful.
-
8:20
So here's a little sneak peak at a pretty powerful collector called grouping by.
-
8:26
So we're gonna say .collect,
-
8:31
and we're gonna say Collectors.groupingBy.
-
8:35
So groupingBy requires a function that takes an item and
-
8:39
returns an item, just like map, okay?
-
8:42
So, we want this is what it's going to be for a key.
-
8:45
Now this is gonna be a little bit dorky but
-
8:47
what we want this is the function that basically just returns itself.
-
8:52
It's gonna get a word, it's gonna return a word.
-
8:54
And then the next perimeter is a collector.
-
8:58
So, there's a handy one called counting.
-
9:00
So we're gonna collectors dot counting.
-
9:03
Basically that does what we did in the other lab.
-
9:05
Then I'm gonna put a semicolon here.
-
9:07
Now that dorky function that we just wrote, that returns exactly what it got,
-
9:11
I got a word, I'm gonna return a word, pretty dorky.
-
9:14
It's so common that you can actually replace it with a handy static method
-
9:18
off of the Function class, and it's named identity.
-
9:20
So then you don't feel so dorky.
-
9:22
So you say Function.identity.
-
9:25
Basically it's like It doesn't do a map, it just returns what's there and that will
-
9:30
save you from writing that obvious lambda that you don't really need to.
-
9:33
This is common enough in functional programming that you just call that
-
9:36
the identity function.
-
9:38
Now one more thing that we can do to really clean this up.
-
9:41
Cuz, if we take a look at our flat map here.
-
9:43
Look what it's doing, it's taking what was here, and
-
9:45
it's just passing it in to another function.
-
9:48
Well, that's a great place, for a static method reference.
-
9:51
Let's do just as the intention action suggests, and replace that there.
-
9:57
Okay, so let's assume, that we have a few jobs here.
-
10:00
I'll just make one of these comments here.
-
10:02
So, we'll say a job that has a snippet.
-
10:07
And it says this is a job.
-
10:10
And then there's another job and it has a snippet.
-
10:13
And it says, also a job.
-
10:16
Not very helpful snippets, but here we go.
-
10:19
So let's walk this really quick.
-
10:21
We open up a stream on these two jobs, okay.
-
10:25
And we transform that item into a snippet.
-
10:28
So now it's just this.
-
10:30
This is a job, there's a string here, and then we map that into an array.
-
10:34
So then it is a four item array of this is a job.
-
10:37
And next we go into the flat map.
-
10:40
And flat map can take anything and
-
10:42
it expects that what is returned is a "stream".
-
10:45
So, pass in our array to "stream.of",
-
10:48
and that makes a new stream of "this is a job".
-
10:53
So the first item that comes through is this, and it passes the filter; we
-
10:57
make it lower case and then we go into the collector process and
-
11:01
it says hey, is there, basically what this code is doing here.
-
11:05
Right? It just checks
-
11:06
to see if it's in there and then puts it in.
-
11:08
So, it passes this word through and
-
11:11
the function identity is basically just saying this is the word here.
-
11:14
Sometimes you might want to translate that.
-
11:16
Like we could've put that in here, we could've put this map in
-
11:18
here to use the function, it's a lower case the thing so this goes in here, and
-
11:22
then it comes back to the flat map which is running, right?
-
11:25
So it's gonna come in and it is going to say the next word which was is.
-
11:30
So is is gonna come through, and then it's gonna do the count.
-
11:32
And then it's gonna go a, and then job.
-
11:35
And then it will come back up to the top.
-
11:37
Pretty cool, right.
-
11:38
So again, flatmap is an interesting concept, take some time to let it sink in.
-
11:43
It's really powerful and exactly what you need, when you need it.
-
11:47
You're doing great, if you ever find yourself with a stream of streams
-
11:51
you're probably missing a flattening step somewhere.
-
11:54
And like I said when you need it, you'll understand it.
-
11:56
It's like stream a little stream with me.
-
11:59
That collector was super powerful, wasn't it?
-
12:01
I love that you can express that so declaratively.
-
12:04
The overarching principle of processing a stream of items and
-
12:08
producing something new through a terminal operation is called reduction.
-
12:13
In the next stage, we'll dive deeper into some more handy ways to reduce your data.
You need to sign up for Treehouse in order to download course files.
Sign up