flatMap

You're at a point right now where you've had enough time with streams and 0:00 you're probably falling in love with them. 0:03 It's okay, I won't say anything, your secret's safe with me. 0:06 Around this emotional time you wanna spent even more time with them and you'll start 0:10 saying, how they make every problem you try to solve so much more beautiful. 0:14 Okay, I was being silly but in reality you were seeing the power of streams. 0:20 And you'll start wanting to use them for just about everything. 0:24 So, I'd like to show you two important things first, 0:28 you need to know how to create a stream from objects that aren't in collections. 0:32 It's really straightforward there's a static method on the stream class called 0:36 Of, it's pretty you'll love it. 0:40 Secondly, now that you can create streams out of thin air, 0:42 you are bound to run into a common problem. 0:46 You'll try to map at them and 0:48 you'll end up pushing a stream through a stream and stuff just gets awkward. 0:50 But don't worry. There is an easy solution. 0:55 In programming, there is a concept called Flattening. 0:58 Imagine for a second that you have a list of lists. 1:01 Now this list is hard to process so what you do is flatten it, 1:04 by pushing the list down into one single list, now it's much easier to process, 1:09 see how the list is now flattened? 1:14 It could've also called it smooshed together. 1:16 We could do a similar thing with streams of streams and 1:18 it's a pretty powerful concept that is going to make you love them even more. 1:21 I know you're falling pretty hard already. 1:25 >> So first off, let me show you how to make a Stream. 1:28 So if we do Stream.of, it's so declarative and so purdy. 1:32 We'll say hello, 1:38 this is a stream. 1:43 It is in the java.util.stream package so there we go. 1:48 And since it's a stream we can do a for 1:51 each on it, our little terminal operation here and we'll to the ever so popular, 1:53 we'll just print that out real quick, system that out. 1:58 Print line. 2:00 And, boom, there it is. 2:03 So that of static method accepts what is known as varargs. 2:05 So it takes in endless amount of parameters, 2:08 as long as they are all the same type. 2:12 So you can also pass that in array. 2:14 Check the teacher's notes for more on var args, so let's do this. 2:17 On the job object, there is a little snippet of what is required for the job. 2:20 Now it's not the full description, it's just part of it. 2:25 Now that's because the indeed API, 2:27 the one that we're using, wants you to come to their site. 2:29 So I was thinking, why don't we do this? 2:33 Why don't we try to make a Word Cloud based on the different words 2:35 in that snipit. 2:38 And then we'll keep account of how many times each word appears across 2:39 each of the 1000 job postings that we have, sound fun? 2:43 Now normally, we've been taking time to build the imperative style of code to 2:46 solve the same problem. 2:49 Now it's gotten to a point where the code is too long to write in a single video. 2:51 I've already written it and it's in the teacher's notes. 2:54 So I'm gonna copy it and I'm gonna paste it right here. 2:56 And I am going to say this is indeed a Map, yes. 3:01 And that is a HashMap, yes. 3:04 Okay, so let's walk this really quick. 3:06 So I'm gonna return a map of string which is the word to the number of counts there, 3:08 so that's what we're gonna do and we're gonna make a new hash map. 3:14 We're gonna loop through the jobs and we're gonna split apart the snippet. 3:18 We're gonna split Split using a regular expression, 3:23 check the teachers notes if this is new to you. 3:25 But this is anything that is not a word, we're gonna split on that, so 3:27 that we just get the words out. 3:30 And then we're gonna loop each of the words. 3:31 So we've got a nested four loop here, we're gonna loop each one of those words, 3:33 and we'll check the length, cuz sometimes this comes back empty. 3:37 So if it is in fact empty, I could actually us is empty there. 3:40 We'll check the link. 3:44 And then we're gonna continue, right, cuz we don't want to run it through here. 3:46 We're just gonna bail out if that exists. 3:48 And then we really only want the word to be lowercase. 3:50 Right, we don't want a capital java and lower case java. 3:53 So we're just gonna lowercase it so 3:55 it's the same word, cuz case doesn't really matter in a word cloud. 3:56 And then, we are going to pull the current count, should it exist. 4:00 And this thing is here. 4:05 If it's null, that means that it's zero, and we're gonna put a new one in there. 4:07 Now, longs are mutable. 4:11 So what will return is the result of a pre-increment, right? 4:13 So this will, because this will, these are all be 1 if there was a 1 in here, 4:17 we count at 2, right? 4:20 Cuz we're doing a pre-increment. 4:21 And then finally what we return that map. 4:22 Okay, now let's use it. 4:27 So it is called getSnippetWordCountsImperatively. 4:28 So let's get rid of this stream example here. 4:32 And so we'll say getSnippetWordCountsImperatively, and 4:34 we will pass in jobs. 4:38 Now, I wanna loop through the keys and values. 4:40 So there is a collection that takes on a map and it's called forEach. 4:43 Now instead of a normal consumer, because we are getting back a key and a value, 4:46 it takes a by consumer. 4:50 So remember that's 2 values right, so 4:51 the two values that we're gonna get back from that are key, and value. 4:53 So each time that will get passed through. 4:58 Pretty nice right? 5:01 And then I'm gonna print, 5:01 let's say the key occurs %d times. 5:05 Okay, so we're gonna do key and value. 5:10 Cool, let's give that a run, let's see what happens. 5:15 Let's go ahead and put a new line on there and make that a little bit prettier. 5:21 Cool, so locations occur six times TCP will occurs 86 times. 5:28 Gotta have a lot of will. 5:35 Security 19 times, cool. 5:37 This is looking pretty nice. 5:39 All right, so let's tackle the problem declaratively using streams. 5:40 So let's copy that signature first, so let's grab this here. 5:44 And I'm gonna just put them right here and we'll say get snip it word counts string. 5:48 Okay, so we need to return a map so let's remember that we're gonna do that we're 5:55 gonna return job dot string we'll collect into a map this time that'll be fun okay, 5:58 so now we want to transform the job. 6:04 Into its snippet. 6:06 So we will say map, and we'll do Job getSnippet. 6:08 So it's gonna pass the job in, and on that job it's going to call getSnippet, cool. 6:12 So now we have a snip it, we have a string so 6:17 let's map that string, let's snip it, right? 6:22 We're gonna do just like we did before we're gonna split snip dot split and 6:26 we're gonna split on anything that is not a word and 6:30 again if that is confusing to you check the teachers note. 6:33 Okay, so what is this saying? 6:37 Probably, that we're not done yet. 6:40 Cool, so yeah, we're not done. 6:42 It says, you can't return this yet. 6:43 What are you trying to do? 6:44 So now, we have an array of words. 6:46 So we can actually create a new stream. 6:49 But we don't want to cross the streams. 6:53 Never cross the streams. 6:55 We wanna flatten it, right? 6:57 Remember that's what we were talking about, flatten it. 6:59 So we're gonna take flat maps. 7:01 So let's do that, so we'll say .flatMap and 7:02 it takes an item and that item here this are the words, all right? 7:06 And what we gonna do is stream of words. 7:10 So, remember I said stream of can take an array, so took an array words and 7:16 made a new stream. 7:19 So, the cool part is you can imagine what comes out of here 7:21 just like a brand new stream. 7:24 Anything that comes out of flatMap is like a brand new stream. 7:25 So what's going to pop out is a word, one at a time. 7:28 Right? So, let's do that. 7:31 We want to make sure, like we saw before that the word isn't empty, right? 7:32 So we'll say word, length, is greater than zero. 7:40 So now we're sure that we have a string and we want to get the lower case of that. 7:44 Right? 7:49 So we'll do .map and String as a toLowerCase. 7:51 Again, that is an instance method on Strings. 7:58 So because Strings coming through here, that's doing that 8:01 Don't be confused by the squiggly, that is just because we can't return yet. 8:04 And now we wanna collect the items. 8:10 Now before we collected into a list. 8:12 But this time remember we wanna collect into a map. 8:14 So collectors are super powerful. 8:18 So here's a little sneak peak at a pretty powerful collector called grouping by. 8:20 So we're gonna say .collect, 8:26 and we're gonna say Collectors.groupingBy. 8:31 So groupingBy requires a function that takes an item and 8:35 returns an item, just like map, okay? 8:39 So, we want this is what it's going to be for a key. 8:42 Now this is gonna be a little bit dorky but 8:45 what we want this is the function that basically just returns itself. 8:47 It's gonna get a word, it's gonna return a word. 8:52 And then the next perimeter is a collector. 8:54 So, there's a handy one called counting. 8:58 So we're gonna collectors dot counting. 9:00 Basically that does what we did in the other lab. 9:03 Then I'm gonna put a semicolon here. 9:05 Now that dorky function that we just wrote, that returns exactly what it got, 9:07 I got a word, I'm gonna return a word, pretty dorky. 9:11 It's so common that you can actually replace it with a handy static method 9:14 off of the Function class, and it's named identity. 9:18 So then you don't feel so dorky. 9:20 So you say Function.identity. 9:22 Basically it's like It doesn't do a map, it just returns what's there and that will 9:25 save you from writing that obvious lambda that you don't really need to. 9:30 This is common enough in functional programming that you just call that 9:33 the identity function. 9:36 Now one more thing that we can do to really clean this up. 9:38 Cuz, if we take a look at our flat map here. 9:41 Look what it's doing, it's taking what was here, and 9:43 it's just passing it in to another function. 9:45 Well, that's a great place, for a static method reference. 9:48 Let's do just as the intention action suggests, and replace that there. 9:51 Okay, so let's assume, that we have a few jobs here. 9:57 I'll just make one of these comments here. 10:00 So, we'll say a job that has a snippet. 10:02 And it says this is a job. 10:07 And then there's another job and it has a snippet. 10:10 And it says, also a job. 10:13 Not very helpful snippets, but here we go. 10:16 So let's walk this really quick. 10:19 We open up a stream on these two jobs, okay. 10:21 And we transform that item into a snippet. 10:25 So now it's just this. 10:28 This is a job, there's a string here, and then we map that into an array. 10:30 So then it is a four item array of this is a job. 10:34 And next we go into the flat map. 10:37 And flat map can take anything and 10:40 it expects that what is returned is a "stream". 10:42 So, pass in our array to "stream.of", 10:45 and that makes a new stream of "this is a job". 10:48 So the first item that comes through is this, and it passes the filter; we 10:53 make it lower case and then we go into the collector process and 10:57 it says hey, is there, basically what this code is doing here. 11:01 Right? It just checks 11:05 to see if it's in there and then puts it in. 11:06 So, it passes this word through and 11:08 the function identity is basically just saying this is the word here. 11:11 Sometimes you might want to translate that. 11:14 Like we could've put that in here, we could've put this map in 11:16 here to use the function, it's a lower case the thing so this goes in here, and 11:18 then it comes back to the flat map which is running, right? 11:22 So it's gonna come in and it is going to say the next word which was is. 11:25 So is is gonna come through, and then it's gonna do the count. 11:30 And then it's gonna go a, and then job. 11:32 And then it will come back up to the top. 11:35 Pretty cool, right. 11:37 So again, flatmap is an interesting concept, take some time to let it sink in. 11:38 It's really powerful and exactly what you need, when you need it. 11:43 You're doing great, if you ever find yourself with a stream of streams 11:47 you're probably missing a flattening step somewhere. 11:51 And like I said when you need it, you'll understand it. 11:54 It's like stream a little stream with me. 11:56 That collector was super powerful, wasn't it? 11:59 I love that you can express that so declaratively. 12:01 The overarching principle of processing a stream of items and 12:04 producing something new through a terminal operation is called reduction. 12:08 In the next stage, we'll dive deeper into some more handy ways to reduce your data. 12:13