Bummer! This is just a preview. You need to be signed in with a Pro account to view the entire video.
Graphical Effects You Didn’t Know Browsers Could Do25:13 with Divya Manian
Browsers have become better and better at image processing and a lot of these advances have been made leading to very interesting filters, blend modes, and even transforms that help in bringing your designs to life without splicing and dicing graphics multiple times.
[MUSIC] 0:00 Well, thanks Adam for the very energetic, welcome. 0:09 So my slides are not going to be as 0:12 pretty as everyone else's slides, I'm sorry for that. 0:14 I started working on it yesterday, but I know my, hopefully I know my stuff. 0:17 So the URL is there,there'll be some links in my talks. 0:22 If you wanna follow along, you can look at the slides online and I work at Adobe. 0:24 And as, and previously for a year, at 0:30 Adobe, I was working with the web platform team. 0:34 So the web platform team works on proposals for features in 0:37 browsers, and works on the standards and the implementation of these features. 0:41 So some of the things that I'm gonna to be talking 0:45 about are stuff that, my team worked on for more than three 0:47 or four years, and finally are seeing some sort of, support, 0:51 browser support, and you know, the progress standards, so, does it move? 0:55 Yes, it does okay, so I'm gonna to be 1:01 talking about three things, one is masks then is 1:02 blend modes, then filters, so these are the three 1:05 graphical effects I want to talk to you about today. 1:07 Now when I am talking about graphical effects then the way we talk about 1:11 web stuff slightly becomes different, it's no 1:15 longer, you know, just about elements so when 1:17 entering graphical text on the web, it's 1:22 really easy to go overboard, and it's really 1:23 early to do things that just really 1:26 slows down your render performance,and especially with graphics. 1:29 So sometimes maybe it's better off doing it on the server, or better off 1:33 doing it like as a Photoshop image or something and then serving a static file. 1:37 But, there are times when it's actually useful to apply 1:41 these graphical effects and these effects are for those times, specifically. 1:44 So, the jargon we use for talking 1:49 about graphical effects also is slightly different. 1:52 You know when you're talking about 1:56 hestimul stuff, you usually say elements, and 1:57 you say, like CSS selectors, and you know, properties, what of our values? 1:59 That kind of stuff, but when you talk 2:04 about graphic effects we talk primarily of images. 2:06 So how do you apply graphical effects to 2:10 images but how do you apply testable elements then? 2:12 What I will do is that I'll talk about [INAUDIBLE] 2:14 as images, so I would talk about them in layers. 2:18 So graphical effects are applied on layers and will talk a lot 2:21 about pixels because that's what we'll be dealing with in graphical effects. 2:25 So it's all about pixel manipulation, the color manipulation for each pixel. 2:28 And finally you know, Paul actually mentioned this briefly about the GPU 2:33 and textures.so, I'm gonna to talk a little bit about that too. 2:38 Because some of the features that I'm gonna be 2:42 talking about, wer-, are, accelerated on the GPU for animations. 2:43 So, when the GPU does stuff it does it as 2:49 a texture and not like a pixel by pixel thing. 2:52 So it's just gonna be like swiping out textures, so 2:55 that's what I'll be talking about in terms of textures. 2:59 Okay, so let's go about masking. 3:03 So, masking as a specification covers two things. 3:05 One is the actual concept of masking and the other is the concept of clipping. 3:09 Now, masking is basically the control of the, 3:14 you display, you, you can control how your 3:17 content is displayed and you can do so, 3:19 using alpha masks or luminance masks or both. 3:22 And, clipping is basically, you can no longer restrict the content to be 3:26 cropped to a rectangle, you can crop it to any shape you want. 3:29 So, here is how masking works. 3:34 So, you'll have an image on the left, and then you'll apply 3:36 some masks, and then you'll get the final image on the right. 3:38 So that image could be Hestimalt [INAUDIBLE] it could 3:41 be just be live text, and then you'll have 3:43 all of these masks supplied, and then you'll get 3:45 live text with the mask of light on the right. 3:48 So the way the browsers do this, is that they will 3:50 immediately, when they see the live text, that will be converted into 3:53 a rasterized image, and then apply all of these masking on that 3:56 rasterized image, and then give you another rasterized image on the right. 4:00 So, when you are talking about masking there are multiple things that happen and 4:06 so you'll have like SVG masks, you can do that in CSS using SVG masks. 4:10 And then you can also do it with 4:15 alpha-transparent images, so you know, you can have 4:17 images that have transparency in them and use 4:19 that for determining how the masks get applied. 4:22 And you can also use a simple path syntax, 4:25 but that's not for masking, exactly, that's more for clipping. 4:27 So this is a very simple syntax for masking don't know if you can 4:32 see it, but basically, you'll just use 4:36 a property called mask-image with the right prefixes. 4:38 And if you're using it for SVG, you will have 4:40 a URL and like to a SVG file and the ID. 4:44 That is being used for the SVG masks. 4:48 Now, the reason why this exists is masking is, as a 4:50 concept on the Web, has been in existent in SVG for 4:54 a very long time and in SVG, you can do a 4:57 lot of crazy masking uh,you can't do as much in CSS. 5:00 So 5:04 here's an example of masking with images. 5:06 And I'm just gonna show you on my browser, if it opens up. 5:09 Let's go, okay, so none of the stuff that I'm gonna show you are stuff that I 5:16 have done, it's all stolen from people who 5:19 are better than me who are doing these things. 5:23 So, here is live text, I have to show you 5:25 because it is live, I'm selecting it, whoo, it's alive. 5:29 So, now I, I am applying a mask. 5:33 So there is a mask and I can show you how the mask looks like. 5:36 It's just a splatter image. 5:39 And as you can see the black things are basically telling 5:41 the browser that you need to show the background through them. 5:44 So we go back, and so you can see that you just 5:48 apply mask imagine it's a URL and you link to a PNG file. 5:52 If you remove that mask, you notice that the text remains as 5:55 it is and when you reapply the mask, the background shows through. 5:59 [BLANK_AUDIO] 6:03 Okay, so then we go into clipping so clipping is pretty interesting. 6:08 Clipping in, as a term, has existed on the web for a long, long time. 6:14 But it only allowed you to do like rectangular clipping, when using exact 6:18 or like a little antigridial razor during clipping that are not very useful. 6:23 So now you can apply simple paths in tags, you can use 6:27 like polygons, you can use rectangles, you can use circles, and so on. 6:30 So, this is interesting because now you, you know, 6:34 pre, the newest trend, and use avatar rendering in circles. 6:36 But what if you wanna be different? 6:40 You wanna be like a hexagon, or something. 6:41 So if you wanna do that, you can use a 6:43 clip art, and do that with, not that, here it is. 6:45 So here is the clip art that's being applied, as a 6:52 polygon shape, and the way that is being done is fairly simple. 6:55 You just, use right clip art, then you specify a polygon, and you notice that 7:00 it's all in percentages too, so it can be responsive, or, you know, be flexible. 7:05 [SOUND] So the other thing that I forgot to mention with masking is that 7:10 it can be used for very cool effects like this text shadow stuff. 7:16 So you may have noticed these kinds of, cool 7:21 text shadows before with the background clipped text property. 7:24 That was, being a part you know said, oh, it's really cool to use that. 7:28 But, I think Leia wrote an article saying you can do this with SVG, some time ago. 7:33 And so, basically, now, you can do it with CSS masking by referring to, an 7:38 SVG image and what you do here, is, where is this? 7:43 Oh, you, you don't even have a need to use SBG, you can just use png and you 7:48 can simply refer to a masked image on the 7:52 duplicated content to give this kind of a cool effect, 7:55 okay. 8:01 So can you use masking? 8:05 Well probably I mean, if you are not bound by 8:07 all browsers and you can provide a good, fall back. 8:12 There's also, some, somebody has done a very, thorough job of actually 8:15 trying to get it to work across all browsers with for an 8:20 object or whatever, say, you really, really wanna be desperately trying to 8:23 get workers all routes as you can, but I wouldn't recommend that. 8:27 So 8:30 the next thing I would like to talk to you about is, blend modes. 8:32 Now, how many of you know about blend modes in general? 8:37 Okay, okay, I need to ask a question to so that will flavor my jokes, but the 8:42 first thing is how many of you work on 8:47 graphical editors primarily in your day to day jobs? 8:49 Like you give away designs for developers who work on okay, a few of you. 8:52 Only a few are the people who are the cursed people who get those designs. 8:58 Okay, a lot of you. 9:04 Okay. 9:05 Good to know. 9:06 So you would have, have to deal with blend modes, the 9:08 designers will tell you, give you some thing that has an 9:11 image or whatever, a carousel with blend modes, and you're gonna 9:13 go and be cursing them and say, no you can't do that. 9:16 And, you know, you'll just be like, how? 9:20 How dare they do this to me, and this and that. 9:22 Well now you no longer have that excuse, at least 9:25 with the blend modes you can do this in the browser. 9:27 So this is one of the first things that my 9:29 team was working on to make it available in browsers. 9:31 So you know in, in, I tried to look in to what 9:35 blend modes actually do and it's not very easy to find out. 9:38 So basically there are two layers and blend modes 9:42 does something to the pixel values of those two layers. 9:45 So you have an active layer. 9:48 And you have a background layer. 9:49 And what the, browser does is, let's go through with an example. 9:52 So let's say you have a light in blend mode on two 9:56 layers so you have a, the browsers will go through the red, 9:59 green and blue channels for each pixel of those two layers and 10:03 then compare the active layer's pixel 10:07 value with the background layer's pixel value. 10:08 Now, if the active layer's pixel value is lighter than the background 10:11 layer's pixel value, then the active layer's pixel is rendered on the screen. 10:14 If it's the opposite, then the background layer's pixel is rendered on the screen. 10:19 So the way this will look like is 10:22 you'll have something like this, you'll see on the 10:24 image on the left there are some dark values, 10:27 especially the scissors you know, the hand holding area. 10:29 But if you notice, that is all replaced by yellow because 10:34 we're cover, combining with an active yellow layer, and so the yellow 10:36 the is lighter than the black that you see on the 10:40 screen, so the yellow gets rendered on the, on the final image. 10:43 Now there are two ways blend modes can happen. 10:49 The one is you can blend background images together. 10:51 Because you have multiple background images, you 10:54 can choose how those background images interact. 10:56 Or you can blend elements that are within one parent together. 10:59 Now, the blending of elements is not something that is completely 11:02 ready for even discussing yet and a lot of things happening there. 11:05 So, I thought I'll just cuz fix myself to talk about background blend modes. 11:09 And the way you would use background blend modes is You'd simply use a 11:15 property called backrground-blend-mode, and you will apply 11:20 whatever blend mode you want to apply. 11:23 So it can be lighten, darken, whatever. 11:25 So you'll have to apply it on two images or like one 11:28 background image and a color so that you can actually see something happening. 11:31 And so here is an example that I would 11:34 like to walk through to show how it'd look like 11:38 and okay. 11:45 So this is something that using background blending. 11:45 So there are two images and as you see, 11:50 the left plus right is equal to the final image. 11:52 So I am using, 11:55 so I have a background blend mode lighten now the, the order 11:59 in which you use these two images, the backgrounds, matters because it 12:04 works it defines what becomes the active layer or what becomes the 12:08 background layer so for example if I removed this you would see well. 12:11 WelI I guess you don't. 12:18 But, nevermind. 12:20 The idea is that when you replace this thing, you see a different 12:21 screen rendering because the kind of order in which you're combining 12:24 these things changes. 12:30 Okay, so there are a lot of blend modes you can use and some of 12:35 these are some things you would be familiar 12:37 to if you are using Photoshop's blend modes. 12:39 It's kinda of similar, in the sense that 12:41 it, you'd, it would work in a similar manner, 12:44 you'd have this, normal screen multiply, overlay, luminosity, all 12:45 of that stuff that you've already seen and used. 12:51 So, but the most interesting thing for me 12:52 is that you can actually apply multiple blend modes. 12:55 Because if you have, like, multiple background layers, 12:57 you can blend multiple ba, background layers differently. 12:59 So suppose you have like three background layers, so the 13:04 one, the first background layer that you define the background 13:07 image, would be the one that's closest to you, and 13:10 the last one would be you know, the farthest from you. 13:13 So the blend mode property can take multiple values too, 13:16 and maybe its best shown with an example let some here. 13:19 Go back here, so here is multiple background layers and they'll all 13:24 being blended so there's a background, big, huge cat image then there's of, 13:30 right in front, I have a small cat image that's being repeated 13:36 multiple times and then I also have a background color that is red. 13:40 Now, [SOUND] so I have, this is my first, second and the last is a color. 13:45 Now I'm doing multiple background blend modes. 13:49 So what happens now is that this lighten 13:52 blend mode gets applied to, to the two layers 13:53 that is the red color and the large 13:57 cat image because they are the last two layers. 13:59 So the cat image becomes the active layer and the red is the, background, layer. 14:02 So once that, those two are blended, then 14:06 they get a final in between rendered image. 14:10 That is being used now to blend with the final, with the first, very 14:12 first background image that is that 300 14:17 by 300 cat that's being repeated multiple times. 14:19 And we're using screen as a background 14:23 blend mode for those two images to interact. 14:25 And now you get the final interaction on the screen here. 14:28 [BLANK_AUDIO] 14:32 So can you use blend mode? 14:37 I guess not, no. 14:39 I mean it is it's all very cool and is very nice to see. 14:40 But it is available in Chrome and Firefox. 14:45 There is well we think it will be available in 14:46 Safari soon, we did a lot of work for that. 14:51 But until it is available I think in six months time this is 14:54 gonna change, but until then I would think not use it for now. 14:59 Okay, so the almost last one that I'll be talking about is filters. 15:05 Now, just like the blend mode, filters also, in 15:12 a way, apply to the pixel values of layers. 15:15 And, now you'll have the pixel value and that you have an image 15:19 processing function that applies to that pixel value to get final pixel value. 15:22 And the only difference is masking has a, a predefined thing that 15:27 you apply on and filters, this is just a function that you apply. 15:31 And there's a lot of, disturbing stuff that I have to read 15:35 for computer engineering, for image processing 15:39 about filters that I almost failed. 15:42 That I do not want to go into, during this time but 15:44 just assume that it's really annoying and really, too much fancy stuff. 15:47 And so how do you use filters in CSS? 15:51 Again, just like, masking, filters are some things that 15:55 you have already seen in SVG, like ten years ago. 15:59 They've been there for a long time. 16:03 In fact, SVG has a lot of crazy filters. 16:05 I think it's because SVG is 16:09 primarily developed by academy, which explains everything. 16:11 And, so if there was, then finally, I think it's about three or four years ago, 16:15 you know the CSS working group and the 16:19 SVG working group came together and they were like. 16:21 Well, I guess we should have these common elements, 16:23 common things be available, available for broader set of views. 16:27 So, they created a task force called a ef- I don't know, Effects Task Force. 16:30 And so, they started working on these common 16:35 specifications, sorta of which, filters and masking came about. 16:37 So, which is why you can use most of these SVG stuff which uses CSSproperties too. 16:41 So, there are two ways to use them. 16:46 One is again, with the SVG file you can refer 16:48 to a folder within an SVG file using the URL shorthand. 16:51 Or there are also filters that are predefined in CSS specifications. 16:55 So browsers like this, these folders are defined 17:01 in terms of their SVG equivalence in the specification. 17:04 So all browsers know exactly what SVG filter to 17:06 apply when you refer to them with the shorthand. 17:09 So what are these various shorthands that you have? 17:13 So you have like blur, brightness, contrast and whatever. 17:15 So all of these shorthands are actually function names, like, it'll 17:19 be like blur, and it's a function that takes you to value. 17:23 So, blur takes in pixel values, and, but the other, you know, 17:26 shorthands take some other values, some of them take percentages, and so on. 17:30 So, there's no consistent way to give them values. 17:33 So, 17:36 the way you would apply filters, is, is you will just say filter, and then 17:38 the function, and the value you want to pass in and it's kinda of very simple. 17:42 And if you're using photos through a URL, you'll be passing them 17:47 through the you know, the same way you refer to things in masking. 17:50 The thing to know is that if you're having 17:54 these SVG defined in line in your HTML, they're gonna 17:56 occupy space so you need to like display a non 17:59 or whatever to make sure they are away from you. 18:02 And you don't have to see it in your HTML. 18:04 Now again like blend modes you can also have 18:08 multiple filters and the filters apply from left to right. 18:11 And they're not commutative, in which case what I mean is the order matters. 18:15 So, you know, if you're applying the blur and the sepia after the 18:20 blur, you cannot, mix them up and hope everything works the same way. 18:25 So here's an example of how that would be. 18:29 So here is two, I have one webkit-filter, that's sepia 18:32 hue-rotate, and that's what's being applied on this. 18:38 But if I change this, hopefully this works 18:44 it needs to apply the hue-rotate and sepia after. 18:49 Great, the internet is really slow okay. 18:53 But ideally it will change in color, but if it worked, but it doesn't. 18:57 So, I guess you just have to take my word for it, what? 19:02 No I, I, I deliberately printed so that it doesn't apply yeah, but, 19:08 again the Internet is not working so it's not, I'm not having good 19:12 fun. 19:19 Okay, anyways, the idea is that would change in 19:19 value so the order matters how you apply your filters. 19:22 [BLANK_AUDIO] 19:25 Okay, so, can you use folders? 19:32 Again it's a maybe folders, I mean, folders are also kinda of pretty intensive 19:34 features to use and so you need to be careful with how you use those filters. 19:42 So if you really wan to use folders, yeah, go 19:47 ahead, but be careful on how, how much you use. 19:49 So this brings me to the most favorite part of 19:54 my talk, I still have 20 minutes, I can't believe it. 19:56 But the, this is that you have 19:58 hardware-accelerated properties in your browser these days. 20:01 And you know, Paul did a great introduction of what 20:06 that means,of what hardware acceleration even means, which is awesome, 20:09 which means this slide I made is totally useless, and 20:13 it is completely embarrassing compared to what Paul just showed you. 20:16 So, well, I'm sorry, anyways, the point to note is 20:19 that the hardware accelerated CSS properties works on the GPU layer. 20:24 Um,that's where they all operate. 20:29 So two of the things that I mentioned today 20:31 are hardware accelerated, the filters and the blend modes. 20:34 Now when you are animating hardware accelerated CSS, it minimizes the CPU/GPU 20:38 communication and so uh,it makes it 20:42 really, really like it completely outsources 20:46 all of that animation to the GPU so you don't have to 20:51 do, like, so the CPU doesn't have to do anything at all. 20:53 So here's an example, it's a very lame 20:56 example, but it's an example nonetheless which hopefully would 20:58 work. 21:06 Let's, 21:06 do I dare refresh, maybe I don't [LAUGH] let's maybe 21:08 I'll just do that. 21:22 Okay, I don't know if it'll load but the this is Safari because I tried 21:23 to find this in Chrome, and I don't know what happened, it used to exist. 21:27 So in Safari you now have like this panel called Layers where it will show 21:30 you what are the GPU axle rated 21:35 layer, content that's being rendered on the screen. 21:37 So here you notice that dev element is being axle rated and is being rendered 21:40 as a GPU texture, because I'm using 21:45 all these perspective transforms, and stuff like that. 21:48 And, I am actually, if you are using such things, and if 21:51 you are just changing the background 21:55 color, you could animate the background color. 21:57 So, you have a background color red, and then you may, maybe 21:59 you have, want to say, like, background color blue, or something like that. 22:02 And if you do that thing, what happens now the CPU needs to 22:05 tell the GPU for every frame what color it needs to use to render. 22:10 So the GPU will create a texture with the red to begin with, render it 22:16 on the screen and then wait for the CPU to tell it, what's the next color 22:19 it needs to render on the screen, so it will get that color, create the 22:23 texture, render it on the screen and then, 22:26 it has to do that, communication multiple times. 22:29 So, that communication bandwidth is kinda severely limited, and if you 22:31 keep doing it multiple times, it's gonna slow down your application. 22:35 So instead of doing that, you could use these hardware 22:39 CSS axle rated properties to do this kinda of animation. 22:42 Which is why, here in this example what is being used is, a hue rotate filter. 22:46 So hue rotate is kind of similar, 22:52 well it's not exactly changing the background color. 22:55 But it basically hue rotate is a photo that changes the 22:57 color values and because it is a filter, a CSS filter. 23:00 It can be completely outsourced to the GPU. 23:05 So in this case, what happens is that the CPU tells the GPU, in the beginning of the 23:08 animation, that hey I need to change the colors of this image 23:12 that I have, I'll be providing you, and you need to change it over the 23:19 course of one second, and this is whatever you need to use to change it. 23:23 And then the GPU is just gonna be like using 23:27 that information and do all of those changes on its own. 23:29 And it, without having to communicate with the CPU. 23:33 So that's gonna happen much more, like it's, that's completely isolated. 23:36 So it doesn't have to keep talking to the CPU. 23:40 And so that's gonna be much, much faster and smoother to animate. 23:42 [COUGH] 23:52 >> So lets wait for did it load well apparently, maybe not. 23:53 Okay, so well, wow I finished it in 15 minutes, oh, my god. 24:01 Please don't kill me John, so basically, that's all I had for you. 24:05 Which means that I really flew by and all of 24:09 the stuff that we've been working on, all the things 24:13 that I just mentioned to you, all of these things 24:16 that's being worked on by the web platform team at Adobe. 24:18 And if you need to know about the latest status of the specifications 24:21 and things like that, you should follow the Adobe Web Team on Twitter. 24:26 And there are two folks who I'd like 24:30 to thank for helping me with this presentation 24:32 and inspiring me with demos, is CJ Gammon, who works with me at Adobe San Francisco. 24:34 And Razvan Caliman, who's actually in London, and who basically did a 24:40 version of the slide deck for his own talk some time ago. 24:44 And they both are working on like really cool demos, that showcase some of these 24:48 features in in manner that is that is 24:51 very persuasive and compelling with hyper-file case studies. 24:54 So if you follow them, you'll get to know what those .docs 24:58 are and do that anyways, so that all I had for you. 25:03 [NOISE] 25:07 [MUSIC] 25:08
You need to sign up for Treehouse in order to download course files.Sign up