Bummer! This is just a preview. You need to be signed in with a Pro account to view the entire video.
Vagrant, Puppet & Chef #FTW: Managing Development Environments, Made Easy40:38 with Thijs Feryn
Everyone has used the term "but, it works on my machine ..." before and it usually resulted in disaster. Managing your local development environment has always been tricky, especially when you want it to be an exact copy of your production environment. Vagrant tries to solve this problem by offering a simple tool for managing lightweight, portable and reproducible development environments and is suitable for both individual developers and development teams. This talk will cover the basics of Vagrant including a brief introduction of configuration management tools like Puppet and Chef to provision your machines. "vagrant up" will be a command you'll learn to love. Aimed at developers looking for an easy way to manage their development environment and sharing it with team members, you'll learn Hassle-free setup of your development environment, how to improve collaboration and avoid dependency clashes between different projects, moving one step closer to continuous delivery.
[MUSIC] 0:00 Hello everyone. 0:11 Welcome to FOWA. 0:13 How are we doing so far? 0:14 Let's see some hands, let's see some thumbs. 0:16 Are we doing fine? 0:17 How was the schedule, how was the coffee? 0:19 How is the vibe, everything doing great? 0:22 Okay let's start off. 0:24 As I mentioned my name is Thijs and if you 0:26 want to follow me on Twitter you can do so. 0:28 This is my Twitter handle my full name. 0:30 I know it's a bit of a weird name but that's a Belgian thing. 0:31 I'm a tech evangelist at a company called Combell we 0:35 do web hosting, and as Ian mentioned I'm also involved with 0:37 the PHP community run a local community called PHP Benelux, 0:41 where we just run the Belgian, Dutch, and Luxembourg PHP community. 0:45 Not much happening in Luxembourg, though. 0:49 So let's talk about Vagrant right now. 0:51 Let's see a show of hands, who uses Vagrant right now? 0:54 [SOUND] Oh, that's interesting, that's good. 0:56 Cuz maybe you'll learn a thing or two. 1:01 Now, Vagrant, and that's a definition I got 1:03 off the site, allows you to create and configure 1:05 lightweight, reproducible, and portable development environments and I've 1:08 marked development environments because that's what it's all about. 1:12 It's not about having a production system up and running or a staging system. 1:15 No, it's about the box you develop your code on. 1:19 Though it's written in Ruby by Mitchell Hashimoto in 2010. 1:22 And it's actually an orchestration layer, on top of, first of all, Virtual Box. 1:26 And in later stages, they added support for vmware. 1:30 So it's basically a scriptable way of maintaining your virtual machine. 1:33 So let's see some more show of hands. 1:37 Who develops codes on its local machine? 1:40 Let's say, WAMP, MAMP kind of stuff. 1:42 Oh. 1:46 That's a lot of people. 1:46 Sorry. 1:48 You are disappointing me with that, but hopefully at 1:48 the end of this talk we will be changing this. 1:50 Now, who has a separate development, virtual environment on its local machine? 1:52 I asked the question, who uses Vagrant. 2:00 I see more hands right now, so that means you are doing it differently. 2:01 How are you doing it? 2:04 Just shout out. 2:05 Who uses Virtual Box without Vagrant? 2:08 VMware fusion or regular VMware. 2:12 Okay. 2:15 So maybe at the end of this talk you'll be using 2:16 this nice little tool to easily deploy, create and manage your VMs. 2:18 So who uses a remote development box at the office? 2:24 Okay. 2:28 It's perfectly viable. 2:28 So it, let's get back to Vagrant. 2:31 Vagrant tries to solve a specific problem, so why would you use Vagrant? 2:33 This is the main reason why we do it, we all 2:36 know the guise it works on my machine, and then you have 2:38 this awful joke, well, backup your laptops if you're gonna put 2:41 your laptop in production, this is what we're trying to avoid here. 2:43 I work at a hosting company. 2:46 I've did six and a half years of tech support, and my, 2:48 I have to worry about other people's code, and that's a bad thing. 2:51 That's a really bad thing. 2:54 And, other people say, well it works on 2:55 my machine, why doesn't it work on your server? 2:56 So, we're trying to avoid that, and that is actually the main goal of this talk. 2:58 This is what we're here for, or what I'm here for. 3:02 So wanna do is, have an environment per project, not 3:05 a monolithic system on which you run all your code. 3:07 Cuz some code is gonna run a specific version 3:10 of PHP, or Ruby, or Python, or whatever you use. 3:12 Some things will be clashing, so you want sort of separate 3:15 silos per project it can easily set up, easily tear down. 3:19 And easily to script. 3:23 And what you wanna do is, if you script 3:24 it, it needs to be in a very understandable way. 3:26 It shouldn't be really complicated, like complicated XML now. 3:28 Just a file with a certain markup that says how you want your machine being done. 3:31 What software should needs be configured. 3:35 All, all sorts of properties. 3:38 And make sure that your development is like test, staging and production. 3:39 I added the tilde. 3:43 It's, you'll never, or hardly ever have an exact copy of production. 3:44 Having an exact 100% copy is, ambitious. 3:49 But if it's very similar to test staging and production, at least you 3:54 can get your hopes up that your code will be working on that system. 3:58 So, easy to define, transport, and also and this is an important one, easy to 4:01 tear down as well, because all these 4:04 images, all these virtual images will consume resources. 4:07 Especially disk. 4:10 And if you're done working on a project for a while, you might wanna tear 4:11 it down and work on something else, And this is an important one as well, here. 4:14 Shared across the team, so that configuration 4:19 which I will show you later on. 4:21 You can just transport that along in your 4:22 version control system, shared amongst your team members. 4:25 And if we would run Vagrant, and I hope you can read this. 4:28 I tried to make it, the print as big as possible. 4:30 If you run the Vagrant command, because it's basically a command line 4:32 tool, if you run it, it will show you a bunch of options. 4:36 And this is what this talk will be about, showing what 4:39 these options mean and how we can reach the end goal. 4:40 So, if you run Vagrant in it, because that's the main one we're gonna do. 4:44 And it's, it's such an easy job, right, being in it? 4:48 In it, using the term in it here. 4:51 I'll be light on the jokes. 4:52 You had plenty of jokes during the opening keynote I guess. 4:53 So so vagrant in it if you run that 4:56 one it will give you some verbose information on how 4:58 you can create your Vagrant file and what does that 5:01 Vagrant file do because that's an important one to know. 5:03 Vagrant file is a, that's scripted way of selecting a base box, 5:06 and a base box is what the next chapter will be about because 5:10 you need a basic VM that is prepped in which you can 5:13 deploy all sorts of things or install or so, all sorts of things. 5:16 And then you can choose your virtualization provider 5:19 that will mainly be either VirtualBox or VMware. 5:21 Twiggle around with VM parameters such as CPUs, number of 5:24 CPUs, you wanna configure RAM settings, et cetera, et cetera. 5:27 Networking settings such as H settings. 5:31 Mounting your local file system. 5:33 Which is very convenient for developers. 5:34 And finally, provisioning the machine. 5:36 So that, that's what we're gonna do. 5:38 So if you run Vagrant in it without any options it will drop you a Vagrant file. 5:40 And if you open it in your text setter, you'll 5:44 basically see that it's, has the Ruby markup for your editor. 5:46 So it's plain old Ruby but in a specific syntax. 5:49 So you're gonna configure a machine. 5:51 The two means that we're gonna use, 5:53 Vagrant verion twowhich is important because it 5:55 adds a lot of nice features, and 5:57 we're gonna namespace all the properties as config. 5:58 And then you could say config VM box is base. 6:00 So there's this sort of convention that you have a base box. 6:03 A base box that represents your, that 6:05 Linux image you're gonna run everything on. 6:08 And that base box is gonna be cloned every time you make a box. 6:11 You could also add some specific parameters, say, 6:16 we're not going to use a base box. 6:19 We're going to name it precise32, which is an Ubuntu version. 6:20 And this is where I'm gonna download it, and it's gonna just put it in the file. 6:24 It's not gonna actually download it. 6:27 That happens when you start it up, but it's gonna pre-provision 6:28 everything you need, and you can add whatever box you want. 6:31 You can have whatever distro you want as long as you stick with Linux. 6:34 And then the magic commands and they, this is like if you have 6:39 to remember one thing from this talk and one thing only is this command. 6:42 Vagrant up, it's all you need in life Vagrant up. 6:45 And then you just press the Return button 6:49 and it will just give you some verbose information. 6:51 Since the precise 32 box is not really on the 6:53 system right now is gonna download it you can see the 6:56 progress here with a sort of an estimation, and once 6:58 it's added, it's going to store it in your system globally. 7:01 So in the future, when you have that box in there and 7:04 you want another project using it, you don't need to download it anymore. 7:06 So, it's there. 7:10 And then it says all the stuff it does to your machine. 7:11 It, imports the bulks. 7:13 Does some netting. 7:15 Setting the name of the VM. 7:16 Port forwarding, networking interfaces. 7:18 This is a specific one I'll mention. 7:21 Booting it up, configuring it, and mounting a folder. 7:23 So you see what it does. 7:25 But back to base boxes, because boxes are important. 7:27 There is a place where you can download them. 7:30 It's a sort of community site called Vagrant Boxes. 7:31 And it has a bunch of those, it's a community initiative. 7:34 If you want to tailor your own boxes, there's 7:37 project called VWii by a guy named Patrick DuBoise. 7:41 He's the guy who invented Devops, the term Devops. 7:44 And he maintains this project and it's an easy way 7:46 to start from a basic ISO and create a Vagrant Box. 7:48 You can tune it you can strip off stuff you don't need. 7:52 It's a really good tool, so when we type in Vagrant Box. 7:54 You're gonna see this kind of stuff happening. 7:58 The basic stuff, right? 8:00 Adding, removing, listing, repackaging. 8:01 And I have some code examples here. 8:04 Like, this slide is heavy on code examples. 8:05 So you'll see lots of this. 8:07 So, if you box list it, you'll, you'll see 8:09 that the, the machine we previously imported is there. 8:11 It's a virtual box machine called Precise 32. 8:13 We could remove it, we can add it again. 8:16 And mention where it needs to be. 8:18 Now. 8:20 Contrary to the previous slides where we did the Vagrant 8:22 in it with this box, this will actually download the box. 8:24 This could be a URL or a local file. 8:27 It will import it and make sure it's available when you boot a machine. 8:29 And if you want to redistribute that box, let's say you 8:32 just did a, a Vagrant box add from a specific file. 8:35 The file is no longer on your local folder. 8:38 It's, it's just stored in, internally in the system. 8:41 You can make it reappear by doing vagrant 8:44 box repackage, and it will come as a packaged 8:46 box and you can put it online, redistribute 8:49 it to your team, whatever you want to do. 8:51 Now, I would like to remind you that all of this, the 8:54 box management or downloading boxes, can be done in your Vagrant file. 8:56 Just mention config.vm box and URL and it will download it if you don't have it. 9:00 And this is what a box look like. 9:04 It's, the box is just a TRGZ, sort of archive, containing an VMDK and 9:06 OVF file, an meta adjacent, and the 9:10 meta adjacent usually contains a virtualization provider. 9:13 Either Virtual Box or VMWare. 9:15 And a Vagrant file that bootstraps other Vagrant 9:17 files, because that's the cool part, when you 9:20 type Vagrant up, in a folder that contains 9:21 a Vagrant file, it will bootstrap the entire thing. 9:24 So now that we know what Vagrant does. 9:26 And now that we know how we can work with base boxes. 9:28 Let's get up and running. 9:31 So, let's get starting. 9:32 Let's start, stop, suspend. 9:34 Let's do all the cool stuff. 9:36 So if you do Vagrant up again. 9:37 That's the command you'll love by the end of this talk. 9:39 Or, or maybe next week, when you toy around with it. 9:41 You'll, just send me a tweet if you like it. 9:44 For the people who haven't tried it, just send me a tweet. 9:46 So if you do Vagrant output, it will do all 9:48 of that stuff again, when the VM does not exist. 9:50 But when it, it exists, it will just show you some basic information, 9:52 like it was already running, I'm not gonna bother making it boot now. 9:56 A reminder, I may be a hint for you. 9:59 Who has a laptop containing an SSD? 10:02 Who doesn't? 10:05 This will take some time doing a Vagrant up, if you don't 10:07 have SSD, could take some time, could take a couple of minutes. 10:10 So, if you don't want it destroyed, because Vagrant destroys 10:14 the command, you're gonna tear down your entire environment with. 10:17 If you just want to suspend it or, or 10:20 halt your machine, there are commands for that as well. 10:22 So with the status you can see 10:24 what is happening, it's running, it's actually running. 10:24 And when you do suspend, it will just pause 10:27 your VM and keep the current machine state on disk. 10:29 And when you resume when you do Vagrant resume or 10:31 just Vagrant up again it will just restart a machine. 10:34 There are cases where you don't want to do that, and you just say 10:36 I want to shut down, and you do Vagrant alt and it shuts the machine 10:39 down, and it's still in your, when you open your virtual box or when 10:42 you open your VMware you'll still see the machine, but it'll just be shut down. 10:45 And as I mentioned before, if we wanna destroy the 10:50 machine, if we wanna tear it down just Vagrant destroy. 10:52 It won't only shut it down, it will remove it from your virtualization provider. 10:55 So it will be completely gone. 11:00 It won't be consuming any RAM CPU nor will 11:01 it be consuming disk, so that's what it basically does. 11:04 So what have we learned so far? 11:07 We've learned how to set up what a Vagrant file does. 11:08 We know how we can import boxes. 11:12 We know how to set it up, tear it down 11:13 but now we have that we need to connect with it. 11:15 So connecting is is an important part of 11:17 Vagrant and a part that is made incredibly simple. 11:19 You just type the vagrant ssh command and it will automatically log in. 11:23 No need to know the IP, no need to care about passwords, or your own keys. 11:26 It will just go in there, do pseudo su, and your root, in the machine. 11:32 Now in the background, so when you go on that machine, and when you root, and you 11:37 go to the all trice key files, you'll 11:40 see that there is a public key automatically provisioned. 11:41 And it says here very specifically, vagrant insecure public key. 11:45 So it's something that Vagrant manages for you. 11:49 And when you type the command Vagrant as 11:50 SSH Config, you will get SSH Config for that. 11:52 So there's been a default there. 11:55 Where there's a default host, default user, a default 11:56 port and what this does actually is make sure 12:00 that you do a local ssh connection to port 12:02 222, and it will be port forwarded to the IP. 12:05 In this case the, the machine itself will log in using the user Vagrant 12:08 and will be using this insecure private key to match the public key. 12:13 On a private key this is my computer user stays on 12:18 Vagrant here, so there's this global space on my laptop here where 12:21 the key is stored and this is what it looks like, 12:25 nothing spectacular, you can use the key there's no privacy involved here. 12:28 The key aspect when we run these Vagrant machines when it's up and 12:34 running when we connect it is also the portability of the Vagrant file. 12:38 I want to briefly also touch on that because that's important. 12:41 I mentioned already that you can use the Vagrant file. 12:43 This is a GitHub project of an open 12:46 source tool or an open source project called Joinedin. 12:48 And a lot of people want to contribute to it. 12:52 And having the machine requirements often requires some documentation. 12:54 Like set up your machine as such you need these versions. 12:57 Instead of going through all that hassle, the maintainers just dropped in a vagrant 13:00 file, which is versionable as well, so if you need to change the specs, 13:04 you could easily do so and just, clone it, clone the GitHub repository, just 13:07 type vagrant up, and you have some puppet manifests that will provision the machine. 13:11 And we'll touch on puppet as well, as well as Chef. 13:15 So, what we wanna do is share that vagrant 13:18 file and as a consequence, share your entire development 13:20 environment with contributors of public open source projects, or 13:23 just people within the company who are on your team. 13:27 So what we're basically saying is infrastructure as code and this is a 13:30 big term coming from the Dev 13:34 Apps community, having infrastructure there as codes. 13:35 So you remember this one? 13:39 Our little Vagrant file only containing the box, and the box URL. 13:40 Well, let's do some actual stuff. 13:44 Let's provision it. 13:46 Let's start with some networking because that's something you want 13:47 to care about, if you want to connect, not only 13:50 to raise its age but with a web browser, let's 13:52 say, you need to make sure how to connect to it. 13:54 There are three ways of networking. 13:57 Public networking, private networking. 13:59 And just doing port forwarding. 14:01 The public networking just bridges to a, a network interface. 14:03 And just let it handle everything networking related. 14:06 So you're part of the public network. 14:09 Everyone else could manage or access your box. 14:10 If you don't want that, you can do some regular port forwarding. 14:13 It doesn't, the next string that we're interfaced to the machine. 14:16 It just says while the port 80, which is by convention 14:19 the web server port, will be linked on your computer to 8080. 14:22 So if I type local host 8080, I will end up pinging within the machine. 14:25 So that's what it does. 14:29 And personally, I prefer private networking because it's simple, it's 14:31 straightforward, you just choose an ip, within the private ip space. 14:35 Watch out if that is the same ip space of your 14:39 internal network or your company that my, the, generate some ip issues. 14:42 And let's see the examples here. 14:48 This is how you do, do it into Vagrant files. 14:49 So private network just add within the global configure. 14:51 At config VM network. 14:55 Private network ip. 14:57 One vagrant up will take care of everything. 14:59 Now, mind you, we didn't choose port forwarding, 15:01 but we're still seeing some port forwarding here. 15:04 Is this still something we see, why? 15:06 If you pay close attention, this means 22 goes to the 2222. 15:08 This is for Vagrant as it is h. 15:12 So, you ssh on this port, and you end up in this machine. 15:14 So, this is required for just ssh connections. 15:16 If you use port forwarding you can see the extra line being added. 15:21 So it's added to adapter 1 port 80 internally is matched to 80/80. 15:24 So it's pretty clear when you do a vagrant number 15:28 you see what's happens and finally when we choice public networking. 15:29 And when you do the vagrant up you will get an option, you 15:33 will get an option, which interface do you want to bind it to? 15:36 And I chose 1, to my WiFi, I tapped Enter. 15:39 And which is linked. 15:42 If you wanna pre-link it, there's an option of bridging, and then 15:44 you have to mention your network interface you wanna link it to. 15:47 I don't consider this very convenient. 15:51 In some cases you might use it. 15:53 I always stick with private networking. 15:54 This is the number one reason synced folders, why you should pick, fragrant 15:57 over constantly deploying to a VM, because it has a built in system, of 16:03 mounting your local file system, of just the local file system of the current 16:08 directory, in which the fragrant file re, resides, just mounting it to the system. 16:11 So when you're in you IBE, you can have your local file system available. 16:15 And you can just save directly from your editor or IDE, 16:20 which is very convenient for developers, because you don't want to change 16:23 a piece of code, upload it to your VM, and then 16:26 test it, and the uploading or deployment process might take some time. 16:29 No, it's instant. 16:32 You press Save, you refresh your browser, you see results. 16:33 And by default, there is a mount a slash vagrant mount. 16:37 So that means your current folder is available on your guest OS, and 16:40 when you go to the slash vagrant you just see your local system. 16:44 And when you change something on your computer, it's going to be changed there. 16:47 So, in a lot of the cases people will link this folder to their web server root. 16:50 What I tend to do is add another sync folder. 16:54 Say you have your project and, public files are located in web. 16:57 Or I'm gonna link them to /var/www, and when you install 17:01 Apache, you immediately have a link and can immediately navigate through it. 17:03 And you can see it here when you do vagrant up, you will see this happening. 17:06 So we did quite a number of things already. 17:11 Let's tune some VM properties. 17:14 This happens also through the syntax. 17:17 This is not a complete vagrant file, this is just the snippets that 17:19 is relevant, you just have to shove it within the main configuration block. 17:21 What we do here, these are some stuff, this is some stuff. 17:25 I put a Vagrant file online because I do some training as well, 17:27 and I added varnish installation and I wanted to distributed it to Vagrant. 17:30 And I did it. 17:34 And then some clever person said, you might want to add these parameters. 17:35 Some of these solve kernel panics, other ones 17:38 help you have a better deal as a result. 17:41 It's just internal magic from from virtual bugs but it, it actually helps. 17:43 The only parameters that are very useful to you is memory and CPU. 17:48 This is how you can tune how big the box 17:51 is, like if you need some extra horse power in it. 17:52 You could-, might want to increase this this is the way it works. 17:54 So these are the basic steps. 17:58 This is how you manage a Vagrant box when you have 18:00 it up and running, you won't have any specific software on it. 18:02 This is the next chapter. 18:06 But you will have it tuned, the machine tuned the way you want. 18:07 And we will be dealing with the most important part, provisioning it. 18:10 Making it lot like production. 18:13 Making it look like test and stage. 18:15 Now, provisioning could mean anything, but what it basically 18:17 means to me is adding specific software to it. 18:21 So, using packages to do that. 18:23 Creating configuration files where you need them. 18:25 Execute certain commands to prep your system. 18:27 Create users. 18:29 Manage services, and this all needs to happen automatically. 18:31 You don't need to care about that in theory. 18:34 You have people in your team who are going to write all the provision 18:36 you need, and just type again, and I'll be repeating that a number of times. 18:39 Vagrant up. 18:43 Machine comes up. 18:44 Provisions automatically. 18:45 So, having an exact copy or a similar or as good 18:46 as an exact copy of your environment is the main goal here. 18:51 And we're gonna use, there's several tools out there. 18:54 There's a bunch of tools out there. 18:56 I just marked the ones that I'm going to mention. 18:57 Now, this talk is called Vagrant, Chef and Puppet for the win. 18:59 While it's gonna be mainly focused on Chef and Puppet. 19:02 But you can do plain old shell, just Linux commands. 19:05 Ansible is getting a lot of traction these 19:08 days, as well, Anyone who's heard of Ansible? 19:10 It seems like the cool thing to use these days. 19:13 I like Chef, I like Puppets. 19:17 I personally prefer Puppets. 19:19 I'd like to standardize on Puppet. 19:20 By Chef is, is pretty awesome, itself. 19:21 You see different kinds here. 19:24 Solo clients apply agent. 19:25 You can do a stand alone, wave provisioning, where 19:27 all the scenarios, how you're gonna set up the software. 19:30 Are also in your code repository. 19:33 You saw the screenshot of the join in project had a Puppet folder, so it locally 19:34 stores all the software you wanna install, all 19:39 the configuration files, services you want to manage. 19:41 When you don't want that and when you just wanna connect to 19:44 a configuration management server, gonna use a Puppet agent or Chef clients. 19:47 So it connects over TCPIP to a centralized 19:51 server, the same server that deploys all the code 19:53 and the infrastructure to your production environment, while 19:56 you're gonna connect and hook into the same one. 19:59 Let's start with the beginning. 20:01 Let's start with Shell. 20:03 Shell is plain, shell is simple, and shell is pretty powerful. 20:04 Like, I run into this problem a colleague of mine who is sitting on the third row 20:08 there, wanted to deploy a Symphony app in 20:12 PHP and it was insanely slow on it's system. 20:14 So I said well the cash folder, which is located in vagrant app cache. 20:17 You wanna have a rundisk for it. 20:21 So instead of just writing a complicated Puppet script for just the config vm 20:23 provision, what kind of provision shell, where 20:27 is the, where does the description reside? 20:30 Well, it's inline and then mount tmpf, 50 mgs of RAM disk good to go. 20:32 Pretty simple, you can do even more than that. 20:38 You can you can op, you can use full Ruby because it's has full Ruby 20:40 syntax, and you can do whatever you want 20:45 there in Ruby to manipulate what's coming in. 20:46 And if you wanna petition it separately, have 20:49 separate files for that, you can either use 20:52 local files using like in this case, script 20:53 dot sh, or you could download it remotely. 20:56 But we're not, or we're no longer gonna talk about that. 21:00 We're gonna talk about the stuff that has 21:02 actually built the provision machines, either Chef or Puppet. 21:03 Anyone using Chef these days? 21:06 Anyone using Puppet these days? 21:09 How do you, have the other people provisioned their machines on their, 21:13 or are you not involved with the deploying of code or configuration. 21:17 How does it happen in your companies, manually or 21:22 have all the hosting company take care of it? 21:26 Probably. 21:29 Okay Chef and Puppet. 21:30 Chef is a tool created by OPSCODE. 21:31 It's written in Ruby. 21:33 In 2009, now it's an open source project but it's company 21:34 backed they have an enterprise revenue model, and the same thing applies 21:37 to Puppet, see the pattern here, this one is written in Ruby 21:41 as well by Puppet Labs in 2005, that's the company backing it. 21:43 Chef and Puppet again, such a, so Vagrant is written In Ruby. 21:48 Chef is written Ruby, Puppet is written in Ruby, a lot of stuff happening in Ruby. 21:53 Now as the introduction went I'm a PHP guy, but I 21:57 have to accept this Ruby is big, Ruby is out there. 22:00 And I think next year, next time around I'm 22:03 gonna be learning Ruby to master this stuff more. 22:05 So both of these tools written in Ruby, 22:08 open source, enterprise model behind it, similar features. 22:11 Both stand alone or server-side. 22:14 You can choose. 22:16 It's supported by a large community. 22:16 So, if I had to say which community is 22:19 better, I wouldn't be able to answer that question. 22:20 It, it depends on the flavor you want. 22:22 It's modularized. 22:26 So, the components you gonna download there are parametrized, modularized. 22:27 So, you don't need to figure out the nitty-gritty. 22:31 It, they offer a nice little interface. 22:34 That you can add parameters. 22:36 Let's say for MySQL server, the root password or the location 22:37 of the data file is just gonna abstract that for you. 22:41 And they're gonna use all kinds of stuff 22:44 for file system, for templating,uh, it's, it's pretty easy. 22:46 If I have to say how they differ, I would say 22:50 in terms of the lingo, so, modules, the way you modualize things. 22:53 Chefs call these cookbooks. 22:57 They're gonna stick to the culinary, naming there. 22:58 You're gonna have cookbooks. 23:01 And within the cookbooks, the specific items are 23:03 recipes Whereas, Puppet just calls 'em modules and manifests. 23:05 Now, the main difference, language wise is that the Puppet 23:09 just has a, it's own DSL you could work with. 23:13 Whereas Chef, it's for Ruby, extended with a DSL. 23:15 So if you're a Ruby guy, in a lot of the cases you're gonna pick Chef. 23:19 Because it's just the way you like doing things. 23:22 The approach here is, it's, it defines actions. 23:25 Whereas Puppet more or less defines states or how, on how you want it. 23:28 It doesn't really say, do this, do that. 23:32 It's says, make sure its like this, it's make sure its like that. 23:34 Whereas it is more procedural on the end of Chef, it 23:38 will be more object oriented on the side of of Puppet. 23:40 So instead of talking about it, let's just do it, right? 23:45 How to use it. 23:48 There's a GitHub repo, by Ox Code itself. 23:49 It contains modules for all the popular open source projects. 23:51 You could figure out web servers, database servers, all the components you want. 23:54 You can just clone it to your, your directory where you want it. 23:58 Make sure you link it in your vagrant file through chef.cookbookpot. 24:01 And if you want to include recipes, just use chef@recipe in your vagrant file. 24:05 I'll show you later on. 24:09 And if you wanna interface with all the attributes you can modify, you can 24:11 just use chef.json, json, and have inline JSON with all the parameters you want. 24:14 And, or, when necessary, you can create your own cookbook. 24:19 But in a lot of cases, I will 24:23 advise you, don't reinvent the wheel unless truly necessary. 24:24 So this is what it looks like in the grand scheme of things. 24:28 Again, this is not the complete Vagrant file, 24:30 this is just a config, that is relevant. 24:32 So we're gonna say, we're gonna provision our 24:35 vm using Chef Solo, name space, everything, under chef, 24:37 and then say, the cookbooks are located in the, 24:40 in the current directory under a directory called tools/chef/cookbooks. 24:42 And the recipe I want to add basically comes from the MySQL Cookbook. 24:46 And it's called server. 24:50 So we're gonna install a MySQL server on the system. 24:51 And then we use chef JSON to to parameterize whatever we need to do. 24:55 And we're gonna say the password for 25:00 application for server, just server authentication using root, 25:02 or the Debian built in user is gonna be foo, and you can connect with foo. 25:06 Now, the layout of the cookbook is 25:10 pretty plate, straightforward, pretty plain and simple. 25:12 There's a lot of directories and files there, 25:14 but I marked the ones that are specially relevant. 25:16 So we get, gonna have the recipes and the default recipe 25:19 is called default.rb, and there you're gonna specify the main one, so. 25:22 In terms of bootstrapping This one is going to get loaded first. 25:25 Templates are stored in the templates default folder, so, it's erb style. 25:29 So, if you can just drop Ruby style templates in there. 25:32 And everything that it needs to 25:36 be parametrized resides in the attributes folder. 25:37 I'm going to show you some quick examples. 25:40 These are not going to be full examples because the 25:41 code base is so extensive, it hardly fits on the slides. 25:43 So this is for MySQL. 25:47 All stuff you could, configure. 25:48 But please note that the configurations might depend on platform families. 25:50 So you're gonna have a difference between Debian style, or, red hat style. 25:54 So you need to make sure, if you download 25:59 one, it fits the infrastructure you have, at the office. 26:01 This is a server recipe, a pretty simple one. 26:06 And this actually shows the power of Chef, in this case. 26:08 So we're gonna have a group. 26:12 So this is gonna define a group on your 26:13 Linux operating system, called MySQL, is gonna create it. 26:15 You're gonna create a user called MySQL. 26:18 It's gonna reside in the MySQL group. 26:20 And the location so the home directory of that user is pre-defined. 26:23 No MySQL data there. 26:27 So that comes from attributes. 26:28 You can tune that. 26:29 And in the end, it's going to have a no login shell. 26:31 And then based on all the packages, that are located in the attributes as well. 26:35 So in the configuration, you can loop through it. 26:39 And it will use the package manager of your operating system to install it. 26:41 So, on Debian systems and Ubuntu systems that would be apt-gets. 26:44 On Red Hat, Fedora, or CentOS, that's going to be yum. 26:48 Packages can also be Ruby, Gems or even PHP para installs or whatever you want. 26:51 You can even configure it and write your own providers for it. 26:56 Templates as mentioned, basic ERB, so you can just 27:00 add variables and they will be dropped into there. 27:03 That's, it's just that simple, it does require a change in mind set to work 27:07 with these kind of tools but this is infrastructure as code in the purest form. 27:10 Most of us are developers. 27:15 Infrastructure sometimes is a strange concept that 27:17 we like to hand over to Cis admins. 27:19 But this is just giving you the power to find the things you need. 27:21 You'll turn your Sysadmin into a developer. 27:24 And a developer into a Sysadmin which is a developers way of thinking. 27:27 Typical resources you can tune. 27:32 This is just a short list I added the three dots, there's more than this. 27:34 So Users Cron's Directories executing command files 27:37 installing packages running services, so everything you need. 27:41 Again, I will advise you to download components that are offered 27:46 through the community, but if you want to do it yourself. 27:50 This is a way of doing it yourself, say we create 27:53 a project cookbook and in the recipes we add a default recipe. 27:55 We're gonna do an, an update of our 27:59 channel, so we have the freshest channel, so we 28:01 know when we install a package called MySQL server, 28:04 or Apache, that these are the most recent ones. 28:07 After installing the MySQL server, you're gonna you're gonna make 28:10 sure the system starts, so you're gonna invoke the service. 28:14 The Apache server, when it's installed, make sure it's started as well. 28:17 You're gonna do this delayed here. 28:21 Why? 28:22 Because there is a next step coming. 28:22 And that will require a restart of the server, as well. 28:24 So you can actually schedule and script what the entire installation procedure is. 28:26 And then we continue to assign a root password to 28:31 the system only if there's not a root password present. 28:33 We're gonna manage a service, so this is an actual 28:37 service definition that allows us to start, stop, restart system. 28:39 So this is your system, your entire 28:43 installation wrapped into a couple of files. 28:45 And when we wanna do it ourselves, this is what our 28:48 Vagrant file, this is what gets exposed to the front looks like. 28:50 So we're gonna say we're gonna use Chef 28:54 slo, solo, this is where the cookbooks are located. 28:56 We have a project recipe, and the root password of the project is foo. 28:59 We just do Vagrant Up boom, MySQL server, Apache web server, and a bit of PHP. 29:03 When you do Vagrant Up, you're gonna see 29:09 that there's a provisioning run happening as well. 29:11 So there's an extra mount being added to 29:13 tmp Vagrant Chef, and it contains all the cookbooks. 29:16 Cuz the local machine needs to run the Chef true. 29:19 And you get a lot of verbose information. 29:21 A hell of a lot of verbose information. 29:23 I marked the ones that are useful to us. 29:25 So, the services, the execution of all this stuff. 29:27 So we can really see what's happening. 29:29 And you can use that to debug, in case something goes wrong. 29:31 That was Chef. 29:35 Puppet is pretty similar. 29:37 I think it would be easier for you to follow along right now. 29:38 Because you already have a vague. 29:40 Idea of what configuration management can do, 29:42 and Puppet is very, very similar to Chef. 29:45 It's just a different flavor. 29:48 You have your cookbooks as well, or modules as they're called. 29:49 And Puppet Labs has it's own GitHub channel as well. 29:53 Everything is stored in there. 29:57 It's a bit different how you interact with them. 29:59 So you have a module path as well, very similar to the cookbook path. 30:01 But you don't add recipes in your Vagrant file, you just include a manifest, 30:05 and that manifest is just your bootstrapping 30:09 ugh file, that does all the installations. 30:11 If you want to transport custom variables, 30:13 custom attributes from your custom Vagrant file. 30:17 To the internal Puppet manifests. 30:19 You can use Puppet dot factor and add custom 30:22 effects, and these will become variables put into your scripts. 30:24 So, Vagrant file looks a bit like this. 30:28 Manifest paths, module paths, manifest file, so the init dot ppp resides 30:31 within the puppet manifest folder and it looks a bit like this. 30:35 So with this single line you can setup a MySQL server, just install it. 30:39 It won't contain a password, so if you connect to 30:42 your local host, just MySQL enter, and you're in the system. 30:45 If you wanna tune it, you have to turn this into an actual 30:48 class, and add a parameter root password equals foo, figuring up, good to go. 30:51 Now the layout of this module, it's pretty similar to chef if you ask me chef 30:56 or, or puppet, it likes choosing between Pepsi 31:00 and Cola, everyone has its own its own preference. 31:03 Manifests instead of recipes. 31:07 You have your init.pp this is the one that's going to get bootstrapped first. 31:08 And instead of having an attributes folder where all the attributes are located, by 31:13 convention people are mostly going to use 31:16 parmas.pp and this contains bunch of parameters. 31:18 Again there is a templates folder, containing all the basic templates. 31:21 I have a screen dump of what this looks like. 31:25 This is, you see a more object oriented approach. 31:28 Oh ho, all the parameters you can tune. 31:31 This is just a, a, a piece of, of the action. 31:33 There's plenty more. 31:36 I added some doves there, you can see there's some differences for Red Hats. 31:37 You can do this for Debians, you can do other stuff. 31:40 It's a huge list of options to tune. 31:42 The manifest itself see MySQL Server, so we're gonna install a server. 31:45 It inherits from the parameters. 31:50 You have some templates too so very, very, very similar. 31:54 And again, the resources are simple too. 31:58 I mean, files, execution, computers, cron, host, whatever 32:00 is in Chef is probably also in Puppet. 32:04 And if you want to do it yourself, this is what it looks like. 32:07 Now there's a very important difference, between Chef and 32:09 Puppet in terms of writing it into a file. 32:11 In Chef it's going to go top down as you expect. 32:14 In Puppet it is not, and that freaked me out when I learned Puppet at first. 32:17 You have to chain events to each other. 32:21 So, this is gonna be executed whenever Puppet 32:23 feels like it, so we're gonna update our channel. 32:27 But the way we gonna make sure there's a logical order 32:29 is by using require and notify the change things to each other. 32:31 So if we install the mySQL server package, before we 32:35 do that we're gonna make sure our channels are updated. 32:38 And after that we're gonna make sure the server is started. 32:40 So that's the way you order things. 32:43 When you install Apache, it could be that MySQL is installed first. 32:45 It could be on the next run that Apache stall, installed first. 32:48 But that doesn't really matter. 32:51 What does matter is when you want to 32:53 ins, install Apache, make sure the channel is updated. 32:55 And right after that, make sure you install PHP. 32:57 And when you install PHP there, make sure in the end 33:00 after you install both of these components that you reload Apache. 33:03 That freaked me out. 33:06 That was really complicated. 33:07 And this is the, the remainder before you assign 33:08 a root password, make sure the MySQL server is installed. 33:12 And do this only if you can just execute that command and get in. 33:16 If you get an access denied, it means you already have a password, 33:19 no need to reconfigure it, and then at the end you have your 33:23 two services, a MySQL service an Apache service whereas Chef was just 33:25 blocks, this is like more curly braces style it depends on what you wanna do. 33:31 And this is just a basic file, manifest 33:36 part, module part, main file you're gonna includes the 33:39 Puppet factory see foo, and foo has used their 33:42 root bother you see, just a dollar sign password. 33:45 You define it here, it finds its way from your vagrant 33:48 all the way up to the system internally, and you're done. 33:51 And this is what happens when you type vagrant 33:55 up two new mounts, a manifest mount, a module mount. 33:57 And you can see all sorts 34:00 of information: services, executions, packets being installed. 34:01 This is the random order that happens. 34:05 That being said so we're true to chef. 34:08 We've already done puppet as well. 34:10 In a lot of cases I said Vagrant up is the main command you need. 34:12 In a lot of the cases the machine will already 34:15 be there and you made an error in your provioning. 34:17 You can just type vagrant provision. 34:20 It won't reboot the machine, it will just run Puppet or 34:22 the Chef or both because you can combine all of them. 34:25 You can have Chef run, a Puppet run. 34:27 A shell run, a ansible run, all run through each other Main 34:29 question at the ends chef or puppet, what are we gonna choose? 34:33 After this, who's gonna prefer chef, puppet? 34:38 A slight majority for puppets, if I can say so. 34:44 But there are problems as well when using chef and 34:47 puppet, and I noticed it, so my colleague Bram who's 34:50 sitting on the third row there asked me, Thijs, make 34:51 me a dev box and this is my list of requirements. 34:53 And, I, I scripted it all, and then, I typed vagrant up, 34:56 and it took bout half an hour to boot the entire system up. 35:00 Because it had so much dependency, that went over the network, cause 35:03 using a package manager, implies that you make connections outbound, and I 35:05 didn't configure my system as such, that it took Geo locality into 35:10 account, so it just, went off, to connect to the US servers. 35:15 And started downloading. 35:19 So, it was very cumbersome, very time consuming. 35:20 So, I decided we needed a different approach. 35:22 So, what I did is, instead of just, installing 35:25 everything on the fly, I created a new base box. 35:28 Because it can be slow from time to time. 35:33 Maybe some other lessons learned before we dive into the final piece. 35:37 The quality of public cookbooks and manifests 35:41 and the support you get may vary. 35:44 So, when you choose a specific cookbook or module for shareware Puppet, make 35:45 sure it fits your requirements so that it is built around your distribution. 35:52 Because I have found some pretty awesome scripts Puppet scripts that only work 35:56 on Red Hat systems and I use a Debian, But make sure it works. 36:00 Make sure the quality is there. 36:06 Make sure it's updated regularly. 36:07 My main advice is stick with, if you stick with Chef 36:08 go to the ops code get up and get one of those. 36:11 These are well maintained, and for puppet it's the very same thing. 36:14 We are getting back to the story of the, the VM I had to create for my colleague. 36:17 It was so time consuming to set it up it wasn't really working. 36:20 We have, we have several projects we have to work on. 36:24 And if you type vagrant up for the enti, for this project, and, and 36:26 in the afternoon you have to work on another project, it doesn't really work. 36:29 So what I did is use the Vagrant package command, 36:32 and the Vagrant package will just take the current vm 36:34 you're working on, and repackages it as a box file 36:37 and that box file, you can reload upload it elsewhere. 36:41 So, what I did, is I installed all the software he 36:43 needed, all the configuration files were in place, and the only thing 36:45 he needed to do, was make sure he had that box in 36:48 his vagrant file and in a full blown box, ready to go. 36:51 There is this is time consuming as well to package it, because it has to export 36:54 the entire VM, turn it into a box file, tar gz it, so it's a trade off. 36:59 And another thing you can do, and this is 37:03 a, a very good practice is do light provisioning. 37:05 So everything that is pretty heavy weight, so in terms of installing 37:08 software, making connections with packages, you can do that in the box file. 37:11 And all the configurations, like 37:15 configuration settings, virtual host files. 37:17 You can still do that using puppet chef 37:19 shell cuz that doesn't really take any time. 37:21 It's just file system action. 37:23 When you have to connect outbound, you might wanna figure out a way. 37:25 There are people who, who just use depth files, so, 37:28 or, or rpm files and just store them on the system. 37:31 So, they don't need to connect through the network. 37:34 They have all the packages at hand. 37:36 There are even people who boot a specific VM that servers 37:38 as a sort of package server and they connect to that. 37:42 It, it's pretty much up to you. 37:45 I have three minutes left in this talk, and I want to dedicate it to multi-machine 37:46 setups which is a very interesting concept when 37:50 you want to take it to the next level. 37:53 When you work at a decent company and, or work on 37:56 a product, it's hardly going to be run on a single machine. 37:58 In a lot of cases you're going to have multiple machines, two web 38:02 servers, two My SQL servers, an Oracle, some red hats, whatever you want. 38:04 So it would be nice to see this in action too. 38:10 I'll show you Vagrant file, I hope this is readable at the back. 38:13 This was usually done in the main piece, but what I 38:17 do now is conflict VM define and I'm given the name. 38:19 We have a web server. 38:22 We have a DB server. 38:23 This is shared along so these are both DB and weezies. 38:25 So, we just say that once, we just mention it once. 38:29 DB and 7 1 0. 38:32 and on every other machine, I'm gonna do a 38:34 shell inline script update a channel, update the packages. 38:36 And then the specifics are noted here. 38:39 We're gonna define a host name for the 38:41 machine because default won't really cut it right now. 38:43 So we're gonna have a webpp node, and a DB node, and 38:45 when you log in you're gonna be vagrant at webpp, Vagrant at db. 38:48 You assign a specific IP address ten, 11, the web 38:52 server is gonna be the mounted folder on var www. 38:56 Then the autorun is gonna require a specific, positive for the MySQL. 39:00 And the only difference, here, is that we connect to web.pp in the manifest folder. 39:05 Or db.pp. 39:11 And these are separate scenarios that run. 39:12 And you can have. 39:13 MySQL server and Apache web server running in parallel, and 39:14 this is what it mainly looks like when you do it. 39:18 We had default and all the other slides that were default here, now it gets 39:21 named to the host name you give the machine, either web or DB in our case. 39:24 And you can see what happens here. 39:28 And the very final thing is controlling these machines, because 39:32 if you have multiple machines and you do vagrant SSH. 39:34 Where we're going to end up. 39:36 These are the, these are all the commands again. 39:39 So, vagrant up will boot up all the machines. 39:41 If you have ten machines, you define in the vagrant 39:43 file, your system is going to set up ten machines. 39:45 Be careful with that cuz it might consume a lot of CPU, might consume a lot of ram. 39:48 So, be careful. 39:52 When you do vagrant ssh, you're going to log in to the primary vm. 39:53 And the primary vm, I'm at its primary here. 39:57 So that's your primary one. 40:01 When you type vagrant SSH, you're gonna end up in this one. 40:02 When you do vagrant destroy, all your boxes will be destroyed. 40:06 And then you could name them. 40:08 Vagrant up web. 40:10 I just wanna, boot up the web vm. 40:11 SSH into the web vm, Vagrant ssh web. 40:14 Vagrant destroy web. 40:18 Vagrant up db. 40:19 Vagrant ssh db. 40:21 And vagrant destroy db. 40:23 And that's all I have to say for today. 40:26 Remember people. 40:28 Vagrant up saves your life. 40:29 Thank you. 40:32 [SOUND] 40:33
You need to sign up for Treehouse in order to download course files.Sign up