Into The Abyss…
It’s no secret I like cloud platforms, hence I work here. The facination stems from my time at Transitive. Even way back in the early 2000s our testing platform evolved into a cloud. Starting from a load of bare-metal servers running jobs, it was realised that we could get better utilisation by leveraging VMWare to spin up machines on the fly, which co-exist on the same physical host.
From a development perspective it’s fantastic to have a limitless number of fresh machines to play with. Gone are the days where your laptop would fill to bursting point with pointless transient applications which were required for the duration of the task at hand. Just fire up a VM, install the software, do the work and throw it away.
Orchestration and automation is the next big leap. Not only do you spin up a VM, but you can have it provision itself, installing software and self configure. All with a single command! This brings me to the topic of this post, how do you perform this voodoo magic?
We as the cloud provider worry about authentication. It is assumed you’ve got the openstack command line clients installed and the environment configured so you can talk to openstack. First task is to set up a network so your VM can talk to the outside world, and you can talk to it.
Here I’ve created a new network called simon, then created a new subnet within it. When you create a subnet, neutron (openstack’s network component) also creates a DHCP server within that broadcast domain. Thus when you create a VM, it will get an IP address allocated to it out of that address pool, learn the netmask, broadcast address, and pick up a name service from google (that’s what the 18.104.22.168 and 22.214.171.124 addresses are all about for my less technical readers). Local area network setup, nice and easy. Next up we need to create a router to forward packets between that and the wide area network i.e. the internet!
Yeah it looks scary but not really. A lot of this is hidden away if you use the web interface, but we’re really interested in scripting all of this and adding more automation! First we are asking neutron about the external networks onto the internet that exist. Next we create a router (which like all commands returns a resource ID). The final two steps connect the router ID to the external network ID, so if it doesn’t know about an address it will just fire it at the internet, and the router to our LAN. Looking at our subnet any VMs will be allocated an address from 10.0.0.2 to 10.0.0.254, it will query google’s name servers for address translation, and all packets no destined for machines within 10.0.0.0/24 get sent to 10.0.0.1, our router, and forwarded on the the internet. Easy!
Creating and Provisioning a Virtual Machine
This is where the fun really begins. My current task was to create a puppet orchestration server within the cloud to manage this website coincidentally. How was this accomplished?
One command… I did warn you! Bit of explanation required here. The flavour of a machine relates to the number of CPUs it has, how much RAM & how much disk. The key relates to my SSH public key, this gets automagically installed on the machine during boot so I can log in later. The image is the ID of a disk image I have uploaded into the glance service, in this case it is an Ubuntu 14.04 amd64 cloud image (the cloud image bit is responsible for adding ssh keys and performing provisioning). The NIC parameter tells the VM to create a network adapter and attach it to my network, user-data is a custom script that is run on first boot, poll basically waits for the server to be created, and finally puppet is the hostname of the machine.
So the real magic is the user data I passed in, this sets the machine up for me automatically. After all isn’t that the dream? Pressing the go button and surfing the web for the rest of the day!
For the more astute of you no that’s not the actual production key, nice try! So basically I am installing puppet, r10k and git, then checking out my puppet repository from GitHub, and applying the manifests for that particular host. This has the result of provisioning a puppet master for my subnet and a name server, as openstack does not provide DNS as a service yet. Finally I override the DNS server supplied by DHCP. You can at this point change this in your subnet configuration so it gets picked up automatically.
Is There Anybody Out There?
Not quite finished yet as we can’t talk to the VM from the internet. The VM has an IP address on the local subnet, but nothing that can be uniquely addressed from the internet. Cue floating IPs:
Floating IPs are allocated from a pool and are expensive, so keep that in mind, the first thing you will want to provision is a VPN endpoint. The floating IP is associated with the network port on my VM. Under the hood this creates a rule to DNAT incoming packets from the internet to my server at 10.0.0.3. The final thing is to create firewall rules that allow access to the floating IP address, and you can now log in with SSH!
So as you can see, I’ve managed to build a LAN, connect it to the internet via a router, create and install a machine which I can log in to in 11 commands. This is the power of the cloud, no more waiting weeks while finance um and ah over buying equipment, IT install it and then finally you can install it and get on with your work. Once the provisioning code was written the whole process end to end took 5 minutes.
The provisioning is the bottleneck now. We want to give a world class experience in simplicity and usability. We don’t want you to have a degree in computer science to use this stuff, so looking forward into the future we want to provide this functionality for you, imagine a world where you click a button and your entire business creates, installs and configures itself. Your infrastructure constantly monitors and heals itself adapting to demands. There are a lot of cool ideas out there and we want to hear them!