Apache’s libcloud is a Python library that allows you to write code that interacts with numerous cloud service providers. From the big players like Amazon AWS, Google and Rackspace to even smaller local providers like the New Zealand based RimuHosting. A full list of supported providers can be found here
The library is not limited to provisioning servers and supports a bunch of things like load balancing, storage and even DNS but here we’ll just focus on cloud server provisioning.
I am assuming you are running Ubuntu or another Debian variant. if not check official installation instructions here. Normally installation is as easy as:
The library is designed to provide a common interface to the different cloud providers, the general process is.
Choose a provider
Request a Driver for that provider
Continue or complete
As an example, if I wanted to see the EC2 instance sizes that were available in Sydney I could do the following.
Include the required libraries:
Request the AWS EC2 Sydney provider: (remember full list here)
Set my credentials
Now I can request a list of node sizes
NOTE: Remember to set the variables AWS_EC2_ACCESS_ID and AWS_EC2_SECRET_KEY to your API credentials.
The variable sizes now contains a list of all the available instance sizes in Sydney. Running that script would output something like so:
Another helpful method is list_images() which returns a list of all the OS images that are available for our current driver (in this case just EC2 AMI’s in Sydney). Simply adding the following to the bottom of our script and running it again
Will output a the size list then a big list of the available node images like following (trimmed)
One thing to note, is that these images and sizes are objects we need to pass through to the create methods, so if you know the size of Node you want unfortunately to make the library portable you cant just parse in the ID, you still need to get the NodeSize object you are after from the list, for example if you knew you wanted to create a ‘m1.small’ instance you’d have to do something like this:
The same is true for your host’s NodeImage so lets update our script from before to specify a NodeSize and a NodeImage.
For this example I am going to use an AMI for an official Ubuntu Instance image in Sydney. This is for m1.small instances and is available in the Sydney pool of AMI’s. ami-934ddea9. You can see all the official Ubuntu ones here.
Perfect, let’s continue.
Creating an instance
The libcloud library provides a create_node method that allows us to provide, a name, NodeSize and NodeImage parameters to fully provision a node. A call to this method could look like so
This will create an EC2 instance called “My Instance”
Simply adding that line to the bottom of our script will create an Amazon EC2 Instance. See:
But that’s not much help, we really want to be able to set up some access (Key Pairs) and ensure that we have at least the SSH port open (Security Groups)
So in the AWS console set up a Key Pair and name it ‘provision’ and then create a Security group called ‘ssh_access’ and allow inbound SSH access. We can now update our script like so.
This script will create a EC2 small instance, in Sydney, add the Public key to the authorized_keys file for root, and make sure that port 22 is open for connection.
You can now SSH into your server, check the AWS EC2 console for the publicly assigned DNS name to connect to.
NOTE: since we are running Ubuntu, you can’t SSH into the new box as root you need to use the user account ubuntu. If you do login with root you will be instructed about this.
So there we have it. Apache libcloud makes it really easy to provision servers.
There is a whole bunch of additional functionality we haven’t covered here. Executing commands on the server during provisioning for installation of software, adding public keys via the library and running other arbitrary shell commands. If you’re interested in knowing more feel free to leave a comment.