Let’s imagine I am a small software vendor that uses Amazon EC2 for his complete IT infrastructure. Let’s say I have around 20 virtual machines on Ubuntu Linux running all my applications. There are internal applications, i.e. applications that only my company uses – like a CRM, code-repository, a document-management tool, a project-management tool, a billing-system or a mail-server. And there are external applications that are shared with the outside such as a file-server to download documents and software, a bug-tracking tool, and a SaaS version of my products. Almost any software is also using a database instance in the backend. Of course, there are also a couple of management tools running and scripted processes for backup, integrity checks, or auditing.
So far so good. Now a new version of Ubuntu comes out that fixes a potential security leak. In addition, I want to use the latest release of MySQL for performance reasons. And there’s this new version of my bugtracking system with the long-awaited new feature. In the classical IT world, I would start software updates (e.g. for Ubuntu) or download and install the latest version of MySQL and my bug-tracking tool. How would I do it in the EC2 Cloud?
I could try it the same way. But I might run into problems like this – a new OS version might require an update of the kernel, which is not possible with EC2. So I do need to start up a new instance from an AMI containing the last Ubuntu version, reinstall and configure my software, reimport my data, and copy and reactivate my scripts. When I choose to do it like that, I must also be prepared for the (rare) case of corrupted instances or unintentional termination of an instance. In such cases, I need to do the whole process from scratch. RightScale helps to automate those kind of updates, since it allows to write and assign reusable scripts to AMIs that are executed on startup or termination of an instance.
With VMWare, I would install the new software and create a snapshot after the upgrade, which allows me to switch back immediately to a certain system state at any time I want. This feature is not available for EC2, but can be somehow simulated: the process of creating a VM snapshot corresponds to the process of bundling a new AMI. Everytime I need to upgrade, I would do my updates on the running instance (or better: a clone of that instance based on the same AMI) and then create a new AMI out of it. Elastic-Server is a service that helps you to do that more easily. Afterwards, I need to shut-down my instance based on the old AMI and start a new one based on the new AMI. However, that only works for application and service software upgrades and for OS upgrades without kernel modifications. For kernel upgrades, I still need to do the whole installation process from scratch…
Ideally, I would like to separate my AMI from the installation information of the software. What about a solution like this: I start from a clean OS-only image and store all modifications on persistent storage i.e. EBS (Elastic Block Storage). On my AMI, there is an installation script executed at startup that checks the current setup against the target setup stored on EBS. Installing MySQL? Store the whole bin+lib+config on EBS. The installation script will check if it needs to apply modifications and gets them directly from the EBS. Upgrading MySQL means simply updating the target state on EBS. Upgrading the OS means: using another AMI (that includes the installation script – that’s the catch).
I will try to formalize those solutions and add them to the list of EC2 Patterns. Every input is welcome (especially, I am still searching for good pattern names)!