Amazon Web Services is usually a good choice when you need to deploy something fast, scale fast or even try things out, for example to set up an instance of SugarCRM without deploying it on your live server.
An alternative to this would be starting a virtual server on your workstation, but this limits you a lot and you can’t do simple things such as showing it to your boss, friends or potential clients / users.
Anyway, the beauty of Amazon services is that you only pay for what you use. And if you just need to try things out, a minimal server costs just cents per hour. If you’re not willing to pay that much, then you’re just playing around without any real need and you might go with a free solution such as VirtualBox and some virtual server installed right on your computer.
I don’t remember exactly how the signup process for the Amazon services was back a few years when I signed up, but I remember that I went right through it and got my servers up and running in minutes. Amazon is doing a really great job at measuring user experience and improving their services as they go along.
Create and launch your server
The first thing you need to do is log into your Amazon AWS console: https://console.aws.amazon.com
After logging in the first thing you see is the list of service that are available for you to use on a pay-as-you-go basis.
I won’t go through each of them right now, but you should know about a few of them:
- EC2 – Virtual Servers in the Cloud: this is the Elastic Cloud 2 virtual server instance that we’re talking about in this article;
- S3 – Scalable Storage in the Cloud: store anything (or everything) and access it how ever you like; there are many libraries out there that you can use to retrieve or upload files to your very own cloud storage, and you can even upload an entire (static) website and serve it right off your S3 storage service;
- DynamoDB – Predictable and Scalable NoSQL Data Store: if you’re familiar with MongoDB or CouchDB, this is their Amazon equivalent; of course you can install MongoDB or whatever database system on your EC2 server instead, DynamoDB and many other services are built to be highly scalable and standardised;
- SES – Email Sending Service: use this to send emails to your users or customers. SES can handle high quantities of emails and is also packed with everything required for your emails to reach the Inbox.
Click on EC2 to go to your virtual servers management panel. This page displays all the resources related to your virtual servers, including how many instances you have, volumes (virtual disk drives), security groups, key pairs, IPs, snapshots and load balancers (you won’t need this until later on but you can surely try them out and learn how they work).
The first thing to do here is select your zone. Amazon has data-centers in several locations around the world, and in order to get the best performance, you’ll need to select the appropriate zone from the upper-right combo, right next to Help (it may be already selected on N. Virginia).
As of 2013, the following zones are available:
- United States – Northern Virginia
- United States – Oregon
- United States – Northern California
- Europe – Ireland
- Asia Pacific – Singapore
- Asia Pacific – Tokyo
- Asia Pacific – Sydney
- South America – Sao Paolo
Select the zone that’s the closest to you (or to your users) and then click Launch Instance to create a new virtual server. When I say “closest to your users” I mean that if you need to try something out that targets users in or near Europe, you should go with Europe Ireland even if you’re from California. This ensures that the service you deploy has the lowest latency possible.
Going through the Launch Instance wizard is really easy and it only takes minutes. First select classic wizard, to go step by step through the process of launching a new instance. Don’t worry, if you screw things up, you can easily destroy the instance at no extra cost.
Since it’s very popular, we’ll go with Ubuntu 12.04 LTS for now as our operating system for the new machine. Scroll down and identify it in the list, then click Select. You can select 64 bit or 32 bit but you probably won’t notice the difference, so go with either one.
The installation process is not the regular “download, start installer, finish” process. There are many images (virtual hard drive files) with pre-installed operating systems that they only copy into your “account” and power up. This makes it possible to create and power up hundreds of machines in just under a minute.
If you want to play around, you can start several instances at once, but usually you’d want to start one server and set it up properly. You should know, however, that working with 100 or 500 instances at once is not as hard as it sounds. There’s a nice piece of software that connects to all instances via SSH and runs the same commands on all of them. And since they’re identical, it’s very likely that they will behave exactly the same. You can even edit files and do any kind of tasks, all at once. The network engineer’s job is in fact a daily challenge, with a great sense of power and also fun. Managing hundreds of servers, all working marvellously brings new meaning to the words “one for all and all for one”. Or team work.
Back to reality, let’s leave 1 as the Number of Instances, and select a T1 Micro type instance (that’s the smallest there is, really great to test things and play around). Bigger instances have higher CPU power and more RAM, and are useful for processing large amounts of data or other resource-intensive tasks.
If you missed the zone selection earlier, you can select the desired zone in this step. Click Continue to get to the Advanced Instance Options.
Here you can select really advanced options such as the RAM Disk ID or Kernel. You’ll usually want to leave these as they are. If you intend to place really important data on this instance, you can check the Prevent against accidental termination check-box to make sure you don’t click something and find your machine recycled. Click Continue to move on.
You are almost done. What you need to choose is the storage drive and some security options.
Usually, a 8 GB drive that will get deleted when you destroy the machine is a good start. But if you need to go wild, feel free to click Edit and have a bigger one. Please note that you will pay a bit more for a bigger drive, and if you don’t really need it, just go with 8 GB. You’ll be able to add a larger drive later on if you suddenly decide that you need 50 GB or so, drive that you can mount on your machine (and even share it between machines). In the cloud, all drives are better called “volumes”.
Click Continue to get to the Tags page.
Name your machine so that you can identify it later, especially if you’ll have several. You can name this whatever you’d like, even Prime Experiment or whatever you find meaningful. You can add several tags to help you manage the server later.
One really great thing about all these instances is that you can even power on and manage instance via API calls, which gives great flexibility to create a really intelligent system that reacts to change in real time. Think about it. You can have a script in place that powers on a machine for every 5000 concurrent users on your site. Automatic configuration of load balancers is just a matter of API calls. Your entire infrastructure can grow and shrink as needed, without wasting resources. Big press release today and you’re expecting ten times the traffic? Just power on some more machines. It’s that easy!
Click Continue to get to the security part.
If you’ve never created a key pair, now is the time. These key pairs are actually files that are used to authenticate you as an authorized user of the machine. It’s called pair, because any pair is composed of two files. One is installed on the machine, and one should be in your possession (on your local computer). Every time you need to connect, you “pair” or “match” the two. If the match is successful, you’re allowed to log in. This means hackers can’t connect to your machine even if they have the password.
Your “part” of the Key Pair is a file that you keep on your computer. If you use PuTTY to connect to your machine, you just have to specify its location and save your connection and you can login without any hassle or even a password. The default user on Ubuntu systems is “ubuntu”, with no password, just the key file.
You can move on without a Key Pair but it’s not recommended. You don’t want your server hijacked and used for malicious purposes.
So, either create a Key Pair or select one you already have and click Continue.
This is the second step regarding security. The Security Groups.
Amazon take security very serious, and this is why the provide an extra layer of protection, before the one that you choose to configure on your machine. Security groups work this way: you create a security group and assign it to one or more machines. After that, you configure the security group based on your needs, using rules, just like the firewall. You can open certain ports for public access, or for access only from specific hosts. This enables you to quickly update your security settings and apply them to all the machines that are part of a certain security group. Otherwise, you’d have to login and update the firewall on each machine, manually.
This also reduces overhead on your instance, because you don’t have to install firewall software on the machine, just add the services that you need and use them.
Of course, you don’t need to decide now what ports you’ll want to have open, you can always do that later.
The next page is a short summary of all the previous questions, and it’s a good idea to go over them and make sure you power up the machine you need.
Here’s your moment of glory. If it all looks ok, click Launch and your machine is up and running!
Here it’s safe to click Close, this will take you to the instances list where you will click on your new machine to get the connection details.
Each Amazon machine, when powered up, it is assigned a new IP, because IPv4 addresses are scarce. Along with the IP, the new machine gets a Public DNS hostname so that you can connect to it from outside the Amazon “cloud”. Scroll down and find your Public DNS hostname. You’ll need this in the next step.
This wasn’t hard, wasn’t it?
Connect to your machine
On Windows, connecting to your Ubuntu machine is really easy. Just fire up PuTTY, the software used to connect securely, and paste the Public DNS hostname into the Host Name text-box.
From the list on the left, click on the SSH leaf and then Auth. This is where the authentication options reside.
Click on Browse and select your Key file that you obtained when you created your Key Pair. Go back up to the Session page, type a name into the Saved Session text box and click Save. This will list your session in the Saved Sessions box, under Default Settings, and each time you want to connect to this machine (if the hostname is not altered by shutdown/restart), just open PuTTY and double-click on your saved connection.
You can make it even simpler, by asking PuTTY to automatically type in the user name, for example ‘ubuntu’. This way, with one double-click you are connected to the virtual server that runs in the cloud.
Pretty cool, right?
The basic setup
Immediately after you fired up your new machine, you’ll want to do some tasks that usually are meant to put things into place and to update your server to the latest version.
To do this, you need to have Aptitude, the package manager, up to date. Run these commands on your new born machine:
sudo apt-get update
sudo apt-get upgrade
This will recheck all the package sources and update all the software already installed on your machine. I usually reboot the machine after this step, because it may contain kernel or other boot software updates, which, if go wrong, may prevent your machine from booting in the future. You don’t want to find out it doesn’t boot anymore after you’ve spent hours setting it up with everything, do you? Run sudo reboot and keep your fingers crossed.
After this, I usually install a bunch of packages that I always use, such as Midnight Commander, htop (a smarter top), and unpacking tools:
sudo apt-get install zip unzip arj htop mc multitail byobu dnsutils dstat
From here on, you can install any software you like, and don’t forget to power off your machine when you’re done with it. And, of course, if your website grows a bit and users rely on it, don’t forget to get some uptime monitoring.
If your project’s architecture is designed to be decoupled, you can fire up a machine just for database storage, and one to server the application to the users. This allows you identify problems within any component independently. For example if the whole application is on one system and that system starts to increase in load, it’s going to be hard to identify which service has gone rogue. But that’s the subject for next week’s article.
So, what tools do you usually install by default on your virtual machines?
Great step-by-step article!
I created my own bash script that install everything nicely on my server, and I only use wget to get it downloaded and executed, in just one line. This saves me minutes every time I have to set up a new server.