Starting an Amazon EC2 Server from scratch using Tim Kay's aws tools

Starting your first server using just command line tools can be a bit intimidating. If you can, try to start your first server from the webinterface, it's much more intuitive and you'll get a better grasp of what has to be done.

Before we start, good luck and godspeed, this will be a long haul.

Installing and configuring the aws tools

First, download the tools. These can be found on http://timkay.com/aws/. Using these requires you to have the CURL libraries installed (use your packet manager to install, e.g. sudo apt-get install curl if you're on a Debian based linux like Ubuntu).

Get the tools like explained on the page linked to above. Installation is optional. Just make sure you're in the folder in which you downloaded the "aws" file when using the tools and don't forget to chmod u+x aws if you prefer not installing the tools. I won't be using an installed version in this tutorial so if you do install, replace all instances of ./aws with aws (this will make bash look for aws in your path instead of your current directory).

Don't forget to create a file in your home directory called .awssecret containing your access key id and secret access key id. This file looks like

5esf4s6e5f4s6e54f6se
qlekfjqselfkj561e6f51se+esjfkse

with the first line containing your api key and the second your api secret.

General notice

One very important thing to note, is that there isn't exactly "one" Amazon EC2. There are several, the one you should choose depends on your location. These are the so-called endpoints I've talked about before. If you're in the US, you can just use the ./aws without doing something extra and you'll be using the (default) US endpoint. If you're in Europe, you probably want to use the European endpoint. Doing this in Tim Kay's aws tools is done by giving the extra parameter --region=eu after every command. That's right, every single one.

If you're like me and forget to do this every other command, you can execute the command

echo "--region=eu" >> ~/.awsrc

exactly once. This will write the option to a file that's used as a set of default options for every aws parameters. This way you don't have to add the --region parameter to the end of every line and endpoints shouldn't bother you too much.

Creating a new SSH key

Don't confuse this key (the SSH key) with the API key you entered in the ~/.awssecret file. The key in the ~/.awssecret is the key you need when talking to the Amazon API, the key you'll generate here is the key you'll need to talk to your instances.

Instead of giving you a password to connect to your EC2 instance, Amazon allows (well, forces) you to use a more secure RSA key to setup an ssh connection to your instance(s).

Generating this key is easy. Execute the command

./aws add-keypair default > defaultawskey.key

To generate a key named "default" (when talking about this key to Amazon, you'll refer to it as "default" or whatever you named the key) and place your private key in a file called "defaultawskey.key" in your current directory. This is a normal run-of-the-mill SSH key. SSH won't accept your key if you give it like this, by default the key file is probably readable by everyone (a security problem which annoys ssh). You can fix the problem by only allowing yourself (so by not allowing other users on your local machine access to this key file) access to the file with

chmod 600 defaultawskey.key

Allowing SSH access to your server by configuring a group

If you want your server to do anything, you'll probably need it to execute commands. When starting a new instance, you'll specify a security group. This group will tell the server's firewall what ports to open.

Practically, you want to open a communication channel on TCP port 22 (SSH traffic). You can do this by either modifying an existing security group (one called "default" is created for you by default). Suppose we want to create a new group called "sshonly" and only allow ssh traffic in this group.

Making a new group called sshonly is done with the command

 ./aws addgrp sshonly -d "Only ssh traffic"

The text in the -d argument is obligatory and contains a short description of your group.

By default, this group is empty (it does not contain any rules). You can confirm this by executing

./aws describe-groups

The output is too wide to display here, but you'll notice nothing is written in the righmost few columns on the "sshonly" row. This is because there are no rules for this group.

Once this new group is created, a new rule can be added to it. This is done with the authorize command.

./aws authorize sshonly -P tcp -p 22 -s 0.0.0.0/0

This authorizes tcp access on port 22 for a security group called "sshonly". The parameter 0.0.0.0/0 is important since this notes that all IP addresses are allowed to ssh to the instance. If you'd like extra security, you can probably enter your own ip address or address range here. Confirm the port was opened succesfully by once again executing

./aws describe-groups

Note the new table containing the columns

 ipProtocol | fromPort | toPort |       groups
 tcp        | 22       | 22     | item= userId=x cidrIp=0.0.0.0/0

A good sign indeed.

Finding an AMI

An AMI, or Amazon Machine Instance, identifies an image you can use to base your instance on. You can request all possible AMI's by executing the command

./aws describe-images > amis.txt

This command writes all available ami's to a file called amis.txt in the current directory. Executing this command can and will take a while (it's a very big list). You're best of grepping through it. When you find an image you like (a simple i386 image with a recent ubuntu version on will probably do fine), take note of the string in the first column (ami-xxxxxxxx), you'll be needing this. The one I chose was ami-d80a3fac (an Ubuntu 10.10 server image).

Starting a new instance

Actually starting a new instance is done by executing

./aws run-instances ami-d80a3fac -g sshonly -k default

This will start a new instance with ami "ami-d80a3fac", security group "sshonly" and the key named "default" (remember, this was the one for which we stored our private key in the file defaultawskey.key). After pressing enter, the new instance will be in status "Pending". This means Amazon is getting the machine ready for usage. Wait about a minute and run the command

ives ~/aws $ ./aws describe-instances --simple
i-44e3d333 running ec2-79-125-77-241.eu-west-1.compute.amazonaws.com

Keep running the describe-instances command until a DNS address appears and the state is "running" (not "pending"). Your new instance can be reached on the address given in the describe-instances output, e.g. "ec2-79-125-77-241.eu-west-1.compute.amazonaws.com" and the instance is named "i-44e3d333".

Executing commands on your instance

We're finally here! We now have SSH access to our running instance. Give the command

ssh -i defaultawskey.key root@ec2-79-125-77-241.eu-west-1.compute.amazonaws.com

with "defaultawskey.key" being the file to which your RSA key was written in the section "Creating a new SSH key" and "ec2-79-125-77-241.eu-west-1.compute.amazonaws.com" being the address your instance can be reached on, given by the describe-instances output.

Doing this should give you something like

RSA key fingerprint is 58:ff:ff:42:f1:8e:46:86:71:ef:a6:66:d5:43:4a:d4.
Are you sure you want to continue connecting (yes/no)? 

Enter "yes", press return and with some luck you can enjoy your brand new instance.

If you get the error

Please login as the ubuntu user rather than root user.

Login as the user called "ubuntu" instead of "root", like this:

ssh -i defaultawskey.key ubuntu@ec2-79-125-77-241.eu-west-1.compute.amazonaws.com

And everything should be fine. If you got to this point, congratulations, I know it wasn't easy! Anyhow, read on and don't forget to terminate your instances! They're not free.

Terminating instances

./aws describe-instances --simple

will tell you what instances you have running at the moment (on your current endpoint, so make sure you didn't start any servers on the USA endpoint and are looking at the EU instance listing -- they won't show up).

The column on the left contains the instance number. Pass this number to ./aws terminate-instance like this:

./aws terminate-instances i-44e3d333

with "i-44e3d333" being the instance number "describe-instances" gave you. After entering this command, the describe instances listing will look like this:

ives ~/aws $ ./aws describe-instances --simple   
i-44e3d333 terminated 

The instance will keep its "terminated" status for a few hours and will then disappear.

Conclusion

This post of Steve Yegge-an proportions (more than 1500 words) should help you when you're trying to start your first instance. It was long and it was hard, but this was probably the hardest part. The next time you're starting new instances, you won't have to repeat the first few steps (you won't need to create a new ssh key (use the existing one), create a new group, add rules for that group, ..., just find a good AMI, execute the run-instances command and you should be on your way).