Virtual Hosting Features

October 21, 2009 | etbe

I’ve just been setting up new virtual servers at Linode [1] and Slicehost [2]. I have previously written a review of both those services [3], based on that review (and some other discussions) one of my clients now has a policy of setting up pairs of virtual servers for various projects, one server at Linode and one at Slicehost.

Now both virtual hosting providers work very well and I’m generally happy with both of them.

But Linode seems to be a better offering.

Linode has graphs of various types of usage, I can look at graphs of disk IO, CPU use, and network IO for the last 24 hours, 30 days, or for previous months. The three graphs have the same scale of the X axis so I can correlate them. The stats on Slicehost just allow you to get the current raw numbers, which doesn’t help if I want to know what happened last night when performance sucked.

When I build a Linode instance I can have multiple filesystems configured (Slicehost can’t do any of this). I can use less disk space than is available to reserve space for other filesystems. Separating filesystems makes it easier to track IO performance and also allows some bounds to be set on the amount of disk space used for various tasks. Nowadays the use of multiple partitions is not as popular as it once was, but it’s still a real benefit. Of course one of the benefits of this is that I can have two partitions on Linode that are suitable for running as the root filesystem. If an upgrade fails then it would be an option to boot with the other filesystem (I haven’t actually done this but it’s good to have the option).

I believe that this feature of Linode could do with some improvements. Firstly when creating or resizing filesystem it should be possible to specify the number of Inodes when using Ext3. The fsck time for a large Ext3 filesystem that has the default number of Inodes is quite unreasonable. It would also be good if other filesystems such as XFS were supported, for some use cases XFS can significantly outperform Ext3 – and choice is always good. When BTRFS becomes stable I expect that every hosting provider will be compelled to support it (any provider that wants my continued business will do so).

Now Linode and Slicehost both allow sharing bandwidth allowances between virtual servers. So if you run one server that uses little bandwidth you can run a second server that needs a lot of bandwidth and reduce the potential for excess bandwidth use problems. The next logical extension to this is to allow sharing disk allocation between servers on the same physical system. So for example I might want to run a web server for the purpose of sending out large files, 360M of RAM as provided by the Linode 360 offering would be plenty. But the 128G of storage and 1600GB per month of bandwidth usage that is provided with the Linode 2880 plan would be really useful. At the same time a system that does computationally expensive tasks (such as a build and test server) might require a large amount of RAM such as 2880MB while requiring little disk space or bandwidth. Currently Linode allows sharing bandwidth arbitrarily between the various servers but not disk space or RAM. I don’t think that this would be a really difficult feature to implement.

Finally Linode has a “Pending Jobs Queue” that shows the last few requests to the management system and their status. It’s not really necessary, but it is handy to see what has been done and it gives the sysadmin a feeling of control over the process.

These management features provide enough value to me that if I was going to use a single virtual hosting provider then I would choose Linode. For certain reliability requirements it simply wouldn’t be a responsible decision to trust any single hosting company. In that case I’m happy to recommend both Linode and Slicehost as providers.

New Servers – a non-virtual Cloud

October 13, 2009 | etbe

NewServers.com [1] provides an interesting service. They have a cloud computing system that is roughly comparable to Amazon EC2, but for which all servers are physical machines (blade servers with real disks). This means that you get the option of changing between servers and starting more servers at will, but they are all physical systems so you know that your system is not going to go slow because someone else is running a batch job.

New Servers also has a bandwidth limit of 3GB per hour with $0.10 per GB if you transfer more than that. Most people should find that 3GB/hour is enough for a single server. This compares to EC2 where you pay $0.10 per GB to receive data and $0.17 to transmit it. If you actually need to transmit 2100GB per month then the data transfer fees from EC2 would be greater than the costs of renting a server from New Servers.

When running Linux the EC2 hourly charges are (where 1ECU is provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor):

NAME	Cost	Desc
Small	$0.10	1.7G, 160G, 32bit, 1ECU, 1core
Large	$0.20	7.5G, 850G, 64bit, 4ECU, 2core
Extra Large	$0.40	15G, 1690G, 64bit, 8ECU, 4core
High CPU Medium	$0.20	1.7G, 350G, 32bit, 5ECU, 2core
High CPU Extra Large	$0.80	7G, 1690G, 64bit, 20ECU, 5core

The New Servers charges are:

NAME	Cost	Desc
Small	$0.11	1G, 36G, 32bit, Xeon 2.8GHz
Medium	$0.17	2G, 273G, 32bit, 2Xeon 3.2GHz
Large	$0.25	4G, 250G, 64bit, E5405 Quad Core 2Ghz
Jumbo	$0.38	8G, 2*500G, 64bit, 2 x E5405 Quad Core 2Ghz
Fast	$0.53	4G, 2*300G, 64bit, E5450 Quad Core 3Ghz

The New Servers prices seem quite competitive with the Amazon prices. One down-side to New Servers is that you have to manage your own RAID, the cheaper servers have only a single disk (bad luck if it fails). The better ones have two disks and you could setup your own RAID. Of course the upside of this is that if you want a fast server from New Servers and you don’t need redundancy then you have the option of RAID-0 for better performance.

Also I don’t think that there is anything stopping you from running Xen on a New Servers system. So you could have a bunch of Xen images and a varying pool of Dom0s to run them on. If you were to choose the “Jumbo” option with 8G of RAM and share it among some friends with everyone getting a 512M or 1G DomU then the cost per user would be a little better than Slicehost or Linode while giving better management options. One problem I sometimes have with virtual servers for my clients is that the disk IO performance is poorer than I expect. When running the server that hosts my blog (which is shared with some friends) I know the performance requirements of all DomUs and can diagnose problems quickly. I can deal with a limit on the hardware capacity, I can deal with trading off my needs with the needs of my friends. But having a server just go slow, not knowing why, and having the hosting company say “I can move you to a different physical server” (which may be better or worse) doesn’t make me happy.

I first heard about New Servers from Tom Fifield’s LUV talk about using EC2 as a cluster for high energy physics [2]. According to the detailed analysis Tom presented using EC2 systems on demand can compete well with the costs of buying Dell servers and managing them yourself, EC2 wins if you have to pay Japanese prices for electricity but if you get cheap electricity then Dell may win. Of course a major factor is the amount of time that the servers are used, a cluster that is used for short periods of time with long breaks in between will have a higher cost per used CPU hour and thus make EC2 a better option.

[1] http://www.newservers.com/
[2] http://luv.asn.au/2009/10/06

Dom0 Memory Allocation and X

May 11, 2009 | etbe

I’ve previously written about memory squeeze problems in a Xen Dom0 when large amounts of memory were assigned to DomUs [1]. In summary the Dom0 would have problems if started with default options and the majority of the RAM was later assigned to DomUs, but if the memory of the Dom0 was limited by the dom0_mem parameter to the Xen kernel then things would work well.

Fatal server error:
xf86MapVidMem: Could not mmap framebuffer (0xfae80000,0x80000) (Invalid argument)

I have since found another exciting bug with Xen. I was in the process of upgrading an AMD64 workstation to using Xen so that I could test other versions of some software in the background. The first stage was to install the Xen kernel and the Xen enabled Linux kernel and boot the machine. Unfortunately I then received the above message when trying to start the X server. I discovered the solution to this in the comments section of Different Colours blog post about Virtualisation on Lenny [2]. It seems that there is a problem gaining mmap access to MMIO memory regions in Xen and that restricting the memory of the Dom0 is a work-around.

My AMD64 workstation has 3G of RAM because the motherboard can’t support more than 3.25G and buying 4G of RAM to have 3.25G usable would be an expensive way of doing it. So I used dom0_mem=1500M and now X works again. I have yet to discover if anything strange and exciting happens when I create DomUs on the machine. I don’t have any immediate plans for running Xen on the machine. It’s main uses at the moment is torcs (a slightly realistic 3D car racing game), supertuxkart (a cartoon 3D car racing game), and mplayer so it doesn’t really need a lot of RAM.

I like to keep my options open and have all my machines capable of virtualisation apart from routers.

Red Hat, Microsoft, and Virtualisation Support

February 19, 2009 | etbe

Red Hat has just announced a deal with MS for support of RHEL virtual machines on Windows Server and Windows virtual machines on RHEL [1]. It seems that this deal won’t deliver anything before “calendar H2 2009” so nothing will immediately happen – but the amount of testing to get these things working correctly is significant.

Red Hat has stated that “the agreements contain no patent or open source licensing components” and “the agreements contain no financial clauses, other than industry-standard certification/validation testing fees” so it seems that there is nothing controversial in this. Of course that hasn’t stopped some people from getting worked up about it.

I think that this deal is a good thing. I have some clients who run CentOS and RHEL servers (that I installed and manage) as well as some Windows servers. Some of these clients have made decisions about the Windows servers that concern me (such as not using ECC RAM, RAID, or backups). It seems to me that if I was to use slightly more powerful hardware for the Linux servers I could run Windows virtual machines for those clients, manage all the backups at the block device level (without bothering the Windows sysadmins). This also has the potential to save the client some costs in terms of purchasing hardware and managing it.

When this deal with MS produces some results (maybe in 6 months time) I will recommend that some of my clients convert CentOS machines to RHEL to take advantage of it. If my clients take my advice in this regard then it will result in a small increase in revenue and market share for RHEL. So Red Hat’s action in this regard seems to be a good business decision for them. If my clients take my advice and allow me to use virtualisation to better protect their critical data that is on Windows servers then it will be a significant benefit for the users.

[1] http://www.redhat.com/promo/svvp/?intcmp=70160000000HiHHAA0

Xen and Lenny

February 16, 2009 | etbe

Debian GNU/Linux 5.0 AKA “Lenny” has just been released [1].

One of the features that is particularly noteworthy is that Xen has been updated and now works fully and correctly on the 2.6.26 kernel (see the Debian Wiki page about Xen for details [2]). This may not sound exciting, but I know that a lot of people put a lot of work into getting this going, and for a long time in Unstable it wasn’t working well. I’ve just upgraded three Xen servers from Etch to Lenny (actually one was Etch kernel with Lenny user-space), and they all worked!

Those three servers were all running the i386 architecture, the next thing to do is to try it out with the AMD64 architecture. One of my plans is to try the latest Debian kernel on the server I use in Germany, but I’ll try on a few other AMD64 machines first.

Physical vs Virtual Servers

November 29, 2008 | etbe

In a comment on my post about Slicehost, Linode, and scaling up servers [1] it was suggested that there is no real difference between a physical server and a set of slices of a virtual server that takes up all the resources of the machine.

The commentator notes that it’s easier to manage a virtual machine. When you have a physical machine running at an ISP server room there are many things that need to be monitored, this includes the temperature at various points inside the case and the operation of various parts (fans and hard disks being two obvious ones). When you run the physical server you have to keep such software running (you maintain the base OS). If the ISP owns the server (which is what you need if the server is in another country) then the ISP staff are the main people to review the output. Having to maintain software that provides data for other people is a standard part of a sys-admin’s job, but when that data determines whether the server will die it is easier if one person manages it all. If you have a Xen DomU that uses all the resources of the machine (well all but the small portion used by the Dom0 and the hypervisor) then a failing hard disk could simply be replaced by the ISP staff who would notify you of the expected duration of the RAID rebuild (which would degrade performance). For more serious failures the data could be migrated to another machine, in the case of predicted failures (such as unexpected temperature increases or the failure of a cooling fan) it is possible to migrate a running Xen DomU to another server. If the server migration is handled well then this can be a significant benefit of virtualisation for an ISP customer. Also Xen apparently supports having RAM for a DomU balloon out to a larger size than was used on boot, I haven’t tested this feature and don’t know how well it works. If it supports ballooning to something larger than the physical size in the original server then it would be possible to migrate a running instance to a machine with more RAM to upgrade it.

The question is whether it’s worth the cost. Applications which need exactly the resources of one physical server seem pretty rare to me. Applications which need resources that are considerably smaller than a single modern server are very common, and applications which have to be distributed among multiple servers are not that common (although many of us hope that our projects will become so successful ;). So the question of whether it’s worth the cost is often really whether the overhead of virtualisation will make a single large machine image take more resources than a single server can provide (moving from a single server to multiple servers costs a lot of developer time, and moving to a larger single server exponentially increases the price). There is also an issue of latency, all IO operations can be expected to take slightly longer so even if the CPU is at 10% load and there is a lot of free RAM some client operations will still take longer, but I hope that it wouldn’t be enough to compete with the latency of the Internet – even a hard drive seek is faster than the round trip times I expect for IP packets from most customer machines.

VMware has published an interesting benchmark of VMware vs Xen vs native hardware [2]. It appears to have been written in February 2007 and while it’s intent is to show VMware as being better than Xen, in most cases it seems to show them both as being good enough. The tests involved virtualising 32bit Windows systems, this doesn’t seem an unreasonable test as many ISPs are offering 32bit virtual machines as 32bit code tends to use less RAM. One unfortunate thing is that they make no explanation of why “Intger Math” might run at just over 80% native performance on VMware and just under 60% native performance on Xen. The other test results seem to show that for a virtualised Windows OS either VMware or Xen will deliver enough performance (apart from the ones where VMware claims that Xen provides only a tiny fraction of native performance – that’s a misconfiguration that is best ignored). Here is an analysis of the VMware benchmark and the XenSource response (which has disappeared from the net) [3].

The Cambridge Xen people have results showing a single Xen DomU delivering more than 90% native performance on a variety of well known benchmarks [4].

As it seems that in every case we can expect more than 90% native performance from a single DomU and as the case of needing more than 90% native performance is rare it seems that there is no real difference that we should care about when running servers and that the ease of management outweighs the small performance benefit from using native hardware.

Now it appears that Slicehost [5] caters to people who desire this type of management. Their virtual server plans have RAM going in all powers of two from 256M to 8G, and then they have 15.5G – which seems to imply that they are using physical servers with 16G of RAM and that 15.5G is all that is left after the Xen hypervisor and the Dom0 have taken some. One possible disadvantage of this is that if you want all the CPU power of a server but not so much RAM (or the other way around) then the Slicehost 15.5G plan might involve more hardware being assigned to you than you really need. But given the economies of scale involved in purchasing and managing the large number of servers that Slicehost is running it might cost them more to run a machine with 8G of RAM as a special order than to buy their standard 16G machine.

Other virtual hosting companies such as Gandi and Linode clearly describe that they don’t support a single instance taking all the resources of the machine (1/4 and 1/5 of a machine respectively are the maximums). I wonder if they are limiting the size of virtual machines to avoid the possibility of needing to shuffle virtual machines when migrating a running virtual machine.

One significant benefit of having a physical machine over renting a collection of DomUs is the ability to run virtual machines as you desire. I prefer to have a set of DomUs on the same physical server so that if one DomU is running slowly then I have the option to optimise other DomUs to free up some capacity. I can change the amounts of RAM and the number of virtual CPUs allocated to each DomU as needed. I am not aware of anyone giving me the option to rent all the capacity of a single server in the form of managed DomUs and then assign the amounts of RAM, disk, and CPU capacity to them as I wish. If Slicehost offered such a deal then one of my clients would probably rent a Slicehost server for this purpose as soon as their current contract runs out.

It seems that there is a lot of potential to provide significant new features for virtual hosting. I expect that someone will start offering these things in the near future. I will advise my clients to try and avoid signing any long-term contracts (where long means one year in the context of hosting) so that they keep their options open for future offers.

LUV Talk about Cloud Computing

November 9, 2008 | etbe

Last week I gave a talk for the Linux Users of Victoria about Cloud Computing and Amazon EC2 [1]. I was a little nervous as I was still frantically typing the notes a matter of minutes before my talk was due to start (which isn’t ideal). But it went well. There were many good questions and the audience seemed very interested. The talk will probably appear on the web (it was recorded and I signed a release form for the video).

Also I have just noticed that Amazon have published a license document for their AMI tools, so I have added the package ec2-ami-tools to my Debian/Lenny repository (for i386 and AMD64) with the following APT sources.list line:

deb http://www.coker.com.au lenny misc

Also the Amazon license [2] might permit adding it to the non-free section of Debian, if so I’m prepared to maintain it in Debian for the next release – but I would prefer that someone who knows Ruby take it over.

[1] http://www.luv.asn.au/2008/11
[2] http://aws.amazon.com/asl/

EC2 and IP Addresses

November 6, 2008 | etbe

One of the exciting things about having a cloud computing service is how to talk to the rest of the world. It’s all very well to have a varying number of machines in various locations, but you need constant DNS names at least (and sometimes constant IP addresses) to do most useful things.

I have previously described how to start an EC2 instance and login to it – which includes discovering it’s IP address [1]. It would not be difficult (in theory at least) to use nsupdate to change DNS records after an instance is started or terminated. One problem is that there is no way of knowing when an instance is undesirably terminated (IE killed by hardware failure) apart from polling ec2-describe-instances so it seems impossible to remove a DNS name before some other EC2 customer gets a dynamic IP address. So it seems that in most cases you will want a constant IP address (which Amazon calls an Elastic IP address) if you care about this possibility. For the case of an orderly shutdown you could have a script remove the DNS record, wait for the timeout period specified by the DNS server (so that all correctly operating DNS caches have purged the record) and then terminate the instance.

One thing that interests me is the possibility of running front-end mail servers on EC2. Mail servers that receive mail from the net can take significant amounts of CPU time and RAM for spam and virus filters. Instead of having the expense of running enough MX servers to sustain the highest possible load even while one of the servers has experienced a hardware failure there is a possibility of running an extra EC2 instance at peak times with the possibility of running a large instance for a peak time when one of the dedicated servers has experienced a problem. The idea of having a mail server die and have someone else’s server take the IP address and receive the mail is too horrible to contemplate, so an Elastic IP address is required.

It is quite OK to have a set of mail servers of which not all servers run all the time (this is why the MX record was introduced to the DNS) so having a server run periodically at periods of high load (one of the benefits of the EC2 service) will not require changes to the DNS. I think it’s reasonably important to minimise the amount of changes to the DNS due to the possibility of accidentally breaking it (which is a real catastrophe) and the possibility of servers caching DNS data for longer than they should. The alternative is to change the MX record to not point to the hostname of the server when the instance is terminated. I will be interested to read comments on this issue.

The command ec2-allocate-address will allocate a public IP address for your use. Once the address is allocated it will cost $0.01 per hour whenever it is unused. There are also commands ec2-describe-addresses (to list all addresses allocated to you), ec2-release-address (to release an allocated address), ec2-associate-address to associate an IP address with a running instance, and ec2-disassociate-address to remove such an association.

The command “ec2-associate-address -i INSTANCE ADDRESS” will associate an IP address with the specified instance (replace INSTANCE with the instance ID – a code starting with “i-” that is returned from ec2-describe-instances. The command “ec2-describe-instances |grep ^INSTANCE|cut -f2” will give you a list of all instance IDs in use – this is handy if your use of EC2 involves only one active instance at a time (all the EC2 API commands give output in tab-separated lists and can be easily manipulated with grep and cut). Associating an IP address with an instance is documented as taking several minutes, while Amazon provides no guarantees or precise figures as to how long the various operations take it seems that assigning an IP address is one of the slower operations. I expect that is due to the requirement for reconfiguring a firewall device (which services dozens or maybe hundreds of nodes) while creating or terminating an instance is an operation that is limited in scope to a single Xen host.

One result that I didn’t expect was that associating an elastic address is that the original address that was assigned to the instance is removed. I had a ssh connection open to an instance when I assigned an elastic address and my connection was broken. It makes sense to remove addresses that aren’t needed (IPv4 addresses are a precious commodity) and further reading of the documentation revealed that this is the documented behavior.

One thing I have not yet investigated is whether assigning an IP address from one instance to another is atomic. Taking a few minutes to assign an IP address is usually no big deal, but having an IP address be unusable for a few minutes while in the process of transitioning between servers would be quite inconvenient. It seems that a reasonably common desire would be to have a small instance running and to then transition the IP address to a large (or high-CPU) instance if the load gets high, having this happen without the users noticing would be a good thing.

[1] http://etbe.coker.com.au/2008/11/04/basics-of-ec2/

Basics of EC2

November 4, 2008 | etbe

I have previously written about my work packaging the tools to manage Amazon EC2 [1].

First you need to login and create a certificate (you can upload your own certificate – but this is probably only beneficial if you have two EC2 accounts and want to use the same certificate for both). Download the X509 private key file (named pk-X.pem) and the public key (named cert-X.pem). My Debian package of the EC2 API tools will look for the key files in the ~/.ec2 and /etc/ec2 directories and will take the first one it finds by default.

To override the certificate (when using my Debian package) or to just have it work when using the code without my package you set the variables EC2_PRIVATE_KEY and EC2_CERT.

This Amazon page describes some of the basics of setting up the client software and RSA keys [2]. I will describe some of the most important things now:

The command “ec2-add-keypair gsg-keypair > id_rsa-gsg-keypair” creates a new keypair for logging in to an EC2 instance. The public key goes to amazon and the private key can be used by any ssh client to login as root when you creat an instance. To create an instance with that key you use the “-k gsg-keypair” option, so it seems a requirement to use the same working directory for creating all instances. Note that gsg-keypair could be replaced by any other string, if you are doing something really serious with EC2 you might use one account to create instances that are run by different people with different keys. But for most people I think that a single key is all that is required. Strangely they don’t provide a way of getting access to the public key, you have to create an instance and then copy the /root/.ssh/authorized_keys file for that.

This Amazon page describes how to set up sample images [3].

The first thing it describes is the command ec2-describe-images -o self -o amazon which gives a list of all images owned by yourself and all public images owned by Amazon. It’s fairly clear that Amazon doesn’t expect you to use their images. The i386 OS images that they have available are Fedora Core 4 (four configurations with two versions of each) and Fedora 8 (a single configuration with two versions) as well as three other demo images that don’t indicate the version. The AMD64 OS images that they have available are Fedora Core 6 and Fedora Core 8. Obviously if they wanted customers to use their own images (which seems like a really good idea to me) they would provide images of CentOS (or one of the other recompiles of RHEL) and Debian. I have written about why I think that this is a bad idea for security [4], please make sure that you don’t use the ancient Amazon images for anything other than testing!

To test choose an i386 image from Amazon’s list, i386 is best for testing because it allows the cheapest instances (currently $0.10 per hour).

Before launching an instance allow ssh access to it with the command “ec2-authorize default -p 22“. Note that this command permits access for the entire world. There are options to limit access to certain IP address ranges, but at this stage it’s best to focus on getting something working. Of course you don’t want to actually use your first attempt at creating an instance, I think that setting up an instance to run in a secure and reliable manner would require many attempts and tests. As all the storage of the instance is wiped when it terminates (as we aren’t using S3 yet) and you won’t have any secret data online security doesn’t need to be the highest priority.

A sample command to run an instance is “ec2-run-instances ami-2b5fba42 -k gsg-keypair” where ami-2b5fba42 is a public Fedora 8 image available at this moment. This will give output similar to the following:

RESERVATION r-281fc441 999999999999 default
INSTANCE i-0c999999 ami-2b5fba42 pending gsg-keypair 0 m1.small 2008-11-04T06:03:09+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc

The parameter after the word INSTANCE is the serial number of the instance. The command “ec2-describe-instances i-0c999999” will provide information on the instance, once it is running (which may be a few minutes after you request it) you will see output such as the following:

RESERVATION r-281fc441 999999999999 default
INSTANCE i-0c999999 ami-2b5fba42 ec2-10-11-12-13.compute-1.amazonaws.com domU-12-34-56-78-9a-bc.compute-1.internal running gsg-keypair 0 m1.small 2008-11-04T06:03:09+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc

The command “ssh -i id_rsa-gsg-keypair root@ec2-10-11-12-13.compute-1.amazonaws.com” will then grant you root access. The part of the name such as 10-11-12-13 is the public IP address. Naturally you won’t see 10.11.12.13, it will instead be public addresses in the Amazon range – I replaced the addresses to avoid driving bots to their site.

The name domU-12-34-56-78-9a-bc.compute-1.internal is listed in Amazon’s internal DNS and returns the private IP address (in the 10.0.0.0/8 range) which is used for the instance. The instance has no public IP address, all connections (both inbound and outbound) run through some sort of NAT. This shouldn’t be a problem for HTTP, SMTP, and most protocols that are suitable for running on such a service. But for FTP or UDP based services it might be a problem. The part of the name such as12-34-56-78-9a-bc is the MAC address of the eth0 device.

To halt a service you can run shutdown or halt as root in the instance, or run the ec2-terminate-instances command and give it the instance ID that you want to terminate. It seems to me that the best way of terminating an instance would be to run a script that produces a summary of whatver the instance did (you might not want to preserve all the log data, but some summary information would be useful), and give all operations that are in progress time to stop before running halt. A script could run on the management system to launch such an orderly shutdown script on the instance and then uses ec2-terminate-instances if the instance does not terminate quickly enough.

In the near future I will document many aspects of using EC2. This will include dynamic configuration of the host, dynamic DNS, and S3 storage among other things.

Types of Cloud Computing

November 3, 2008 | etbe

The term Cloud Computing seems to be poorly defined at the moment, as an example the Wikipedia page about it is rather incoherent [1].

The one area in which all definitions of the term agree is that a service is provided by a varying number of servers of which the details are unknown to the people who use the services – including the people who program them.

There seem to be four main definitions of cloud computing. The first one is Distributed Computing [2] (multiple machines in different administrative domains being used to perform a single task or several tasks), this definition does not seem to be well accepted and will probably disappear.

The next one is Software As A Service [3] which is usually based on the concept of outsourcing the ownership and management of some software and accessing it over the Internet. An example is the companies that offer outsourced management of mail servers that are compatible with MS Exchange (a notoriously difficult system to manage). For an annual fee per email address you can have someone else run an Exchange compatible mail server which can talk to Blackberries etc. The main benefit of SAAS is that it saves the risk and expense of managing the software, having the software deployed at a central location is merely a way of providing further cost reductions (some companies that offer SAAS services will also install the software on your servers at greater expense).

The last two definitions are the ones that I consider the most interesting, that is virtual machines (also known as Cloud Infrastructure [4]) and virtual hosting of applications (also known as Cloud Platforms [5]). The Amazon EC2 (Elastic Cloud Computing) service [6] seems to be the leading virtual machine cloud service at the moment. Google App Engine [7] seems to be the leading cloud application hosting service at the moment.

It is also claimed that “Cloud Storage” is part of cloud computing. It seems to me that storing data on servers all over the world is something that was done long before the term “Cloud Computing” was invented, so I don’t think it’s deserving of the term.

etbe – Russell Coker

Linux, politics, and other interesting things

Category Archives: Virtualisation