Bandwidth

Bandwidth allotment is an important consideration in choosing a dedicated
or colocated server host.  There are different methods of allocation,
and it is important to understand the differences among them.

First, let us begin by defining bandwidth.  What is often referred
to as bandwidth is really data transfer, which is the total of how much
data has been transferred from a site over a certain period of time. Every
time someone makes a request for an HTML page (views it in a browser),
that HTML file is transferred to the user's computer, which takes bandwidth.
If there are any images on that page, each of those images must also be
transferred to the users browser, which takes more bandwidth.  Bandwidth
consumption on specific sites can be minimized by making the data set
to be transferred smaller.  Some of the ways to accomplish this include
removing extraneous tags and spaces in your HTML code, compressing images
more, using less colors in them, and using fewer images.  Also, not
using Microsoft Front Page. Not only does it generate sloppy and bloated
code, but it adds unnecessarily to bandwidth consumption.

Sizes defined

8 bits = 1 byte.

1,024 bytes = 1 kilobyte (Kb).

1,024 kilobytes (Kb) = 1 megabyte (mb or meg)

1,024 megabytes = 1 gigabyte (gb or gig)

1,024 gigabytes = 1 terabyte. (tb)

The above are the correct and exact definitions.  Note, however,
that some hosts will use 1000 as the multiplier rather than 1024. Be sure
to check with any host regarding how they define these terms.

Bandwidth patterns

Bandwidth allocation and costs

Bandwidth allocation methods

There are three basic methods used in the allocation of bandwidth. 
They are the capped line (or rate limited) method, average usage method,
and the x percentile method.  It is important to understand each
one and what the differences among them are.  One in particular,
the percentile method, is greatly misunderstood and there is much false
information spread about it by people who simply do not understand it.

Capped Line

Payment is a flat fee based on the size of the dedicated line you get.
The amount of bandwidth that can be transferred is capped, or limited,
to a specified amount.  This can contain the cost of bandwidth to
the specified limit and keep it at a flat fee. You get a data pipe of
a specified (limited) size. A peak in bandwidth (several users accessing
the site simultaneously) would mean slowdown or inaccessibility of the
server. Some users would have to wait longer to access sites and some
might get a "server timed out" message and not reach the site.

This method is also referred to as rate limiting as the rate of transfer
is limited.

Average Usage

Bandwidth measurements are taken at various intervals and averaged together
to provide the bandwidth consumption figure. Any bandwidth in excess of
the allotted amount is charged the stated amount per gigabyte.  Some
hosts have bandwidth plans in blocks, whereas others charge per gb. 
Crossing the threshold into the next level could add substantially to
the costs in block plans, especially for those hosts with a high cost
per gb, so be especially aware of the cost of higher levels. 

Note also that some hosts charge an estimated usage amount which may
be higher than actual usage.  Some may choose to use an average figure,
others might use the median. 

Overall costs tend to be higher at hosts that use the average usage method. 
You are either subsidizing someone else's high usage, or they are subsiding
yours, and you are all paying higher costs on other services to cover
the costs to the host. 

It probably averages out for the hosting companies if they have enough
customers.  The way bandwidth traffic patterns usually work out,
some 75% of them will probably overpay and cover the other 25% that underpay.

Percentile Method

This is the most misunderstood of the bandwidth methods. Though any percentile
can be used, the 95th percentile method is most frequently mentioned.
Here's the reason providers use the 95th percentile system - If you let
your customers burst, it doesn't matter if you're paying for X bandwidth,
or paying for X with burstable. 

If your customers burst, then you must make sure that you have
bandwidth available above their highest burst rate, not just equal
to it, above it. If a customer uses 5 kbit for 5 minutes, you have
to have 6 kbit for the entire month as they can burst at any time. If
a customer uses 5 MBit for 5 minutes, you have to have 5.1 Mbit the entire
month. Again, they can burst at any time.

In order to do that, you must either subsidize bandwidth elsewhere, or
bill them based on that burst rate. There is no in-between, not if you
want to last as a business.

Most of the upper end bandwidth companies such as AboveNet, Exodus Communications,
and UUNet use it.  They do so in order to prevent situations where
they're getting congestion due to lack of bandwidth.  They must always
keep the 100th percentile amount of bandwidth in reserve.

The percentile method allows a server to accommodate most any increase
in traffic without slowing access speeds for the visitors.  The chart
below is an actual one from the port of a host using the 95%ile method. 
In this case, it also indicates a change in clients, which is why there
is no real activity from about July 21 to July 31 (the new customer took
over on the 31st).

However, there are several spikes driving usage way up on a repeated
basis. Basically, what happens is that more bandwidth is allocated so
that traffic is not slowed down when it increases, even as much as it
did here (and in other spikes). Essentially, the 95th percentile system
is close to your maximum sustained bandwidth usage for any part of the
month, and you are billed for that amount for the entire month if you
are on this system.

The top 5% on the spikes is knocked off. Note the highest point in this
graph is 1.21m Multiply that by 30 days and you would have 36.3 gb for
the month. However, the maximum bandwidth is 33 gb. Obviously, part of
that 1.21m spike is the 5% lopped off the top end.

What's the alternative? Systems that limit the bandwidth. They can contain
your bandwidth costs to what you expect, but at the drawback of slowing
access at peak periods. Imagine a hosting company with capped bandwidth
and several sites at peak demand simultaneously.

Anyone care to read War and Peace or Hawaii while waiting
for a web site to load in your browser?

Here we have another actual example (taken on May 30, 2001). Note the
overall more uniform level than the example above.

So, to recap, we have 95%ile, with the advantage of fairly uniform access
speed regardless of the increase in traffic and the disadvantage that
the price of bandwidth will rise commensurately with that increase.

Since most hosting companies don't want the reputation of having slow
servers, you should understand why they would prefer this method, even
if they have a choice. However, many bandwidth providers charge by the
95th%ile method, so some of the hosting companies charge their customers
the same way.

Misconceptions About the Percentile Method

It is unfair.

Is the 95%ile system unfair? No. It is a system that ensures that
no matter how much increased bandwidth you have (which is related to traffic
but not synonymous with it), that the speed will not be impaired. It can
keep companies and servers from getting a reputation as a "slow server".
In some cases, that could be detrimental for business. If you try people's
patience, they may try other sites.

Why, then, do some people think this method is unfair?  Aside from
not understanding it, it is probably because they are billed for near
the highest amount of usage for the entire month.

It discriminates against high bandwidth usage sites.

It does no such thing.  It is the high spikes in bandwidth,
not high bandwidth usage of itself, that trigger the charges some people
don't understand.  High usage sites pay more because they use more.
There is nothing the least bit discriminatory about that very fair practice.

It triples your "real" bandwidth usage.

Another falsehood. Some people have had sustained spikes triple,
or even quadruple, that of their level usage, in excess of the allotted
amount, and state this as fact.  It simply isn't true.  The
variation from level use to sustained spike height can vary from server
to server.

One thing must happen in order to incur additional bandwidth charges
under the 95th %ile method.  The sustained spike level must exceed
the allotted amount of bandwidth. If that doesn't happen, it doesn't matter
how high the spikes are.

If the sustained spike level does exceed the allotted amount, then the
height of the sustained spikes is a factor in additional charges. 
The frequency of the spikes, their height, and how long they spike, all
help determine to what extent additional charges are incurred. It is far
from so simple a statement as tripling one's usage.