File storage and video hosting on Amazon S3

Posted on by

Amazon S3 (Simple Storage Service) provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. Each object is stored in a bucket and retrieved via a unique, developer-assigned key. It’s possible to write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited.

The pricing plan for Amazon S3 is based on “Pay only for what you use”. There is no minimum fee. The prices vary across geographic regions and are based on the location of the bucket. For Europe the pricing effective november 2008 is as follows :

Storage :

  • $0.180 per GB – first 50 TB / month of storage used
  • $0.170 per GB – next 50 TB / month of storage used
  • $0.160 per GB – next 400 TB / month of storage used
  • $0.150 per GB – storage used / month over 500 TB

Data Transfer :

  • $0.100 per GB – all data transfer in
  • $0.170 per GB – first 10 TB / month data transfer out
  • $0.130 per GB – next 40 TB / month data transfer out
  • $0.110 per GB – next 100 TB / month data transfer out
  • $0.100 per GB – data transfer out / month over 150 TB

Requests :

  • $0.012 per 1,000 PUT, POST, or LIST requests
  • $0.012 per 10,000 GET and all other requests

The Amazon S3 service is part of the Amazon Web Services, an infrastructure web services platform in the cloud. The direct link to the S3 login page is aws.amazon.com/s3/. I created my account on Amazon S3 on october 13th, 2008.

Some test files to verify the correct configuration are accessible at the following links :

The sender of a request to an AWS web service is authorised by Access Identifiers. Access identifiers are also used to authenticate requests to AWS. For services that require authenticated requests, where you need to prove that you are authorized to make the request, you must sign the request by including a “signature” in the request. A request signature is calculated using the pair of public / private Access Identifiers. AWS supports two types of Request Identifiers: AWS Access Key Identifiers or X.509 Certificates.

The Access Key ID is used as the value of the AWSAccessKeyId parameter in requests you send to Amazon Web Services and identifies you as the party responsible for the request. Since the Access Key ID is not encrypted in requests to AWS, it could be discovered and used by anyone. Services that are not free require you to provide additional information, a request signature, to verify that a request containing your unique Access Key ID could only have come from you. You use your Secret Access Key to calculate a signature to include in requests to web services that require authenticated requests.

Uploading and managing files on the Amazon S3 distributed system is not straight forward. Amazon offers documentation, release notes, sample code & libraries, developer tools, articles & tutorials and a community forum as resources to developers.

A comfortable user interface to upload and manage files on the 3S servers called “Bucket Explorer” (version 2008.06.30.01) is offered by Chambal.com.Inc for Windows, Linux and Mac (website www.bucketexplorer.com). The price for one licence is USD 49.95, a 30 day free trial version is available. A free plugin for the Firefox browser called “S3Fox Organizer” (version 0.4.1) has been developped by Rjonna of Suchi Software Solutions from India.

Another Firefox plugin (Gspace) developped by the same author and distributed by Fon Labs allows you to use your Gmail Space (2700 MB) as a remote machine. You can transfer files between your hard drive and gmail. Your gmail account looks like a FTP host and you can upload and download your files. After you install, you get an option called “GSpace” in your “tools” menu clicking on which opens the window for transfer. You have to be aware that this extension is not endorsed by Google and that this usage is not compliant with the Google Terms of Service.