File storage and video hosting on Amazon S3

Amazon S3 (Simple Storage Service) provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. Each object is stored in a bucket and retrieved via a unique, developer-assigned key. It’s possible to write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited.

The pricing plan for Amazon S3 is based on “Pay only for what you use”. There is no minimum fee. The prices vary across geographic regions and are based on the location of the bucket. For Europe the pricing effective november 2008 is as follows :

Storage :

  • $0.180 per GB – first 50 TB / month of storage used
  • $0.170 per GB – next 50 TB / month of storage used
  • $0.160 per GB – next 400 TB / month of storage used
  • $0.150 per GB – storage used / month over 500 TB

Data Transfer :

  • $0.100 per GB – all data transfer in
  • $0.170 per GB – first 10 TB / month data transfer out
  • $0.130 per GB – next 40 TB / month data transfer out
  • $0.110 per GB – next 100 TB / month data transfer out
  • $0.100 per GB – data transfer out / month over 150 TB

Requests :

  • $0.012 per 1,000 PUT, POST, or LIST requests
  • $0.012 per 10,000 GET and all other requests

The Amazon S3 service is part of the Amazon Web Services, an infrastructure web services platform in the cloud. The direct link to the S3 login page is aws.amazon.com/s3/. I created my account on Amazon S3 on october 13th, 2008.

Some test files to verify the correct configuration are accessible at the following links :

The sender of a request to an AWS web service is authorised by Access Identifiers. Access identifiers are also used to authenticate requests to AWS. For services that require authenticated requests, where you need to prove that you are authorized to make the request, you must sign the request by including a “signature” in the request. A request signature is calculated using the pair of public / private Access Identifiers. AWS supports two types of Request Identifiers: AWS Access Key Identifiers or X.509 Certificates.

The Access Key ID is used as the value of the AWSAccessKeyId parameter in requests you send to Amazon Web Services and identifies you as the party responsible for the request. Since the Access Key ID is not encrypted in requests to AWS, it could be discovered and used by anyone. Services that are not free require you to provide additional information, a request signature, to verify that a request containing your unique Access Key ID could only have come from you. You use your Secret Access Key to calculate a signature to include in requests to web services that require authenticated requests.

Uploading and managing files on the Amazon S3 distributed system is not straight forward. Amazon offers documentation, release notes, sample code & libraries, developer tools, articles & tutorials and a community forum as resources to developers.

A comfortable user interface to upload and manage files on the 3S servers called “Bucket Explorer” (version 2008.06.30.01) is offered by Chambal.com.Inc for Windows, Linux and Mac (website www.bucketexplorer.com). The price for one licence is USD 49.95, a 30 day free trial version is available. A free plugin for the Firefox browser called “S3Fox Organizer” (version 0.4.1) has been developped by Rjonna of Suchi Software Solutions from India.

Another Firefox plugin (Gspace) developped by the same author and distributed by Fon Labs allows you to use your Gmail Space (2700 MB) as a remote machine. You can transfer files between your hard drive and gmail. Your gmail account looks like a FTP host and you can upload and download your files. After you install, you get an option called “GSpace” in your “tools” menu clicking on which opens the window for transfer. You have to be aware that this extension is not endorsed by Google and that this usage is not compliant with the Google Terms of Service.

Layers over objects using z-index

last update : june 13, 2011

I noticed that a layer with an embedded webpage as an object is not covered by an other layer having a higher z-index in Internet Explorer, whereas it is correctly displayed in Firefox. The same problem has been reported in the webmasterworld forum. I found a workaround (see www.artgallery.lu/expositions/expo_samuel_martin.html) to integrate the text from the second layer in the embedded webpage, but this works only without links.

Another tough nut to crack for developers is the z-index working with Flash, because the Flash Player always writes the SWF on the topmost layer of the page, regardless of z-index order. It works with swfobject if the parameter wmode is set to opaque (see an example on the website of Leslie’s Artgallery).

A demopage showing how z-index works is available at the tjkdesign.com website.

Email Encoder and Spam Blocker for Web Forms

Spam is a great problem for all website owners. A simple method to stop email harvesters from picking up e-mail addresses from web pages consist in encoding the email address in hexadecimal characters. By encoding the entire e-mail address including the “mailto” portion, most e-mail harvesters (except the most sophisticated) can’t find it. If they can’t find it, they can’t send spam to it.

A free hexadecimal email encoder is offered on the website from K’nechtology Inc

To protect web-forms from spam abuse, more sophisticated solutions are required. Spam Blocker For Web Forms is  a powerful, easy-to-use application ideal for protection of web forms from being parsed and used for sending spam emails. Includes adjustable spam filters and anti spam source code protection. I use a registered version since more than a year with great success.

Spam Blocker For Web Forms is developped by Dmitry Konovalov from White Stranger Group. The current version is 1.1, a single user license is available for 40 US$, a business license for 60 US$.

SWISH : alternative Flash animation tool

SWiSHzone.com is the world leader in alternative Flash™ animation tools. The philosophy of SWISHzone Pty Ltd is to make Flash authoring accessible to everyone, with software that is both easy to use and affordable.

Founded in 1999 and based in Sydney, Australia, SWiSHzone’s product range has grown from a single text animation tool to a suite of products that make it easy to create Flash-based video, photo albums, PowerPoint™ presentations and more. The flagship product SWiSH Max is a professional Flash authoring tool with enough power and flexibility for the most demanding of users.

CHMOD

CHMOD (abréviation de change mode) est une commande exécutable dans un environnement de type Unix qui permet de changer les permissions d’accès (spéciales ou non) sur un fichier ou dossier. Il arrive fréquemment qu’on lit dans un guide d’installation d’une application sur un serveur qu’il faut configurer un fichier ou dossier en chmod 777 ou chmod 644 ou -wx.

Sur un serveur, on trouve, sous propriétés d’un fichier ou dossier, un tableau avec des champs à cocher avec trois colonnes “lire ( r = read), écrire (w = write) et exécuter (x = execute)” et trois rangées “propriétaire (owner), groupe (group) et autres (all users)”. Lire a la valeur 4, écrire a la valeur 2 et exécuter a la valeur 1.

Par exemple CHMOD 764 veut donc dire :

  • le propriétaire peut lire, écrire et exécuter: 4+2+1=7
  • le groupe peut lire et écrire:4+2=6
  • les autres ne peuvent que le lire: 4

Il n’est pas recommandé de mettre les permissions des fichiers et répertoires sur un serveur sur 777, car tout le monde peut alors les modifier s’il a accès au serveur. Il est conseillé de mettre les fichiers sur 664 et les répertoires sur 775, sauf exceptions justifiées.

Sitemap’s

Last update : June 16, 2013
Sitemap’s are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. Sitemaps also offer human visitors an overview of a website to easily navigate. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.

The sitemap xml protocol 0.90, available on www.sitemaps.org,  is offered under the terms of the Attribution-ShareAlike Creative Commons License and has wide adoption, including support from Google, Yahoo!, and Microsoft.

There are several tools available to generate sitemaps. My favorite is the online sitemap generator from xml-sitemap.com.

A quick guide for creating sitemaps is available on www.sitemapxml.co.uk. Detailled user guides are edited by the search engines providers, for example by Google.

On 13th august 2009, I installed the standalone sitemap generator from XML-Sitemaps.com succesfully.

Pagepeel, pageear and pageflip

The first flash scripts to  show a flip book animation on a website appeared a few years ago. The most advanced script was programmed by IpariGrafika in Hungary. A dynamic version with an external xml file based on IpariGrafika’s script was published in 2005 by Steve Palmer on the Blogspace of 76design in Canada.

Today, IpariGrafika still offers the version 2.25 of the PageFlip script for free, but the new scalable versions 3.5 are associated with a paid licence ranging from 25 US$ for one website without source code to 395 US$ for unlimited commercial use with source code included. A special website www.pageflip.hu bas been created to promote and distribute the different versions. Several demonstrations of the pageflip script for artworks catalogs can be seen at the website of Leslie’s Artgallery in Luxembourg.

Two years ago, a script called pagepeel related to the pageflip animation appeared as a new interesting advertising form on websites. Pagepeel is a small animated pageear waving on the top right side of a website. It reveals a hint of something behind it. When a visitor moves the mouse over it, it pulls down like peeling the page down to reveal the eye catching ad design, while not effecting the website.

The pagepeel script is also named pagecurl, magiccorner, adpeel, peelad, eselsohr, flash peel, … There are a lot of more or less well designed pagepeel scripts available on the net, but only a single one is available for free. It’s the pagepeel flashscript developped by Christian Harz from Webpicasso Media in Germany. A scalable version 1.4 was released on june 29th, 2008. A tutorial about programming a pagepeel script is shown on www.pagepeel.info.

In the IE browser, animated GIF’s are stopped running when the pageear closes. Tip : The small image has an x-offset of 3 pixels and y-offset of 1 pixel.

Website visitor tracking

There are several tools available to analyse and monitor visits to a website or blog in real-time. The common method for reliable tracking consist in inserting a simple piece of javascript code on every webpage and saving a first party cookie on the visitors computer. The cookie allows to distinguish between returning visitors and first time visitors. The sum of returning and first time visitors are called unique visitors.
IP + User Agent tracking typically reports higher numbers than cookie-based tracking because of dynamically assigned IP addresses and spider and robot visits. 3rd party cookies are set by a source other than the website being visited and are often blocked by browsers and security software, whereas 1st party cookies are considered as harmless by spyware protection software.

The most used free tracking and statistics packages are StatCounter, SiteMeter and Google Analytics. A comparison of StatCounter versus Google Analytics has been published by Andy Atkinson on his blog “Pain in the Tech“.

Google Analytics

I prefer Google Analytics, which is based on Urchin, and I use it for different websites since 2006. Google Analytics uses 1st partie cookies, counts a visit even if the page is pulled from a user’s or server’s cache, but it registers only one visit if a user comes to a site twice within thirty minutes without closing their browser. The Google Analytics reports include a dashboard with different site usage overviews and detailed informations about visitors, content, traffic sources and goals. All the data are anonymous and the privacy of the visitors is guaranteed.

Google provides a help page, an official blog and a conversion university for Google Analytics. An unofficial Google Analytics Blog is provided by ROI Revolution. A forum about Website Analytics – Tracking and Logging is available at WebmasterWorld.com. A solid tutorial “8 stupid things webmasters do to mess up their analytics” has been edited by Linda Bustos on Marketing Pilgrim. SiteScan offers a Google Analytics Diagnostic tool designed to provide you with a complete audit of your Google Analytics setup.

Nicolas Lierman is the author of the Google Analytics Reporting Suite developed in Adobe AIR. He published on his website AboutNico.be 5 tips for using Google Analytics.

Webalizer

A famous log file analysis programm for web servers is Webalizer. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. Written in C by Bradford L. Barrett who started the development in 1997, the program is open source and was designed for running on Linux systems. Version 2.20-01 was released on july 12th, 2008. “What is your webserver doing today ?” is the motto of the webalizer website.