Wrapping Things Nicely with HTML5 Local Storage

HTML5 is here to turn the web from a web of hacks into a web of applications – and we are well on the way to this goal. The coming year will be totally and utterly awesome if you are excited about web technologies.

This year the HTML5 revolution started and there is no stopping it. For the first time all the browser vendors are rallying together to make a technology work. The new browser war is fought over implementation of the HTML5 standard and not over random additions. We live in exciting times.

Starting with a bang

As with every revolution there is a lot of noise with bangs and explosions, and that’s the stage we’re at right now. HTML5 showcases are often CSS3 showcases, web font playgrounds, or video and canvas examples.

This is great, as it gets people excited and it gives the media something to show. There is much more to HTML5, though. Let’s take a look at one of the less sexy, but amazingly useful features of HTML5 (it was in the HTML5 specs, but grew at such an alarming rate that it warranted its own spec): storing information on the client-side.

Why store data on the client-side?

Storing information in people’s browsers affords us a few options that every application should have:

• You can retain the state of an application – when the user comes back after closing the browser, everything will be as she left it. That’s how ‘real’ applications work and this is how the web ones should, too.
• You can cache data – if something doesn’t change then there is no point in loading it over the Internet if local access is so much faster
• You can store user preferences – without needing to keep that data on your server at all.

In the past, storing local data wasn’t much fun.

The pain of hacky browser solutions

In the past, all we had were cookies. I don’t mean the yummy things you get with your coffee, endorsed by the blue, furry junkie in Sesame Street, but the other, digital ones. Cookies suck – it isn’t fun to have an unencrypted HTTP overhead on every server request for storing four kilobytes of data in a cryptic format. It was OK for 1994, but really neither an easy nor a beautiful solution for the task of storing data on the client.

Then came a plethora of solutions by different vendors – from Microsoft’s userdata to Flash’s LSO, and from Silverlight isolated storage to Google’s Gears. If you want to know just how many crazy and convoluted ways there are to store a bit of information, check out Samy’s evercookie.

Clearly, we needed an easier and standardised way of storing local data.

Keeping it simple – local storage

And, lo and behold, we have one. The local storage API (or session storage, with the only difference being that session data is lost when the window is closed) is ridiculously easy to use. All you do is call a few methods on the window.localStorage object – or even just set the properties directly using the square bracket notation:

if('localStorage' in window && window['localStorage'] !== null){
	var store = window.localStorage;

// valid, API way store.setItem(‘cow’,‘moo’); console.log( store.getItem(‘cow’) ); // => ‘moo’

// shorthand, breaks at keys with spaces store.sheep = ‘baa’ console.log( store.sheep ); // ‘baa’

// shorthand for all store[‘dog’] = ‘bark’ console.log( store[‘dog’] ); // => ‘bark’

}

Browser support is actually pretty good: Chrome 4+; Firefox 3.5+; IE8+; Opera 10.5+; Safari 4+; plus iPhone 2.0+; and Android 2.0+. That should cover most of your needs. Of course, you should check for support first (or use a wrapper library like YUI Storage Utility or YUI Storage Lite).

The data is stored on a per domain basis and you can store up to five megabytes of data in localStorage for each domain.

Strings attached

By default, localStorage only supports strings as storage formats. You can’t store results of JavaScript computations that are arrays or objects, and every number is stored as a string. This means that long, floating point numbers eat into the available memory much more quickly than if they were stored as numbers.

var cowdesc = "the cow is of the bovine ilk, "+
		"one end is for the moo, the "+
		"other for the milk";

var cowdef = { ilk“bovine”, legs, udders, purposes front“moo”, end“milk” }
};

window.localStorage.setItem(‘describecow’,cowdesc);
console.log( window.localStorage.getItem(‘describecow’)
); // => the cow is of the bovine…

window.localStorage.setItem(‘definecow’,cowdef);
console.log( window.localStorage.getItem(‘definecow’)
); // => [object Object] = bad!

This limits what you can store quite heavily, which is why it makes sense to use JSON to encode and decode the data you store:

var cowdef = {
	"ilk":"bovine",
	"legs":4,
	"udders":4,
	"purposes":{
	"front":"moo",
	"end":"milk"
	}
};

window.localStorage.setItem(‘describecow’,JSON.stringify(cowdef));
console.log( JSON.parse( window.localStorage.getItem(‘describecow’) )
); // => Object { ilk=“bovine”, more…}

You can also come up with your own formatting solutions like CSV, or pipe | or tilde ~ separated formats, but JSON is very terse and has native browser support.

Some use case examples

The simplest use of localStorage is, of course, storing some data: the current state of a game; how far through a multi-form sign-up process a user is; and other things we traditionally stored in cookies. Using JSON, though, we can do cooler things.

Speeding up web service use and avoiding exceeding the quota

A lot of web services only allow you a certain amount of hits per hour or day, and can be very slow. By using localStorage with a time stamp, you can cache results of web services locally and only access them after a certain time to refresh the data.

I used this technique in my An Event Apart 10K entry, World Info, to only load the massive dataset of all the world information once, and allow for much faster subsequent visits to the site. The following screencast shows the difference:

For use with YQL (remember last year’s 24 ways entry?), I’ve built a small script called YQL localcache that wraps localStorage around the YQL data call. An example would be the following:

yqlcache.get({
	yql: 'select * from flickr.photos.search where text="santa"',
	id: 'myphotos',
	cacheage: ( 60*60*1000 ),
	callback: function(data) {
		console.log(data);
	}
});

This loads photos of Santa from Flickr and stores them for an hour in the key myphotos of localStorage. If you call the function at various times, you receive an object back with the YQL results in a data property and a type property which defines where the data came from – live is live data, cached means it comes from cache, and freshcache indicates that it was called for the first time and a new cache was primed. The cache will work for an hour (60×60×1,000 milliseconds) and then be refreshed. So, instead of hitting the YQL endpoint over and over again, you hit it once per hour.

Caching a full interface

Another use case I found was to retain the state of a whole interface of an application by caching the innerHTML once it has been rendered. I use this in the Yahoo Firehose search interface, and you can get the full story about local storage and how it is used in this screencast:

The stripped down code is incredibly simple (JavaScript with PHP embed):

// test for localStorage support
if(('localStorage' in window) && window['localStorage'] !== null){

var f = document.getElementById(‘mainform’);
// test with PHP if the form was sent (the submit button has the name “sent”)

// get the HTML of the form and cache it in the property “state” localStorage.setItem(‘state’,f.innerHTML);

// if the form hasn’t been sent…

// check if a state property exists and write back the HTML cache if(‘state’ in localStorage){ f.innerHTML = localStorage.getItem(‘state’); }

}

Other ideas

In essence, you can use local storage every time you need to speed up access. For example, you could store image sprites in base-64 encoded datasets instead of loading them from a server. Or you could store CSS and JavaScript libraries on the client. Anything goes – have a play.

Issues with local and session storage

Of course, not all is rainbows and unicorns with the localStorage API. There are a few niggles that need ironing out. As with anything, this needs people to use the technology and raise issues. Here are some of the problems:

• Inadequate information about storage quota – if you try to add more content to an already full store, you get a QUOTA_EXCEEDED_ERR and that’s it. There’s a great explanation and test suite for localStorage quota available.
• Lack of automatically expiring storage – a feature that cookies came with. Pamela Fox has a solution (also available as a demo and source code)
• Lack of encrypted storage – right now, everything is stored in readable strings in the browser.

Bigger, better, faster, more!

As cool as the local and session storage APIs are, they are not quite ready for extensive adoption – the storage limits might get in your way, and if you really want to go to town with accessing, filtering and sorting data, real databases are what you’ll need. And, as we live in a world of client-side development, people are moving from heavy server-side databases like MySQL to NoSQL environments.

On the web, there is also a lot of work going on, with Ian Hickson of Google proposing the Web SQL database, and Nikunj Mehta, Jonas Sicking (Mozilla), Eliot Graff (Microsoft) and Andrei Popescu (Google) taking the idea beyond simply replicating MySQL and instead offering Indexed DB as an even faster alternative.

On the mobile front, a really important feature is to be able to store data to use when you are offline (mobile coverage and roaming data plans anybody?) and you can use the Offline Webapps API for that.

As I mentioned at the beginning, we have a very exciting time ahead – let’s make this web work faster and more reliably by using what browsers offer us. For more on local storage, check out the chapter on Dive into HTML5.