localForage: Offline Storage, Improved

Web apps have had offline capabilities like saving large data sets and binary files for some time. You can even do things like cache MP3 files. Browser technology can store data offline and plenty of it. The problem, though, is that the technology choices for how you do this are fragmented.

localStorage gets you really basic data storage, but it’s slow and can’t handle binary blobs. IndexedDB and WebSQL are asynchronous, fast, and support large data sets, but their APIs aren’t very straightforward. Even still, neither IndexedDB nor WebSQL have support from all of the major browser vendors and that doesn’t seem like something that will change in the near future.

If you need to write a web app with offline support and don’t know where to start, then this is the article for you. If you’ve ever tried to start working with offline support but it made your head spin, this article is for you too. Mozilla has made a library called localForage that makes storing data offline in any browser a much easier task.

around is an HTML5 Foursquare client that I wrote that helped me work through some of the pain points of offline storage. We’re still going to walk through how to use localForage, but there’s some source for those of you that like learn by perusing code.

localForage is a JavaScript library that uses the very simple localStorage API. localStorage gives you, essentially, the features of get, set, remove, clear, and length, but adds:

  • an asynchronous API with callbacks
  • IndexedDB, WebSQL, and localStorage drivers (managed automatically; the best driver is loaded for you)
  • Blob and arbitrary type support, so you can store images, files, etc.
  • support for ES6 Promises

The inclusion of IndexedDB and WebSQL support allows you to store more data for your web app than localStorage alone would allow. The non-blocking nature of their APIs makes your app faster by not hanging the main thread on get/set calls. Support for promises makes it a pleasure to write JavaScript without callback soup. Of course, if you’re a fan of callbacks, localForage supports those too.

Enough talk; show me how it works!

The traditional localStorage API, in many regards, is actually very nice; it’s simple to use, doesn’t enforce complex data structures, and requires zero boilerplate. If you had a configuration information in an app you wanted to save, all you need to write is:

// Our config values we want to store offline.
var config = {
    fullName: document.getElementById('name').getAttribute('value'),
    userId: document.getElementById('id').getAttribute('value')
};

// Let's save it for the next time we load the app.
localStorage.setItem('config', JSON.stringify(config));

// The next time we load the app, we can do:
var config = JSON.parse(localStorage.getItem('config'));

Note that we need to save values in localStorage as strings, so we convert to/from JSON when interacting with it.

This appears delightfully straightforward, but you’ll immediately notice a few issues with localStorage:

  1. It’s synchronous. We wait until the data has been read from the disk and parsed, regardless of how large it might be. This slows down our app’s responsiveness. This is especially bad on mobile devices; the main thread is halted until the data is fetched, making your app seem slow and even unresponsive.

  2. It only supports strings. Notice how we had to use JSON.parse and JSON.stringify? That’s because localStorage only supports values that are JavaScript strings. No numbers, booleans, Blobs, etc. This makes storing numbers or arrays annoying, but effectively makes storing Blobs impossible (or at least VERY annoying and slow).

A better way with localForage

localForage gets past both these problems by using asynchronous APIs but with localStorage’s API. Compare using IndexedDB to localForage for the same bit of data:

IndexedDB Code

// IndexedDB.
var db;
var dbName = "dataspace";

var users = [ {id: 1, fullName: 'Matt'}, {id: 2, fullName: 'Bob'} ];

var request = indexedDB.open(dbName, 2);

request.onerror = function(event) {
    // Handle errors.
};
request.onupgradeneeded = function(event) {
    db = event.target.result;

    var objectStore = db.createObjectStore("users", { keyPath: "id" });

    objectStore.createIndex("fullName", "fullName", { unique: false });

    objectStore.transaction.oncomplete = function(event) {
        var userObjectStore = db.transaction("users", "readwrite").objectStore("users");
    }
};

// Once the database is created, let's add our user to it...

var transaction = db.transaction(["users"], "readwrite");

// Do something when all the data is added to the database.
transaction.oncomplete = function(event) {
    console.log("All done!");
};

transaction.onerror = function(event) {
    // Don't forget to handle errors!
};

var objectStore = transaction.objectStore("users");

for (var i in users) {
    var request = objectStore.add(users[i]);
    request.onsuccess = function(event) {
        // Contains our user info.
        console.log(event.target.result);
    };
}

WebSQL wouldn’t be quite as verbose, but it would still require a fair bit of boilerplate. With localForage, you get to write this:

localForage Code

// Save our users.
var users = [ {id: 1, fullName: 'Matt'}, {id: 2, fullName: 'Bob'} ];
localForage.setItem('users', users, function(result) {
    console.log(result);
});

That was a bit less work.

Data other than strings

Let’s say you want to download a user’s profile picture for your app and cache it for offline use. It’s easy to save binary data with localForage:

// We'll download the user's photo with AJAX.
var request = new XMLHttpRequest();

// Let's get the first user's photo.
request.open('GET', "/users/1/profile_picture.jpg", true);
request.responseType = 'arraybuffer';

// When the AJAX state changes, save the photo locally.
request.addEventListener('readystatechange', function() {
    if (request.readyState === 4) { // readyState DONE
        // We store the binary data as-is; this wouldn't work with localStorage.
        localForage.setItem('user_1_photo', request.response, function() {
            // Photo has been saved, do whatever happens next!
        });
    }
});

request.send()

Next time we can get the photo out of localForage with just three lines of code:

localForage.getItem('user_1_photo', function(photo) {
    // Create a data URI or something to put the photo in an img tag or similar.
    console.log(photo);
});

Callbacks and promises

If you don’t like using callbacks in your code, you can use ES6 Promises instead of the callback argument in localForage. Let’s get that photo from the last example, but use promises instead of a callback:

localForage.getItem('user_1_photo').then(function(photo) {
    // Create a data URI or something to put the photo in an  tag or similar.
    console.log(photo);
});

Admittedly, that’s a bit of a contrived example, but around has some real-world code if you’re interested in seeing the library in everyday usage.

Cross-browser support

localForage supports all modern browsers. IndexedDB is available in all modern browsers aside from Safari (IE 10+, IE Mobile 10+, Firefox 10+, Firefox for Android 25+, Chrome 23+, Chrome for Android 32+, and Opera 15+). Meanwhile, the stock Android Browser (2.1+) and Safari use WebSQL.

In the worst case, localForage will fall back to localStorage, so you can at least store basic data offline (though not blobs and much slower). It at least takes care of automatically converting your data to/from JSON strings, which is how localStorage needs data to be stored.

Learn more about localForage on GitHub, and please file issues if you’d like to see the library do more!

About Matthew Riley MacPherson

Matthew Riley MacPherson (aka tofumatt) is a Rubyist living in a Pythonista's world. He's from Canada, so you'll find lots of odd spelling (like "colour" or "labour") in his writing. He has a serious penchant for pretty code, excellent coffee, and very fast motorcycles. Check out his code on GitHub or talk to him about motorcycles on Twitter.

More articles by Matthew Riley MacPherson…

About Robert Nyman [Editor emeritus]

Technical Evangelist & Editor of Mozilla Hacks. Gives talks & blogs about HTML5, JavaScript & the Open Web. Robert is a strong believer in HTML5 and the Open Web and has been working since 1999 with Front End development for the web - in Sweden and in New York City. He regularly also blogs at http://robertnyman.com and loves to travel and meet people.

More articles by Robert Nyman [Editor emeritus]…

About Angelina Fabbro

I'm a developer from Vancouver, BC Canada working at Mozilla as a Technical Evangelist and developer advocate for Firefox OS. I love JavaScript, web components, Node.js, mobile app development, and this cool place I hang out a lot called the world wide web. Oh, and let's not forget Firefox OS. In my spare time I take singing lessons, play Magic: The Gathering, teach people to program, and collaborate with scientists for better programmer-scientist engagement.

More articles by Angelina Fabbro…


33 comments

  1. Lucas Holmquist

    This looks really cool,
    We, AeroGear, also have something similar called datamanager in our JS library here: https://github.com/aerogear/aerogear-js

    February 12th, 2014 at 07:02

  2. Joe Larson

    Nice! Is it too late to bake in namespacing? https://github.com/joelarson4/LSNS

    February 12th, 2014 at 07:34

    1. tofumatt

      The library is still what I’d call “in-flux”, in that the features I’d want it to ship with for a stable, 1.0 aren’t all there yet. It handles these simple cases all fine, but if you’d like to see further features implemented file an issue on GitHub and we can talk about it.

      February 12th, 2014 at 09:17

  3. Patrik

    How about permanent/persistent storage? I.e. storage that doesn’t get cleared if the browser is closed or the computer turned off.

    February 12th, 2014 at 08:13

    1. tofumatt

      All of the storage libraries used are persistent; for instance: MDN mentions explicitly that localStorage is persistent.

      February 12th, 2014 at 09:22

  4. Felipe N. Moura

    Really cool!
    Helps a lot and make it simpler to use and even implement fallbacks!

    Cross domain is always a problem…
    Any ideias on maybe using some kind of proxy?

    February 12th, 2014 at 08:20

  5. Matěj Cepl

    High-fidelity is not the best example of anything, I would say … https://travis-ci.org/mcepl/high-fidelity :( And yes, I would LOVE to have a working podcatcher for my Peak. So far, https://github.com/colinfrei/Podcast/ is the best we have.

    February 12th, 2014 at 09:06

    1. tofumatt

      The Podcasts’ app’s failing tests are less a sign of it being a bad example and more a sign of me writing tests in a strange way that don’t run well on Travis yet (running those tests locally actually works, for what it’s worth).

      In addition, high-fidelity’s code inspired localForage, but isn’t using the library directly.

      February 12th, 2014 at 09:18

      1. Matěj Cepl

        It is not only about Travis-CI. When I run High-fidelity on my Peak (with 1.1hd … anything more recent won’t have phone connection; https://bugzilla.mozilla.org/show_bug.cgi?id=924999) I get just a black screen without anything more. So, unfortunately, as Podcast is bad for me it is the best podcatcher available (or bashpodder on computer and rsync via USB cabel).

        February 16th, 2014 at 13:20

  6. Stu

    I wonder if a simple filesystem type API would work – these wouldn’t be files on the client itself, but inside some sort of virtual space, per app ?

    [Possibly these would live in an image file with a filesystem].

    February 12th, 2014 at 09:20

    1. tofumatt

      There is a File System API in the works, but it’s not nearly as widely-available and covers a more niche use case. When it’s better standardized it will make for a better choice for storing files than localForage, for sure.

      One of the aims of this library is intense cross-browser support, as it’s one of the things I felt missing in a good, offline storage API.

      February 12th, 2014 at 09:26

  7. Mindaugas J.

    Nice solution. We recently needed exactly that for our extension. localStorage API is dead simple and we already had async wrappers on it but the problem was storage limits. https://github.com/jensarps/IDBWrapper was the closest I could find in simplicity that uses IDB.

    An IDB polyfill was needed for Safari, obviously.

    I supposed it’s now time to try localForage instead :)

    February 12th, 2014 at 09:23

    1. tofumatt

      The lack of Safari support always bothered me, and the IndexedDB polyfills I tried would be CSP nightmares at best. Let me know how things go, and please don’t be shy about filing issues!

      February 12th, 2014 at 09:28

      1. Mindaugas J.

        So straight from the bat there are evident problems with localForage. It’s all about namespacing. Firstly, you don’t allow for custom DB names. I see an issue in github has already been filed. Secondly, even in the same app, it’s customary to namespace key names with localStorage, say ‘myapp.foo.bar.baz’. Since iterating over all keys asynchronously is a pain (found another issue filed), it would be nice to at least have an ability to clear all keys in any namespace, e. g. myapp.foo.bar.*, myapp.foo.* or the whole myapp.*. Sure there’s a clear() method but it’s rather useless as it can only dump the whole database which also brings us back to issue number 1.

        This is how easily I implemented it with IDBWrapper: http://pastebin.com/VBhRQDir

        February 27th, 2014 at 09:23

  8. Fawad Hassan

    Isn’t this similar to lawnchair?
    http://brian.io/lawnchair/

    February 12th, 2014 at 11:43

    1. tofumatt

      It looks similar in some ways, yes. Seems like it’s more of a modular system, but it looks nice.

      February 12th, 2014 at 12:22

  9. James

    I’m a little confused. You list the supported browsers as: “IE 10+, IE Mobile 10+, Firefox 10+, Firefox for Android 25+, Chrome 23+, Chrome for Android 32+, and Opera 15+.”

    But then immediately under that you wrote: “In the worst case, localForage will fall back to localStorage” and according to http://caniuse.com/#feat=namevalue-storage localStorage is supported by IE 8+, Firefox 3.5+, Chrome 4+, Opera 10.5+, and Safari 4+.

    So which is correct?

    February 12th, 2014 at 12:06

    1. tofumatt

      Sorry, I should clarify: asynchronous storage is supported in those browsers. But you’re right: any browser that supports localStorage is supported.

      February 12th, 2014 at 12:14

  10. Brock

    This is exactly how I always wished localStorage worked. great job.

    February 12th, 2014 at 16:42

  11. Ido Green

    Great job!
    I would love to see it covering the File system API as well and use it only in cases that the browser support it.
    Thank you for the hard work!

    February 14th, 2014 at 03:38

    1. tofumatt

      This is probably something I’ll implement eventually, as storing things like images or MP3s would likely be much slicker with APIs better suited to the task.

      February 14th, 2014 at 11:23

  12. Sumeet

    Great Job!
    I am working on a proposal and the RFP needs are to have a web application with offline capabilities. The Offline capabilities are required for an extensive data entry process where there are about 5 Chapters, tons of Grids and hundreds of Questions. This data entry process needs to be implemented through a web interface ensuring that the users from the countries who have unreliable internet connectivity can work offline for most part of the data entry work (may be for days and weeks) and once they are ready with the data entered, then the application should be able to sync it back to the main database….

    Do you think HTML5 with localForage is mature enough to handle such complex and high volume offline data??

    thanks and regards
    Sumeet

    February 14th, 2014 at 05:11

    1. tofumatt

      localForage is pretty stable and has lots of tests, so I’d think you’d be fine. Of course, open source software like this doesn’t really come with a warranty, but I think you’d be okay.

      February 14th, 2014 at 11:22

  13. Glintch

    Great! Exactly what I was looking for. But is the storage size still limited to 5MB when localStorage is used?

    February 14th, 2014 at 06:00

    1. tofumatt

      It is, though there’s sadly not much we can do about that. Fortunately, most browsers will use a non-localStorage driver in the first place, so you’re mostly safe!

      February 14th, 2014 at 11:20

      1. Glintch

        Good to know. Thank you.

        February 14th, 2014 at 12:26

  14. Sam

    It seems that localStorage was never designed/implemented properly in the first place since it is slow (especially on [Android] tablets) and blocking. The API however is cleaner than IndexedDBs, so they got that part right.

    February 14th, 2014 at 10:34

  15. Agustin Lopez

    One question… I just downloaded this and I was trying it out but I noticed safari pops up a message asking for allowing the website to use 10MB.

    This happens for the default DB_SIZE used in this library…

    var DB_SIZE=5*1024*1024;

    but it does not happen if I set it manually to a lower size…

    var DB_SIZE=4.5*1024*1024;

    Is this correct? I don’t want my users to see a popup asking for that and I guess 5MB is probably the limit that Safari uses to ask for permission or not.

    February 16th, 2014 at 09:04

    1. tofumatt

      Looks like 5MB is the limit, yes, and just under is fine. I’ll push a change today that fixes this error.

      February 16th, 2014 at 10:01

  16. Agustin Lopez

    One more question…

    How do you delete an item from the storage? Do we have to set it as null or do we have a way like…

    localforage.deleteItem or something?

    Thanks in advance.

    February 16th, 2014 at 13:44

  17. Simon

    GO !! Matt, Im looking at building the first of what i hope to be many offline apps for mobile using the browser rather than native and had just discovered the issues around mobile Safari vs the rest of the browser world :) so anything like this to level the browser playing field is fantastic !!!!!

    February 17th, 2014 at 02:21

  18. Andrea Giammarchi

    Thanks for creating exactly what I’ve proposed in June 2012 and published in github since then: https://github.com/WebReflection/db#asyncstorage–a-developer-friendly-asynchronous-storage I am sure now that Mozilla talked about it, developers will use this more.

    I also wonder if you are also planning to make the API conflicts prone as my old solution does, so that a `storage.clear()` performed by library-A won’t destroy potentially 50 MB of data from library-B

    In my case, I’ve used a storage name, as internal namespace for DB operations, so that creation of the namespaced DB will be asynchronous (or in your case promise based) too.

    Best Regards

    February 18th, 2014 at 14:23

  19. Dheeraj

    Is there an easy way to plugin third-party drivers? Lawnchair (http://brian.io/lawnchair/) does allow adapters. This is required when using PhoneGap plugins in hybrid mobile apps.

    February 19th, 2014 at 23:18

Comments are closed for this article.