Introducing Web Activities

One of the more powerful things lately for apps on various mobile phones have been intents. Register your app for handling certain types of actions, or specify in your app what kind of support you are looking for, for the thing you are trying to do.

This is especially important in the case of Firefox OS. No matter how good your web app is to begin with, what really takes it to the next level is interaction with other apps and activities on the device.

This is where Web Activities come into place.

It is basically one of the new WebAPIs we’ve been working on to make the web even more powerful as a platform, and the idea is to have a simple API to both specify intent with your activity, but also declare that your app will be able to handle actions from other apps.

If you haven’t looked into web apps yet, by the way, the best thing is probably to read Getting started with making apps. In their simplest form, a web app is just HTML5, CSS and JavaScript, with an app manifest added.

Getting started with Web Activities

There are a few ways you can work with Web Activities:

  • Call an activity and get presented with apps that can handle that
  • Register your activity support for your web app in the manifest file
  • Register activity support on-the-fly
  • Attach a handler to your app for when that activity occurs

Calling an activity

Let’s say you have a button in your app, and you want to be able to get a picture – either from the Gallery, Camera or any other app in Firefox OS that supports that activity. You can then call the pick activity, like this:

var pick = new MozActivity({
   name: "pick",
   data: {
       type: ["image/png", "image/jpg", "image/jpeg"]
   
}
});

In this example, we specify the name pick and the data to be PNG or JPEG images. You will then be presented with a menu of available activities in Firefox OS:

As a user, you choose the app you want to pick an image from – or take a picture with the Camera – and once you’ve done so, the result will be posted back to the requesting app (Note: it is chosen from within the app handling the activity what, and if, something will be returned).

Handling the response

For most WebAPIs, including Web Activities, you will have onsuccess and onerror event handlers. In the case of an image/file you will get a blob back. You can then represent that returned image visually directly in your app:

pick.onsuccess = function () {

    // Create image and set the returned blob as the src
    var img = document.createElement("img");
    img.src = window.URL.createObjectURL(this.result.blob);

    // Present that image in your app
    var imagePresenter = document.querySelector("#image-presenter");
    imagePresenter.appendChild(img);
};

pick.onerror = function () {

    // If an error occurred or the user canceled the activity
    alert("Can't view the image!");
};

Register your app for an activity

As mentioned above, you can also set your app as a handler for certain activities. There are two ways to do that:

Through the manifest file – declaration registration

{
    "name": "My App",
    "description": "Doing stuff",
    "activities": {
       "view": {
            "filters": {
                "type": "url",
                "url": {
                    "required": true,
                    "regexp":"/^https?:/"
                }
            }
        }
    }
}

Register an activity handler – dynamic registration

var register = navigator.mozRegisterActivityHandler({
    name: "view",
    disposition: "inline",
    filters: {
        type: "image/png"
    }

});



register.onerror = function () {
    console.log("Failed to register activity");

}

and then handle the activity:

navigator.mozSetMessageHandler("activity", function (a) {

    var img = getImageObject();

    img.src = a.source.url;

    /*
      Call a.postResult() or a.postError() if 

      the activity should return a value

    */
});

Available activities

The available activities to choose from at this time are:

  • configure
  • costcontrol/balance
  • costcontrol/data_usage
  • costcontrol/telephony
  • dial
  • new (e.g. type: “websms/sms”, “webcontacts/contact”)
  • open
  • pick (e.g. type: “image/png”)
  • record
  • save-bookmark
  • share
  • test
  • view

A few examples:

Dial

var call = new MozActivity({
    name: "dial",
    data: {
        number: "+46777888999"
    }
});

New SMS

var sms = new MozActivity({
    name: "new",
    data: {
        type: "websms/sms",
        number: "+46777888999"
    }
});

New Contact

var newContact = new MozActivity({
    name: "new",
    data: {
        type: "webcontacts/contact",
        params: { // Will possibly move to be direct properties under "data"
            giveName: "Robert",
            familyName: "Nyman",
            tel: "+44789",
            email: "robert@mozilla.com",
            address: "Sweden",
            note: "This is a note",
            company: "Mozilla"
        }
    }
});

View URL

var openURL = new MozActivity({
    name: "view",
    data: {
        type: "url", // Possibly text/html in future versions
        url: "http://robertnyman.com"
    }
});

Save bookmark

var savingBookmark = new MozActivity({
    name: "save-bookmark",
    data: {
        type: "url",
        url: "http://robertnyman.com",
        name: "Robert's talk",
        icon: "http://robertnyman.com/favicon.png"
    
}
});

Try it out now!

You can try this out right now, by putting together a web app and calling web activities. You can then test it in the Firefox OS Simulator!

Work in progress

Web Activities are a work in progress, and the activity names, data types etc are subject to change. However, currently most of the above works right now (with the exception for mozRegisterActivityHandler and mozSetMessageHandler, that haven’t been implemented yet).

I hope you share my excitement with all the possibilities Web Activities are offering us, and can probably think of a number of use cases, making your app much more powerful through interaction with other apps!

About Robert Nyman [Editor emeritus]

Technical Evangelist & Editor of Mozilla Hacks. Gives talks & blogs about HTML5, JavaScript & the Open Web. Robert is a strong believer in HTML5 and the Open Web and has been working since 1999 with Front End development for the web - in Sweden and in New York City. He regularly also blogs at http://robertnyman.com and loves to travel and meet people.

More articles by Robert Nyman [Editor emeritus]…


33 comments

  1. Steve

    What controls are in place to avoid exploitive usage of premium SMS and dialing?

    Has this necessitated an further security in the structure of the browser itself to avoid malware penetration of these tools?

    While I am excited to see HTML5 treated as a native app, it is also cause for concern.

    Imagine a desktop browser allowing a webpage to launch live application from a user’s start menu (or dock, or any of the available Linux equivalents, etc).

    Certainly, as a developer, I would love to see native integration everywhere… and I *do* think intents at the best way to go about handling it.

    As a user, however, I am still concerned.

    January 24th, 2013 at 06:31

    1. Robert Nyman [Editor]

      A valid question. In general, let’s say someone calls the web activity for dialing a number. If more than one app supports that activity, the user is presented with a menu where they get to choose which app should open it.

      If only one app supports it, it sends the request to that app. However, no call is being placed/initiated (same for SMS), so the only action being triggered is showing the phone (or SMS) screen with the passed number pre-filled out for the user.

      January 24th, 2013 at 12:59

      1. Steve

        Understood.

        As we all know, the web has forever been rife with malware, even before the advent of mobile.

        With mobile apps, the intent typically opens up a permissions window which must be accepted before installation… web malware, however, has become far more stealthy, injecting malicious scripts through Flash, Java, and even images.

        My concern is really that (especially on Android, where Java is king) that as long as a path between web script and hardware exists, there is potential for exploit.

        Android has already had numerous cases where installed apps were able to send stealth SMS to premium numbers that the handset owner was never notified of.

        Part of this may be due to the fact that, given elevated permissions, an app may embed itself extremely closely to the firmware, even intercepting notifications before they are propagated throughout the system for example.

        Looking at the plethora of web exploits, and the variety of mobile exploits… the combination between the two must be considered carefully.

        Would it be possible, for example, to create an SMS app in browser?

        If, for example, the app was able to get permission from a user to run Java, would it be potentially be possible to run as a relatively native Android app if they found a hack to elevate permissions? Or would it still be under the traditional permissions for an applet?

        Especially as we open the web to hardware access… Cameras, SMS, dialers, memory access, etc.

        Besides the typical authorization request popup, are there any additional layers of security between a site and the device? Not sure if one would really be able to implement a sandbox in a case like this, for example.

        Love the plan, would love to implement it, but concerned because I know how many smart, tricky people there really are out there.

        January 24th, 2013 at 15:24

        1. Jonas Sicking

          You seem to be making a lot of assumptions about what it means to be an application.

          Just because we support creating applications using HTML, doesn’t mean that we support those applications being able to do anything that an application on Windows or OSX can do.

          We have very consciously limited the initial release of our WebApps platform to a feature set which we could make safe. Surely there are bugs just like in any software, so I won’t claim that Firefox OS is impenetrable.

          So for example we do not permit any applications other than the built-in ones to use the Telephony or SMS APIs for now. This might change in the future, but only once we can make that safe.

          Likewise, all applications are very strongly separated and WebActivities is one of the few ways they can interact. So web applications can’t embed each other or read/write each others data for example. This separation is much stronger than in for example Android.

          While I agree that as long as there is a path from Javascript to hardware, there is an increased risk of exploits. But that holds just as true as long as there is a path between Java and the hardware, or C++ and the hardware.

          January 24th, 2013 at 21:18

          1. Steve

            Absolutely true. Glad you guys are thinking of it.

            As for the assumptions, I disagree.

            It may not be happening now, and as you state, you will work towards it once it can become safer.

            But it *is* coming.

            With the advent of HTML5, we finally have a single, secure cross platform language that is powerful and flexible enough to actually create real applications… especially as Harmony is released and NodeJS evolves in the back end.

            We are already seeing implementations of geolocation, camera access, et al… and with the prominence of mobile, I do not believe it will be long before pressure mounts to bake other capabilities into the framework.

            More than likely, someone (Chrome, Explorer, Safari, whoever) use some semi-proprietary means of access, and that will then be used by developers.

            By that stage of the game, it is too late to plan for a unified API..rather than break existing code, we will ‘pave the cow paths’… even if they are poorly planned out. We have seen it before, and I am sure we will see it again.

            While that is a pain in the butt as a developer for most things, this is different.

            Our last greatest cross platform language, Java, has fallen due to security flaws… Flash as well.

            There is no other place in the developing web that I see the same potential for catastrophe.

            HTML5 is the only remaining cornerstone of the web.

            While it *is* true that any language which has direct access to the hardware is susceptible to exploitation… in no other case do we find this unique confluence between native connectivity and potential for both loss of intellectual property and identity, but also direct monetary loss, especially on mobile.

            To be honest, I would love to see full access… HTML5 running as native code across all available platforms without modification or restriction… I would love to see real disk access, I would love to see the ability to launch external apps, I would love to see the ability to run telephony directly through HTML5.

            I think this would have the potential to spark another revolution in the evolving web. I believe that this *will* happen, and probably more quickly than we are prepared for.

            I am simply hoping to do whatever I can to ensure that it is well planned out, and that everyone is on board with the same, secure, unified API when it finally comes along…. even if it means pestering you guys with my concerns.

            The average developer has no other direct access to the powers on high that shape the web.

            I trust the Mozilla team… they are pretty solid Open Source. I trust that they will do their utmost to ensure that they are compliant with whatever the emerging standard becomes, and to help ensure that it is secure. I especially appreciate that they have shown the penchant for staying on top of technology without rushing it to market… that must be a difficult balance to maintain.

            I am less impressed with the Chrome team, as Dart and other technologies have shown that they are willing to run in their own direction to be first to market… similarly I am wary of Apple and Microsoft.

            I am concerned of the long term security effects if one of these teams rushes the API into existence, resulting in a cobbled together API.

            I sincerely hope that for once, each of these teams can put their individual interests aside to agree on a real Open Source solution, with an eye for the eventual expansion into the platform that I see coming… and are able to do so in a unified, secure way.

            All I can do is voice my concerns, and hope they reach the right ears.

            Thank you for your responses… it has assured me that the Mozilla team is on top of the issues that I have brought up. I hope the other teams take it as seriously.

            January 24th, 2013 at 22:42

          2. Robert Nyman [Editor]

            Thanks for your thoughts!
            I believe with HTML5 – as opposed to Java and Flash – that since it is completely open and not run/owned by one company, there’s a much bigger chance of working together to find the flaws, make sure it’s secure etc.

            We definitely take it seriously, and we’ll do whatever we can to make sure it’s secure.

            January 25th, 2013 at 02:06

  2. Joshua Ols

    Awesome, really glad you guys decided to support intents. I use them all the time on my Android phone and tablet, so I see them as a killer feature. ;)

    January 24th, 2013 at 07:36

    1. Robert Nyman [Editor]

      Thanks! I compltely agree, I think the intents approach plays a major role!

      January 24th, 2013 at 12:55

  3. Fabrice

    Note that the dynamic registration is not yet implemented.

    January 24th, 2013 at 10:30

    1. Robert Nyman [Editor]

      Yep, that’s why I wanted to touch on that in the last paragraph. That part isn’t in there yet, but hopefully soon!

      January 24th, 2013 at 12:55

  4. Mikko Ohtamaa

    Love this. We are on stop closer useable HTML5 applications now.

    January 24th, 2013 at 14:00

    1. Robert Nyman [Editor]

      Thanks! And I agree, this is a very important part.

      January 24th, 2013 at 14:08

  5. Ron

    I hope you guys are at least talking to the Chrome guys about this.

    Why do your “Web Activities” look so similar to Web Intents (http://webintents.org/), just with an infuriatingly different name?

    Smells like not-invented-here-syndrome. :S

    I think I’ll just wait for this to be standardised across at least 2 browsers before I bother.

    January 24th, 2013 at 14:45

    1. Robert Nyman [Editor]

      We talk to them all the time. :-)
      Web Intents and Web Activities have been discussed for a long time, and Web Activities are a counter-proposal to Web Intents, with a limited scope. Additionally, since November last year, Google has put Web Intents on hold.

      January 24th, 2013 at 15:00

    2. Jonas Sicking

      The reason we didn’t go with WebIntents is that they are still in the research stage. WebIntents has a bigger feature set and tries to solve more complex problems. This is great, but also means that it takes a longer time to come up with a working solution.

      In fact, I believe that newer versions of Chrome has deprecated some of the WebIntents support since the initial approach didn’t work as well as they had hoped.

      Since we wanted to put something into Firefox OS we decided to go with something simpler. Hence WebActivities.

      But we have definitely talked to the Google guys, and will continue to do so. Last time we talked we were able to bring our proposals much closer to each other so I have no doubt that we’ll be able to arrive at a common proposal.

      January 24th, 2013 at 21:11

  6. Pic

    Unclosed bracket in that one :p
    ‘Through the manifest file – declaration registration’

    Any chance on seeing that in the browser? Or only in the apps downloaded from the Marketplace?

    January 24th, 2013 at 14:45

    1. Robert Nyman [Editor]

      Thanks, updated that!
      At this time, it works in apps but also in the regular web browser on Firefox OS. For the future, long-term, I’d love to see this in Firefox on Android and any other mobile web browser.

      January 24th, 2013 at 15:04

  7. Steve

    Seeing this on mobile is nice… but expected.

    More to my interest is seeing whether or not we will ever see this for the desktop.

    I wouldn’t necessarily have to interface with other apps immediately, but I know that Mozilla, Chrome, and the rest of the crew are working on implementing access to the client side hardware: webcams, microphones, hard drives, etc.

    Thinking intents should be pretty simple way of wrapping all of these permissions intensive features all together under one API.

    Any chance of us seeing this on the desktop in the near recent future?

    January 24th, 2013 at 15:04

    1. Robert Nyman [Editor]

      I can only agree. Like you say, this could provide a simple way to access webcams and much more. To my knowledge, though, I don’t think we will see it in the near future on desktop.

      For accessing webcam etc in general, though, we will have WebRTC – where we’re making progress – but personally I think Web Activities would be a very easy option as well.

      January 24th, 2013 at 15:08

      1. Steve

        Would have to agree.

        Would love to see everything wrapped under a single permissions system…unfortunately, too many web technologies have been clobbered together piecemeal, leaving multiple API’s where a single one would make sense (apple-touch-icon on Android??).

        Hopefully, Mozilla will lead the pack, here, and it will all fall together seamlessly.

        January 24th, 2013 at 15:29

        1. Robert Nyman [Editor]

          I think both applies, though. In some cases, a simple API would’ve made more sense, but at the same time, the scope for support, updates etc becomes much bigger. Then smaller, more dedicated APIs are great.

          January 25th, 2013 at 02:08

  8. Joshua Ols

    Had a sort of related question. Will Firefox OS support sharing/copying rich text? That is one really annoying limitation on android that keeps me going back to my PC whenever I have to do work between email, google docs, browser, etc.

    January 25th, 2013 at 04:24

    1. Robert Nyman [Editor]

      Great question. I don’t know about copying, but there is a “share” activity where sharing text in a richer format might be an option (although right now, I believe it only supports sharing files/blobs).

      January 25th, 2013 at 04:38

  9. Axel Hecht

    I was wondering, can one use the background page of an app to handle activities, to implement a web api that doesn’t pop up a new app?

    Also, are the activity names restricted to the current set, or can any two web apps just make up a new activity and talk to each other?

    January 25th, 2013 at 05:45

    1. Robert Nyman [Editor]

      We’re not really using background pages like, for instance, Google Chrome extensions. But something that will be implemented is support for the disposition property, with the values “inline” or “window”. Basically, “window” is going to the other app, while “inline” is displaying it in the current requesting app.

      About names and restrictions, that’s a great question!
      Right now I believe it’s restricted to the current set, but it could be an interesting way to extend that!

      January 25th, 2013 at 05:50

  10. Jeff Walden

    I see a "regexp":"/^https?:/" in there. Presumably both forward-slashes are required. More consistent with HTML’s pattern attribute on input elements would be for the string to match the Pattern production from ECMAScript. Any chance that could be changed?

    January 27th, 2013 at 03:36

    1. Jeff Walden

      And, looking more closely at the HTML language, perhaps the pattern should imply a ^(?: at the start and a )$ at the end, so that the string matches against the entire input string, rather than against any subcomponent of it (which seems faintly bug-prone, e.g. if someone mistakenly used "https://" as their input string).

      January 27th, 2013 at 03:39

      1. Robert Nyman [Editor]

        Thanks for the feedback! I’ll talk with the team about it.

        January 28th, 2013 at 08:45

    2. Jonas Sicking

      We wanted to support passing regexp flags, so you can write something like:

      regexp: “/^https?:/i”

      I do agree that implying full-string-match would be nice though. Something like:

      regexp: “https?:”
      “regexp-flags”: “i”

      might have been a better solution. Don’t think we can change this for v1, but maybe for v1.x

      January 29th, 2013 at 02:45

  11. Sebastian Ortiz

    Hi,

    I am looking for a list of apps with registered activities. By now i’m trying to share an URL, but looks like the only registered app is bluetooth wich doesn’t run on the FirefoxOS Simulator.

    So, what are my options to test my app with the share activity?

    January 29th, 2013 at 18:36

    1. Robert Nyman [Editor]

      Apps that have registered activities is sort of a moving target, and yes, you’re right about Share at this moment.
      Basically, your options right now are:

      – Go through the Gaia (user interface for Firefox OS) code and see what you can find
      – Wait a while till more apps support it
      – Build a second app where you register for the Share activity yourself

      January 30th, 2013 at 02:12

  12. Robin Berjon

    It’s great to finally see some movement around this. Any chance that you could boost up standardisation on this feature? It’s been lingering for way too long.

    Also, why not on the desktop? This is a killer feature that really opens up the Web. There seems to be no good reason to delay it!

    February 5th, 2013 at 04:30

    1. Robert Nyman [Editor]

      Thanks Robin!
      I’d love to have them standardized soon. In all honesty, I believe that and desktop support are next to come, but just that there are tons of things happening right now and we haven’t gotten to it properly, yet.

      February 5th, 2013 at 07:17

Comments are closed for this article.