Tag: HTML5

Lumbar: Modular Javascript Build

by on Jan.17, 2012, under Web Dev

Since joining @WalmartLab’s mobile web team close to a year ago we have been very busy rebuilding the mobile web platform using Backbone, Handlebars, Stylus and other awesome open source tools and frameworks.

Even though these tools allow for some pretty impressive feats of javascript, we kept hitting inefficiencies both in execution and developer time due to resource loading and management.

When dealing with handlebars for example, we consistently found that we were trying to fight against the system to manage the templates used by a view. We neither wanted to manually inline the template in the javascript as we all know that escaping a language within a language is just painful, ala Java running SQL, nor did we want to add the additional overhead of an additional request for deferred loading of templates or worse inlining all application templates within the html file.

Being both constrained by the mobile environment and needing to operate at the scale of Walmart it became apparent very quickly that we needed to have some sort of build-time utility to manage this so the developer can focus on creating the application logic, throw it into the build tool and out comes a nicely optimized module of code and styles.

Lumbar along with the associated Lumbar projects such as lumbar-loader and lumbar-long-expires are the fruits of these efforts.

Overview

The Lumbar suite focuses on the management of all resources required to generate complex javascript applications across many different platforms and environments. It’s primary goals are to easily package, optimize, link to resources such as javascript, html, templates, styles, images, and any other client-side resources that web application may require.

From it’s initial implementation, Lumbar has focused on making client performance the number one goal, to this end it allows for chunking resources into distinct code modules, easy minification, inlining, and cache management.

Configuration

Lumbar is configured through a simple JSON config file, defining modules which are simple, demand loaded, segments of the application.

A base module that loads the core application:
    "base": {
      "scripts": [
        {"src": "js/lib/zepto.js", "global": true},
        {"src": "js/lib/underscore.js", "global": true},
        {"src": "js/lib/backbone.js", "global": true},
        {"src": "js/lib/handlebars.js", "global": true},
        {"src": "js/lib/thorax.js", "global": true},
        {"src": "js/lib/script.js", "global": true},
        {"src": "js/lib/lumbar-loader.js", "platform": "web"},
        {"src": "js/lib/lumbar-loader-standard.js", "platform": "web"},
        {"src": "js/lib/lumbar-loader-backbone.js", "platform": "web"},
        "js/init.js",
        {"src": "js/bridge.js", "platforms": ["iphone", "ipad", "android"]},
        {"src": "js/bridge-android.js", "platform": "android"},
        {"src": "js/bridge-ios.js", "platforms": ["ipad","iphone"]},
        {"module-map": true}
      ],
      "styles": [
        "styles/base.styl",
        {"src": "styles/iphone.styl", "platform": "iphone"},
        {"src": "styles/android.styl", "platform": "android"},
        {"src": "styles/ipad.styl", "platform": "ipad"},
        {"src": "styles/web.styl", "platform": "web"}
      ],
      "static": [
        {"src": "static/#{platform}/index.html", "dest": "index.html"}
      ]
    },

A router-linked module. When using backbone/thorax integration, this module will automatically load when the # or #hello routes are navigated:
    "hello-world": {
      "routes": {
        "": "index",
        "hello": "index"
      },
      "scripts": [
        "js/views/hello-world",
        "js/routers/hello-world.js"
      ],
      "styles": [
        "styles/hello-world.styl"
      ]
    }

This config file also defines the platforms that lumbar will generate, allowing for customizing the modules for different environments ("platforms": [ "android", "iphone", "ipad", "web" ]). At @WalmartLabs we utilize this to serve customized experiences for the native clients utilizing the same codebase.

Walmart Checkout Designs

Single stylus codebase styled for iPhone, Android, and mobile web platforms.

Walmart Checkout - iPad

More dramatic customizations of the same codebase utilizing platform conditional code.

The final component of the configuration is the packages component, which allows for bundling specific modules into singular responses when a particular use case only uses a subset of modules, such as the native checkout implementations utilized by @WalmartLabs.

  "packages": {
    "web": {
      "platforms": [ "web" ],
      "combine": false
    },
    "native-hello-world": {
      "platforms": [ "android", "iphone", "ipad" ],
      "modules": [ "base", "hello-world" ],
      "combine": true
    }
  },
Package declarations defining normal loading for web and combined for native platforms.

On building this will generate the following structure:

    $ find . -type f
    ./android/index.html
    ./android/native-hello-world.css
    ./android/native-hello-world.js
    ./android/native-hello-world@1.5x.css
    ./ipad/index.html
    ./ipad/native-hello-world.css
    ./ipad/native-hello-world.js
    ./iphone/index.html
    ./iphone/native-hello-world.css
    ./iphone/native-hello-world.js
    ./iphone/native-hello-world@2x.css
    ./web/base.css
    ./web/base.js
    ./web/base@2x.css
    ./web/hello-world.css
    ./web/hello-world.js
    ./web/hello-world@2x.css
    ./web/index.html

Which can be deployed as static resources to any web server, served via a CDN, or distributed by many other means.

While not a core feature, the lumbar-long-expires plugin takes this a step further by allowing resources to include expires tokens in their names automatically. When enabled the above may generate content like the following, allowing for the application resources to be served with extended Expires headers.

    $ find . -type f
    ./android/cb188f8/native-hello-world.css
    ./android/cb188f8/native-hello-world.js
    ./android/cb188f8/native-hello-world@1.5x.css
    ./android/index.html
    ./ipad/cb188f8/native-hello-world.css
    ./ipad/cb188f8/native-hello-world.js
    ./ipad/index.html
    ./iphone/cb188f8/native-hello-world.css
    ./iphone/cb188f8/native-hello-world.js
    ./iphone/cb188f8/native-hello-world@2x.css
    ./iphone/index.html
    ./web/cb188f8/base.css
    ./web/cb188f8/base.js
    ./web/cb188f8/base@2x.css
    ./web/cb188f8/hello-world.css
    ./web/cb188f8/hello-world.js
    ./web/cb188f8/hello-world@2x.css
    ./web/index.html

The @WalmartLabs mobile team believes strongly in open source software which is why in addition to open sourcing Lumbar we are also open sourcing our Thorax framework. If you are a backbone user looking to ease common tasks such as data binding and linking data operations to a particular route I urge you to take a look. It even integrates with Lumbar!

Comments Off on Lumbar: Modular Javascript Build :, , , , more...

border-image-generator: Local File Access

by on May.31, 2010, under border-image-generator, Web Dev

This article covers the techniques used to implement the local file access feature that is included, along with tiling support, alternate styling, and parameter optimization, in the most recent push to border-image-generator.

One of the biggest short comings that I felt like border-image-generator was that all files had to be posted on a HTTP server due to restrictions on the access to the file:// protocol. In my personal workflow this was somewhat of a pain, so I set out to find a way around this issue.

During the design of this feature, I quickly decided that all data must remain on the client as I did not want to upgrade my hosting service to include more bandwidth, felt like responsiveness could be a concern, and do not want to tangle with the potentially messy privacy issues that could arrise from storing user generated images on my server.

While this sounded like a daunting task, it turns out that this was possible, in sufficiently advanced browsers, using a variety of HTML5 techniques, including Data URIs, the File API and the Web Storage API.

Display

The problem of displaying the image was the easiest to solve as all of the targeted browsers support data URIs as part of the border-image property. This allowed the application to generate URLs like the following

    border-width: 46px;
    border-image: url("data:image/png;base64,iVBORw0KGgoAAA....") 46 stretch;

And operate as expected. Data URIs have been covered fairly extensively elsewhere, so I will leave this to the reader to investigate.

Reading

With the actual display issue solvled, there was still the non-trivial issue of actually loading the content. My initial investigations involved Flash, which provides FileReference.load API which allowing for this functionality, but under the version I was testing on this API is only usable if invoked in response to user input. Being a HTML guy and lacking functional knowledge of the Flash development process I quickly ruled this technique out.

Continuing my research I came across an article on MDC that covered this exact use case, using the draft File API specification. This worked like a charm, even exposing the ability to read the image contents directly into a data URI.

The API is very clean for use cases such as this:
function loadFile(file) {
    var reader = new FileReader();
    reader.onload = function(event) {
         updateImageURI(file.name, reader.result);
     };
     reader.readAsDataURL(file);
}

Where the file variable above is a File object retrieved from a

<input type="file">
or the dataTransfer object passed to the drop html event.

        $("body").bind("dragenter dragover", function(event) {
            // We have to cancel these events or we will not recieve the drop event
            event.preventDefault();
            event.stopPropagation();
        });
        $("body").bind("drop", function(event) {
            event.preventDefault();
            event.stopPropagation();
            var dataTransfer = event.originalEvent.dataTransfer,
                file = dataTransfer.files[0];

            loadFile(file);
        });
        $("#localImage").bind("change", function(event) {
            var file = this.files[0];

            loadFile(file);
        });

This unfortunately is not without it’s issues. The File API is very much bleeding edge at this point and support is somewhat limited. As of this writing Firefox is the only browser which features this in production (Version 3.6). Support landed in the WebKit trunk literally earlier this month and can be used in their nightlies, but so far has not made it into any production releases.

The site is currently designed to progressively enhance as it detects support for the File API, so no need to worry about being left out of the site as a whole if you are not on one of these browsers yet.

History

After loading the content using the File API, the original implementation utilized the same #hash storage method for the data URIs, but this proved to be problematic as these strings can become quite large and interacting with these URLs was unwieldily. Needing another data store and being forced to maintain a cache of all local images due to the security model employed by the File API, we were left the options of storing all data in Javascript space or using the new Web Storage APIs implemented as part of HTML5.

Examining both options it seemed the the best course was to utilize the sessionStorage object when available and fail over to the javascript data model when not.

    // Check for browser support of session storage and that it is accessible
    // This may be inaccessible under certain contexts such as file://
    function supportsSessionStorage() {
        try {
            return !!window.sessionStorage;
        } catch (err) {
            return false;
        }
    }
    var localDataBinding = (function() {
        if (supportsSessionStorage()) {
            // If they support FileReader they really should support storage... but who knows (With the exception of file://)
            return {
                storeImage: function(name, data) {
                    var entryId = (parseInt(sessionStorage["imageList-count"])||0)+1;
                    sessionStorage["imageList-count"] = entryId;
                    sessionStorage["imageList-src-" + entryId] = data;
                    sessionStorage["imageList-display-" + entryId] = name;
                    return entryId;
                },
                getImage: function(entryId) {
                    return { src: sessionStorage["imageList-src-" + entryId], displayName: sessionStorage["imageList-display-" + entryId] };
                }
            };
        } else {
            // Fail over to plain js structures, meaing that refresh, etc will cause failures.
            var cache = [];
            return {
                storeImage: function(name, data) {
                    cache.push({ src: data, displayName: name });
                    return cache.length-1;
                },
                getImage: function(entryId) {
                    return cache[entryId];
                }
            };
        }
    })();

Working with this API it seems as though it is still somewhat rough. The vendor documentation all state that supported datatypes are only string values (Mozilla, WebKit), but the spec implies that this will change be final release and allow for storage of a much broader set of datatypes.

A consequence of this design is that boomark and URL sharing for local files is not a possibility. For users who need to share or store application states the image in question will need to be stored on a HTTP server and accessed by this route.

These changes as well as some incremental changes have been pushed live on border-image.com and are available in the github repository.

1 Comment :, , more...

Visit our friends!

A few highly recommended friends...