End Point

News

Welcome to End Point's blog

Ongoing observations by End Point people.

Locally served YUI3

For the vast majority of sites serving static JS and CSS files such as those required by YUI, or jQuery, etc. from a CDN makes great sense, improved performance through caching and geography, reduced load, improved uptime, leveraging some large corporations' resources, etc. Unfortunately as soon as you hit an SSL secured page you can't use a CDN's resources securely, a common thing with e-commerce sites and administration panels. In the YUI case that means doing at least a little bit of extra configuration and maintenance of an installed library base (an all too common task in typical server side development that's becoming more common as libraries are maintained for client side usage as well). Toss in "combo loading" and all of a sudden it feels like the software development cycle can't just be discarded cause you are working on the web, maybe this is really what they meant by web 2.0. But I digress...

Since I'm familiar with and working on YUI3, here is how we have it working for non-CDN based serving for an administration panel developed for a newer generation of an Interchange site. Our custom application uses YUI3 (core), modules from the YUI3 Gallery, and a few modules that are pulled in using the YUI 2in3 version compatibility layer. Now down to business...

Each of the libraries is provided by the YUI team through github.com. Since our project has git under it (you are using git, right?) we can set up submodules to help us with tracking new versions as they are released (or really be on the bleeding edge ahead of releases). To start we choose a location to serve the files, this is fairly arbitrary I happened to choose /vendor. Then set up new submodules for each of the libraries in use,

  • git submodule add http://github.com/yui/yui3 vendor/yui3
  • git submodule add http://github.com/yui/yui3-gallery vendor/yui3-gallery
  • git submodule add http://github.com/yui/2in3 vendor/yui-2in3

Follow that with the normal git submodule init, git submodule update cycle to pull down the HEAD version of each library. At this point the library files are available for local serving, and the current version is pegged against the superproject.

With the files locally stored they can be served locally. To do so we need to adjust the "seed" file for the YUI core, the loader, and any CSS files loaded on the page. For instance,

<!-- YUI Seed, Loader, and our primary source --> <script type="text/javascript" src="/combo?vendor/yui3/build/yui/yui.js&vendor/yui3/build/loader/loader.js"></script>

(For now ignore the "/combo" portion of this, more on that later.)

With the yui.js and loader.js files loading from a local resource YUI will automatically know where to look for other YUI3 core files. But for non-core files (such as gallery modules or 2in3 loaded modules) we need to provide more information to the loader through the YUI_config global variable.

The above configuration has parameters for both combo and non-combo loading. Lines 4-7 tell the loader where to find core modules explicitly, though the loader should default to these based on the location of the seed file. Starting on line 9 there are definitions for the additional groups of files that we now want locally served. The first group, lines 11-20, are needed to load our local gallery files. Lines 25-44, as the comment above them indicates, are for loading specific CSS files that were not properly handled by the first group, though I've yet to track down why (more robust skin support may fix this issue). Lines 46-62 bring in YUI 2-in-3, note the specific minor version of 2 is pegged in line 47 and 50. With this configuration in place, and properly located files on the filesystem you should be able to use any core, gallery, or 2in3 modules loaded from your local system.

But what about "combo loading." "Combo loading" allows us to reduce the number of roundtrip requests made to pull down the dependencies by concatenating a set of files together (either JS or CSS but not together), when serving from the CDN you get this because Yahoo! has provided a combo loading service. You can see a PHP based loading service provided by the YUI team in the phploader project, https://github.com/yui/phploader/ . Because we are working with Interchange and aren't using PHP I've written a similar script to be used as plain ole' boring CGI, it can be found at github. In the example code this script would be setup to run at the root of our URL space, specifically as "/combo" and the script itself will need to know where to find the local root, "vendor", on the physical file system. (It is important to note that combo loading CSS files takes some special parsing to make images and other referenced files work correctly. Also note that my version does not work with includes in the CSS.)

Finally one of the biggest benefits to this set up is being able to test literally any commit in any of the modules while leaving all else the same. This can allow you to determine when a bug first entered the picture (see git bisect) or test against development branches ahead of releases. To test against the latest and greatest between releases, you simply have to go to one of the submodule paths, fetch any recent updates, then check out the master (or any) branch, merge from the origin branch, and test. If all your tests pass, then you git add the submodule path in the superproject, git commit, and know that your stack is now up to date. Hopefully some day the gallery modules themselves will be git submodules so that maintaining a local gallery will not involve pulling down all modules and an application can peg (or update) a specific module as needed by the application, essentially you'd have your own gallery.

3 comments:

Ryan Grove said...

Great post!

For folks running Node.js on the server instead of PHP or CGI, this Node.js combo handler might be useful. See the readme for instructions on using it with YUI 3.

Steve Maring said...

Great thoughts. I had similar thoughts about this time last year and created a few projects to do a few js related things:

1) perform Maven based artifact dependency loading of YUI js and css files, based on the YUI Loader dep tree, into your own project war via a war overlay.

2) have a locally run (i.e. non-CDN) js and css combo service, in the form of a servlet, that creates a single response for the js and css requests.

3) perform the same sort of js minification (min, raw, and debug) on your own js files, to be compatible with YUI Loader and the local combo service

Check them out at http://github.com/smaring

called:
yui-war-overlay
javascript-combo-service-maven-plugin
javascript-combo-service
javascript-builder-maven-plugin

Einar said...

Thanks for this post.

I'm wondering about this part:

"It is important to note that combo loading CSS files takes some special parsing to make images and other referenced files work correctly."

Do you have any tips on or rules for how this should be done? Struggling with this myself right now in a C# combo handler I have written for serving YUI locally.