End Point

News

Welcome to End Point's blog

Ongoing observations by End Point people.

Defense in Depth

“Defense in depth” is a way to build security systems so that when one layer of defense fails, there is another to take its place. If breaking in is hard enough for an attacker, it’s likely that they’ll abandon their assault, deciding it’s not worth the effort. Making the various layers different types of defense also makes it harder to get in, so that an attacker is less likely to get through all the layers. It can also keep one mistake from causing total security failure.

For example, if you have an office with one door and leave it unlocked by accident, then anyone can just walk in. However, if you have an office building with a main door, then a lobby and hallways, and then additional inner locked doors, then the chance of accidentally leaving both unlocked is small. If you accidentally leave the inner door open or unlocked, someone can only get into the office if they first get through the outer door.

Another example of defense in depth is making and maintaining offsite computer backups. The chance of an office being destroyed and all your data with it is low, but not zero. If you maintain offsite backups, then your losses in a catastrophe are reduced.

Another way to decrease risk of data loss is to keep multiple backups in widely separated locations, held by different people or companies. This ensures you still have access to data even when one account is locked out, a storage facility goes out of business, or an account is hijacked by an attacker.

What about the rogue insider threat? You can protect each backup with different credentials, and ensure few or no employees have access to more than one backup.

A security scheme is only as strong as its weakest link, so your various defenses need to be on par with each other. If your office is locked up nice and tight, but anyone can get into your computer over the network, then all your office security isn’t going to do much good and vice versa.

When considering security, it’s important to think about the cost compared to the potential loss. Picture a house. With a normal house, it’s not too hard to get in; all you have to do is break a window. Considering that, is it really worth spending lots of money on fancy locks for the doors? You can add bars to the windows and install multiple fancy door locks, but is it worth the inconvenience and ugliness? Perhaps moving to a new neighborhood is a better solution.

Recognizing that no defense is foolproof, and that a combination of cheap defenses may be more effective than one expensive defense, consider the concept of defense in depth whenever looking at protection, whether it be of an office building, website, or castle.

These articles go into more detail on the topic:

Test Web Sites with Internet Explorer for Free

Browser Testing

While many Web Developers prefer to build sites and web applications with browsers like Chrome or Firefox it's important for us to keep an eye on the browser market share for all web users. Internet Explorer (IE) still owns a large piece of this pie and because of this, it is important to test sites and applications to ensure they work properly when viewed in IE. This poses a potential problem for developers who do not use Windows.

Although I use OS X on my desktop, I have Windows virtual machines with IE 6,7,8,9 and 10 installed. I also have a Linux virtual machine running Ubuntu so I can check out Chrome/Chromium and Firefox on that platform. In the past I had tried solutions like MultipleIEs but wasn't satisfied with them. In my experience I've found that the best way to see what users are seeing is to have a virtual machine running the same software they are.

I did some IE8 testing for a colleague a short time ago and suggested she should give VirtualBox a shot. Her response was "You should write a blog post about that!". So here we are.

Free Tools

VirtualBox is a free virtualization application similar to Parallels or VMWare. These applications aim to solve this problem by allowing us to run different "guest" operating systems on the "host" operating system of our choice. For example, I run several versions of IE and Windows on my computer which is running OS X.

Microsoft have begun in recent years to release Virtual PC (VPC) images for free to enable Web Developers to test their sites and applications with various versions of Internet Explorer. The most recent images were released on July 30, 2012 and you can check them out here. Although these images are created to run on Microsoft's Virtual PC application, with a little bit of effort we can also use them on Linux, OS X or Windows via VirtualBox.

For our purposes, we will be building a Windows 7 virtual machine running Internet Explorer 8. However, there are several images available with various Windows OS and Internet Explorer version combinations:

  • Windows 7 / IE8
  • Windows 7 / IE9
  • Windows Vista / IE7
  • Windows XP / IE6
There will likely be some differences creating virtual machines for the various images but the process will be similar to what we'll document here for Windows 7 / IE.

Linux and OS X Users

In researching this article I came across a very helpful project on GitHub. The ievms project by xdissent is a bash script that automates this process. I was able to run the script and run my newly created Windows 7 / IE virtual machine shortly afterward.

Requirements

To start we'll need to download and install VirtualBox. Be sure you have the most recent version available as it gets updated quite often. We'll also need to install the VirtualBox Extension Pack. Both are available on the VirtualBox Downloads page.

Because the VPC images are quite large (~2.6GB in our case) they have been split up into several files. For the Windows 7 / IE8 image we'll download the following files:

Once the files have been extracted we have created our virtual machine the disk will take up close to 10 GB. Make sure you have enough disk space available. I run my virtual machines on an external USB hard drive which works well if your local hard drive is starved for space.

Extract the VPC Files

Once you have all of the the VPC image files downloaded you'll need to extract them:
Windows users: double-click the Windows_7_IE8.part01.exe file
Linux / OS X users: You will need to install unrar. I did so on OS X with Homebrew and there are packages available for Ubuntu as well.

Once you have unrar installed, issue the following command in the directory containing all of the VPC image files:

`unrar e -y Windows_7_IE8.part01.exe`

This will extract the files and combine them into a single .vhd file. The -y flag tells unrar to say "yes" to the EULA from Microsoft so be aware of that.

Create a new Machine

Once the .vhd file is ready we can create a new machine in VirtualBox. Start up VirtualBox and click the "New" button:
Vbox new

Click "Continue" to proceed.

VM Name and OS Type

We'll enter a name (IE8 / Windows 7) and select "Microsoft" and "Windows 7" in the Operating System and Version drop-downs:
Vbox name and type
Click "Continue" to proceed.

Memory

Specify how much memory (RAM) you'd like to allocate to the virtual machine. This really depends on how much you have available but it is best to go with the minimum at the very least: Vbox mem config
Click "Continue" to proceed.

Disk

Select the "Do not add a virtual hard drive" option.: Vbox do not add hdd 1
The reason we skip this for now is because we want to add the hard disk as an IDE drive. For some reason VirtualBox adds the disk as a SATA drive which causes the blue screen of death (BSOD) to appear when the virtual machine is booted up.
Click "Continue" to proceed and click "Continue" again when presented with the warning about not attaching a hard disk.

Review Settings and Create the VM

At this point you can review the settings you've made and click "Create" to complete the process:
Vbox review and create

Configure Storage Settings

Highlight your newly created virtual machine and click the "Settings" button. From there, click the "Storage" tab and add a hard disk to the IDE controller: Vbox storage add hd

Absolute Pointing Device

In the Settings, click on the "System" tab and uncheck "Enable absolute pointing device". I had to do this in order to to get the mouse working properly.

Start it Up

We're almost there: next we need to start up the VM by clicking the "Start" button. You'll see Administrator and IEUser accounts and the password for both is "Password1". Log in as Administrator and check it out! It's important to note when the virtual machine has the keyboard and mouse focus. When you click inside the virtual machine window, it will capture the input from your mouse and keyboard. To return focus back to your computer you will need to press the "host key" sequence on your computer:
Vbox host key
The "host-key" sequence can be configured in the VirtualBox preferences. On my computer, the host-key sequence is the left CTRL key (displayed as Left ^). When the arrow icon is illuminated, the guest computer – Windows 7 in this case, has the focus. When I press the left CTRL key on my computer the arrow icon goes tray to indicate focus has been returned to my computer.

Activation

As soon as you log in you will be presented with the Windows Activation wizard. Follow the steps to activate this copy of Windows. Once that is complete you will be prompted to restart your computer. Go ahead and do that.
Vbox activate now
The screen resolution in the virtual machine will be quite low at first — I believe 800x600 is the default. Don't worry about this as it will be addressed in the next step.

Guest Additions

Log in as Administrator (Password1) and Install the VirtualBox Guest Additions. To do this, navigate to Devices in the VirtualBox menu on your computer (the "host" machine"). Choose the menu option to "Install Guest Additions". Each time you are prompted with the "Would you like to install..." dialog, choose "Install":
Vbox install guest additions

This will install some drivers optimized by VirtualBox which allow you to change the screen resolution, improve the interaction between your host computer and the Windows guest operating system. Once the install process is complete you will be prompted to restart your virtual machine once more.

Finished Product

With Windows activated and the Guest Additions installed you should be able to log in to your very own Windows 7 / IE8 testing machine!
Vbox ie8 mr

This free copy of Windows will operate for 30 days and the trial period can be extended twice by running the following at the command prompt:

slmgr –rearm

Check out the documentation at Microsoft for more details.

Piggybak: An Update on End Point's Ruby on Rails Ecommerce Engine

With the recent release of one of our client sites running on Piggybak, Piggybak saw quite a few iterations, both for bug fixes and new feature development. Here are a few updates to Piggybak since its announcement earlier this year.

Admin: Continues to Leverage RailsAdmin

Piggybak continues to leverage RailsAdmin. RailsAdmin is a customizable admin interface that automagically hooks into your application models. In the case of the recent project completion, the admin was customized to add new features and customize the appearance, which can be done in RailsAdmin with ease.

As much as I enjoy working with RailsAdmin, I think it would be great in the future to expand the admin support to include other popular Rails admin tools such as ActiveAdmin, which has also gained popularity in the Rails space.

Refund Adjustments

When Piggybak first came out, there was little in the way to allow orders to be monetarily adjusted in the admin after an order was placed. One requirement that came out of client-driven development was the need for recording refund adjustments. A new model for "Adjustments" is now included in Piggybak. An arbitrary adjustment can be entered in the admin, which is tied to a specific user, an amount, and a recorded note. This functionality allows for site administrators to record adjustments given against orders.

At the moment, the creation of an adjustment is not tied to a payment gateway to refund against the original transaction, because ActiveMerchant payment gateways have varied refund support.

AJAX Queueing for Shipping Requests on One-Page Checkout

Discussed in this article, AJAX queuing was added for shipping method generation on the one-page AJAX-driven checkout.

Order Notes

Another feature that originated from client needs was the need to record order changes over time. Now included in Piggybak are "Order Notes". Order notes are automatically created when attributes or nested attributes are changed on the order:

And arbitrary order notes can be added to record data not represented in attribute changes:

All order notes have a created_at attribute and belong to a user, which is tied to the administrator who made the change to the order.

Masked CC Number Storage

While we don't want to store unencrypted credit card numbers in the database per PCI compliance, Piggybak now includes storage of the masked credit card number to the payments table. This allows the site administrators to examine the credit card type and reference a particular card if needed. This was accomplished using the following method added to the String class:

class String
  def mask_cc_number
    masked = ''

    if self.gsub(/\D+/i, '').match(/^(\d\d)(.+)(\d\d\d\d)$/)
      masked = $1 + $2.length.times.inject('') { |s, i| "#{s}*" } + $3
    end

    masked
  end
end

Upgrade to Rails 3.2.8

The Piggybak demo has recently been updated to Rails 3.2.8. No changes were required, as this was considered a minor Rails update.

Conclusion

It's exciting to see the progress of Piggybak over the last several months, as well as to see Piggybak launch for a site with complex custom needs. I believe it accomplishes the vision I had in mind for it (summarized in this presentation), without requiring fighting against assumptions that traditional monolithic ecommerce platforms make. Granted, Piggybak still makes specific ecommerce assumptions, but they are limited to cart, checkout, and order functionality rather than systematic application-level behavior.

The future likely holds incremental improvements to Piggybak, but there are no plans to change Engine-like structure or principles of Piggybak.

Rails 4 Highlights

I watched this recent video What to Expect in Rails 4.0 presented by Prem Sichanugrist to the Boston Ruby Group. Here are a few high-level topics he covered in the talk:

  • StrongParameters: replaces attr_accessor, attr_protected, moves param filtering concern to the controller rather than the model. Moving param filtering concern to the controller allows you to more easily modify user attribute change-ability in controllers (e.g. customer-facing vs admin).
  • ActiveSupport::Queue: Discussed at RailsConf, add queueing support to Rails, e.g.:
    # Add to queue
    Rails.queue.push UserRegistrationMailerJob(@user.id)
    
    # Control queue configuration (asynchronous, synchronous or resque, e.g.
    config.queue = [:asynchronous, :synchronous, :resque]
    
  • Cache Digests: Rails 4.0 introduces cache key generation based on an item and its dependencies, so nested cache elements properly expire when an item is updated.
  • PATCH verb support: Support of HTTP PATCH method (_method equals "patch"), which will map to your update action is introduced in Rails 4.0.
  • Routing Concern: Rails 4.0 introduces some methods to help clean up your duplicate routes.
  • Improvements to ActiveRecord::Relation
    • Relation.all: returns ActiveRecord::relation object. User.all => User.to_a
    • Relation.none: Returns ActiveRecord::NullRelation, still chainable
    • Relation.___!: mutates current relation, e.g. @users.where!, @users.include!
  • Deprecations
    • AR::Base.scoped
    • Dynamic Finder Methods: e.g. find_all_by_*
    • Hash-based Finders: e.g. User.find(:first)
    • Eager Evaluated Scope: scope will require a lambda
    • ActiveRecord::SessionStore
    • ActiveResource
    • Rails::Plugin
  • New Deprecation Policy: Many of the above deprecations will still work in Rails 4.0 and included as gem dependencies, but will be removed in the jump to Rails 4.1. This means that the upgrade to Rails 4.1 may be more painful than the upgrade to Rails 4.0.

Check out the video or read the official current Rails 4.0 release notes here. Also, check out this post I came across about PostgreSQL array support in Rails 4, which may be pretty interesting to our PostgreSQL experts.

Insidious List Context

Recently, I fell into a deep pit. Not literally, but a deep pit of Perl debugging. As a result, I'm here to warn you and yours about "Insidious List Context(TM)".

(Note: this is a fairly elementary discussion, for people early in their Perl wizardry training.)

Perl has two contexts for evaluating expressions: list and scalar. (All who know this stuff cold can skip down a ways.) "Scalar" context is what non-Perl languages just call "normal reality", but Perl likes to do things ... differently ... so we have more than one context.

In scalar context, a scalar is a scalar is a scalar, but a list becomes a scalar that represents the number of items in the list. Thus,

@x = (1, 1, 1);  # @x is a list of three 1s
# vs.
$x = (1, 1, 1);  # $x is "3", the list size

In list context, a list of things is still a list of things. That's pretty simple, but when you are expecting a scalar and you get a list, your world can get pretty confused.

Okay, now the know-it-alls have rejoined us. I had a Perl hashref being initialized with code something like this:

my $hr = {
  KEY1 => $value1,
  KEY2 => $value2,
  KEY_TROUBLE => (defined($foo) ? mysub($foo) : 1),
  KEY3 => $value3,
};

So here is the issue: if mysub() returns a list, then the hashref will get extra data. Remember, Perl n00bs, "=>" is not a magical operator, it's just a "fat comma". So a construction like this:

1 => (2, 3, 4)
is really the same as:
1, 2, 3, 4

Here's a complete example to illustrate just what size and shape hole I fell into:

use strict;
use Data::Dumper;

my($value1,$value2,$value3,$foo) = qw(value1 value2 value3 foo);

my $hr = {
  KEY1 => $value1,
  KEY2 => $value2,
  KEY_TROUBLE => (defined($foo) ? mysub($foo) : 1),
  KEY3 => $value3,
};

print Data::Dumper->Dumper($hr);

sub mysub {
  return qw(junk extrajunk);
}
This outputs:
$VAR1 = 'Data::Dumper';
$VAR2 = {
          'extrajunk' => 'KEY3',
          'KEY2' => 'value2',
          'KEY1' => 'value1',
          'value3' => undef,
          'KEY_TROUBLE' => 'junk'
        };

Now, the actual subroutine involved in my little adventure was even more insidious: it returned a list context because it was evaluating a regular expression, in a list context. Its actual source:

sub is_yes {
   return( defined($_[0]) && ($_[0] =~ /^[yYtT1]/));
}

So watch those expression-evaluation contexts; they can turn fairly harmless expressions into code-busters.

AJAX Queuing in Piggybak

AJAX is inherently asynchronous; for the most part, this works fine in web development, but sometimes it can cause problem if you have multiple related AJAX calls that are asynchronous to eachother, such as the use case described in this article.

In Piggybak, a Ruby on Rails open source shopping cart module developed and maintained by End Point, the one page checkout uses AJAX to generate shipping options. Whenever state and zip options change, the shipping address information is sent via AJAX and valid shipping methods are returned and rendered in a select dropdown.


Event listeners on the state and zip code inputs trigger to generate shipping options via AJAX.

While working on development for a client using Piggybak, I came across a scenario where AJAX asynchronous-ity was problematic. Here's how the problematic behavior looked on a timeline, picking up as the user enters their shipping address:

  • 0 seconds: User changes state, triggers AJAX shipping lookup with state value, but no zip code entered (Let's refer to this as AJAX REQUEST 1).
  • 1 second: User changes zip code, triggers AJAX shipping lookup with state and zip value present (Let's refer to this as AJAX REQUEST 2).
  • 2 seconds: AJAX REQUEST 2 returns valid shipping options.
  • 3 seconds: AJAX REQUEST 1 returns invalid shipping options, because no zip code provided in this data set, and overwrites shipping options returned by AJAX REQUEST 2.

The result is that the user has finished entering a valid shipping address, but sees that no valid shipping options can be chosen:

To address this issue, I researched AJAX queuing, with the requirement that AJAX requests should be performed synchronously and existing AJAX requests could be aborted if needed. After experimenting with a few different plugins, I found the most success with the jQuery-ajaxq plugin. It's simple to use:

  • To append a new AJAX call on to a queue, you call $.ajaxq(queue_name, options), where options includes the standard AJAX arguments.
  • To cancel or abort AJAX calls currently running in a queue, you call $.ajaxq(queue_name).

The event listener on state and zip changes now looks like the code shown below, in simplified form. The piggybak.update_shipping_options cancels AJAX requests on the shipping_options queue and then adds a new request to the queue which will execute immediately. This does not affect other asynchronous AJAX requests on the page.

var piggybak = {
    update_shipping_options: function() {
        $.ajaxq("shipping_options");
        $.ajaxq("shipping_options", {
            url: ...,
            cached: false,
            data: ...,
            dataType: "JSON",
            beforeSend: function() {
                 ...
            },
            success: function(data) {
                ...
            }
        });
    }
}

Here's how this looks on a timeline:

  • 0 seconds: User changes state, triggers AJAX shipping lookup with state value, but no zip code entered (Let's refer to this as AJAX REQUEST 1).
  • 1 second: User changes zip code, triggers AJAX shipping lookup with state and zip value present (Let's refer to this as AJAX REQUEST 2). This signals to abort all current AJAX requeusts on the shipping_options queue.
  • 2 seconds: AJAX REQUEST 2 returns valid shipping options.

This was an interesting technical problem to solve, but the needs were not surprising. The jQuery-ajaxq plugin offers a simple, elegant solution for handling AJAX queing in jQuery.

Company Presentation: Ecommerce as an Engine




Today, I gave the presentation to my coworkers entitled "Puppies & Ecommerce as an Engine". The presentation is strongly coupled with my work on Piggybak, and includes a discussion of traditional ecommerce platforms versus a lightweight ecommerce approach through modularity (Rails Engine). It also provides some code examples as how this does work in Piggybak.

Below are a few more related articles to my work on Piggybak. Check them out!

Three Things: Times Two

It’s been a while since I’ve written up a “Three Things” article where I share a few featured web development tidbits picked up recently. So I made this a double episode!

1. event.stopPropagation() and event.stopImmediatePropagation()

I recently came across these two methods in jQuery, described here and here. Both of these methods [prevent the event from bubbling up the DOM tree, preventing any parent handlers from being notified of the event]. In my web application, my $('html') element had a listener on it, but I added specific listeners to children elements that when clicked on calls event.stopPropagation to cancel the event on the $('html') element. See the code below for a simplified example:

jQuery(function() {
    jQuery('html').click(function() {
        jQuery.hideSomething();
    });
    jQuery('.popup').click(function(event) {
        event.stopPropagation();
    });
})

2. alias_attribute

The alias method in Rails is one that I use frequently. But I recently came across the alias_attribute method as well. This might make the most sense to use when using shared views for multiple models with varying attributes.

3. Excel behavior in jQuery

Recently, there was a bit of discussion about jQuery tools that emulate spreadsheet behavior. A couple of the tools that came up were Handsontable and DataTables. They are worth checking out if you are looking to add Excel-like behavior to your web application!

4. Rack::SslEnforcer

I recently had a need on a Rails application to force some pages as secure, but have other pages be forced as non-secure. Instead of the common practice of adding controller before filters to force a redirect, this was included via the Gemfile, bundle install, and then configured in config/application.rb. Here’s an example configuration setup that I’m using:

config.middleware.use Rack::SslEnforcer,
        :only => [/\/checkout\/$/, /\/users$/, ‘/admin’],
        :strict => true

The one interesting caveat I found in working with this is that you absolutely must have all CSS and JavaScript assets precompiled in order for them to be served via SSL. The JS and CSS assets would not be forced to SSL, so they must exist in the Rails public directory via the precompiling, or this gem will redirect https requests on a secure page to http, resulting in the browser reporting that some non-secure elements are being served from a secure page.

5. Setting a viewport

I was recently troubleshooting a CSS issue for a client who was examining their website on an iPad. The iPad was setting the width of the viewport to a value that resulted in mis-alignment of floating elements. After a bit of research, I found that setting the viewport to the desired static width of my page fixed this issue. Here is a nice overview of the viewport attribute.

6. Line Specific Substitution in vi

Several of my coworkers are vi experts. When I’m in a shared screen with them, I pick up small tips for improving efficiency. One that I picked up on a few months ago and practiced enough to remember was line-specific substitution, e.g. 4,10s/moo/meow/g will substitute all occurrences of “moo” to “meow” in lines 4 through 10. I use this technique frequently.

Cannot parse Cookie header in Ruby on Rails

Yesterday I resolved a client emergency for a Ruby on Rails site that continues to leave me scratching my head, even with follow-up investigation. In short, the emergency came up after an email marketing campaign was sent out in the morning, and resulted in server (HTTP 500 Status Code) errors for every customer that clicked on the email links. Despite the fact that Rails exception emails are sent to the client and me, the errors were never reaching the exception email code, so I was unaware of the emergency until the client contacted me.

Upon jumping on the server, I saw this in the production log repeatedly:

ArgumentError (cannot parse Cookie header: invalid %-encoding (...)):
ArgumentError (cannot parse Cookie header: invalid %-encoding (...)):
ArgumentError (cannot parse Cookie header: invalid %-encoding (...)):

The URLs that the production log was complaining about had a bunch of Google Analytics tracking variables:

  • utmcmd=Email
  • utmcct=customeremail
  • utmccn=New Site Sale 70% off
  • etc.

After a user visits the site, these variables are typically stored as cookies for Google Analytics tracking. Upon initial investigation, the issue appeared to be triggered from any Google campaign variable that contained a '%' character.

After follow-up investigation today, the more complete story looks like this:

  1. Email blast sent
  2. User clicks on link in email. That link goes to the email marketing company first for tracking, then is redirected to the website.
  3. According to the email marketing campaign (after chatting with them today), Google Analytics tacks on their own tracking here, which is the source of the non-parseable URLs.
  4. Rack receives the request and tries to parse the query, utilizing the Ruby URI module:
    def self.decode_www_form_component(str, enc=Encoding::UTF_8)
        if TBLDECWWWCOMP_.empty?
          tbl = {} 
          256.times do |i|
            h, l = i>>4, i&15 
            tbl['%%%X%X' % [h, l]] = i.chr
            tbl['%%%x%X' % [h, l]] = i.chr
            tbl['%%%X%x' % [h, l]] = i.chr
            tbl['%%%x%x' % [h, l]] = i.chr
          end  
          tbl['+'] = ' '
          begin
            TBLDECWWWCOMP_.replace(tbl)
            TBLDECWWWCOMP_.freeze
          rescue
          end  
        end  
        raise ArgumentError, "invalid %-encoding (#{str})" unless /\A[^%]*(?:%\h\h[^%]*)*\z/ =~ str
        str.gsub(/\+|%\h\h/, TBLDECWWWCOMP_).force_encoding(enc)
      end 
    
  5. The argument error on line 18 trickles up the pipeline in rack, and is not handled elegantly, so a rack-originated server (HTTP 500 Status Code) error is triggered. Again, the '%' character in the URL appears to be the problem here likely based on the regexp match on line 18 — the error is not triggered when the Google variable does not contain a '%' character.
  6. Customer sees server error page and is unhappy :(

At the time of the emergency we tried solving the problem on multiple avenues:

  • Investigated removal of Google Analytics tracking URLs from email blast links, but this wouldn't help all the customers who already received the emails.
  • Remove CGI parameters or sanitize them via nginx.
  • Make web application changes to ignore or handle the ArgumentError.

Ultimately, I ended up added a begin/rescue statement to the rack code to skip escaping URLs where decode_www_form_component was raising an error:

def unescape(s, encoding = Encoding::UTF_8)
  begin
    URI.decode_www_form_component(s, encoding)
  rescue
    Rails.logger.warn "DECODING on #{s.inspect} with #{encoding.inspect} FAILING."
  end 
end 

While this is a reasonable fix, I'm still puzzled for a number of reasons:

  • The email marketing company (via chatting) claims that Google Analytics is tacking on the tracking variables, however, there is only one redirect from the email marketing company to the website. I don't understand the mechanism for which Google Analytics tracking variables are added to the URL, and if this process can be cleaned up to ensure proper URL encoding.
  • I'm not sure if the issue happens immediately upon a customer landing on the site, or after a cookie is stored.
  • At the moment, I'm not able to reproduce this issue in development mode, which makes it difficult to troubleshoot on my development instance.
  • When I use the URI module directly in a console, no ArgumentError is raised:
    >> URI.decode_www_form_component("url_with_google_campaign_variables")
    >> #happy dance
    

My best advice at this point is to tell the client not to use '%' character in the Google Campaign ID, but I'm still putting all the pieces together in the virtual code map in my head. I think a fix is more likely needed on the Ruby and rack side to handle URL parameters with the '%' character, and to elegantly handle situations where the URI.decode_www_form_components method dies.

Enforcing Transaction Compartments with Foreign Keys and SECURITY DEFINER

In support of End Point's evolving offering for multi-master database replication, from the precursor to Bucardo through several versions of Bucardo itself, our code solutions depended on the ability to suppress the actions of triggers and rules through direct manipulation of the pg_class table. Most PostgreSQL database developers are probably familiar with the construct we used from the DDL scripts generated by pg_dump at one time.

Disable triggers and rules on table "public"."foo":

UPDATE pg_class SET
        relhasrules = false,
        reltriggers = 0
    FROM pg_namespace
    WHERE pg_namespace.oid = pg_class.relnamespace
        AND pg_namespace.nspname = 'public'
        AND pg_class.relname = 'foo';

Re-enable all triggers and rules on "public"."foo" when finished with DML that must not fire triggers and rules:

UPDATE pg_class SET
        reltriggers = (
            SELECT COUNT(*) FROM pg_trigger
            WHERE pg_class.oid = pg_trigger.tgrelid
        ),
        relhasrules = (
            SELECT COUNT(*) > 0
            FROM pg_rules
            WHERE schemaname = 'public'
                AND tablename = 'foo'
        )
    FROM pg_namespace
    WHERE pg_namespace.oid = pg_class.relnamespace
        AND pg_namespace.nspname = 'public'
        AND pg_class.relname = 'foo';

In practice, the simple usage described for trigger and rule suppression worked reasonably well. However, it didn't always work well. In particular, there is a somewhat concerning state that exists between the two previously described events. The actions of disabling triggers/rules, then manipulating those relations affected, and then re-enforcing triggers/rules, must happen within the confines of a single transaction, and they must happen, period. The risk is that, at some point between the "open and shut" on pg_class, the transaction is committed and the "shut" never fires. If that happens, all database activity against the relations with disabled triggers and rules continues. I don't recall that we ever isolated the reasons why, on rare occasion, this happened; I only know that it did happen, and it was never welcome news.

In an effort to curb the worst aspect of this issue, I started with a simple question: how can I limit the transaction to a "safe compartment", thinking in terms of perl's Safe.pm? In this case, the "unsafe" action is "commit the transaction with triggers and rules disabled". But in reality, the unsafe list can be any conditions the developer needs to have exposed, but cannot make visible to the rest of the database.

An ancillary issue we faced, too, was the fact that any app code needing to suppress triggers and rules (beyond syncing, there were any number of DML requirements where it was undesirable for syncing to occur, and the pg_class manipulations were quite common) had to operate as the super user. While we had not had an incident where the postgres user for mundane operations had burned down the database, there was certainly concern about that potential.

The resolution I settled on was to construct a pair of functions that made use of the following features:

  • PostgreSQL's SECURITY DEFINER function attribute
  • Deferred foreign keys
  • The ON COMMIT DROP option for CREATE TEMP TABLE

The first function, safe_disable_trigrules(schema_name text, table_name text), is called after beginning a transaction and makes the necessary modifications to pg_class on behalf of schema_name.table_name. After the work within the transaction is finished, the second function--safe_reenable_trigrules(schema_name text, table_name text)--is called before issuing the commit. It, of course, puts pg_class back to the proper state.

Under the hood, the two functions create a dependency that only each other can satisfy when used as a non super user. Before safe_disable_trigrules() will manipulate pg_class, it creates a temp table with a self-referencing, deferred foreign key. Then, based on the schema and table args, it will insert a record for the relation defined by the args that violates the FK. Once the transaction's work is finished, but before committing, safe_reenable_trigrules() is called for every relation that safe_disable_trigrules() was called against and it will delete out the offending record for that relation alone. If the two functions are used properly, by the time of commit, the temp table is empty, thus having no foreign key violations, at which point the transaction can be safely committed. In the process of ensuring the temp table has no foreign key violations, pg_class has been fully restored to its pre-transaction state.

How each of the identified features is used:

  • Creating the functions with the SECURITY DEFINER attribute, we have now opened an access point for non-privileged users specifically for the purpose of the proscribed interaction with pg_class--and nothing more.
  • The temp table is created and owned by user postgres. There is no chance of the non-privileged user manipulating this table directly, accidentally or otherwise. Thus, the only positive escape for the transaction is through the use of the reenabling function.
  • Deferring the foreign key on the temp table allows us to ensure the transaction is in an invalid state at all times while pg_class is in its vulnerable condition without aborting the transaction.
  • ON COMMIT DROP allows the function to clean up after itself without having to make an explicit decision on the right time to drop the table. It allows a single temp table to be utilized per transaction, regardless of how may different relations will be passed through the trigger and rule disabling process.
  • Before the temp table actually drops, its deferred foreign keys are evaluated. If any rows are left in the table, it means for at least one relation we failed to call the reenable function and the entire transction is aborted rather than risk committing pg_class in the disabled state.

Converting to this system of pg_class manipulation completely eliminated the instances of finding pg_class in a committed state with triggers and rules disabled for various relations. It also allowed us to convert a number database-dependent scripts and applications from using the postgres user down to the appropriate application users.

CREATE FUNCTION safe_disable_trigrules (
        schema_name TEXT,
        table_name TEXT
    )
RETURNS void
LANGUAGE plpgsql
STRICT
SECURITY DEFINER
AS $$
DECLARE

    text_table_pk TEXT NOT NULL := '';
    text_fk_table TEXT NOT NULL := '';
    text_cur_min_msg TEXT;

BEGIN

    -- Stop any malicious shenanigans by user overloading
    -- relations or operators in a different schema and
    -- manipulating search_path to use them.

    PERFORM
        pg_catalog.set_config(
            'search_path',
            'pg_catalog, '
            OPERATOR(pg_catalog.||)
            (SELECT
                pg_catalog.current_setting(
                    'search_path'
                )
            ),
            TRUE
        );


    -- Shared PK for table to hold FK in violated state.
    -- This naming convention must not change without also
    -- reflecting the convention in safe_reenable_trigrules()
    -- so that both can immutably create the same name given
    -- the same arguments.

    text_table_pk :=
        schema_name || '_' ||
        table_name || '_' ||
        TO_CHAR(
            NOW(),
            'DHH24MISSMS'
        );

    -- Allowing for the same relation to have triggers and rules
    -- disabled and reenabled multiple times within the same
    -- transaction. On subsequent calls, the temp table will
    -- already exist.

    SELECT setting
    INTO text_cur_min_msg
    FROM pg_settings
    WHERE name = 'client_min_messages';

    UPDATE pg_settings
    SET setting = 'error'
    WHERE name = 'client_min_messages'
        AND text_cur_min_msg IS DISTINCT FROM 'error';

    -- Attempt to create the temp table. If first function call for
    -- transaction, it succeeds; otherwise, it fails silently unless
    -- error is something other than re-creating extant table.

    BEGIN
        -- Temp table for this transaction, with same shared
        -- convention as the PK above.
        text_fk_table :=
            'trigrules_' ||
            TO_CHAR(
                NOW(),
                'DHH24MISSMS'
            );

        -- Use ON COMMIT DROP so PG will garbage collect
        -- all such temp tables created within the transaction.

        EXECUTE
            'CREATE TEMP TABLE ' ||
                quote_ident(text_fk_table) || ' (
                id TEXT PRIMARY KEY NOT NULL,
                fk_id TEXT NOT NULL
                    CONSTRAINT "Must Call safe_reenable_trigrules() Before Commit"
                    REFERENCES ' ||
                    quote_ident(text_fk_table) || '
                    DEFERRABLE
                    INITIALLY DEFERRED
            )
            ON COMMIT DROP';

    EXCEPTION
        WHEN DUPLICATE_TABLE THEN
            -- Ignore
    END;

    UPDATE pg_settings
    SET setting = text_cur_min_msg
    WHERE name = 'client_min_messages'
        AND text_cur_min_msg IS DISTINCT FROM 'error';

    -- Insert new record that violates FK. Allowing for
    -- the function to be gracefully recalled on the same
    -- relation between calls to re-enable triggers and rules.

    EXECUTE '
        INSERT INTO ' ||
        quote_ident(text_fk_table) || '
        SELECT ' ||
            quote_literal(text_table_pk) ||
            ', ' ||
            quote_literal(text_table_pk || 'X') || '
        WHERE NOT EXISTS (
            SELECT 1
            FROM ' ||
            quote_ident(text_fk_table) || '
            WHERE id = ' ||
            quote_literal(text_table_pk) || '
        )';

    -- Disable all rules and triggers on target relation
    UPDATE pg_class SET
        relhasrules = false,
        reltriggers = 0
    FROM pg_namespace
    WHERE pg_namespace.oid = pg_class.relnamespace
        AND pg_namespace.nspname = schema_name
        AND pg_class.relname = table_name;

    -- Abort transaction if relation doesn't exist
    IF NOT FOUND THEN

        RAISE EXCEPTION
            'Table %.% does not exist',
            schema_name,
            table_name;

    END IF;

    -- reset search_path for users legitimately overloading
    -- operators or relations.

    PERFORM
        set_config(
            'search_path',
            (SELECT
                SUBSTRING(
                    current_setting('search_path')
                    FROM
                    '^pg_catalog, (.*)'
                )
            ),
            TRUE
        );
END;
$$
;


CREATE FUNCTION safe_reenable_trigrules (
        schema_name TEXT,
        table_name TEXT
    )
RETURNS void
LANGUAGE plpgsql
STRICT
SECURITY DEFINER
AS $$
DECLARE

    text_fk_table TEXT NOT NULL := '';
    text_table_pk TEXT NOT NULL := '';
    int_num_del INTEGER;

BEGIN

    -- Stop any malicious shenanigans by user overloading
    -- relations or operators in a different schema and
    -- manipulating search_path to use them.

    PERFORM
        pg_catalog.set_config(
            'search_path',
            'pg_catalog, '
            OPERATOR(pg_catalog.||)
            (SELECT
                pg_catalog.current_setting(
                    'search_path'
                )
            ),
            TRUE
        );

    -- Re-enable rules and triggers on target
    UPDATE pg_class SET
        reltriggers = (
            SELECT COUNT(*) FROM pg_trigger
            WHERE pg_class.oid = pg_trigger.tgrelid
        ),
        relhasrules = (
            SELECT COUNT(*) > 0
            FROM pg_rules
            WHERE schemaname = schema_name
                AND tablename = table_name
        )
    FROM pg_namespace
    WHERE pg_namespace.oid = pg_class.relnamespace
        AND pg_namespace.nspname = schema_name
        AND pg_class.relname = table_name;

    -- Shared PK for table to hold FK in violated state.
    -- This naming convention must not change without also
    -- reflecting the convention in safe_disable_trigrules()
    -- so that both can immutably create the same name given
    -- the same arguments.

    text_table_pk :=
        schema_name || '_' ||
        table_name || '_' ||
        TO_CHAR(
            NOW(),
            'DHH24MISSMS'
        );

    -- Temp table for this transaction, with same shared convention
    -- as the PK above.
    text_fk_table :=
        'trigrules_' ||
        TO_CHAR(
            NOW(),
            'DHH24MISSMS'
        );

    -- Remove pertinent row so FK is no longer in violated state
    EXECUTE
        'DELETE FROM ' ||
        quote_ident(text_fk_table) || '
        WHERE id = ' ||
        quote_literal(text_table_pk);

    GET DIAGNOSTICS int_num_del = ROW_COUNT;

    IF (int_num_del > 0) IS NOT TRUE THEN
        RAISE EXCEPTION
            'No entry for %.% set by safe_disable_trigrules()',
            schema_name,
            table_name;
    END IF;

    -- reset search_path for users legitimately overloading
    -- operators or relations.

    PERFORM
        set_config(
            'search_path',
            (SELECT
                SUBSTRING(
                    current_setting('search_path')
                    FROM
                    '^pg_catalog, (.*)'
                )
            ),
            TRUE
        );

END;
$$
;

Pl/Perl multiplicity issues with PostgreSQL - the Highlander restriction

I came across this error recently for a client using Postgres 8.4:

ERROR: cannot allocate multiple Perl 
interpreters on this platform

Most times when you see this error it indicates that someone was trying to use both a Pl/Perl function and a Pl/PerlU function on a server in which Perl's multiplicity flag is disabled. In such a case, only a single Perl interpreter can exist for each Postgres backend, and trying to create a new one (as happens when you execute two functions written in Pl/Perl and Pl/PerlU), the error above is thrown.

However, in this case it was not a combination of Pl/Perl and Pl/PerlU - I confirmed that only Pl/Perl was installed. The error was caused by a slightly less known limitation of a non-multiplicity Perl and Postgres. As the docs mention at the very bottom of the page, "...so any one session can only execute either PL/PerlU functions, or PL/Perl functions that are all called by the same SQL role". So we had two roles both trying to execute some Pl/Perl code in the same session. How is that possible - isn't each session tied to a single role at login? The answer is the SECURITY DEFINER flag for functions, which causes the function to run as if it was being invoked by the role that created the function, not the role that is executing it.

There is still a bit of a gotcha here, because Perl interpreters are created as needed, and thus the order of operations is very important. In other words, you may be able to run function foo() just fine, and run function bar() just fine, but you cannot run them together in the same session! This applies to both the Pl/Perl and Pl/PerlU limitation, as well as the Pl/Perl multiple user limitation.

While Postgres will validate functions as you create them, this is subject to the same in-session limitation. All of the below examples assume you have a non multiplicity-enabled Perl (see the perlguts manpage for gory details on what multiplicity means in Perl) . To see what state your Perl is, you need to determine if the 'usemultiplicity' option is enabled. The -V option to the perl executable tells it to output all of its configuration parameters. While the canonical way to check is to issue a perl -V:usemultiplicity, that's a hard string to remember, so I simply use grep:

$ perl -V | grep multi
 useithreads=define, usemultiplicity=define

The above indicates that Perl has been compiled with multiplicity and thus not subject to the Postgres limitations - you can mix and match Perl functions in your database with abandon. The only problem occurs if the output looks like this:

$ perl -V | grep multi
 useithreads=undef, usemultiplicity=undef

Technically, you can also prevent the issue by setting ithreads on, but there really is no reason to not just keep things simpler by setting the multiplicity on.

Watch what happens when we try to create two Perl functions using Postgres 9.2:

postgres=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# create language plperl;
CREATE LANGUAGE

test=# create language plperlu;
CREATE LANGUAGE

test=# create or replace function test_perlver()
test-# returns text
test-# language plperl
test-# AS $$ return "Running test_perlver on Perl $^V"; $$;
CREATE FUNCTION

test=# create or replace function test_perlverU()
test-# returns text
test-# language plperlU
test-# AS $$ return "Running test_perlverU on Perl $^V"; $$;
ERROR:  cannot allocate multiple Perl interpreters on this platform
CONTEXT:  compilation of PL/Perl function "test_perlveru"

What's going on here? We've already used a perl (Pl/Perl) in *this session*, so we cannot create another one, even if just to compile (but not execute) the function. However, if we start a new session, we can create our Pl/PerlU function!

test=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# create or replace function test_perlverU()
test-# returns text
test-# language plperlU
test-# AS $$ return "Running test_perlverU on Perl $^V"; $$;
CREATE FUNCTION

This Highlander restriction ("there can be only one!") applies to both creation and execution of functions. Notice that we have both the Pl/Perl and Pl/PerlU versions installed, but we can only use one in a particular session - and which one depends on which is called first!:

test=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# select test_perlver();
             test_perlver
--------------------------------------
 Running test_perlver on Perl v5.10.0

test=# select test_perlverU();
ERROR:  cannot allocate multiple Perl interpreters on this platform
CONTEXT:  compilation of PL/Perl function "test_perlveru"

test=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# select test_perlverU();
             test_perlveru
---------------------------------------
 Running test_perlverU on Perl v5.10.0

test=# select test_perlver();
ERROR:  cannot allocate multiple Perl interpreters on this platform
CONTEXT:  compilation of PL/Perl function "test_perlver"

As you can imagine, the nondeterministic nature of such functions can make discovery and debugging of this issue on production servers tricky. :) Here's the other variant we talked about, in which only the first of two functions - both of which are Pl/Perl - will run:

postgres=# create database test;
CREATE DATABASE

postgres=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# create language plperl;
CREATE LANGUAGE

test=# create or replace function foo()
test-# returns text
test-# language plperl
test-# security invoker
test-# AS $$ return "Running as security invoker"; $$;
CREATE FUNCTION

test=# create or replace function bar()
test-# returns text
test-# language plperl
test-# security definer
test-# AS $$ return "Running as security definer"; $$;
CREATE FUNCTION

Now let's run as the user who created the function - no problemo, because we are the same user that created the function:

test=# \c test postgres
You are now connected to database "test" as user "postgres".

test=# SELECT foo();
           foo
-----------------------------
 Running as security invoker
(1 row)

test=# SELECT bar();
           bar
-----------------------------
 Running as security definer
(1 row)

All is well. However, if we try it as a different user, the Highlander restriction creeps in:

test=# \c test greg
You are now connected to database "test" as user "greg".

test=# SELECT foo();
           foo
-----------------------------
 Running as security invoker
(1 row)

test=# SELECT bar();
ERROR:  cannot allocate multiple Perl interpreters on this platform
CONTEXT:  compilation of PL/Perl function "bar"

test=# \c test greg
You are now connected to database "test" as user "greg".

test=# SELECT bar();
           bar
-----------------------------
 Running as security definer
(1 row)

test=# SELECT foo();
ERROR:  cannot allocate multiple Perl interpreters on this platform
CONTEXT:  compilation of PL/Perl function "foo"

This one took me a while to figure out on a production system, as somewhere in a twisty maze of trigger functions there was one that was set as security definer. Normally, this was not a problem, as the user that created that function did much of the updates, but a different user invoked a non- security definer function and then the security definer function, causing the error at the top of this article to show up.

So what can one do to prevent this problem from occurring? Luckily, for most people this will not be a problem, as many (if not all) distros and operating systems have the multiplicity compile flag for Perl enabled. If you do have the restriction, one option is to simply be careful about the use of security definer functions. You could either declare everything as security definer, or perhaps make sure that it is only called in a separate session if it really needs to be called by a different user.

A better solution is to recompile your Perl to enable multiplicity. I am not aware of any drawbacks to doing so. In theory, one could even recompile Perl in-place and then restart Postgres, but I have never tried this out. :)

Musica Russica Launches with Piggybak


The new home page for Musica Russica.

Last week, we launched a new site for Musica Russica. The old site was running on an outdated version of Lasso and Filemaker and was approximately 15 years old. Although it was still chugging along, finding hosting support and developers for an outdated platform becomes increasingly challenging as time goes on. The new site runs on Ruby on Rails 3 with Nginx and Unicorn and uses open source Rails gems RailsAdmin, Piggybak, CanCan and Devise. RailsAdmin is a great open source Rails Admin tool that I've blogged about before (here, here, and here). Piggybak is End Point's home grown light-weight ecommerce platform, also blogged about several times (here, here, and here). Below are a few more details on the site:

  • The site includes Rails 3 goodness such as an elegant and thorough MVC architecture, advanced routing to encourage clean, user-friendly URLs, the ability to integrate modular elements (Piggybak, RailsAdmin) with ease, and several built-in performance options. The site also features a few other popular Rails gems such as Prawn (for printing order and packing slip PDFs), Rack-SSL-Enforcer (a nice tool for enforcing SSL pages), exception_notification (a small tool to configure sending emails on Rails exceptions), and Paper Trail (a gem to track changes on your models).
  • The site features complex, feature-rich search run by Sphinx with the popular Rails gem ThinkingSphinx. The search includes standard features such as selecting items listed per page, sort by various product attributes and wildcard text search. Although we've used Solr on several recent Rails projects, we wanted to go with Sphinx here to avoid the Tomcat and Java requirements. After working with both in depth, I'd conclude that both Sphinx and Solr offer very similar features.
  • The site has a very custom data model. With the use of Piggybak, any Rails model can become a sellable item, which gives us the ability to customize each data model without affecting all sellable products. This opportunity suits Musica Russica well because they offer very different product types such as books, sheet music, CDs, and sheet music collections. Additional features (taxonomy, cross-sell, upsell) in the application are not constrained to assumptions that traditional monolithic ecommerce applications make.
  • The site introduces downloadable products, described more in this article. A user purchasing downloadable products is required to register, and then has access to their purchased downloadables after purchase. Downloadable orders also include free shipping. Piggybak needed minor customization to support downloadable products.
  • The site also features integration with the Elavon payment gateway. This is the first time that End Point has worked with Elavon. Elavon is supported by ActiveMerchant, so adding support for Elavon simply required hooking into ActiveMerchant's Elavon module.

It was exciting to build a custom Rails ecommerce site from the ground-up, and the site certainly showcases End Point's wide range of ecommerce expertise ranging from hosting with modern cutting-edge servers to development of popular front-end ecommerce features with advanced open source tools. Don't hesitate to contact us today to get started on development of your Rails application!