Tuesday, December 21, 2010

Integrate your Perl application with Google Apps Marketplace

I spent most of the last week trying to figure out how to take a Perl web app and integrate it with the Google Apps Marketplace. This is where the supposedly 3 million businesses who signed up for Google Apps go for third-party integrations.

You have to sign up as a vendor in order to make your web application available to Google Apps customers. The other requirement is that your app supports OpenID Single Sign-on. This is where the integration turned difficult for me.

I assumed you would use Net::OpenID::Consumer to handle the consumer-side processing. However, after only a little headway, and asking around on StackOverflow as well as the Google Marketplace forums, I was stuck. I could not close the OpenID circuit and continue on to my app.

I eventually solved the problem by switching modules. I changed to the skimpily documented Net::Google::FederatedLogin, and finally got things working.

The code is as follows (substitute example.com below for your actual developer's domain).

First, you have to login your Google Apps Marketplace vendor profile, and add the URL to index.cgi in your application manifest, with the required ${DOMAIN_NAME} variable. ${DOMAIN_NAME} will be replaced by the domain of the user who installs your app. This parameter is integral to the authentication scheme.
The application manifest is like the installer for your web app. It's detailed here, but is kind of outside of the scope of this post.

Once you've gotten the application manifest done, add the following code to your servers.

use CGI;
use Net::Google::FederatedLogin;

my $q = CGI->new();

my $domain = $q->param('domain');
if (!$domain) {
    print $q->header(), 'Provide domain please.';
    exit 0;

my $fl = Net::Google::FederatedLogin->new(
    claimed_id => 
        'https://www.google.com/accounts/o8/site-xrds?hd=' . $domain,
    return_to =>
    extensions => [
            ns          => 'ax',
            uri         => 'http://openid.net/srv/ax/1.0',
            attributes  => {
                mode        => 'fetch_request',
                required    => 'email',
                type        => {
                    email => 'http://axschema.org/contact/email'
    ] );

print $q->redirect($fl->get_auth_url());
Note that $domain above is used in the claimed_id parameter and is sent to Google for verification. The extensions parameter informs Google what user data to send back to your site when it redirects to return_to. Which, in this case, is

use CGI;
use Net::Google::FederatedLogin;
use LWP::UserAgent;
use HTTP::Request::Common;
use URI;
use URI::Escape qw(uri_escape);
use Net::OAuth;

# OAuth (to access user's Google data)
# You get these from your vendor profile in Google Apps. Same place
# where you edit the application manifest.
my $CONSUMER_KEY = '??????????????.apps.googleusercontent.com';
my $CONSUMER_SECRET = '??????????????????';

# We want to get some calendar data from the user
my $URL = 

my $q = CGI->new();
print $q->header();

# OpenID final step
my $fl = Net::Google::FederatedLogin->new(  
    cgi => $q,
    return_to =>
        'http://www.example.com/return.cgi' );

eval { $fl->verify_auth(); };
if ($@) {
    print 'Error: ' . $@;
else {
    my $ext = $fl->get_extension('http://openid.net/srv/ax/1.0');

# OAuth
sub get_calendar_oauth {
    my $email = shift;

    my $oauth_request =
              consumer_key => $CONSUMER_KEY,
              consumer_secret => $CONSUMER_SECRET,
              request_url => $URL,
              request_method => 'GET',
              signature_method => 'HMAC-SHA1',
              timestamp => time,
              nonce => nonce(),
              extra_params => {
            'xoauth_requestor_id' => $email
    my $req = HTTP::Request->new(
        GET => $URL . '?xoauth_requestor_id=' . uri_escape($email) );

    $req->header('Content-type' => 'application/atom+xml');
        'Authorization' => $oauth_request->to_authorization_header);

    my $ua = LWP::UserAgent->new;
    my $oauth_response = $ua->simple_request($req);
    while($oauth_response->is_redirect) {

      my $url = URI->new($oauth_response->header('Location'));


      my %query = $url->query_form;
      foreach my $param (keys %query) {
        $oauth_request->{extra_params}->{$param} = $query{$param};

      $url->query(undef); # clear out the query parameters
      $oauth_request->{request_url} = $url;
      $oauth_request->sign; # resign
        'Authorization' => $oauth_request->to_authorization_header );

      $oauth_response = $ua->simple_request($req);

    print $oauth_response->as_string;

} # get_calendar_oauth

sub nonce {
  my @a = ('A'..'Z', 'a'..'z', 0..9);
  my $nonce = '';
  for(0..31) {
    $nonce .= $a[rand(scalar(@a))];

The final OpenID step is quite minimal, as you can see above. You simply create a new Net::Google::FederatedLogin object and pass it the CGI object plus return_to value. Then you verify, and if there isn't an error, you should be able to access the extension data via the call to get_extension().

Much of the above script is devoted to doing OAuth in order to access the user's Google data, in this case his calendar. If you only need to authenticate a user and not access Google data, you could omit the call to get_calendar_oauth() entirely.


When you create your app in the vendor section of Google Apps Marketplace, it will be assigned a Consumer Secret and a Consumer Key. These must be present in the parameters when you instantiate your Net::OAuth object. In the above code, you would set $CONSUMER_KEY and $CONSUMER_SECRET to these values.

The data is returned as Atom/XML. In the above code I do nothing with it except print it out. The code in get_calendar_oauth has been borrowed almost directly from this blog post by Jeremy Smith.

That's basically it. This was intended to be a sparse example covering the two main points for integrating with Google Apps from Perl -- OpenID to grant access to your app via Google credentials, and OAuth for accessing Google data on behalf of the user.

Tuesday, November 30, 2010

Bookmarklets versus Man-In-The-Middle attacks

Let me start off by saying I don't consider myself a security expert. As a web systems developer I've had to become knowledgeable about security, e.g client-side password hashing, salted hashes, PKI, etc. But like many I've relied quite a bit on TLS/SSL to ensure that data moving between my systems and users is safe. In fact, if I were completely honest, I'd say it's been something of a crutch. If we point users to an https link, we feel like we've done what's necessary for security.

TLS/SSL has a pretty serious weakness, however, the Man-In-The-Middle attack. And MITM is a fairly trivial thing to do, thanks to the Address Resolution Protocol, which is used nearly everywhere for one physical device to find another on a network.

MITM is also trivial to do because smart and devious people like Moxie Marlinspike have exploited these weaknesses and created tools like sslstrip. With available tools and only a reasonable amount of knowledge, a "script kiddie" can implement MITM in around 2 minutes, and pretty easily trick you into giving away information. Think about that the next time you want to do your banking while sitting in a coffee shop.

The MITM attack is very difficult to circumvent programmatically, because (if you watched the video at the sslstrip link above) the attacker has the page first, and is able to manipulate it in subtle ways that are hard to detect. For instance, stripping out https links so when you login somewhere, you send your credentials for the attacker to view and capture.


Recently, I began to think about safe ways to do logins, assuming that a MITM attack was under way.

You could use public/private key encryption like RSA to encrypt the username and password with a public key before sending. However, if someone is in the middle, they could just as well manipulate the code to use a public key of their own, and then decrypt your credentials. It makes the attack harder, but not impossible.

So how can you ensure that the key (and code) you've obtained from the server hasn't been tampered with? This is where I thought of using bookmarklets. Bookmarklets are typically a bit of compressed JavaScript code that is stored in a link. When clicked, the JavaScript runs in the context of the current page. My idea was to do the following:

1. Create a login page that uses public key encryption to encrypt credentials before sending. Embed the public key in the page.

2. Use a hash function to generate a signature for the login page.

3. Embed the hash function along with the hash from Step 2 in a bookmarklet that you make available to users. Ask them to add it to their Bookmarks.

4. When users visit your login page, they would click the bookmarklet from their Bookmarks. It would process the current page and generate a hash, and compare it to the one in the bookmarklet. If the hashes didn't match, the user would be alerted.


I ended up doing the following to test my theory. First, I created a bookmarklet for developers. When clicked, it traverses the page you're currently visiting and extracts text, elements, and attributes, generating a SHA1 hash from the combined values. It then outputs the JavaScript code along with the hash, which you can turn into a bookmarklet for your users.

To use it, drag the below link to your Bookmarks, visit your [login] page, and click the link. The code for the bookmarklet of your page will pop up in a new window. Use that code to make a bookmarklet to add to your site.

Generate SHA1 Validation Bookmarklet

Using the above validation-bookmarklet-generator-bookmarklet :), I've created the following example of a pretend login page that uses RSA to encrypt credentials before sending.

Here's the validation bookmarklet for my pretend login page:


Drag it to your Bookmarks. Then, visit the login page below and click the CheckPage! bookmarklet to verify the login form hasn't been tampered with:

Fake login page with RSA encryption

(You can even click Login to see the encrypted info that will be sent to the server.)

As long as a user clicks the validation bookmarklet and heeds its warning (that is, doesn't continue regardless), an MITM attack could be mitigated, if not altogether prevented for this page.

Issues and miscellany

There are a few issues I see with this idea. There may (probably?) be more, but these are the ones that immediately come to mind.

1. A MITM attack may modify the page to remove the user hint to click the bookmarklet first, relying on users to be forgetful. The page isn't disabled in any way, so if it was compromised and the user goes ahead, their credentials may be captured.

2. An attacker may trick the user into 'updating' the bookmarklet to do nothing but alert them the page they are viewing is validated.

3. If every login page for every service created a bookmarklet, users wouldn't be able to manage them very well, and they'd become rather cumbersome.

For developers, it also becomes a hassle if the login page changes, even slightly. They will be dealing with a lot of users who suddenly can't validate the login page, and are possibly panicking.

I don't in any way think this is a panacea, but I do feel like the idea has some merit, and possibly there are other, better ways to implement something in a similar vein.

Saturday, October 2, 2010

Google Document List API NetBeans sample

I've recently been doing work in Java using Google's Document List API v3.0. It's well documented and there are some basic "Hello World" samples available, but not a single sample that fully demonstrates the basics of using the API.

In teaching myself the API, that's where I started. I took a lot of the code from their documentation pages, and created a single NetBeans project that connects to Google Docs, lists your documents, creates a new folder, creates a new file in that folder, etc. It was a good starting point for the app I was building, and is probably a good starting point for anyone else who wants to build an app that interfaces with Google Docs.

One of the most annoying parts of creating the project was figuring out just the dependencies that were needed, and getting those .jar files in the right place. For convenience, I include all the necessary .jar files in the download. They're in the Libraries/ directory, and are already referenced by the project.

You can download the sample here: http://www.arbingersys.com/dnlds/GDocsSample.zip

If you have NetBeans installed, simply choose File | Open Project and browse to the GDocsSample/ folder that you've extracted from the archive.

Under Source Packages double-click Main.java and it will open in the editor. On line 59 you'll want to modify the client.setUserCredentials() call with your Gmail credentials. Then you should be ready to build and run the project.


Friday, June 25, 2010

I'm a believer - Chrome + JavaScript = fast

Update 7/6/2010: I re-ran all the tests on a Windows 7 machine in order to include the IE9 Preview, which I can't install on Windows XP. The results were consistent with my previous tests - Chrome wins.

In working on an API with a requirement to process a large amount of data (> 5MB) client-side in a browser, I needed to find a way to make JavaScript behave in a thread-like manner. I came across the setTimeout() function, and the following pattern from Julien Lecomte's excellent blog.

This pattern effectively allows you to execute long-running processes without locking up your browser and making it unresponsive.

It worked as expected, but it seemed slow running under Firefox. I was developing on a VM of Ubuntu, so I'm sure that had something to do with it. However, I kept tweaking parameters and optimizing my code to see if I could get a bit more response from the pattern. I did, but it was marginal. I was processing 5MB of data client-side in around 1.7 minutes under VM, so I reasoned it would probably be faster in real life.

Then, I decided to try Chrome. I was completely stunned. It ran in about 15 seconds. I tested more, but the results were consistent. I also have Opera installed, so I started it up, and the results were even worse than Mozilla. In fact, I got tired of waiting for it to finish, so I just killed it.

But now I was curious why it was happening. Was it the pattern itself, or was the process my code was running simply taking longer in the other browsers? I know Chrome's V8 is supposed to be faster, but I wonder why it's so noticeable on this particular pattern.

I decided to try a different test. I borrowed Julien's example code from the above post and saved it to a Windows XP workstation with IE, Firefox, and Chrome installed. I set the length of Julien's array to be sorted to length = 5000; so it would take longer in all browsers. Then I launched the page and let it sort. Chrome, again, is the clear winner. Here are the results, from fastest to slowest:



IE 9 Preview:

IE 8:

I ran each browser a couple of times just to be sure the results were consistent. (Hardly rigorous testing, I know, but enough to satisfy me.)

So I've heard that Chrome has the fastest JavaScript engine, but now I've actually experienced it for myself. However, I'm left to wonder why it's so apparent in this code? My guess is the way in which the 'continuation' of the anonymous function is implemented in the various engines. Perhaps somebody with a deeper knowledge of the internals knows better?


It turns out, somebody did know better. I asked on the Chrome forums, and Erik Kay, one of the Chrome hackers, indicated that the speed increase is most likely due to Chrome's timer implementation. Here's his response:


He pointed me to the following blog post that gives a detailed account of how the Chrome team developed the timer system, and why it's so fast. It's totally worth the read:


One more thing. There's also this page, which tests the frequency of the timer implementation in your browser:


Thursday, April 29, 2010

In RE: Thoughts on Flash by Steve Jobs

A friend of mine sent me the Steve Jobs open letter to Adobe concerning Flash. I replied to him via email, but I thought it might be good fodder for the blog, which is desperately in need of some love. Here's my response to him, without edits:

Interesting. He makes some good points, but there's also plenty of hubris, in my opinion.

This -- "Though the operating system for the iPhone, iPod and iPad is proprietary, we strongly believe that all standards pertaining to the web should be open" -- just sounds hypocritical and self-serving, not to mention bitter that Flash managed to become the de facto standard of web media.

Personally, I'd say that Apple and Adobe are pretty much the same. Flash could be considered 'open' because the SWF format is published & well-known, e.g. there are other players for it, just check out Linux. Adobe controls the Flash player, but doesn't control how flash files are made, exported, converted, etc. Apple makes no apologies for locking down what they can, so why expect Adobe to?

His points about Flash being designed for PCs and mice are spot on, though. And his 6th point makes sense just from a strategy stand-point. I think this is less about Mr. Job's "ideals", and more about severing a dependency that seems dangerous to Apple.

He's not just fighting Adobe, though. It's like Windows. Part of its staying power is all the third-parties that bought in to the platform, and have created things people want, and who will continue to create things for it. He not only upsets Adobe, but millions of Flash developers.

(Ok, now I gotta go back to work.)

Monday, October 5, 2009

A minimal jQuery source for a fade behind pop-up

I recently wanted to do one of those nice trendy popups that stay within the current web page and fades everything behind the pop-up. I wanted to use it to allow a user to view a demo, a Flash animation. Pretty typical usage from what I've seen.

I figured this was something done handily by jQuery, but I had some trouble finding a minimal, complete source to start with. Everyone seemed to want to force you to go through the tutorial they wrote, step by step. Well, I usually want the code, and then the tutorial.

I found this tutorial which was at least succinct. Soon I had a very small (i.e. minimal), working .html document that behaved how I wanted. For instance, it automatically figures out the horizontal and vertical positioning of the pop-up so it comes up in the center of the screen.

Here you go:
#popup {
height: 100%;
width: 100%;
background-color: #000000;
position: absolute;
top: 0;
#window {
width: 500px;
height: 400px;
margin: 0 auto;
border: 1px solid #000000;
background: #ffffff;
position: absolute;
top: 10%;
left: 15%;
<script type="text/javascript" 
<script type="text/javascript"> 

function Show_Popup(action, userid) {

var hpos = ($(window).height()/2) - (400/2);
var wpos = ($(window).width()/2) - (500/2);
$('#window').css('top', hpos + 'px').css('left', wpos + "px").fadeIn('fast');
// I added a function call here to insert my demo into the #window div

function Close_Popup() {
        <div onclick="Show_Popup()" 
          View demo
<div id="popup" style="display: none;"></div> 
<div id="window" style="display: none;"> 
<div id="popup_content">
<a href="#" onclick="Close_Popup();" >Close</a>

And now for the tutorial, also minimal:

(1) Make sure that the <div id="popup" ... </div> section is placed into your page just prior to the </body> tag.

(2) It's unlikely that your popup height and width will be the same as mine. You'll need to modify in two places to change this - inelegant I know - in the #window style declaration, and in the Show_Popup() function, where hpos and wpos are calculated.

Here's the demo page.

Wednesday, July 22, 2009

The Missing GObject Tutorial Sample

Well, perhaps not exactly, but still -- I think it should work.

I've recently started poking around the GObject library, which is part of GLib. GObject is a C library aimed at providing OOP programmability that easily integrates with (usually) dynamic third-party languages. Basically, it allows you to write "glue" code between $your_dynamic_language and the GObject library just once, and then hook into any libraries created with GObject without writing further glue.

A good, detailed tutorial is available here, which I've been working through. After getting the gist of something, I get itchy for some sample code to play around with. So I Googled and found this how-to. An older version of the documentation mentioned some sample code that I was never able to find.

After giving up on that, I decided that I should be able to use the tutorial to scrap together my own sample. The tutorial was pretty detailed, after all, and apparently referenced a sample that did (or does) exist somewhere. So that's what I did. I've created a fully functioning sample based more-or-less on the tutorial mentioned above. The code is below, with comments.

I have compiled this on my Ubuntu 8.10 and 9.04 machines using the following command:

gcc `pkg-config --libs gtk+-2.0` `pkg-config --cflags gtk+-2.0` maman-bar.c


* Copyright/Licensing information.
* Reference:
* http://library.gnome.org/devel/gobject/unstable/howto-gobject.html
* http://library.gnome.org/devel/gobject/unstable/chapter-gobject.html

/* inclusion guard */
#ifndef __MAMAN_BAR_H__
#define __MAMAN_BAR_H__

#include <glib-object.h>

* Potentially, include other headers on which this header depends.

* Type macros.
#define MAMAN_TYPE_BAR (maman_bar_get_type ())
#define MAMAN_BAR_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST ((klass), MAMAN_TYPE_BAR, MamanBarClass))

typedef struct _MamanBar MamanBar;
typedef struct _MamanBarClass MamanBarClass;

* Private instance fields
* Uses the Pimpl method:
* http://www.gotw.ca/gotw/024.htm
* http://www.gotw.ca/gotw/028.htm
typedef struct _MamanBarPrivate MamanBarPrivate;

/* object */
struct _MamanBar
GObject parent_instance;

/* public */
int public_int;

/*< private >*/
MamanBarPrivate *priv;

/* class */
struct _MamanBarClass
GObjectClass parent_class;

/* class members */

/* Virtual public method */
void (*do_action_virt) (MamanBar *self, gchar *msg);


* Non-virtual public method
void maman_bar_do_action (MamanBar *self, gchar *msg /*, other params */);

/* Virtual method call declaration */
void maman_bar_do_action_virt (MamanBar *self, gchar *msg /*, other params */);
/* Virtual method default 'super' class method */
void maman_bar_do_action_virt_default (MamanBar *self, gchar *msg);

#endif /* __MAMAN_BAR_H__ */


#include "maman-bar.h"


A convenience macro for type implementations, which declares a class
initialization function, an instance initialization function (see GTypeInfo
for information about these) and a static variable named t_n_parent_class
pointing to the parent class. Furthermore, it defines a *_get_type()
function. See G_DEFINE_TYPE_EXTENDED() for an example.
G_DEFINE_TYPE (MamanBar, maman_bar, G_TYPE_OBJECT);

/* Define the private structure in the .c file */

struct _MamanBarPrivate
int hsize;
gchar *msg;

/* Init functions */
static void
maman_bar_class_init (MamanBarClass *klass)
g_type_class_add_private (klass, sizeof (MamanBarPrivate));

/* Setup the default handler for virtual method */
klass->do_action_virt = maman_bar_do_action_virt_default;

static void
maman_bar_init (MamanBar *self)

g_print("maman_bar_init() - init object\n");

/* Initialize all public and private members to reasonable default values. */

/* Initialize public fields */
self->public_int = 99;

g_print(" initializing public_int to %d\n", self->public_int);

/* Initialize private fields */
MamanBarPrivate *priv;
self->priv = priv = MAMAN_BAR_GET_PRIVATE(self);
priv->hsize = 42;

g_print(" init'd private variable priv->hsize to %d\n", priv->hsize);

/* If you need specific construction properties to complete initialization,
* delay initialization completion until the property is set.


/* Object non-virtual method */
void maman_bar_do_action (MamanBar *self, gchar *msg) {
/* First test that 'self' is of the correct type */
g_return_if_fail (MAMAN_IS_BAR (self));

// Assign to private 'msg'
self->priv->msg = msg;

g_print("maman_bar_do_action() - %s\n", self->priv->msg);


/* Object virtual method call - performs the override */
void maman_bar_do_action_virt (MamanBar *self, gchar *msg) {
/* First test that 'self' is of the correct type */
g_return_if_fail (MAMAN_IS_BAR (self));

g_print("maman_bar_do_action_virt() -> ");
MAMAN_BAR_GET_CLASS (self)->do_action_virt(self, msg);

/* Object virtual method default action (can be overridden) */
void maman_bar_do_action_virt_default (MamanBar *self, gchar *msg) {

g_print("maman_bar_do_action_virt_default() - %s\n", msg );


main (int argc, char *argv[])
* Prior to any use of the type system, g_type_init() has to be called
* to initialize the type system and assorted other code portions
* (such as the various fundamental type implementations or the signal
* system).

/* Create our object */
MamanBar *bar = g_object_new (MAMAN_TYPE_BAR, NULL);

bar->public_int +=1;
g_print("incremented bar->public_int: %d\n", bar->public_int);

/* Call object method */
maman_bar_do_action(bar, "helowrld");

/* Call virtual object method - we could subclass and override... */
maman_bar_do_action_virt(bar, "HELOWRLD");

return 0;

And here's what I get when I run a.out:
ok ./a.out
maman_bar_init() - init object
initializing public_int to 99
init'd private variable priv->hsize to 42
incremented bar->public_int: 100
maman_bar_do_action() - helowrld
maman_bar_do_action_virt() -> maman_bar_do_action_virt_default() - HELOWRLD
You can download the source files directly from here.