I've tried multiple approaches to storing application configuration with Node.js but most have been painful until I found this solution.

tl;dr: Executable configuration FTW via exports.

With Node.js you can export an object or function for use within another module. This is key to keeping your Node.js application readable and structured which as I've found can be an art-form in itself.

My current approach is as follows.

I have a config.js which looks like this.

var url = require('url')


var config = {}


config.google = {};
config.redis = {};


config.google.id = process.env.GOOGLE_ID || 'DEVELOPMENT.googleusercontent.com';
config.google.secret=  process.env.GOOGLE_SECRET || 'DEVELOPMENT';
config.google.callback=  process.env.GOOGLE_CALLBACK || 'http://127.0.0.1:3000/google/callback';


config.redis.url=  url.parse(process.env.REDISTOGO_URL || 'http://127.0.0.1:6379');


module.exports = config;

Because it's a standard JavaScript module, when Node.js loads the file it will be executed with the config hash created. This has two important aspects.

Firstly, I can include other helper modules, such as being able to return a parsed url.

Secondly, I can take advantage of the current environment to determine if production or development values should be returned. This is great when combined with Heroku.

With the configuration defined, the logic to access the configuration looks like this:

var config = require('./config')
var id = config.google.id
var url = config.redis.url

No if statements, no redefining variables, no swapping files around. Clean, simple, effective.

If I wanted to have additional confidence then if NODE_ENV equaled production I could ensure all environment variables are not undefined.

This is working for me, but has anyone found any better solutions? Leave a comment or tweet me @Ben_Hall

Storing data in the session state is a little bit naughty as HTTP should remain stateless but there is a trade-off. In this case I wanted to test an assumption around a potential feature for Mayday. I felt it was more important to release than over architecture the entire solution with the possibility of dropping the  feature the next day.

When it comes to Node.js, session state is stored in-memory meaning any restarts or new deployments will delete the data.

Enter Redis

Redis is an ultra-fast, open source, advanced key-value store. To make it even easier, RedisToGo offer a hosted version with a Heroku plugin. After adding the plugin, an account will be created with a database URL provided as a configuration variable.

$ heroku config
REDISTOGO_URL   => redis://redistogo:PASSWORD@SERVER.redistogo.com:9712/
NODE_ENV        => production

To use this as your session store you will need to configure the middleware by defining a RedisStore from the connect-redis npm.

The require statements should look like this:

var express = require('express');
var RedisStore = require('connect-redis')(express);

var url = require('url')


For development you will want to use your local server.
app.configure('development', function(){         
  app.use(express.session({ secret: "password", 
                            store: new RedisStore({
                                          host: "127.0.0.1",
                                          port: "6379",
                                          db: "name_of_my_local_db"
                                        })  
          }));
}); 


For production you should use the RedisToGo URL provided. 
app.configure('production', function(){
 var redisUrl = url.parse(process.env.REDISTOGO_URL);
 var redisAuth = redisUrl.auth.split(':');


 app.use(express.session({ secret: "password", 
                           store: new RedisStore({
                                        host: redisUrl.hostname,
                                        port: redisUrl.port,
                                        db: redisAuth[0],
                                        pass: redisAuth[1]
                                      })  
         }));
});

Node.js and Connect-Redis will do the rest for you.

The key will be the session id for the user with the value being a JSON serialised object of req.session.

When writing a file modification script I find it annoying (read: boring) to write the logic of saving the file under a different name, moving to override original and then cleaning up. This is what I recently had to do and instead decided to look around at what other options were available, one of which was sed.

The following command will replace the word testdomain with proddomain.

$ echo 'www.testdomain.com' > test.txt
$ cat test.txt 
www.testdomain.com  
$ sed -i '' s/testdomain/proddomain/ test.txt
$ cat test.txt
www.proddomain.com

The key to how this works is the -i '' argument of the sed command. -i specifies the file extension, by passing an empty string it uses the same file.

I have a lot of respect for Node, but sometimes certain npm packages become out of sync with the latest version and break. Today, that npm package was node-compress.

While the package looked to build, when attempting to require the module it would error.


$ node
> var c = require('compress')
Error: Cannot find module 'compress'


After a quick look around on GitHub I found a fork with the fix https://github.com/elliotttf/node-compress/

In your package.json, simply reference the tarball, clean out the node_modules directory, install and everything should work again.

Package.json

    , "compress":"https://github.com/elliotttf/node-compress/tarball/1edaa48bf33f7c836f1e275691e1d8645f0a71c3"

Wow. Chris Kemp (Advance technical solutions team at Salesforce.com) has create a great short video based on my blog post about How To Deploy a .NET App with Mono on Heroku. A huge thank you!



Link: http://www.youtube.com/watch?v=LkdiTzgqqsk&hd;=1

To help with development of Mayday we are asking people to fill out a questionnaire about how they use Google Analytics. If you use Google Analytics and have a moment to fill out the questionnaire then we would be most grateful.
Visit our Google Analytics Questionnaire 
Thank you.

While working on the new version of Mayday, I wanted to show a message if no data was returned from Google Analytics. To add to the complexity, I wanted to be able to override the default message on a per page basis.

I'm already using the ICanHazJS client-side tempting engine which has a method for each template block on a page. However, if the page doesn't have the block then the method won't exist and an error will be thrown.

What I needed was functionality similar to Ruby's respond_to. With this method I can ask the object if it will respond to the method call. For example:

Object.respond_to? 'test' #=> false
Object.respond_to? 'to_s' #=> true

Luckily this is just as easy to do in JavaScript using 'in'

> 'test' in Object
false
> 'toString' in Object
true

This allowed me to write the following:

var e = $('#data')
if('nodata' in ich)
   e.append(ich.nodata());
else
   e.append('No Data Found');



If a nodata ICanHaz template block appears on the page then it will be rendered, otherwise it will fall back to the default. Problem solved.


Update: 


Problem almost solved. As pointed out on Twitter by @nmosafi and @theprogrammer, just using 'in' along is not enough.  For example:

> Object.test = "test"
"test"
> 'test' in Object
true
> Object.test()
TypeError: Property 'test' of object function Object() { [native code] } is not a function



What you need to do is ensure that the property is also a function. Something I had assumed previously.


> 'test' in Object && typeof(Object.test) == "function"
false

> 'toString' in Object && typeof(Object.toString) == "function"
true

When responding to a web request it's common for servers to tell the client various bits of information. The one they enjoy most is some promotion around the name and version "powering" the site. Sadly, hackers also love this as it gives them more information for an attack vector.

By default, ExpressJS with NodeJS will return a X-Powered-By header.


$ curl -I 0.0.0.0:3000/
HTTP/1.1 302 Moved Temporarily
X-Powered-By: Express


I wasn't overly impressed by this but it's easy to remove. In your application configuration, at the top, add a new middleware function which removes the header.


  app.configure(function(){
      app.use(function (req, res, next) {
        res.removeHeader("X-Powered-By");
        next();
      }); 


      app.set('views', __dirname + '/views');
      app.set('view engine', 'jade');
      app.use(express.bodyParser());
      app.use(express.methodOverride());
      app.use(express.cookieParser());
      app.use(express.static(__dirname + '/static'));
  });


Simple.

While trying to solve a problem, we have all sometimes done a little console.log outputting to help us gain additional understanding. While it's far from the most effect way of debugging, it's more annoying when those statements appear in your production logs. That was my scenario today.

Thankfully, with a simple grep command I was able to identified all of my "debugging" statements and the lines they occurred on.

$ grep -nr console * | grep -v "node_modules/*" | grep -v "static/"

endpoints/browser_stats.js:35:        if(error) { console.log(error); }
endpoints/browser_stats.js:51:        if(error) { console.log(error); }
endpoints/browser_stats.js:67:        if(error) { console.log(error); }
endpoints/events.js:16:    console.log(eventsUrl);
endpoints/events.js:17:    console.log(hitsUrl);
endpoints/events.js:20:        if(error) { console.log(error); }
endpoints/events.js:27:            if(error) { console.log(error); }
passport.js:23:      console.log("Auth");

In this case, I filtered out any matches from node_modules and static javascript files. Keep in mind that in some cases it's a node module causing the output.

A simple command to help keep your production logs clean and readable.

With Javascript becoming increasingly popular, I'm still shocked at how bad the online documentation and examples are.  Let's take the search phase "setting a cookie in javascript", a fairly common problem which I always forget the correct syntax for.

The first four results are as follows:
http://www.w3schools.com/js/js_cookies.asp
http://www.quirksmode.org/js/cookies.html
http://techpatterns.com/downloads/javascript_cookies.php
http://www.javascripter.net/faq/settinga.htm

Sorry to the original authors, but each one of them IS AWFUL!

Let's take W3Schools as an example. It defines a variable called ARRCookies are could be for Pirate Cookies (via @Blowdart) or ARRay (via @AndrewVos) - both of which are great examples of creating Clean Code. To make matters even worse, this example has spread like a virus with Google returning 288,000 results and GitHub returning 398.

The best link which describes handling cookies, and generally any Javascript documentation, can be found on the Mozilla Developer Network (MDN).

https://developer.mozilla.org/en/DOM/document.cookie

While I'm not an SEO expert, this looks to be light on SEO (Search Engine Optimisation) related to phases people search for. As a result the link was found in the lonely spot on page 3 which is should be number 1.

Wouldn't it be great if MDN had SEO focused content to support the excellent reference material it already has available? The PromoteJS movement has made a start but I think it needs a kick-start.

Who's with me?

UPDATE (20/1/12):

In a moment of great timing, the Mozilla WebDev team started a AMA (Ask Me Anything) thread on Reddit. I pointed out the above comments and had some great responses from the team.

It turns out that MDN is a wiki so it's in the community (and my hands) to make a difference.

You can read the comments here:

http://www.reddit.com/r/IAmA/comments/oonrg/iama_member_of_the_mozilla_webdev_team_ama/c3iup1b

Almost every time I have to deal with the Date object in Javascript I always hit the same WTF.

The documentation is clear but the API has some strange aspects that personally I don't think reflect the real world and how we naturally handle dates.

For example:
      getDay() returns the day of the week (0-6) leaving getDate() to return 1-31. Personally, I would have imagined getDate() to return DMY with getDay() returning 1-31.
      getMonth() returns 0-11.  Classic computer science with starting the count at 0 while the Gregorian calendar is 1-12. This is also inconsistent with getDate() which starts at 1.
      getYear() returns the year (usually 2-3 digits). For example, 2012 is obviously 112. You need to use getFullYear() to return 2012. The logic about how the method came to 112 is found at https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Date/getYear


JavaScript 1.0 - we love you and you have left an impact in many ways.

Last week I mentioned how I had successfully deployed Nancy / Mono (C# frameworks) onto Heroku.  This post covers how you can deploy your own Mono projects onto Heroku.

As before, this approach is completely unsupported by Heroku and I'm not 100% sure if it's production ready. I would love to hear your experiences.

0) Prerequisites 
There are a few requirements before deploying a project.
    - The solution can compile using Mono on Linux (Ubuntu 10.04 - Lucid Lynx). 
    - The launch application must be a self-hosted executable.
    - The self-hosted executable must stay alive and not exit. Example: https://github.com/BenHall/nancy-demo-hosting-self/blob/master/src/Program.cs#L20-23

1) Update heroku gem
Before you continue, make sure your running the latest heroku gem as you need the buildpack option which is only supported in later versions. Thanks Andy Pike for pointing this out.

$ gem update heroku

2) Listening to Heroku requests
This is the only part which I'm not too happy with. In order for Mono to accept Heroku requests, you need to bind the listener to the full hostname provided by Heroku - for example http://deep-moon-1452.herokuapp.com.

The application also needs to listen on a particular port randomly defined by Heroku. This is provided via an environment variable. 

Example:
https://github.com/BenHall/nancy-demo-hosting-self/blob/master/src/Program.cs#L12

Ironically this is the part which blocked me and after multiple combinations I was left with hard-coding the name as my last resort - see the commit https://github.com/BenHall/nancy-demo-hosting-self/commit/4d9e07bed92559a53b1463c37e837045d59c3a9a

3) How projects are built during deployment
Projects are built on Heroku during the git push process.  To find a solution to build, the command "ls $BUILD_DIR/*.sln | head -1" is used. This will return the name of the first solution file found.
Once the name has been determined, xbuild is called with the output streamed back. If the build fails, the deployment is stopped.

4) How the application is launched - Procfile
Once the project has been built, the self-hosted executable needs to be executed via a Procfile.

An example can be found here:
https://github.com/BenHall/nancy-demo-hosting-self/blob/master/Procfile

The web property specifies the command used by Heroku. The first part is the path to mono which is defined in the buildpack and shouldn't be change. The second part is the relative path to your compiled executable.

In the example I also defined a local environment command allowing me to replicate the hosted environment locally via "foreman start local". 

5) Creating the Heroku environment - Cedar and buildpacks
When creating your Heroku application to host your project, it's important to specify the stack as cedar along with the buildpack for Mono. The buildpack defines the steps required to download Mono from my S3 account, configure the environment and build your solution.

$ heroku create --stack cedar --buildpack http://github.com/BenHall/heroku-buildpack-mono 

Remember to change the hostname your exe listens on otherwise you'll received a Bad Request (Invalid Host) response.

If you receive the error "!Name must start with a letter and can only contain lowercase letters, numbers, and dashes" then make sure your using the latest Heroku gem (currently 2.18.1).

6) Push
A simple push should now be all you need.

$ git push heroku master

Done!  Technical details about how the buildpack works in a future post.

Important links:
Buildpack:  https://github.com/BenHall/heroku-buildpack-mono
Example: https://github.com/BenHall/nancy-demo-hosting-self

tl;dr: The main steps are
0) $ gem update heroku
1) $ git clone https://github.com/BenHall/nancy-demo-hosting-self
2) $ cd nancy-demo-hosting-self
3) $ heroku create --stack cedar --buildpack http://github.com/BenHall/heroku-buildpack-mono
4) Replace deep-moon-1452 in src/Program.cs to the application name created above
5) $ git commit -am "Changed application name"
6) $ git push heroku master
7) Tweet to say how amazing it is.

Over the past couple of days I've been trying to get Mono running on top of Heroku.

I'm pleased to report that I have successfully deployed Nancy (a C# web framework) on top of Heroku.

It's far from a production ready solution and is completely unsupported by Heroku but it's an interesting experiment. More technical details in future posts but here's some proof.

Server response headers:
$ curl -i http://deep-moon-1452.herokuapp.com/
HTTP/1.1 200 OK
Content-Type: text/html
Date: Tue, 03 Jan 2012 16:33:02 GMT
Nancy-Version: 0.9.0.0
Server: Mono-HTTPAPI/1.0

Screenshot:















This is the output when you push your git repository to Heroku. The source code is built on Heroku.

$ git push heroku master
Counting objects: 9, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 514 bytes, done.
Total 5 (delta 3), reused 0 (delta 0)


-----> Heroku receiving push
-----> Fetching custom buildpack... done
-----> Mono app detected
-----> Fetching Mono binaries
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
tar: Ignoring unknown extended header keyword `SCHILY.dev'
tar: Ignoring unknown extended header keyword `SCHILY.ino'
tar: Ignoring unknown extended header keyword `SCHILY.nlink'
-----> Vendoring mono 2.10.8
-----> building via /tmp/mono-GssQ/bin/mono /tmp/mono-GssQ/lib/mono/4.0/xbuild.exe /tmp/build_19y6xv4nr43ic/Nancy.Demo.Hosting.Self.sln
XBuild Engine Version 2.10.8.0
Mono, Version 2.10.8.0
Copyright (C) Marek Sieradzki 2005-2008, Novell 2008-2011.


Build started 01/03/2012 16:28:24.
__________________________________________________
Project "/tmp/build_19y6xv4nr43ic/Nancy.Demo.Hosting.Self.sln" (default target(s)):
Target ValidateSolutionConfiguration:
Building solution configuration "Debug|Mixed Platforms".
Target Build:
Project "/tmp/build_19y6xv4nr43ic/src/Nancy.Demo.Hosting.Self.csproj" (default target(s)):
Target PrepareForBuild:
Configuration: Debug Platform: x86
Created directory "bin/"
Created directory "obj/x86/Debug/"
Target CopyFilesMarkedCopyLocal:
Copying file from '/tmp/build_19y6xv4nr43ic/lib/Nancy.dll' to '/tmp/build_19y6xv4nr43ic/src/bin/Nancy.dll'
Copying file from '/tmp/build_19y6xv4nr43ic/lib/Nancy.Hosting.Self.dll' to '/tmp/build_19y6xv4nr43ic/src/bin/Nancy.Hosting.Self.dll'
Target GenerateSatelliteAssemblies:
No input files were specified for target GenerateSatelliteAssemblies, skipping.
Target CoreCompile:
Tool /tmp/mono-GssQ/bin/dmcs execution started with arguments: /noconfig /debug:full /debug+ /optimize- /out:obj/x86/Debug/Nancy.Demo.Hosting.Self.exe Program.cs TestModule.cs /target:exe /define:"DEBUG;TRACE" /main:Nancy.Demo.Hosting.Self.Program /platform:x86 /reference:/tmp/mono-GssQ/lib/mono/4.0/System.dll /reference:../lib/Nancy.dll /reference:../lib/Nancy.Hosting.Self.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/System.Xml.Linq.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/System.Data.DataSetExtensions.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/Microsoft.CSharp.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/System.Data.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/System.Xml.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/System.Core.dll /reference:/tmp/mono-GssQ/lib/mono/4.0/mscorlib.dll /warn:4
Target _CopyDeployFilesToOutputDirectoryAlways:
Creating directory '/tmp/build_19y6xv4nr43ic/src/bin/Views'
Copying file from '/tmp/build_19y6xv4nr43ic/src/Views/staticview.html' to '/tmp/build_19y6xv4nr43ic/src/bin/Views/staticview.html'
Target _CopyAppConfigFile:
Copying file from '/tmp/build_19y6xv4nr43ic/src/app.config' to '/tmp/build_19y6xv4nr43ic/src/bin/Nancy.Demo.Hosting.Self.exe.config'
Target DeployOutputFiles:
Copying file from '/tmp/build_19y6xv4nr43ic/src/obj/x86/Debug/Nancy.Demo.Hosting.Self.exe.mdb' to '/tmp/build_19y6xv4nr43ic/src/bin/Nancy.Demo.Hosting.Self.exe.mdb'
Copying file from '/tmp/build_19y6xv4nr43ic/src/obj/x86/Debug/Nancy.Demo.Hosting.Self.exe' to '/tmp/build_19y6xv4nr43ic/src/bin/Nancy.Demo.Hosting.Self.exe'
Done building project "/tmp/build_19y6xv4nr43ic/src/Nancy.Demo.Hosting.Self.csproj".
Done building project "/tmp/build_19y6xv4nr43ic/Nancy.Demo.Hosting.Self.sln".


Build succeeded.
0 Warning(s)
0 Error(s)


Time Elapsed 00:00:01.0730530
-----> Discovering process types
Procfile declares types -> local, web
-----> Compiled slug size is 78.3MB
-----> Launching... done, v18
http://deep-moon-1452.herokuapp.com deployed to Heroku


To git@heroku.com:deep-moon-1452.git
4d9e07b..6ce44e4 master -> master

When I started this post I wanted to reflect on the interesting year I've had. I wanted to talk about how launching a start-up had taught me so much about business, technology and myself. I wanted to go on and cover how it has also meant making sacrifices, such as missing Glastonbury even when I had a ticket and cutting back on spending, blogging, open source and conference speaking - the things I really enjoy. Finally, I wanted to end with how start-up life has sometimes meant working in isolation and at times being demotivated and stressed with how everything was going.

However, while it's important for me to look back I also want to focus on looking forward which can be summed up by this tweet:

#2012 Release more. Blog more. Drink more. Earn more.
— Ben Hall (@Ben_Hall) January 1, 2012

To get the ball rolling, interested in hiring me? Send me an email at blog@benhall.me.uk