Start with the Express project

I created a new copy of the Express project from the previous tutorial, source of which you can find on GitHub.

This has got our simple route and single page of info, but the data list for the teams is hard-coded as a JSON object in the ‘routes/index.js’ file.

Install Connect and Mongoose

The easiest way to do this is to use the package.json file in the root of the project and npm. This method allows you to install all of the modules you need at once, or you can add some in later, like we are doing now. This is a useful habit to get in to.

Opening up the package.json file you should see something like this:

{
 "name": "application-name",
 "version": "0.0.1",
 "private": true,
 "scripts": {
  "start": "node app"
 },
 "dependencies": {
  "express": "3.0.0rc5",
  "jade": "*"
 }
}

We are going to add a couple of lines to the ‘dependencies’ section to include Connect and Mongoose, so that it now looks like this:

{
 "name": "application-name",
 "version": "0.0.1",
 "private": true,
 "scripts": {
  "start": "node app"
 },
 "dependencies": {
  "express": "3.0.0rc5",
  "jade": "*",
  "connect": "*",
  "mongoose": "*"
 }
}

Save this file and open a terminal prompt at the project folder and run:

npm install

This will read the dependency list from the package.json file and install any new packages into the project.

Setting up the Mongoose connection and defining the data model

In the root directory of the application we’ll create a ‘model’ folder to contain our database connection, schema information and any interactions with it.

In this folder we will start off by creating a file db.js with the following content:

var mongoose = require( 'mongoose' );

var teamSchema = new mongoose.Schema({
 country: String,
 GroupName: String
});
mongoose.model( 'Team', teamSchema );

mongoose.connect( 'mongodb://localhost/euro2012' );

Here we are doing four things:

1. “Requiring” Mongoose so that we can use it
2. Defining the schema for a team
3. Building a ‘model’ of this schema, called “Team”
4. Connecting to the database, in this case a MongoDB database called ‘euro2012′ on the ‘localhost’ server

The schema and model should both be familiar if you’ve seen my Mongoose & Node.js tutorial.

Getting data from the database

In the ‘model’ folder create a file teams.js. We will use this to search the database for teams with a given ‘GroupName’ and return a JSON response. If you had other ‘team’ related queries they could be placed in here – e.g. all data about a given team.

Our file will need an export function that:

1. Takes a ‘GroupName’ value as a parameter
2. Takes a callback function as a second parameter
3. Searches the MongoDB database for teams with the provided ‘GroupName’
4. Send the database output to the callback function.

We created a “Team.find()” function last time round, so by taking that, bringing in the ‘Team’ model as a local variable and  requiring Mongoose, teams.js should look something like this:

var mongoose = require('mongoose');

exports.teamlist = function teamlist(gname,callback){
 var Team = mongoose.model( 'Team' );
 Team.find({'GroupName':gname}, function (err, teams) {
  if(err){
   console.log(err);
  }else{
   console.log(teams);
   callback("",teams);
  }
 })// end Team.find
}// end exports.teamlist

Establish the Mongoose connection

Mongoose is designed to have a reusable connection created on application startup. It also makes sense to only define the schemas once. So as we start our app by calling app.js this is where we need to require db.js. Let’s add it in to the require section at the top of the file, just after requiring Express.

var express = require('express')
 , db = require('./model/db')
 , routes = require('./routes')
 , http = require('http')
 , path = require('path');

(Snipped for brevity, keep the rest of the app.js file intact).

Displaying the data on the webpage

When we created the static data version in Express we hardcoded the JSON object into routes/index like so:

exports.index = function(req, res){
 var strGroup = "D";
 res.render('index', {
  title: 'Test web page on node.js using Express',
  pagetitle: 'Hello there',
  group: strGroup,
  teams: [{"country":"England"},{"country":"France"},{"country":"Sweden"},{"country":"Ukraine"}]
 });
};

Replacing this with the MongoDB data via Mongoose is surprisingly easy. We need to:

1. Require the teams.js model file
2. Call the ‘teamlist’ export function, sending it:
   1. The ‘GroupName’ of teams we are searching for
   2. The res.render functions as a callback build the page – including the list of teams – when the database query completes

So let’s modify routes/index.js like so:

var teamdata = require('../model/teams');

exports.index = function(req, res){
 var strGroup = "D";
 teamdata.teamlist(strGroup,function(err,teamlist){
  res.render('index', {
   title: 'Test web page on node.js using Express and Mongoose',
   pagetitle: 'Hello there',
   group: strGroup,
   teams: teamlist
  });
 });
};

Again, this is very similar code to that used in my earlier Mongoose tutorial, but here the Express res.render() function is a much neater way of building the resulting web page.

See it working

In terminal cd into the root folder for this site and run it by typing

node app

Head over to localhost:3000 in your browser and you should see something like this:

If you’ve kept the console.log statements in the code, you should see verification of the data being retrieved from the database. Similar to this:

So there it is. Mongoose and Express together, greatly simplifying some of the tasks of development in Node.js

Now that we have the building blocks of Node.js, Express, MongoDB and Mongoose in place, next time we’ll start building a web app we can actually use in the real world.

What is Express?

Express is a popular web application framework for node. The first thing you’ll enjoy is that it simplifies the web server setup – including routing for static files – and creates a directory structure as a start point for organising your  code.

We’re going to use it as we’re looking at building a web application, whether that’s a one page app or a multipage site isn’t important right now.

Installing Express

First of all we need to install Express globally. So in terminal:

sudo npm install -g express

Creating a project

Now that we’ve got Express installed globally, we let’s use it to create a new project. Here’s we create a new folder for our Express project, and install Express and it’s dependencies.

mkdir testpage-express
cd testpage-express
express #this installs an instance of express into the directory using the default options
npm install #this installs the dependcies

You can override some of the default install options when creating a new project, for example to specify a different HTML templating engine or CSS compiler.

See the official Excpress.js guide for more info.

Start the project and view it in a browser

In terminal:

node app

This should display a confirmation in terminal “Express server listening on port 3000″. 3000 is the default port used by Express, so head over to localhost:3000 in your browser and you should see something like this:

Nice. That was pretty easy right?

If you look at the page you may notice that it’s not using the default browser font, meaning there must be a stylesheet included.

Take a look back into the terminal that you use to start the process and you should see confirmation that it has delivered the URL ‘/’ and the stylesheet ‘/stylesheets/style.css’.

So there we go, a few terminal commands and you’ve got a node.js webserver and framework in place. More excitingly we’ve got that basis for a web-app just waiting for us!

Next time we’ll take a look under the hood and build our simple site in Express, using the built in routing and template engines.

Get the source code

The source code for this and my other tutorials on Node.js is available on GitHub here: https://github.com/simonholmes/knowing-node/

You get can get them here: https://github.com/simonholmes/knowing-node/

Here’s an easier way.

Introducing Mongoose!

In their own words Mongoose provides – “elegant mongodb object modeling for node.js” and “solves common problems for real-world applications”. Essentially, for our purposes here, it exposes only what you need to see and does everything else behind the scenes.

Installing Mongoose

First things first, you’ll need to install Mongoose (assuming you already have Node.js, npm and MongoDB installed).

cd into the root directory of your site – I’ve made a copy of my “testpage-site” folder called “testpage-mongoose” – and enter:

sudo npm install mongoose

Now we’re ready to go!

Taking our code from last time as a start point, we are going to need to look at our controller file, and our model file.

Changing the model file

Most of our work here will be in the ‘model’ file, as we are dealing directly with the database, so let’s start there. For this I have created a new file ‘mongoose-data.js’ using ‘mongo-data.js’ from the previous tutorial as a template.

Connecting to MongoDB using Mongoose

This will be where Mongoose is going to manage the connection to our MongoDB database, so let’s start off with this code snippet:

var mongoose = require('mongoose'),
 db = mongoose.createConnection('localhost', 'euro2012');
db.on('error', console.error.bind(console, 'connection error:'));

Note that we now require ‘mongoose’ instead of ‘mongodb’. We have also very easily created a connection to our database ‘euro2012′ on the server ‘localhost’. Finally, we have created a default error handler to catch any connection errors.

A note on connecting and disconnecting

As Aaron Heckmann has pointed out in the comments – and he knows what he’s talking about – it is “ best to open your db connection once, at application startup, and leave it open until your app shuts down”.

Using Node.js and Mongoose to query MongoDB

Let’s dive straight into the meat of the code and see what is happening in our ‘export’ function. The export function itself expects two parameters, a variable ‘gname’ that we will search for in the database, and a callback function that we’ll execute when the database query is complete.

exports.teamlist = function(gname,callback){
 db.once('open', function(){
  var teamSchema = new mongoose.Schema({
   country: String,
   GroupName: String
  });
  var Team = db.model('Team', teamSchema);
  Team.find({'GroupName':gname}, function (err, teams) {
   if(err){
    onErr(err,callback);
   }else{
    mongoose.connection.close();
    console.log(teams);
    callback("",teams);
   }
  })// end Team.find
 });// end db.once open
};

First, we open the database in the db.once wrapper, and send the rest of the code through as a callback to be executed once the database connection is open.

Setting a schema and building a model

Mongoose requires a schema to be set for the data. This enables Mongoose to then build a model framework for the information we will be dealing with. Ours is pretty simple here, using just a couple of strings – see the Mongoose docs for more info on schemas.

Here’s our schema:

var teamSchema = new mongoose.Schema({
 country: String,
 GroupName: String
});

And here’s the simple one liner to build a model ‘Team’ from the schema:

var Team = db.model('Team', teamSchema)

Using Mongoose “find” to query MongoDB

The “find” method of Mongoose is very similar to that of the native driver, and is happy to accept the same filter object as an argument. You simply apply the find method to the model you’ve just created, sending the filter object/search query as the first parameter, and a callback function as the second. This callback function has two possible parameters, an error object and the returned data following the query.

Team.find({'GroupName':gname}, function (err, teams) {
 if(err){
  onErr(err,callback);
 }else{
  mongoose.connection.close();
  console.log(teams);
  callback("",teams);
 }
})// end Team.find

The Mongoose ‘find’ method automatically returns an array of JSON objects, reducing the amount of coding effort you have to go to in order to make the data more usable. Our console.log from the above snippet returns something along the lines of:

[ { _id: 50011ff173ca2c992a7bae46,
 country: 'England',
 GroupName: 'D' },
 { _id: 500122cb73ca2c992a7bae48,
 country: 'France',
 GroupName: 'D' },
 { _id: 5001230673ca2c992a7bae49,
 country: 'Sweden',
 GroupName: 'D' },
 { _id: 5001231073ca2c992a7bae4a,
 country: 'Ukraine',
 GroupName: 'D' } ]

Once we have the data we send it through to the callback function provided when the exports function was called.

Catching errors

The last part of our model file deals with catching and returning errors. You can see that if there is an error with the find operation we call this function onErr(err,callback).

var onErr = function(err,callback){
 mongoose.connection.close();
 callback(err);
};

This simply closes the database connection and then sends the error object back to the original callback function.

Using the data in the web page

To get the returned JSON data into our website we need to set up the controller. I have set up a new copy of the controllers/home-mongo.js file and called it home-mongoose.js.

Here is the full source code for that file:

var template = require('../views/template-main');
var mongo_data = require('../model/mongoose-data');

exports.get = function(req, res) {
 var strGroup = "D";
 mongo_data.teamlist(strGroup,function(err,teamlist){
  if(!err){
   var strTeam = "", i = 0, teamCount = teamlist.length;
   for (i = 0; i<teamCount;) {
    strTeam = strTeam + "<li>" + teamlist[i].country + "</li>";
    i = i+1;
   }
   strTeam = "<ul>" + strTeam + "</ul>"
   res.writeHead(200, {'Content-Type': 'text/html'});
   res.write(
    template.build("Test web page on node.js","Hello there","<p>The teams in Group " + strGroup + " for Euro 2012 are:</p>" + strTeam));
   res.end();
  }
  else{
   res.writeHead(200, {'Content-Type': 'text/html'});
   res.write(
    template.build("Oh dear","Database error","<p>Error details: " + err + "</p>"));
   res.end();
  }
 });
};

First of all, we need to require the new model file.

Secondly, we have changed the way the main ‘get’ function works very slightly by providing the name of the group of teams we want. In this case “D”. This means that we will search the database for this and return the relevant teams (this is what we did in the model at the start of this post).

After that, looping through the list and sending the output to the template is pretty much the same as before.

That’s it!

Well, that’s nearly it. Don’t forget to change the router to require the ‘home-mongoose’ controller instead of ‘home-mongo’.

Now we have Mongoose acting as an interface to our MongoDB database, taking much of the coding pain away and simplifying the process.

Get the source code

The source code for this and my other tutorials on Node.js is available on GitHub here: https://github.com/simonholmes/knowing-node/

Fortunately there is a very easy way of managing your node version, using the Node binary manager module ‘n’.

1: Check your current version of Node.

$node -v
v0.6.12

2: Clear your npm cache

sudo npm cache clean -f

3: Install ‘n’

sudo npm install -g n

4: Upgrade to a later version (this step can take a while) You can specify a particular version like so:

sudo n 0.8.11

Or you can just tell the manager to install the latest stable version like so:

sudo n stable

5: Check the running version of Node to verify that it has worked:

$node -v
v0.8.11

If the version doesn’t number output in step 5 isn’t what you were expecting, you may need to reboot.

If you have just arrived at this page, and want the context and framework for the rest of the files, check out my posts Creating an MVC framework for our Node.js page – getting ready for scalability and How to create a MongoDB database

In the MVC approach we will need to look at the ‘model’ code and the ‘controller’ code.

Make your code ‘non-blocking’

We will start off by looking at the controller. In a traditional stack you say:

1. I’m going to go and get the data from the database
2. When I’ve got it I will do this with it

This is no good for Node.js as it is a ‘blocking’ operation – nothing else can happen until you have the data back. This is essentially what we did in the MVC framework with this bit of code in home.js.

var teamlist = test_data.teamlist;
var strTeam = "", i = 0;
for (i = 0; i<teamlist.count;) {
 strTeam = strTeam + "<li>" + teamlist.teams[i].country + "</li>";
 i = i+1;
}

In that particular instance it was correct to do it this way, as we knew that the data-retrieval operation was non-blocking – we instantly pushed back a JSON object into the teamlist variable.

As pulling data from a database is a ‘blocking’ operation we need to handle it differently. We need to say:

1. Go get this data from the database
2. When you’ve got it, do this with it
3. Don’t disturb me until you’re done, I’ve got other things to do

Sound like a callback function to you? It does to me!

Creating the controller

So let’s take a look at the new controller code and see what’s going on. I have created a new file for this, called home-mongo.js.

var template = require('../views/template-main');
var mongo_data = require('../model/mongo-data');

exports.get = function(req, res) {
 mongo_data.teamlist("D",function(err,teamlist){
  if(!err){
   var strTeam = "", i = 0;
   for (i = 0; i<teamlist.count;) {
    strTeam = strTeam + "<li>" + teamlist.teams[i].country + "</li>";
    i = i+1;
   }
   strTeam = "<ul>" + strTeam + "</ul>"
   res.writeHead(200, {'Content-Type': 'text/html'});
   res.write(
   template.build("Test web page on node.js","Hello there","<p>The teams in Group " + teamlist.GroupName + " for Euro 2012 are:</p>" + strTeam));
   res.end();
  }
  else{
   res.writeHead(200, {'Content-Type': 'text/html'});
   res.write(
   template.build("Oh dear","Database error","<p>Error details: " + err + "</p>"));
   res.end();
  }
 });
}

This is very similar to our home.js file, just a few important differences.

Firstly, we ‘require’ a different model file, mongo-data.js that we will look at shortly.

Secondly, rather than pulling the data and doing something with it, we are using a callback function to say “do this with the data when you’ve got it”. What we are doing with the data is exactly the same as before. Note that we are sending the string “D” through as a variable; this is the string we will be searching for in the database.

Thirdly, through the callback function we are also sending instructions for what to do if an error occurs getting the data.

Using Node.js to query the MongoDB data

Before we look at the code, let’s think about what we’re looking to achieve:

1. Connect to the MongoDB database
2. Have a function that is expecting a string variable and a callback
3. Search the database for the variable (a specific key of ‘GroupName’)
4. Create a JSON object if successful
5. Catch the error if not successful
6. Pass the JSON object or the error to the provided callback function
7. Close the database connection

Easy? Not as hard as you might think. So here’s the code for the mongo-data.js file:

var mongo = require('mongodb'),
 Server = mongo.Server,
 Db = mongo.Db;
var server = new Server('localhost', 27017, {auto_reconnect: true});
var db = new Db('euro2012', server);

var onErr = function(err,callback){
 db.close();
 callback(err);
};

exports.teamlist = function(gname,callback){
 db.open(function(err, db) {
  if(!err) {
   db.collection('teams', function(err, collection) {
    if(!err){
     collection.find({'GroupName':gname}).toArray(function(err, docs) {
      if(!err){
       db.close();
       var intCount = docs.length;
       if(intCount > 0){
        var strJson = "";
        for(var i=0; i<intCount;){
         strJson += '{"country":"' + docs[i].country + '"}'
         i=i+1;
         if(i<intCount){strJson+=',';}
        }
        strJson = '{"GroupName":"'+gname+'","count":'+intCount+',"teams":[' + strJson + "]}"
        callback("",JSON.parse(strJson));
       }
      }
      else{onErr(err,callback);}
     });//end collection.find
    }
    else{onErr(err,callback);}
   });//end db.collection
  }
  else{onErr(err,callback);}
 });// end db.open
};

The first five lines are all about setting up the MondoDB server and connecting to our database. You can learn more about that on the MongoDB Native Driver GitHub page.

Then we have a function onErr that we will call if we catch an error at any point.

Now comes the fun part! The MongoDB Native driver comes with a number of non-blocking functions built in. We are using three, to:

1. Open our database (db.open)
2. Find the ‘collection’ we specify (db.collection)
3. Find the documents in that collection that match the query we specify (collection.find().toArray)

These all run as nested callbacks, which is where the code can get tricky to follow. To help with this I recommend adding a comment at the end of each callback function, like you can see in the code.

Counter-intuitively, the collection.find method doesn’t actually execute the query, that’s done by the toArray method. So that’s where we pass our next callback function. This part is where we take the docs returned by the database and use them to create a JSON object in the format we want.

The finished JSON object should look like this:

{"GroupName":"D","count":4,"teams":[{"country":"England"},{"country":"France"},{"country":"Sweden"},{"country":"Ukraine"}]}

When the JSON object is complete we finally use the callback function, sending the object through.

The last thing we need to do – if you’ve been playing along at home – is update our router file to call the controller.

So in router.js file just change the home controller to be home-mongo, like so:

if (path === '/' || path === '/home') {
 require('./controllers/home-mongo').get(req, res);
}

If you run the site, you shouldn’t really see any change, as all of the data manipulation is happening behind the scenes.

Next up we’ll see if we can simplify this process by using Mongoose, Node.js and MongoDB together.

Get the source code

The source code for this and my other tutorials on Node.js is available on GitHub here: https://github.com/simonholmes/knowing-node/

Step 1: don’t create a database

Hang on Simon, are you sure you’ve got that right? Yep! There is no create database command in MongoDB. If you come from a traditional stack, like me, this is quite a weird concept. Surely the first thing to do is create the actual file and define a schema? Not so with MongoDB. With MongoDB you create a database the first time you save data into it.

Let’s try that out.

When we installed MongoDB we tested the installation by displaying a list of the databases on the machine. So let’s drop into the Mongo shell and do that. A couple of terminal commands (don’t type the ‘$’ or the ‘>’ symbols, here ‘$’ indicates a terminal prompt and ‘>’ a Mongo prompt):

$ mongo
> show dbs

You should see something like this get returned:

Step 2: “Use” the database you wish to create

As we mentioned earlier, rather than explicitly creating a database, MongoDB will create a database when it first has data saved to. In order to save data, we need to connect to our new database even though it doesn’t exist yet. Sounds odd, but it’s really easy. Simply enter this command into the Mongo shell:

use euro2012

In this instance we are going to use a database called “euro2012″.

If you use the show dbs command again, you will see that the database hasn’t been created yet.

Step 3: Insert some data into the MongoDB database

Here’s another difference of MongoDB from traditional relational databases: there are no tables, rows or columns. In their place, MongoDB uses collections and objects. Think of collections as tables, and objects as table rows. The main difference is that we’re not stuck into a rigid column structure.

Here’s how the MongoDB guys explain it themselves (source):

• A database holds a set of collections
• A collection holds a set of documents
• A document is a set of fields
• A field is a key-value pair
• A key is a name (string)
• A value is a
  • basic type like string, integer, float, timestamp, binary, etc.,
  • a document, or
  • an array of values

You can explicitly create collections, but they can also be created the first time you use them. We’ll do the latter here. In the Mongo shell enter the following:

db.teams.save({country:"England",GroupName:"D"}) 

That will create a new collection called “teams”, and save an object with two fields “country” and “GroupName”. Crucially, this will now also save our database. Check that it has been created by running show dbs  again.

You should now see that our new database has been added to the list.

Now add the rest of the data using the following command in the Mongo shell:

db.teams.save({country:"France",GroupName:"D"})
db.teams.save({country:"Sweden",GroupName:"D"})
db.teams.save({country:"Ukraine",GroupName:"D"})

Step 3: testing by reading the data back out.

Having put some data in, and tested that the database has been saved, we should now verify that the data we entered has been saved as we expected it to be.

Run a simple “find” on the collection to return all objects by entering this into the Mongo shell:

db.teams.find()

This should return you something like this:

Perfect! We’ve created a MongoDB database and saved our information into it

Next up, we’ll take a look at how to use this data in our site, by connecting MongoDB to Node.js.

There are a whole bunch of databases that we could use, but we’re going to go with a popular choice for Node.js development, MongoDB. In their own words:

MongoDB (from “humongous”) is a scalable, high-performance, open source NoSQL database.

Installing MongoDB on Ubuntu

If you Google how to do this you might get put off, as you’ll find a load of pages with different workarounds to get MongoDB installed correctly, Q&A threads on fixing install errors etc. Fortunately this has all been fixed, and it’s now super easy to install MongoDB on Ubuntu.

Open a terminal window and enter:

sudo apt-get install mongodb

That’s it! As soon as that’s finished downloading and processing you’ll have MongoDB installed.

Testing your MongoDB install

You want to test to make sure that it’s installed correctly? Fair enough. Let’s start it up as an Upstart service (more on Upstart at another time).

Start the MongoDB service

In terminal enter:

sudo service mongodb start

You should now see a confirmation along the lines of “mongodb start/running, process 15251″

Using the MongoDB command line shell

MongoDB comes with a command line shell. We’ll use this to verify everything is running correctly.

To enter the MongoDB shell, in terminal simply enter:

mongo

This should tell you the shell version and connect you to the ‘test’ database that ships with Mongo. You should see something like this:

The “>” indicator at the start of a new line shows that you are now working in the MongoDB shell, not the Ubuntu command line.

Listing all databases

While we’re here, let’s see what MongoDB database have been created during installation.

In the MongoDB shell, enter this command to list all databases:

show dbs

You should then see something like this screenshot, showing two databases and their sizes.

Exiting the MongoDB shell

Press Ctrl+D to exit the MongoDB shell and return to the terminal command line interface.

How to install the native MongoDB drivers for Node.js

So having got MongoDB installed, we want to make sure that our Node.js applications will be able to use it. For this we need to install the native driver. Note: there are actually loads of middleware modules to simplify and extend the possibilities which we’ll cover at another time. For now we’ll stick to the basics so that we’ve got a solid foundation to build upon.

To install the native MongoDB drivers we’ll use NPM, so head to terminal and enter:

npm install mongodb --mongodb:native

This will run through the installation process, downloading everything we need to get MongoDB and Node.js talking to each other!

Happy days! Next up we’ll be creating a MongoDB database so that we can actually use it with our Node.js MVC site.

If you’ve spent much time as a developer you’ll know that the key to maintainability is separation of concerns across your projects. If you’ve been developing a long time, since before such architectures were really discussed, then you probably learned the hard way. Anyhow, what we mean, and what we want to achieve is to separate out our data, logic and presentation.

MVC – The Model-View-Controller approach

A very popular approach for web sites & web apps – and the one we’ll be working with – is MVC, which stands for model, view, controller. This makes distinctions between the model (data), view (presentation) and controllers (logic).

The diagram below shows how a standard MVC request/response loop looks:

In short these are the steps:

1. Request comes in to the Controller
2. Controller makes a request to the model
3. Model returns a response to the controller
4. The controller (normally) performs some operations using the response from the model
5. The controller sends a response to the view
6. The view renders the response

In Node.js, because we are also creating the webserver too, we need to add three steps in at the beginning:

1. Request comes in to server
2. Server sends request to router
3. Router sends request to appropriate controller

Looking at this, we can see that we are going to need the following objects (or collections of objects):

1. Server – to listen to and respond to HTTP requests
2. Router – to send the incoming requests to the correct controller
3. Controllers – to perform operations & interrogate the data
4. Model – to provide the data
5. Views – to provide the HTML rendering we’re going to see in the browser

Building our framework

To keep things clean, lets create a new folder to put our site in.

Creating the node.js server

Create a file called server.js with the following content:

var http_IP = '127.0.0.1';
var http_port = 8899;
var http = require('http');
var server = http.createServer(function (req, res) {
 require('./router').get(req, res);
});// end server()
server.listen(http_port,http_IP);
console.log('listening to http://' + http_IP + ':' + http_port);

The only new thing there, is that rather than outputting anything directly here, we are calling in a ‘router’ module and sending it the request & response object. This makes sense if you look back at the steps the application needs to take: server sends request to router.

Building the router

In the same folder as the server.js file, create a new file called router.js and save it with the following content:

var url = require('url');
var fs = require('fs');

exports.get = function(req, res) {
 req.requrl = url.parse(req.url, true);
 var path = req.requrl.pathname;
 if (/.(css)$/.test(path)){
  res.writeHead(200, {'Content-Type': 'text/css'});
  fs.readFile(__dirname + path, 'utf8', function (err, data){
   if (err) throw err;
   res.write(data, 'utf8');
   res.end();
  });
 } else {
  if (path === '/' || path === '/home') {
   require('./controllers/home').get(req, res);
  } else {
   require('./controllers/404').get(req, res);
  }
 }
}

In the router we are doing a few different things:

1. Importing the URL and FileSystem (fs) modules that come as part of your node.js install
2. Exporting a function called “get” – this allows our server file to use this function, passing the request and response objects through
3. Getting the path of the URL request
4. Testing to see whether the request is for a CSS file and loading it in if so. There are a number of different ways of handling static files, but this method is fine for what we’re doing here
5. Sending the request to the correct controller, if it’s not for a CSS file

At this point in our development, it’s number 5 we are interested in. We are checking for two URL paths ‘/’ and ‘/home’ which will route to a controller called ‘home’. We then send any other URL requests to a controller called ’404′. Let’s take a look at these controllers.

Building the controllers

Create a folder in the root of your site, called ‘controllers’. Inside this folder create a file called ‘home.js’ with the following content:

var template = require('../views/template-main');
var test_data = require('../model/test-data');

exports.get = function(req, res) {
 var teamlist = test_data.teamlist;
 var strTeam = "", i = 0;
 for (i = 0; i<teamlist.count;) {
 strTeam = strTeam + "<li>" + teamlist.teams[i].country + "</li>";
 i = i+1;
 }
 strTeam = "<ul>" + strTeam + "</ul>"
 res.writeHead(200, {'Content-Type': 'text/html'});
 res.write(
 template.build("Test web page on node.js","Hello there","<p>The teams in Group " + teamlist.GroupName + " for Euro 2012 are:</p>" + strTeam));
 res.end();
}

Right at the top of this file you can see that we are including the rest of our MVC structure the model and the view. Referring back to the diagram we remember that the controller should do two main things:

1. The controller (normally) performs some operations using the response from the model
2. The controller sends a response to the view

Performing operations on model data

Our task here is quite simple: get a data object (teamlist) and create a HTML string putting the items into an unordered list. In the next step we’ll see how we create this data.

Sending a response to the view

We will use a view (think template) called template-main, which we will create in a few minutes. Note that through the template.build function that we are only sending three content items; within the controller there is no reference to how this content will be displayed. In this example you will notice that we are sending through the HTML for the <ul> list, there are arguments against having even this level of markup in a controller, but we’re going for a simple example here so let’s keep it in!

Getting data from the model

In a future post we’ll see how we can extend this demo by getting the data out of a database, but for the sake of remaining focused on the point right now we’ll fake it by creating and returning a JSON object.

So, create a folder in the root of your site called ‘model’. Inside this folder create a new file called ‘test-data.js’ with the following content:

var thelist = function() {
 var objJson = {"GroupName":"D","count":4,"teams":[{"country":"England"}, {"country":"France"}, {"country":"Sweden"}, {"country":"Ukraine"}]};
 return objJson;
};

exports.teamlist = thelist();

Through the export function we are exposing the function thelist() which itself is returning a JSON object. This is how – for now – we will return a data object to the teamlist in our controller.

Using a view template to create the HTML page

Now that we have our controller pulling in some data from the model and doing something with it, we want to show the outcome in a webpage. We’ve seen that we’re going to send the data to a build function in a view template, so let’s create that now.

Create a folder in the root of your site called ‘views’. Inside this folder create a new file called ‘template-main.js’ with the following content:

exports.build = function(title, pagetitle, content) {
 return ['<!doctype html>',
 '<html lang="en">\n<head>n<meta charset="utf-8">\n<title>{title}</title>',
 '<link rel="stylesheet" href="/assets/style.css" />\n</head>',
 '<body><h1>{pagetitle}</h1>',
 '<div id="content">{content}</div>\n</body>\n</html>'
 ].join('\n')
 .replace(/{title}/g, title)
 .replace(/{pagetitle}/g, pagetitle)
 .replace(/{content}/g, content);
}

Here we are directly exposing the build function, which takes three parameters: title, pagetitle and content.

The first thing we are doing here is joining an array of strings, placing a new line ‘n’ between them. This creates one long string of HTML that will render on separate lines leaving the source code more readable. This string is the basis of our template, and contains a number of placeholders for the content items {title}, {pagetitle} and {content}.

Naturally then, the second step here is to replace these placeholders with the actual content sent through via the function arguments. The three replace functions are doing exactly that, running a very simple regular expression to globally change all occurrences found in the string. This means that you can have {title} in several places in your template, and each will get replaced. Note that we are also chaining the join and replace commands; the separate lines are for readability. Seeing as it works when laid out like this there is little sense in putting them all on one line and making the script harder to read.

Using a static CSS file

You may have noticed that in the HTML view template that we are referencing a static CSS file; remember in the router how we created a statement checking for the .css file extension? Now is the time to implement it.

In the root of your site create a folder called ‘assets’. Inside this folder create a text file called ‘style.css’ with the following content:

* {font-family:arial, sans-serif;}

This isn’t the most complex CSS file in the world, but the change in font is enough for us to know that it is working!

Creating a 404 page

The final piece of our site is to create our 404 page. We already have a suitable template – our template-main view – so we’ll re-use that. Our router is also already set to send people to a 404 if it doesn’t recognise the URL path requested. So all that remains for us to do is to create the controller.

In your ‘controllers’ folder create a new file called ’404.js’ with the following content:

var template = require('../views/template-main');
exports.get = function(req, res) {
 res.writeHead(404, {'Content-Type': 'text/html'});
 res.write(
 template.build("404 - Page not found","Oh noes, it's a 404","<p>This isn't the page you're looking for...</p>"));
 res.end();
}

You should be able to see what we’re doing here. Using the same template - views/template-main - we send it three items of content. The only difference this time is that we have changed the status code of the response – in the res.writeHead function – from 200 (Ok) to 404 (not found).

Let’s run the site!

That’s all the pieces of our MVC framework in place, so let’s try out our site. In terminal cd into the root folder of your site and run it with the command:

node server.js

Now head to 127.0.0.1:8899 in browser and you should see something like this:

Let’s test our other path from the router, by navigating to 127.0.0.1:8899/home

Finally, let’s test our 404 page, by going to a URL that we haven’t added to the router, for example 127.0.0.1:8899/test

Success!

So there we have it. We now have a working website in node.js written in a maintainable and scalable MVC framework. In future posts I will take a look at how to remove some of the pain by using the express.js framework, and also how to get the data from a MongoDB database.

Get the source code

The source code for this and my other tutorials on Node.js is available on GitHub here: https://github.com/simonholmes/knowing-node/

For these early tests it’s fine to just use a simple text editor, I’ve been using Gedit on Ubuntu. Just type gedit into terminal to get it started.

Create your file

Create a new text document and save it as app.js.  Enter in the text below and then we’ll take a look at what’s going on.

var http = require('http');
http.createServer(function (req, res) {
 res.writeHead(200, {'Content-Type': 'text/html'});
 res.write('<!doctype html>\n<html lang="en">\n' +
 '<head>\n<meta charset="utf-8">\n<title>Test web page on node.js</title>\n' +
 '<style type="text/css">* {font-family:arial, sans-serif;}</style>\n' +
 '</head>\n<body>\n<h1>Euro 2012 teams</h1>\n' +
 '<div id="content"><p>The teams in Group D for Euro 2012 are:</p><ul><li>England</li><li>France</li><li>Sweden</li><li>Ukraine</li></ul></div>' +
 '\n</body>\n</html>');
 res.end();
}).listen(8888, '127.0.0.1');
console.log('Server running at http://127.0.0.1:8888');

Ok great, so what are we looking at here?

Creating the webserver

The first thing to remember – and a big change to get used to if you’re coming over from a Microsoft IIS stack, or an Apache based stack – is that with Node.js there is no webserver until you create it. And that’s what our first two lines are doing.

Calling the HTTP module

Starting at the top, the very first thing we do is load in the pre-prepared HTTP module that ships as part of node. You’ll see this type of syntax a lot, as Node.js programming uses a lot of modules. As a best practice it’s a good idea to set the variable name to match the module name.

var http = require('http');

Create the server

Stripping out the callback function to make it a little more clear, we can look at the createServer method of HTTP like this:

http.createServer(function(req, res){ }).listen(8888, '127.0.0.1');

What we are doing here is creating a HTTP server to listen to requests (req) made to port 8888 on the IP address 127.0.0.1, and also send response objects (res)  back to the same IP and port.  The IP address 127.0.0.1  is common across many OS’s as the ‘localhost’ server.

We also output a message to the console to show that we’ve reached the end of the script and that the server is running:

console.log('Server running at http://127.0.0.1:8888');

But what about the middle bit?!

Okay, so in a nutshell what we’re doing here is responding to any request in exactly the same way. Let’s break that response down.

res.writeHead()

The first part of the response object is another item that you’re probably not used to really thinking about if you’re coming from a traditional stack where the web-server does all this for you. We need to send the HTTP headers. Here we’re  just sending the HTTP status code and content type. So to say that the request was successful, and that we’re sending back HTML content we type:

res.writeHead(200, {'Content-Type': 'text/html'});

res.write()

Next we want to send some content back to the response object. Here we’re doing it all as one long string:

res.write('<!doctype html>\n<html lang="en">\n' +
 '<head>\n<meta charset="utf-8">\n<title>Test web page on node.js</title>\n' +
 '<style type="text/css">* {font-family:arial, sans-serif;}</style>\n' +
 '</head>n<body>\n<h1>Euro 2012 teams</h1>\n' +
 '<div id="content"><p>The teams in Group D for Euro 2012 are:</p><ul><li>England</li><li>France</li><li>Sweden</li><li>Ukraine</li></ul></div>' +
 '\n</body>\n</html>');

res.end()

To tell the server that we’ve reached the end of the response object we type:

res.end()

Run your node.js site!

Starting your node.js server is nice is simple. In terminal cd into the directory that you saved your app.js file and type:

node app.js

All being well you’ll see your console message output to the terminal:

Open your node.js site in a browser

Go to your browser and navigate to http://127.0.0.1:8888. Cross your fingers, blink twice and you should see this:

Hurrah and huzzah! You’ve got a webpage running in Node.js

Next time we’ll take a look at how to build this in a more scalable way, creating a basic MVC framework in Node.js!

Get the source code

The source code for this and my other tutorials on Node.js is available on GitHub here: https://github.com/simonholmes/knowing-node/