Hey dudes, In this post I will show you how to create a simple push notification server using nodejs and socket.io. Generally if we want update some contents inside a web application, we are used to refreshing the page using a javascript timer or simply we do some iterated AJAX requests to a web service so we can fetch constantly updated data. This could be a good solution for a small number of users , but what is going to happen if you got a big amount of geeks at the same time on your website?
As you should know, you can get the weight of any AJAX request summing the header (client) request + the header (server) response + the weight of data retrieved (you can get more info here websocket performance). So if you have a huge number of users on your web application doing many AJAX requests trying to load updated content, even if there are some AJAX requests that could not contain any new data, your server is forced to support a big number of unnecessary requests simulating a long polling connection. To solve this big issue you may have just two solution:
- you are GOOGLE so your have a big server farm that allows you to do any kind of rubbish
- you can smartly use a crossbrowser solution building a clean push notification server
I will show you how to build a simple and clean pushing notification server so let’s start to code. First you need to install nodejs on your server. Then for this example we need to import socket.io and xml2js nodejs modules in our project. You can use “npm” (node package manager) command by your console to install any kind of nodejs module like this:
$ npm install xml2js
So you have to create the server.js file that contains all the nodejs javascript that will run on the server. Here it is the source code:
, io = require('socket.io').listen(app)
, xml2js = require('xml2js')
, parser = new xml2js.Parser()
, fs = require('fs');
// creating the server ( localhost:8000 )
app.listen(8000);
// on server started we can load our client.html page
function handler ( req, res ) {
fs.readFile( __dirname + '/client.html' ,
function ( err, data ) {
if ( err ) {
console.log( err );
res.writeHead(500);
return res.end( 'Error loading client.html' );
}
res.writeHead( 200 );
res.end( data );
});
};
// creating a new websocket to keep the content updated without any AJAX request
io.sockets.on( 'connection', function ( socket ) {
// watching the xml file
fs.watch( 'example.xml', function ( curr, prev ) {
// on file change we can read the new xml
fs.readFile( 'example.xml', function ( err, data ) {
if ( err ) throw err;
// parsing the new xml datas and converting them into json file
parser.parseString( data );
});
});
// when the parser ends the parsing we are ready to send the new data to the frontend page
parser.addListener('end', function( result ) {
// adding the time of the latest update
result.time = new Date();
socket.volatile.emit( 'notification' , result );
});
});
Then you are ready to create a simple example.xml file containing all your data:
<test>
<sample>Hello world!</sample>
</test>
At last you are free to build your frontend page client.html where your data will be pushed anytime your example.xml will change.
<head>
<!--
* Author: Gianluca Guarini
* Contact: gianluca.guarini@gmail.com
* Website: http://www.gianlucaguarini.com/
* Twitter: @gianlucaguarini
-->
<title>Push notification server</title>
</head>
<body>
<div id="container">Try to change your xml data to update this content</div>
<script src="socket.io/socket.io.js"></script>
<script src="http://code.jquery.com/jquery-1.7.1.min.js"></script>
<script>
// creating a new websocket
var socket = io.connect('http://localhost');
// on every message recived we print the new datas inside the #container div
socket.on('notification', function (data) {
$('#container').html(data.test.sample[0]);
$('time').html('Last Update:' + data.time);
});
</script>
</body>
</html>
I made this video so you can see how it should work:
This is a simple and crossbrowser solution just because thanks to socket.io, we create a full duplex connection between the html page and the server so it will push new data only when the xml file changes, by using the new websocket API and then the flash technology for the fallback ( read socket.io support for more info ).
Download the files used in this tutorial
If you liked this post you can read my new article about how to bring your Nodejs push notification server to the next level reading from a MySql database.
Wow. Excellent example
Great post, thanks! Can’t wait to try it.
Though I wish you had a mic for the excellent video.
thanks bro..my english is still not perfect for the mic..but I will
i have a little problem, can you help me please?
text in web didn’t change to “hello world” and i got this on server.js
debug – setting request GET /socket.io/1/xhr-polling/15051029411184320447?t=1330421918273
debug – setting poll timeout
debug – discarding transport
debug – cleared close timeout for client 15051029411184320447
debug – clearing poll timeout
debug – xhr-polling writing 8::
debug – set close timeout for client 15051029411184320447
Can you tell me on which browser and OS you are you running this example?
Firefox 10.0.1 on Ubuntu 10.4
I think it is a Socket.io bug on Linux + Firefox. Follow this post
https://github.com/LearnBoost/socket.io/issues/548
Hey Gianluca,
I am trying out your example and everything works only on localhost. In another words, the nodeJS or socket.io didn’t work with remote clients, anyone who isn’t on the same machine. I am quite puzzled at the moment.
I tried to host it on localhost on Windows XP, and have one of my friend access through the Lan, or I host on a Node server with CentOS 5.5 and tried to access it.
I would end up with following
info – socket.io started
debug – served static content /socket.io.js
No matter how I modify the example.xml, both log and content do not change at all.
I tried to get rid of all the file change checking and parsing, just plain
socket.volatile.emit( ‘notification’ , ‘test’);
and it worked…so either fs or parser failed, I suspected that the fs module has some issues but this doesn’t feel right because it works with localhost.
Do you have any ideas in mind about why this might happen?
Resolved the issue just now. Sorry for posting the long post. But stuck at fs now. I’m trying to see if this is related to some bug in fs module.
fs.js:663
throw errnoException(errno, ‘watch’);
^
Error: watch ENOSYS
at errnoException (fs.js:636:11)
at FSWatcher.start (fs.js:663:11)
at Object.watch (fs.js:691:11)
at SocketNamespace. (/server.js:27:6)
Hello Tim,
I am having the same problem that you had…remote client do not work. My local host is ok. You mentioned in your next post that you have resolve the issue but you did not post your answer.
May I know how you solve this problem?
Hey Dude,
this was excellent work, i got an error
/var/www/nodejs/node_modules/socket.io/lib/manager.js:915
match = req.url.substr(0, resource.length);
^
i’m using Debian 6 OS.
can you please suggest me something.
Nitin ray
Thank you Nitin,
I think this could be a soket.io issue, try to submit your problem here https://github.com/LearnBoost/socket.io/issues
Then Let me know about it
Pingback: NodeJS « AgNetPro
Grande. Funziona a meraviglia.
Unico appunto, a riga 27,
Nodejs si incazza e vuole al posto di:
fs.watchFile( ‘example.xml’, function ( curr, prev ) {
questa:
fs.watch( ‘example.xml’, function ( curr, prev ) {
Altrimenti butta un throw.
Grazie della guida.
D.
Grazie per il consiglio Daniele
i try this example and… it works but i got this console log
data is null
[Parar en este error]
$(‘#container’).html(data.sample);
can you tell me what’s wrong?
The problem is fixed, I was using watchFile API instead of watch API, right now I just update the article and the repository.
Thanks to Daniele for the tip.
Cool, thanks a lot.. works perfect.
But think you have to change the client code var socket = io.connect(‘http://localhost’); to var socket = io.connect(‘http://localhost:8000′);
this is so awesome! nice and easy explanation.. thanks..
why do you have “node module” folder into socket.io folder..which extra package are you installing … i followed you blog and write code but it is giving me error!!! but after runing ur code which u attach with this blog this working fine.. only difference in extra node module folder(means extraa package) in socket.io folder..
m using window 7 and chrome browser
Got it.. you install socket.io-client package in to socket.io…which you forgot to tell in blog!!! BTW good blog i was searching such simple tutorial for socket.io …thnks a lot!!
It is in the first instruction of this tutorial “npm install socket.io”! The default socket.io module, installable by npm, it is already with socket.io-client package in to socket.io. I am glad you’ve appreciated the tutorial
Hey,
I just wondering if you can give some gud and simple tutorial about how to use ajax and node.js together.. i unable to find gud and correct tutorial abt it. if u can thn plz write some gud tutorial for it..it wil realy helpful..
Great work dude.
Actually, I’ve something complex a bit. I want the notifications to be delivered just for specific users (friends). How could that be? Friends are stored in a “Friend” table which contains each “member_id” with his “friend_id”s. Some keywords or general idea maybe enough for me to get started.
I suggest you to take a tour on http://nodetuts.com/. BTW it depends on what kind of DB you are using, with nodejs it would be better to use one of its DB modules https://github.com/joyent/node/wiki/Modules#wiki-database (I like mongoDB) and then trigger the events directly from the DB.
I’m using MySQL. I’ll check the link you’ve given me, if you don’t have something special about using MySQL with NodeJS, maybe some cons.
Thanks again Gianluca Guarini.
I have test your code and it works perfect. However, I have one problem that I can’t fix.
I need to listen instead of example.xml one file designated on the fly. That is, if the user open a socket on localhost:8000/site/entity/1234/ then, the file I will listen to is:
/var/nodejs/push/site/entity/1234.xml
How can I do it? Do you have any idea?
Excellent! Was thinking to myself: how does Apple’s push notifications work. Something must be listening. Can’t we do the same thing in the browser? Can’t the client run a server? So I started out on a search, specifically for use with node.js and came across this. Seems like the perfect starting point for something I want to do. Thanks!
This seems so cool – just not working for me!
I followed the instructions here, except that I changed the listening port (to 8888). When I hit the port, I get the message “Try to change your xml data to update this content”, so the (Linux Red Hat) server’s running OK. Now when I change the content of the XML file and save it on the server, nothing happens on the client browser. On the console, I do see heartbeat checks and, when I edit the file and save it, I see messages of this format:
debug – websocket writing 5:::{“name”:”notification”,”args”:[{"test":{"sample":["Hello! Changed? No? Come on!"]},”time”:”2012-09-07T14:41:45.884Z”}]}.
But I don’t see any change on the client browser.
I don’t know if this matters, but among the console messages, I see this:
debug – websocket writing 2::
debug – set heartbeat timeout for client ID7HJqkn0WIwHd9ULRrI
debug – got heartbeat packet
debug – cleared heartbeat timeout for client ID7HJqkn0WIwHd9ULRrI
Meaning, the number after “websocket writing…” is different where it seems to checking the heartbeat from where it is writing the notification.
What am I doing wrong?
You rock! Thank you kindly Mr. Guarini.
Hello world!
After taking out , the example works on my CentOS and WinXP machines.
Thanks Guarini.
Indeed tutorial is excellent!
But it not working for me! I am using Window 7 and Google Chrome. Whenever I change example.xml content in in browser does not changing at all.
Any comments?
I think it is not a problem related to the tutorial itself but It could be more related to your windows7 environment. Please check if you have all the assets installed correctly to run this example.
Agreed, great tutorial. Although I am having the same issue on Win7 Pro/Chrome (v23.0.1271.64).
Date/Time is updating everytime the XML is changed, but the other content in the browser doesn’t change. I can however see my updated text in the log … it just never gets pushed to the browser.
Any thoughts?
Scratch that – I have figured it out. I removed the elements from my xml just to leave:
Hello
This now works.
Thanks for the tutorial. it is helpful to me however, my problem is that whenever some records are inserted in MySQL table. i want to notify to particular user who is associated with that record. so, is it possible to keep watch/trigger on mysql data tables rather than file.
thanks
I have received many requests to show you how to realize push notification polling on a db so I am planning to make a new tutorial. In the meanwhile I could say that the only way that I know to watch a MySQL db is by a timer that could be set trough node anyway I would avoid this approach and I suggest you a Redis db for this kind of stuff.
Very nice tutorial, straight to the point.
But, (and this is missing on most tuts) how to send data to different clients that are connected? How to send right push to the right client with the right data?
Please could you help on this matter?
thanks again!
Hi thank you R.Dylan, this is just a demo to show you how node.js is able to broadcast data to different devices connected at same time on the same web page (this is perfect for news websites and so on..). Starting from this tutorial you could easily try to send the right data to each client connected using user credentials ( like username and password, or client id for example), obviously this should be done serverside thanks to node.js that deals with each client connection (you can collect all the connections in an Array or an object…). Soon I will publish another tutorial about Push notification from a MySql database and probably it could help you to understand what you could do with node.js under the hood!
Let’s keep in touch.
Pingback: Push notification server streaming on a MySQL database | Gianluca Guarini's Blog
Hi, It’s good tutorial… I have few questions, some are following..
How many web client(web socket) connections can persist by this server (depends on host machine, i guess)? Suppose if it is 1000, how to overcome?
and do you know, what is the life span of web socket persistance?
Hi and thanks for the interesting questions:
Yes you are right, the number of connection that a server could support depends on the host machine but I suggest you to read this post http://stackoverflow.com/questions/10161796/how-many-users-nodejs-socket-io-can-support for a deeper explanation of that.
You shouldn have big problems, Anyway you must stay under the ~3000 connections at same time
A websocket is persistent by its definition http://en.wikipedia.org/wiki/WebSocket . A websocket connection is closed when a user leaves the application (when the browser tab gets closed) or you can close it from the server after a certain period of time.
I hope it was clear enough
Your socket.io server is notified of the Node server event with the fs.watch() handler.
Is there any other way for the socket.io server to be notified of an event sent by the Node server ?
My Node server is doing a:
io.sockets.emit(‘message’, “The NodeJS http server says firstname: ” + postedData.firstname + ” lastname: ” + postedData.lastname);
in an http handler triggered on a curl request.
But the socket.io server is not picking up the message.
Have you read also this article http://www.gianlucaguarini.com/blog/push-notification-server-streaming-on-a-mysql-database/? Anyway your code seems to be wrong, try with something like that:
io.sockets.emit(‘message’,{
firstname: “The NodeJS http server says firstname: ” + postedData.firstname,
lastname: ” lastname: ” + postedData.lastname
});
Hi I have a small issue, I am running this under SuSE Linux, I get this error if I edit the xml and refresh the page:
debug – client authorized for
debug – websocket writing 1::
debug – websocket writing 5:::{“name”:”notification”,”args”:[{"test":{"sample":["Hi there friend! Whts up"]},”time”:”2013-02-07T05:34:35.030Z”}]}
/node_modules/xml2js/lib/xml2js.js:226
throw ex;
^
Error: Text data outside of root node.
Line: 6
Column: 10
Char: H
at error (/node_modules/xml2js/node_modules/sax/lib/sax.js:616:8)
at strictFail (/node_modules/xml2js/node_modules/sax/lib/sax.js:636:22)
at Object.write (/node_modules/xml2js/node_modules/sax/lib/sax.js:933:13)
at Parser.exports.Parser.Parser.parseString (/node_modules/xml2js/lib/xml2js.js:221:31)
at Parser.__bind [as parseString] (/node_modules/xml2js/lib/xml2js.js:6:61)
at /noderoot/push_notification_server/server.js:32:10
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
I am not using localhost however, I am using a different IP in the html page:
var socket = io.connect(“http://xxx.xxx.xxx.xxx”);
I suspect this could be causing the problem, any ideas on how to do this but not via localhost? Thanks!
I exactly have same error above. Do you have any idea?
*Sorry, please delete my previous comment. I intended to reply this comment.*
Hi I don’t know what are you doing there , but I have tried it on my ubuntu 12.10 http://tinypic.com/view.php?pic=iee1xz&s=6 and everything works correctly, Have you set the right port inside the server.js file ( app.listen(8000); ) ? What does it mean “I am not using localhost however, I am using a different IP in the html page”? Which nodejs version are you using?
We’ve got the same problem and i think problem is caused by inotify under Linux.
fs.watch or fs.watchFile uses
On Linux systems, this uses inotify.
So I logged the events, and i saw fs.watch sends 2 times change event in one time. So first change is true and it processes the data but second one is not true and it comes null.
So you cannot use this function because, it breaks the websocket connection when you use try catch
This problem could be easily solved using a debounce function but this kind of hacks are out of the scope of this tutorial.
Same guy as above ^^
Error I seem to be getting is:
/noderoot/push_notification_server/server.js:30
if ( err ) throw err;
^
Error: ENOENT, open ‘example.xml’
However the example.xml file is in the same directory as server.js, what could be wrong?
Hi. I have problem running this app on the other computers in my network and on phone, which is connected via wireless on my network. It is only work on localhost, where is server running. I’m not familiar with javascript, so do you have any suggestions to solve this problem?
tnx.
Ciao Luka,
if you cannot connect your mobile to localhost, just use an emulator on your computer or run the script on your own server directly online. I am sorry but this is an advanced javascript hack and probably you could start from something simpler. Good luck
Pingback: Why to consider Node.js for your mobile web apps? - langoor.mobi Blog
do not create any effect on the browser in 7 as well as xp
try to download the newest version
You really make it appear so easy along with your presentation however I find this topic to be actually something which I feel I might by no means understand. It sort of feels too complex and extremely vast for me. I am having a look ahead in your next post, I will try to get the dangle of it!
When I changed the xml file I got this error in the console (I’m using FF 20 on Windows 7)
result.time = new Date();
^
TypeError: Cannot set property ‘time’ of null
at Parser. (C:\opt\cygwin\exe\home\george\dev\node\server.js:39:17)
at Parser.EventEmitter.emit (events.js:95:17)
at Parser.exports.Parser.Parser.parseString (C:\opt\cygwin\exe\home\george\dev\node\node_modules\xml2js\lib\xml2js.js:254:14)
at Parser.parseString (C:\opt\cygwin\exe\home\george\dev\node\node_modules\xml2js\lib\xml2js.js:6:61)
at C:\opt\cygwin\exe\home\george\dev\node\server.js:32:14
at fs.js:266:14
at Object.oncomplete (fs.js:107:15)
Which nodejs version are you using? To me it looks like a really easy problem due to your dev environment. Have you also tried an npm update inside the project folder?
Hello Gianluca,
I am sure there is a simple fix but I am stuck at the moment. I tried your sample but it only works for local client within my PC. I cannot get the push notification to work on a remote cleint. How to resolve this problem?
Thank you in advance for your help.
Oo
Thanks for the Excellent example! However i had issue when i run it 2nd times and displayed the error msg below:
events.js:72
throw er; // Unhandled ‘error’ event
^
Error: Text data outside of root node.
Line: 6
Column: 10
Char: w
at error (/Users/admin/Sites/opc/node_modules/xml2js/node_modules/sax/lib/sax.js:616:8)
at strictFail (/Users/admin/Sites/opc/node_modules/xml2js/node_modules/sax/lib/sax.js:636:22)
at Object.write (/Users/admin/Sites/opc/node_modules/xml2js/node_modules/sax/lib/sax.js:933:13)
at Parser.exports.Parser.Parser.parseString (/Users/admin/Sites/opc/node_modules/xml2js/lib/xml2js.js:264:29)
at Parser.parseString (/Users/admin/Sites/opc/node_modules/xml2js/lib/xml2js.js:77:45)
at /Users/admin/Sites/opc/server.js:32:14
at fs.js:266:14
at Object.oncomplete (fs.js:107:15)
It was no prob when execute 1st time and it can update immediate. Just happen 2nd time update the xml file and appear the error.