Friday 31 July 2015

Node.js with MySQL


Hi everybody, in this blog we will cover MySQL connection with Node.js.

Node.js and MySQL is one of the necessary binding needed for any web application. MySQL is one of the most popular open source database in world and efficient as well. Almost every popular programming language like Java or PHP provides driver to access and perform operations with MySQL.

In this tutorial i am trying to cover code for learning and code for production. So if you know this already and looking for ready made code for production.

In this tutorial i am going to cover following points related to Node.js and MySQL.

Sample code to get started.
Code for Production.
Pool connection in MySQL

Sample code to get started


Create a file name package.json in your project directory:

package.json

{
  "name": "node-mysql",
  "version": "0.0.1",
  "dependencies": {
    "express": "^4.10.6",
    "mysql": "^2.5.4"
  }
}

Install dependencies using

npm install

Create a file connect.js and write code which connects to Database and perform SQL query.

var mysql      = require('mysql');
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : '< MySQL username >',
  password : '< MySQL password >',
  database : '<your database name>'
});

connection.connect();

connection.query('SELECT * from < table name >', function(err, rows, fields) {
  if (!err)
    console.log('The solution is: ', rows);
  else
    console.log('Error while performing Query.');
});

connection.end();

Make sure you have started MySQL on default port and changed the parameter in above code then run this code using

node connect.js

Code for production :


Above code is just for learning purpose and not for production payload. In production scenario is different, there may be thousands of concurrent users which turns into tons of MySQL queries. Above code won’t run for concurrent users and here is a proof. Let’s modify our code little bit and add Express routes in that, here it is.

test.js ( Change database settings in code )

var express    = require("express");
var mysql      = require('mysql');
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'root',
  password : '',
  database : 'address_book'
});
var app = express();

connection.connect(function(err){
if(!err) {
    console.log("Database is connected ... \n\n");  
} else {
    console.log("Error connecting database ... \n\n");  
}
});

app.get("/",function(req,res){
connection.query('SELECT * from user LIMIT 2', function(err, rows, fields) {
connection.end();
  if (!err)
    console.log('The solution is: ', rows);
  else
    console.log('Error while performing Query.');
  });
});
app.listen(3000);


Install siege in your system. I use this command to install it in Ubuntu.

apt-get install siege

then run our node and server and following command on two different terminal.

Run this command on one terminal

node test.js

On another terminal run this command

siege -c10 -t1M http://localhost:3000

Assuming you are running Node server on Port 3000.

Here is the output.



In above code, we are allowing it to run for standalone connection i.e one connection at a time but reality is bit different. You may get 100 or 1000 connection at one particular time and if your server is not powerful enough to serve those request then at least it should put them in queue.

Pool connection in MySQL :


Connection Pooling is mechanism to maintain cache of database connection so that connection can be reused after releasing it. In Node mysql, we can use pooling directly to handle multiple connection and reuse the connection. Let’s write same code with pooling and check whether it can handle multiple connection or not.

test.js

var express   =    require("express");
var mysql     =    require('mysql');
var app       =    express();

var pool      =    mysql.createPool({
    connectionLimit : 100, //important
    host     : 'localhost',
    user     : 'root',
    password : '',
    database : 'address_book',
    debug    :  false
});

function handle_database(req,res) {
   
    pool.getConnection(function(err,connection){
        if (err) {
          connection.release();
          res.json({"code" : 100, "status" : "Error in connection database"});
          return;
        }  

        console.log('connected as id ' + connection.threadId);
       
        connection.query("select * from user",function(err,rows){
            connection.release();
            if(!err) {
                res.json(rows);
            }          
        });

        connection.on('error', function(err) {      
              res.json({"code" : 100, "status" : "Error in connection database"});
              return;    
        });
  });
}

app.get("/",function(req,res){-
        handle_database(req,res);
});

app.listen(3000);

Run the app using

node test.js

and fire 10 concurrent users for 1 minute using siege by using this command.

siege -c10 -t1M http://localhost:3000

Here is output.



Final comments :

Siege is really powerful tool for testing server under pressure. We have created 100 connection limit in code, so you might be wondering that after 100 concurrent connection code will break. Well let me answer it via code. Fire 1000 concurrent user for 1 minute and let’s see how our code reacts.

siege -c1000 -t1M http://localhost:3000

If your MySQL server is configured to handle such traffic at one socket then it will run and our code will manage the scheduling of concurrent connection. It will serve 100 connection a time but rest 900 will be in queue. So code will not break.

Conculusion :

MySQL is one of widely used database engine in world and with Node it really works very well. Node-mysql pooling and event based debugging is really powerful and easy to code.

Thanks for read this blog, if you have your own thoughts please share it with me.

Friday 24 July 2015

How to Send e-mail using Node.js

Hi Everybody!!!!

In this blog i am going to discuss about sending e-mail with Node.js, Express.js and NodeMailer package. In many forums and blogs people used to ask about sending e-mail’s using Node.js for account verification, password recovery and promotion. So let’s begin with tutorial and at the end you will have Node application which is able to send e-mail’s to any one’s account.

Create package.json file and keep it in any folder. Put the below code in it.

package.json

{
"name": "email-node",
"version": "1.0.0",
"dependencies": {
"nodemailer": "~0.7.1",
"express": "~4.5.1"
}
}

Once done. Switch to that folder using Command prompt or Terminal and type npm install. It will download and install the dependencies our project need.

npm install

After completion you will able to see node_modules folder in your project directory.

Implementing Server.js:

Let’s begin with creation of our Server file. Paste code shown below in file named as “Server.js”.

Server.js

var express=require('express');
var nodemailer = require("nodemailer");
var app=express();
app.listen(3000,function(){
console.log("Express Started on Port 3000");
});

This is basic block of our app. If you run it in Terminal you will see console message. Now let’s add some routing logic to tell our app what is supposed to do when Request comes from browser. Add these code just above the app.listen line.

app.get('/',function(req,res){
res.sendFile('index.html');
});

Now when you try to open your app from browser, it will return the index.html as response. Let me show you basic snippet of HTML code.

index.html

<div id="container">
<div></div>
Node.JS Email application
<div>
<h1>Mailer In Node.JS</h1>
<input id="to" type="text" placeholder="Enter E-mail ID where you want to send" />
<input id="subject" type="text" placeholder="Write Subject" />
<textarea id="content" cols="40" rows="5" placeholder="Write what you want to send"></textarea>
<button id="send_email">Send Email</button>
<span id="message"></span>
</div>

Now next thing to do is to call our Server from HTML page when user hit on ‘Send Email’ Button. To do so at client end i am using jQuery. Here is code snippet of jQuery inside HTML page only.

index.html

<script>
$(document).ready(function(){
    var from,to,subject,text;
    $("#send_email").click(function(){     
        to=$("#to").val();
        subject=$("#subject").val();
        text=$("#content").val();
        $("#message").text("Sending E-mail...Please wait");
        $.get("http://localhost:3000/send",{to:to,subject:subject,text:text},function(data){
        if(data=="sent")
        {
            $("#message").empty().html("Email is been sent at "+to+" . Please check inbox !");
        }

});
    });
});
</script>

Notice : At $.get we are calling our app with ‘/send’ as handler so now we have to add router in our Server.js file in order to deal with this request. So let’s add some code to handle this route request. Add these code just below of app.get(‘/’) line. File name : Server.js

Server.js

app.get('/send',function(req,res){
//code to send e-mail.
//Will be shown soon.
});

Adding NodeMailer code:

Okay we are almost there, now we have to handle the Mail system. First of all we have to Define Mail transport System (SMTP) so that from that E-mail box our e-mail will be sent. For ease, you can add your Gmail account and password. Add this code just in Server.js just below the ‘var app=express()’ line.

var smtpTransport = nodemailer.createTransport("SMTP",{
service: "Gmail",
auth: {
user: "yourID@gmail.com",
pass: "Your Gmail Password"
}
});

We will use this Object to send e-mail. Our app design is when user click on ‘Send email’ Button then only e-mail should be sent, and we have did that part in “app.get(‘/send’)” router. So in that block only paste the code shown below.

var mailOptions={
to : req.query.to,
subject : req.query.subject,
text : req.query.text
}
console.log(mailOptions);
smtpTransport.sendMail(mailOptions, function(error, response){
if(error){
console.log(error);
res.end("error");
}else{
console.log("Message sent: " + response.message);
res.end("sent");
}
});

In above code we have read GET variables send from HTML page and we have call “sendMail()” function using Transport object we have created above. In case there is any confusion here is a complete Server.js file.

Server.js

var express=require('express');
var nodemailer = require("nodemailer");
var app=express();
/*
Here we are configuring our SMTP Server details.
STMP is mail server which is responsible for sending and recieving email.
*/
var smtpTransport = nodemailer.createTransport("SMTP",{
service: "Gmail",
auth: {
user: "yourID@gmail.com",
pass: "Your G-mail password"
}
});
/*------------------SMTP Over-----------------------------*/

/*------------------Routing Started ------------------------*/

app.get('/',function(req,res){
res.sendFile('index.html');
});
app.get('/send',function(req,res){
var mailOptions={
to : req.query.to,
subject : req.query.subject,
text : req.query.text
}
console.log(mailOptions);
smtpTransport.sendMail(mailOptions, function(error, response){
if(error){
console.log(error);
res.end("error");
}else{
console.log("Message sent: " + response.message);
res.end("sent");
}
});
});

/*--------------------Routing Over----------------------------*/

app.listen(3000,function(){
console.log("Express Started on Port 3000");
});

And this is how HTML page looks like with all styles and JS.

index.html

<html>
<head>
<title>Node.JS Email application</title>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script><script>// <![CDATA[
$(document).ready(function(){
    var from,to,subject,text;
    $("#send_email").click(function(){     
        to=$("#to").val();
        subject=$("#subject").val();
        text=$("#content").val();
        $("#message").text("Sending E-mail...Please wait");
        $.get("http://localhost:3000/send",{to:to,subject:subject,text:text},function(data){
        if(data=="sent")
        {
            $("#message").empty().html("

Email is been sent at "+to+" . Please check inbox !

");
        }

});
    });
});
</script>
</head>
<body>
<div id="container">
<h1>Mailer In Node.JS</h1>
<input id="to" type="text" placeholder="Enter E-mail ID where you want to send" />
<input id="subject" type="text" placeholder="Write Subject" />
<textarea id="content" cols="40" rows="5" placeholder="Write what you want to send"></textarea>
<button id="send_email">Send Email</button>
<span id="message"></span>
</div>
</div>
</body>
</html>
That’s it for now. If you have any queries do let me know in comments.

Friday 17 July 2015

Real Time Chat With NodeJS, Socket.io and ExpressJS

This blog show you how to build a web chat application, using NodeJS, ExpressJS and Socket.io.

Setup Environment

Of course, the first thing to do is get NodeJS installed on your system.Once you have NodeJS installed, you're ready to setup the needed instruments.

ExpressJS - this will manage the server and the response to the user
Jade - template engine
Socket.io - allows for real time communication between the front-end and back-end

Continuing on, within an empty directory, create a package.json file with the following content.

{
    "name": "RealTimeWebChat",
    "version": "0.0.0",
    "description": "Real time web chat",
    "dependencies": {
        "socket.io": "latest",
        "express": "latest",
        "jade": "latest"
    },
    "author": "developer"
}

By using the console (under Windows - command prompt), navigate to your folder and execute:

npm install

Within a few seconds, you'll have all the necessary dependencies downloaded to the node_modules directory

Developing the Backend

Let's begin with a simple server, which will deliver the application's HTML page, and then continue with the more interesting bits: the real time communication. Create an index.js file with the following core expressjs code:

var express = require("express");
var app = express();
var port = 3700;
 
app.get("/", function(req, res){
    res.send("It works!");
});
 
app.listen(port);
console.log("Listening on port " + port);

Above, we've created an application and defined its port. Next, we registered a route, which, in this case, is a simple GET request without any parameters. For now, the route's handler simply sends some text to the client. Finally, of course, at the bottom, we run the server. To initialize the application, from the console, execute:

node index.js

The server is running, so you should be able to open http://127.0.0.1:3700/ and see:
It works!

Now, instead of "It works" we should serve HTML. Instead of pure HTML, it can be beneficial to use a template engine. Jade is an excellent choice, which has good integration with ExpressJS. This is what I typically use in my own projects. Create a directory, called tpl, and put the following page.jade file within it:

<!DOCTYPE html>
html
    head
        title= "Real time web chat"
    body
        #content(style='width: 500px; height: 300px; margin: 0 0 20px 0; border: solid 1px #999; overflow-y: scroll;')
        .controls
            input.field(style='width:350px;')
            input.send(type='button', value='send')

The Jade's syntax is not so complex, but, for a full guide, I suggest that you refer to jade-lang.com. In order to use Jade with ExpressJS, we require the following settings.
app.set('views', __dirname + '/tpl');
app.set('view engine', "jade");
app.engine('jade', require('jade').__express);
app.get("/", function(req, res){
    res.render("page");
});

This code informs Express where your template files are, and which template engine to use. It all specifies the function that will process the template's code. Once everything is setup, we can use the .render method of the response object, and simply send our Jade code to the user.

The output isn't special at this point; nothing more than a div element (the one with id content), which will be used as a holder for the chat messages and two controls (input field and button), which we will use to send the message.

Because we will use an external JavaScript file that will hold the front-end logic, we need to inform ExpressJS where to look for such resources. Create an empty directory, public, and add the following line before the call to the .listen method.

app.use(express.static(__dirname + '/public'));

So far so good; we have a server that successfully responds to GET requests. Now, it's time to add Socket.io integration. Change this line:
app.listen(port);

to:
var io = require('socket.io').listen(app.listen(port));

Above, we passed the ExpressJS server to Socket.io. In effect, our real time communication will still happen on the same port.

Moving forward, we need to write the code that will receive a message from the client, and send it to all the others. Every Socket.io application begins with a connection handler. We should have one:
io.sockets.on('connection', function (socket) {
    socket.emit('message', { message: 'welcome to the chat' });
    socket.on('send', function (data) {
        io.sockets.emit('message', data);
    });
});

The object, socket, which is passed to your handler, is actually the socket of the client. Think about it as a junction between your server and the user's browser. Upon a successful connection, we send a welcome type of message, and, of course, bind another handler that will be used as a receiver. As a result, the client should emit a message with the name, send, which we will catch. Following that, we simply forward the data sent by the user to all other sockets with io.sockets.emit.

With the code above, our back-end is ready to receive and send messages to the clients. Let's add some front-end code.
Developing the Front-end

Create chat.js, and place it within the public directory of your application. Paste the following code:
window.onload = function() {
 
    var messages = [];
    var socket = io.connect('http://localhost:3700');
    var field = document.getElementById("field");
    var sendButton = document.getElementById("send");
    var content = document.getElementById("content");
 
    socket.on('message', function (data) {
        if(data.message) {
            messages.push(data.message);
            var html = '';
            for(var i=0; i<messages.length; i++) {
                html += messages[i] + '<br />';
            }
            content.innerHTML = html;
        } else {
            console.log("There is a problem:", data);
        }
    });
 
    sendButton.onclick = function() {
        var text = field.value;
        socket.emit('send', { message: text });
    };
 
}

Our logic is wrapped in a .onload handler just to ensure that all the markup and external JavaScript is fully loaded. In the next few lines, we create an array, which will store all the messages, a socket object, and few shortcuts to our DOM elements. Again, similar to the back-end, we bind a function, which will react to the socket's activity. In our case, this is an event, named message. When such an event occurs, we expect to receive an object, data, with the property, message. Add that message to our storage and update the content div. We've also included the logic for sending the message. It's quite simple, simply emitting a message with the name, send.

If you open http://localhost:3700, you will encounter some errors popup. That's because we need to update page.jade to contain the necessary JavaScript files.

head
    title= "Real time web chat"
    script(src='/chat.js')
    script(src='/socket.io/socket.io.js')

Notice that Socket.io manages the delivery of socket.io.js. You don't have to worry about manually downloading this file.

We can again run our server with node index.js in the console and open http://localhost:3700. You should see the welcome message. Of course, if you send something, it should be shown in the content's div. If you want to be sure that it works, open a new tab (or, better, a new browser) and load the application. The great thing about Socket.io is that it works even if you stop the NodeJS server. The front-end will continue to work. Once the server is booted up again, your chat will be fine too.

Friday 10 July 2015

Deploying with Envoy Laravel 5.1

We'll use Laravel's Envoy to deploy a PHP application to a production server. This will make a new release directory, clone the repository into it, run any needed build steps, and then finally swap the new code out with the older code, so that Nginx and PHP-FPM serve the new code.


Install Envoy Globally

# Install envoy globally. May need to ensure that
# the composer vendor/bin directory is in your PATH
# so the envoy command is found

composer global require "laravel/envoy=~1.0"

# fails, if the PATH variable is not set with composer's
# bin directory

which envoy
vim ~/.profile

If the which envoy test fails (if there's no output saying the path to the envoy command), edit .profile, .bashrc or whatever is appropriate for your system. Within this file, we'll append the following near the bottom. Adjust the file path as needed for your development server.#
Prepend the composer bin directory to your PATH

# This path is specific to my vagrant server

export PATH=/home/vagrant/.composer/vendor/bin:$PATH

Finally we can create an envoy file for our project:

cd /vagrant/app
vim Envoy.blade.php

Here's a sample we can use to test that Envoy can connect to our server:

@servers(['web' => 'deploy-ex'])

<?php
$repo = 'https://{username}:{password}@github.com:Servers-for-Hackers/deploy-ex.git';
$release_dir = '/var/www/releases';
$app_dir = '/var/www/app';
$release = 'release_' . date('YmdHis');
?>

@macro('deploy', ['on' => 'web'])
    fetch_repo
    run_composer
    update_permissions
    update_symlinks
@endmacro

@task('fetch_repo')
    [ -d {{ $release_dir }} ] || mkdir {{ $release_dir }};
    cd {{ $release_dir }};
    git clone {{ $repo }} {{ $release }};
@endtask

@task('run_composer')
    cd {{ $release_dir }}/{{ $release }};
    composer install --prefer-dist;
@endtask

@task('update_permissions')
    cd {{ $release_dir }};
    chgrp -R www-data {{ $release }};
    chmod -R ug+rwx {{ $release }};
@endtask

@task('update_symlinks')
    ln -nfs {{ $release_dir }}/{{ $release }} {{ $app_dir }};
    chgrp -h www-data {{ $app_dir }};
@endtask

Run that and test it via:

envoy run deploy

Now enjoy, your code will deploy with one command from your local.

Friday 3 July 2015

10 improvements in PHP 5.5.0 for web developers


PHP 5.5.0 has been released, bringing with it a host of new features and fixes. TechRepublic has rounded up 10 improvements that web developers should look out for in the new release.

One key difference to note before upgrading is that support for Windows XP and 2003 has been dropped as of 5.5.0. Other backwardly incompatible changes introduced by the release can be found here.

#1 Generators are now available

Generators provide a simple way to iterate through data without having to write a class implementing the Iterator interface.

Just like any other function a generator is defined with the function keyword, but unlike a normal function that just returns a result once, a generator can send back as many results as needed using the yield keyword.

A simple example of using a generator function to print out a positive sequence of integers is shown below.

function xrange($start, $end, $step = 1) {

for ($i = $start; $i <= $end; $i += $step) {

yield $i;

}

}

echo 'Single digit odd numbers: ';

foreach (xrange(1, 9, 2) as $number) {

echo "$number ";

}

echo "\n";

This would print "Single digit odd numbers:1,3,5,7,9".

A generator allows you to iterate over a set of data using a foreach loop without needing to build an array in memory. Not having to create an array reduces memory usage and processing time.

For example, using the range() function to generate a sequence between one and one million by calling range(0,1000000) within a foreach loop will create an array of more than 100MB in size, according to the PHP Manual. In comparison, creating a sequence using a generator function will never need to consume more than 1KB.

#2 Finally keyword added

The addition of the "finally" keyword refines the way that PHP deals with exception handling.

Like other high level languages PHP allows you to wrap code in a try and catch block. Any exception that is thrown by code within the try block will be passed to code within the catch block to be handled.

The finally keyword allows you to define a block of code, to be placed after the catch block, that will always be executed after the try and catch blocks, regardless of whether an exception was thrown.

The PHP manual gives this example:

function inverse($x) {

    if (!$x) {

        throw new Exception('Division by zero.');

    }

    return 1/$x;

}

try {

    echo inverse(5) . "\n";

} catch (Exception $e) {

    echo 'Caught exception: ',  $e->getMessage(), "\n";

} finally {

    echo "First finally.\n";

}

#3 New password hashing API

The new password hashing API allows you to use one line of code to generate a salted password hash using bcrypt. For example:

$hash = password_hash($password, PASSWORD_DEFAULT);

password_hash() takes two arguments here, first the password as a string and second a constant setting the encryption algorithm to use.

The password will be automatically salted and can be verified using the following code:

 password_verify($password, $hash);

The current default encryption algorithm used is bcrypt, although this is expected to change as new and stronger algorithms are added to PHP.

It is recommended to store the result in a database column that can expand beyond 60 characters.

#4 Array and string literal deferencing added

Both array and string literals can now be dereferenced directly to access individual elements and characters.

For example:

echo 'Array dereferencing:';

echo [4,5,6][0];

echo "\n";

selects the first element in the array to be printed, producing "Array dereferencing:4".

echo 'String dereferencing:';

echo 'HELLO'[0];

echo "\n";

selects the first element in the string to be printed, producing "String dereferencing:H".

#5 Easier class name resolution

The class keyword can now be used to retrieve the fully qualified name of a class, including the namespace it sits within.

For example:

namespace NS {

    class ClassName {

    }

    echo ClassName::class;

}

will print out the both the name of the class and the namespace, producing "NS\ClassName".

#6 Empty() function accepts expressions

The empty() function, used to determine whether a variable is empty or a value equals false, can now be passed an expression and determine whether the variable that expression returns is empty.

For example:

function send_false() {

    return false;

}

if (empty(send_false())) {

    echo "False value returned.\n";

}

if (empty(true)) {

    echo "True.\n";

}

will print "False value returned."

#7 Support for the Zend Optimiser+ opcode cache added

The Zend Optimiser+ opcode cache has been added to PHP as the new OPcache extension.

OPcache improves performance of scripts by removing the need for PHP to load and parse scripts every time a request is made. It achieves this by storing precompiled script bytecode in shared memory.
#8 foreach loops now support the list() construct

Values insides nested arrays can now be assigned to variables using a foreach() loop and the list() construct.

List() can be used to easily assign values taken from inside an array to variables, like so:

$animals = array('dog', 'fox');

// Listing all the variables

list($animal1, $animal2) = $animals;

echo "The quick $animal1 jumped over the lazy $animal2\n";

to produce the familiar, "The quick fox jumped over the lazy dog".

Now list() can be used with foreach() loops to be assigned values from inside nested arrays, for example:

$array = [

    [10, 20],

    [30, 40],

];

foreach ($array as list($a, $b)) {

    echo "First: $a; Second: $b\n";

}

to produce:

First: 10, Second: 20

First: 30, Second: 40

#9 New features added to GD library

PHP's GD extension for creating and manipulating images has gained new capabilities. These include flipping support using the new imageflip() function, advanced cropping support using the imagecrop() and imagecropauto() functions, and WebP read and write support using the imagecreatefromwebp() and imagewebp() functions.

#10 foreach loops now support non-scalar keys

When iterating through an array using a foreach loop, element keys are now able to have a non-scalar value, that is a value other than an integer or a string.