As we all know NodeJs works asynchronously (i.e., Non-Blocking). It will not execute the programming code line by line and it will not wait for any function, i/O or DB operations to finish executing. This feature of nodejs makes the process more efficient, but we cannot prefer this asynchronous nature in all scenarios. For example, In case of Database interaction we must need Synchronous method to execute the code line by line.
Plus, There are 3 ways to handle the asynchronous nature of the nodejs.
1) Callbacks
2) Promises
3) Async module
since callbacks and promises are beyond the scope of this blog, now we will see how to handle database interactions using the async module.
Async:
Async is a utility module which provides straight-forward, powerful functions, that helps us out with the asynchronous tasks that are so often found in Node.js, and in the browser too. Async provides around 70 functions that includes the usual ‘functional’ suspects (map, filter, each…) as well as some common patterns for asynchronous control flow (parallel, series, waterfall…) all designed around asynchronous functions and callbacks, to make our life easier. Some of the important methods are discussed here. We can avail the Async via npm.
npm install --save async
Below are the steps for a sample node API that uses the async module ’s method to interact with the database. To make this application work, Make sure the Mongo database installed in your system. To illustrate this example, consider we are going to upload a post with two fields named as ‘title’, ‘content’ into Mongo database, and also edit, retrieve the same. For this, create a database in mongo db with the name of your choice. For this example, I created database name as “blog_posts.”
Step1:
Create a folder named “sample.” Locate the folder in the terminal and give “npm init”. Now, this will prompt you many questions and continuously press enter for all the questions so at the end it would create package.json in the folder.
Step 2:
Paste the below code into the package.json and save it. In the terminal run “npm install.” This will install the packages defined in the dependencies section of package.json to the project.
Package.json
{
"name": "sample",
"version": "1.0.0",
"description": "",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"async": "^2.6.0",
"mongodb": "2.2.33"
}
}
Step 3:
If you have the records in Database you can able to fetch and update it using this Waterfall method. For example, We already have record named “electron” in database so we tried to fetch and update with below code.
Now, create a file api.js and paste the following code there.
const express = require('express');
var async = require('async');
const ObjectID = require('mongodb').ObjectID;
var MongoClient = require('mongodb').MongoClient
const router = express.Router();
const url = 'mongodb://localhost:27017/blog_posts'
router.post('/asyncwaterfall',(req,res) =>{
async.waterfall([
function ConnectToDb(callback){
MongoClient.connect(url, function(err, conn) {
if (err) callback(err)
callback(null,conn)
})
},
function getPost(db,callback) {
console.log('retrieving the post before update --> ');
const query = {"title":"electron"}
db.collection('posts').findOne(query,function(err, post){
if(err){
callback(err,'There is a error in getting all the post')
} else {
console.log(post)
callback(null,db,post)
}
})
},
function updatePost(db,post,callback){
console.log("updating the post --->")
query = {_id: new ObjectID(post._id)}
newValues = { $set: {content: "Electron is an open-source framework allows for the development of desktop GUI applications "} }
db.collection('posts').updateOne(query,newValues , function(err, item) {
if (err) {
callback(err)
}
else{
console.log('retrieving the post after update --> ');
const query = {"_id":new ObjectID(post._id)}
db.collection('posts').findOne(query,function(err, updatedPost){
if(err){
callback(err,'There is a error in getting all the post')
} else {
callback(null,db,updatedPost)
}
})
}
});
},
function CloseDbConnection(db,updatedPost,callback){
if (db) {
db.close(function(err, result) {
if(err){
callback(err)
}
else{
db = null;
callback(null,updatedPost)
}
})
}
}
], function (err, result) {
console.log(result);
res.json(result)
});
})
module.exports = router;
async.waterfall Method
When you have a group of functions that depend on each other (i.e., when a function expects a previous function output as input to it) and wanted to execute in order (one after the other), then async.waterfall should be used. Here each function passes its result as input to the next function through callback. If any of the asynchronous function passes an error to its callback, the final callback is immediately called with error.
Step 4:
Create a file named ‘server.js’ and paste the following code there. This will create a server and make it listen to the port specified here in the file (3001)
const express = require('express');
const bodyParser = require('body-parser');
const http = require('http');
const app = express();
const api = require('./api');
// API location
app.use('/api', api);
//Set Port
const port = process.env.PORT || '3001';
app.set('port', port);
const server = http.createServer(app);
server.listen(port, () => console.log(`Running on localhost:${port}`));
Now if you save all the files and run “npm start” from the terminal, you can see server listening to the port 3001.
In postman call the API like, Http Method: POST URL: “localhost:3001/api/asyncwaterfall”
Why async.waterfall method is used:
The above code uses Async’s waterfall method to establish a connection with the database, Then use that connection to query the database, update records, and then close the connection. The async’s waterfall method will help us to pass the connection from the first function ConnectToDb() to the next following functions.
Let’s see how to access the database through some other most commonly used async’s methods like async.series, async.map, async.each.
async.series:
When you have series of independent asynchronous function, which does not depend on each other to execute in order(one after the other) async.series should be used. When all asynchronous function finished executing, the final callback is called with results from all asynchronous functions as array. If any asynchronous function passed error to its callback, the next function is not executed, and the final callback is called with the error.
To illustrate database interaction using async.series, consider we are going to upload a post with two fields title and content and retrieve the same post by its “Title” field. For this we have three independent functions, ConnectToDb() which will connect to MongoDB, addPost() which will add title and content of the post to the database and other function getPost() which will retrieve the uploaded post by its title field. The three function will execute in the order they are written and send their results back to the final callback function through the callback.
[ Related: Top 7 Node.js Development Tools For Developers ]
Sample
router.post('/asyncseries',(req,res) =>{
var db = null;
async.series([
function ConnectToDb(callback){
MongoClient.connect(url, function(err, status) {
if (err) callback(err)
db = status
callback(null,"db connected created")
})
},
function addPost (callback) {
const post = {"title":"osrm","content":"The Open Source Routing Machine or OSRM is a C++ implementation of a high-performance routing engine for shortest paths in road networks."}
console.log('uploading post --> ',post);
db.collection('posts').save(post, function(err, uploadedPost) {
if (err) {
callback(err,'There is a error in uploading the post');
} else {
callback(null, "post is uploaded");
}
})
},
function getPost(callback) {
console.log('retrieving the uploaded post --> ');
const query = {"title":"osrm"}
db.collection('posts').findOne(query,function(err, item){
if(err){
callback(err,'There is a error in getting the post')
} else {
callback(null,item)
}
})
},
function CloseDbConnection(callback){
if (db) {
db.close(function(err, result) {
if(err){
callback(err)
}
else{
db = null;
callback(null,'db connection closed')
}
})
}
}
],
function (err, result) {
console.log(result);
res.json(result)
});
})
Note:
The significant difference between async.series() and async.waterfall() is that in async.series once all the series functions have finished executing will send the results to the final callback whereas in waterfall each function passes its output to next function and only the result of the last function is passed to the final callback.
[ Related: 10 JavaScript concepts every Node.js developer should know ]
async.each
When we want a some function to apply for all items in an array and do something once the function finish executing for all elements in the array, then async.each should be used. If the function passes an error, then maincallback is called with error. async.each runs in parallel, but if you want to run it in series, you can use async.eachSeries instead of async.each.
Sample
var db = null;
var SaveToDb = function(post,callback)
{
db.collection('posts').save(post, function(err, uploadedPost) {
if (err) {
callback(err,'There is a error in uploading the post');
} else {
callback();
}
})
}
router.post('/asynceach',(req,res) =>{
MongoClient.connect(url, function(err, status) {
if (err){
callback(err)
}
else{
db = status
async.each([
{"title":"mongo","content":"Classified as a NoSQL database program, MongoDB uses JSON-like documents with schemas"},
{"title":"angular","content":"Angular is a TypeScript-based open-source front-end web application platform"},
{"title":"express","content":"Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications."}
],SaveToDb, function(err) {
if(err){
res.json('Failed to process')
}
db.collection('posts').find({}).toArray(function(err, posts){
if(err){
res.json('There is a error in getting the post count')
} else {
res.json(posts.length)
}
})
});
}
})
})
In the above code, after establishing a connection to the database, We have an array of items that we want to save to our database. Once they’re all saved, we execute a function that calculates the total no of posts in the database.
Result:
async.map:
async.map runs a function on each element in the array resulting in a new array with the updated items. Once the function finishes executing for all the items in the array, the result array will be passed to the final callback. If iterator function passes an error to its callback, the final callback is immediately called with the error. The async.map applies the function to each item in parallel, and there is no guarantee that the functions will complete in order. However, the results array will be in the same order as the original array.
[ Related: Top Node.js Development Libraries For Developers ]
Sample
var db = null;
var getFromDb = function(title,callback)
{
const query = {"title":title}
db.collection('posts').findOne(query,function(err, post){
if(err){
callback(err,'There is a error in getting all the post')
} else {
callback(null,post.content)
}
})
}
router.get('/asyncmap',(req,res) =>{
MongoClient.connect(url, function(err, status) {
if (err){
callback(err)
}
else{
db = status
async.map(['mongo','express','angular','node'],getFromDb, function(err, results) {
if(err){
res.json(err)
}
else{
res.json(results)
}
});
}
})
})
In the above code, after establishing a connection to the database, we are retrieving a list of post by its title. For this, we have an array with the list of post titles. The aync.map function will loop through title array, apply our function getFromDb() to each title in the array and store the retrieved values in the array to be returned in the order we expect, to the main callback.
Note:
The async.map is almost same as async.each but the only difference is that async.map will pass the results array back to the final callback where async.each will not pass.
Hope you find it helpful and similarly you could explore more about technologies with us. Our pool of experts evolving technologies up-to-date so stay updated with us for more technical updates and blogs. On the go, Don’t forget to post your interest and impressions in the comment section.
For more info reach us at info@agiratech.com
