Fixing com.couchbase.client.java.error.TranscodingException: Could not encode document with ID cache:EMPLOYEE_:1 – Couchbase

If you are using couchbase spring cache then you might get following exception:


Error occured: Details com.couchbase.client.java.error.TranscodingException: Could not encode document with ID cache:EMPLOYEE_:1

This exception is raised by couchbase client when document is not able to serialized, possible use case is when you are caching the resource which contain nested objects and does not implement Serializable, for example:

Employee.java


public class Employee {

   private int id;
   private String name;
   private Address address;
}

Address.java


 public class Address {
   private String city;
   private String country;
}

If you are facing the same problem as me then you can fix it implementing Serializable in all the classes, as follows:

Employee.java

import java.io.Serializable;

public class Employee implements Serializable {

  private static final long serialVersionUID = 1L;
  private int id;
  private String name;
  private Address address;
}

Address.java

import java.io.Serializable;

public class Address implements Serializable {

  private static final long serialVersionUID = 1L;
  private String city;
  private String country;

}

Fixing com.couchbase.client.java.error.DocumentDoesNotExistException – Couchbase

If you are using couchbase spring cache then you might get following exception:


12:16:58,338 ERROR [com.arpit.common.util.Resource] (http-/127.0.0.1:8080-6) Error occured: Details com.couchbase.client.java.error.DocumentDoesNotExistException

This exception is raised by couchbase client when document does not exist in cache and a replace operation is used, possible use case is when you are caching the resource which return null (which was happening for me), for example:

@Override
@Cacheable(value = "EMPLOYEE_", key = "#id")
public Employee getEmployee(int id) {
	return null;
}

If you are facing the same problem as me then you can fix it using the unless attribute of @Cacheable which is available as of Spring 3.2, as follows:

@Override
@Cacheable(value = "EMPLOYEE_", key = "#id", unless = "#result == null")
public Employee getEmployee(int id) {
	return null;
}

Setting TTL for @Cacheable – Spring

Today I was asked to set the expiry time of cache for some of the keys our application is using, so I quickly started looking for all of the options provided by Spring @Cacheable notation to set up the expiry time or time to live. Since Spring does not provide any configurable option to achieve it, I build one implementation leveraging @Scheduled annotation with a fixed delay, as follows:


@CacheEvict(allEntries = true, cacheNames = { "EMPLOYEE_", "MANAGER_" })
@Scheduled(fixedDelay = 30000)
public void cacheEvict() {
}

Now the problem is I want cache names as well as the fixed delay time period should be populated from environmental values instead of hard coded values. To achieve the same I declared variables(populated from properties file) at class level and populated the values of the key in the annotation which made IDE to complain that values must be constant, as follows:

“The value for annotation attribute CacheEvict.cacheNames must be a constant expression”

Then I started looking for other options to get the values for the keys from environment and came across fixedDelayString element of @Scheduled notation which helped me to achieve my goal, as follows:


@Scheduled(fixedDelayString = "${couchbase.cache.flush.fixed.delay}")
public void cacheEvict() {
}

Looking for the similar element for @CacheEvict annotation as well, which is unfortunately not available given me a hint after reading the the comment on a bug SPR-10778 which says:

“The cache abstraction supports the CacheResolver abstraction now and it can be specified globally, per class and/or on a specific operation. The CacheResolver give you the ability to compute the caches to use in code so you have all the flexibility that you want.”

As bug says to make use of CacheResolver to dynamically populate the cache names, I used AbstractCacheResolver implementation of it to get the names of cache from environment, as follows:


@Value("#{'${couchbase.cache.flush}'}")
private String couchbaseCacheFlush;

@Bean(name = "customCacheResolver")
	public CacheResolver cacheResolver() {
		CacheResolver cacheResolver = new AbstractCacheResolver(cacheManager()) {
			@Override
			protected Collection<String> getCacheNames(
					CacheOperationInvocationContext<?> context) {
				return Arrays.asList(couchbaseCacheFlush.split(","));
			}
		};
		return cacheResolver;
	}

Modifying the cacheEvict() to use custom cache resolver instead of cache names completed my task for the day, as follows:


@CacheEvict(allEntries = true, cacheResolver = "customCacheResolver")
@Scheduled(fixedDelayString = "${couchbase.cache.flush.fixed.delay}")
public void cacheEvict() {
}

Complete source is available on github.

Permission denied executing shell script on remote host using ssh – Jenkins

Today I was setting up a Jenkins pipeline for one of my project and noticed deploy job fails every time it try to execute the script on a remote Linux machine stating the reason – “Build step ‘Execute shell script on remote host using ssh’ marked build as failure”, as follows:

executing script:

./stage-deployment/scripts/stage-node_deploy.sh
bash: line 1: ./stage-deployment/scripts/stage-node_deploy.sh: Permission denied
[SSH] exit-status: 126
Build step 'Execute shell script on remote host using ssh' marked build as failure

If you are facing the same problem as me then you can fix it after changing the permissions of a directory containing executable, as:

chmod --recursive a+rwx /stage-deployment/scripts/

Expected Exception Rule and Mocking Static Methods – JUnit

Today I was asked to consume a RESTful service so I started implementing it following Robert Cecil Martin’s rules for TDD and came across a new way (atleast for me) of testing the expected exception along with the error message so thought of sharing the way I implemented it as part of this post.

To start with let’s write a @Test and specify rule that our code will throw a specific exception for our example it’s EmployeeServiceException which we will verify it using ExpectedException which will provide us more precise information about the exception expected to be thrown with the ability to verify error message, as follows:


@RunWith(PowerMockRunner.class)
@PrepareForTest(ClassWithStaticMethod.class)
public class EmployeeServiceImplTest {

	@InjectMocks
	private EmployeeServiceImpl employeeServiceImpl;

	@Rule
	public ExpectedException expectedException = ExpectedException.none();

	@Before
	public void setupMock() {
		MockitoAnnotations.initMocks(this);
	}

	@Test
	public void addEmployeeForNull() throws EmployeeServiceException {
		expectedException.expect(EmployeeServiceException.class);
		expectedException.expectMessage("Invalid Request");
		employeeServiceImpl.addEmployee(null);
	}

}

Now we will create an implementing class for our @Test which will throw EmployeeServiceException whenever the request is null, for me it’s EmployeeServiceImpl as follows:

EmployeeServiceImpl.java


public class EmployeeServiceImpl implements IEmployeeService {

	@Override
	public String addEmployee(final Request request)
			throws EmployeeServiceException {
		if (request == null) {
			throw new EmployeeServiceException("Invalid Request");
		}
		return null;
	}
}

Next we will write a @Test where we will mock static method which accepts input parameters with return type using PowerMockito.mockStatic(), verify it using PowerMockito.verifyStatic() and finally do an Assert to record test pass or failure status, as follows:


@Test
	public void addEmployee() throws EmployeeServiceException {
		PowerMockito.mockStatic(ClassWithStaticMethod.class);
		PowerMockito.when(ClassWithStaticMethod.getDetails(anyString()))
				.thenAnswer(new Answer<String>() {
					@Override
					public String answer(InvocationOnMock invocation)
							throws Throwable {
						Object[] args = invocation.getArguments();
						return (String) args[0];
					}
				});
		final String response = employeeServiceImpl.addEmployee(new Request(
				"Arpit"));
		PowerMockito.verifyStatic();
		assertThat(response, is("Arpit"));
	}

Now we will provide the implementation for our @Test inside EmployeeServiceImpl itself. To do that, lets modify the EmployeeServiceImpl to have static method call as part of else statement of addEmployee, as follows:


public class EmployeeServiceImpl implements IEmployeeService {

	@Override
	public String addEmployee(final Request request)
			throws EmployeeServiceException {
		if (request == null) {
			throw new EmployeeServiceException("Invalid Request");
		} else {
			return ClassWithStaticMethod.getDetails(request.getName());
		}
	}
}

Where getDetails is a static method inside ClassWithStaticMethod:


public class ClassWithStaticMethod {

	public static String getDetails(String name) {
		return name;
	}
}


 
Complete source code is hosted on github.

Sending E-mails with Node.JS

In post Writing Restful Service Using Node, Express and Mongodb we created a rest api which creates, delete and get employees from database, this time we will send e-mail over Gmail SMTP server using Nodemailer, as follows:

Create a directory and execute npm init, as follows:

$ cd /Users/ArpitAggarwal/
$ mkdir node-js-send-mail
$ cd /node-js-send-mail
$ npm init

Next, install Nodemailer as a dependency and create server.js which is the default entry point for Node.js to send email:

$ cd /Users/ArpitAggarwal/node-js-send-mail
$ npm install --save nodemailer
$ touch server.js

Copy below content in server.js:


var nodemailer = require('nodemailer');

var smtpConfig = {
    host: 'smtp.gmail.com',
    port: 465,
    secure: true, // use SSL
    auth: {
        user: 'aggarwalarpit.89',
        pass: 'XXXXXX'
    }
};

var transporter = nodemailer.createTransport(smtpConfig);

// setup e-mail data with unicode symbols
var mailOptions = {
    from: '"Arpit Aggarwal 👥" <aggarwalarpit.89@gmail.com>', // sender address
    to: 'aggarwalarpit.89@gmail.com, aggarwalarpit.89@gmail.com', // list of receivers
    subject: 'Hello from Nodemailer ✔', // Subject line
    text: 'Hello Node', // plaintext body
    html: '<b>✌</b>' // html body
};

transporter.sendMail(mailOptions, function(error, info){
    if(error){
        return console.log(error);
    }
    console.log('Message sent: ' + info.response);
});

Next, move to directory /Users/ArpitAggarwal/node-js-send-mail start the app following below command:

$ cd /Users/ArpitAggarwal/node-js-send-mail
$ node server.js

Reference: https://github.com/nodemailer/nodemailer

Writing RESTful Service using Node, Express and MongoDB

Looking at the popularity of Node.js, I thought of learning and creating a rest api that should create, delete and get employees from database and fortunately I did it.

With this post, I will try to replicate the steps I followed while writing it along with references.

Assuming Node.js is already installed in our system, let’s start creating nodejs-rest-api, following below steps:

Step 1: Create a directory which will contain all the files related to our app, for me it’s /Users/ArpitAggarwal/nodejs-rest-api and execute npm init:

$ cd /Users/ArpitAggarwal/
$ mkdir nodejs-rest-api
$ npm init

npm init command specified above will create package.json which helps us to manage dependencies.

Step 2: Install Express as a dependency:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ npm install --save express

Step 3: Create default entry point for Node.js i.e server.js, inside the same directory we created earlier:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ touch server.js

Copy below content in server.js:

const express = require('express');
const app = express();

app.listen(9999, function() {
  console.log('Node server listening on 9999')
})

app.use(express.static(__dirname + '/public'));

Let’s understand what does each line mean.

require – imports modules into the current file.
const express = require(‘express’) – Creates an Express application.
const app = express() – Is app object conventionally denotes the Express application.
app.listen(9999) – Starts a UNIX socket and listens for connections on the given path.
app.use(express.static(__dirname + ‘/public’)) – Express serve static content for the app from the “public” directory in the application directory.

Step 4: Create index.html inside directory /Users/ArpitAggarwal/nodejs-rest-api/public as follows:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ mkdir public
$ cd public
$ touch index.html

Copy below content in index.html:

Hello Node!

Step 5: Modify server.js to add GET mapping to display the index.html we just created:

app.get('/', function(req, res){
  res.sendFile('/index.html')
});

Step 6: Move to directory /Users/ArpitAggarwal/nodejs-rest-api and start the app following below command:

$ cd /Users/ArpitAggarwal/nodejs-rest-api
$ node server.js

Now, opening http://localhost:9999/ in browser or executing command:

curl http://localhost:9999

will show us “Hello Node!” as response.

Step 7: Now, we will install MongoDB as a dependency moving to the directory /Users/ArpitAggarwal/nodejs-rest-api/:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ npm install --save mongodb

Step 8: Create config.js which will store all the configurable parameters of the app like username, password, urls, etc. It basically helps us to specify the input to an app at runtime based on different environments on which we are going to run the app:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ touch config.js

Copy the below content in config.js:

var config = {};

config.mongodb = {};
config.server = {};

config.server.port = process.env.WEB_PORT || 9999;

config.mongodb.username = process.env.MONGODB_USERNAME || 'arpit';
config.mongodb.password= process.env.MONGODB_PASSWORD || 'xxxx';
config.mongodb.host= process.env.MONGODB_HOST || 'ds047752.mlab.com';
config.mongodb.port = process.env.MONGODB_PORT || 47752;
config.mongodb.databaseName = process.env.MONGODB_NAME || 'my-database';

module.exports = config;

Do replace the properties with your’s or export from command line before starting the app, for example:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ export MONGODB_PASSWORD = p@ssword
$ node server.js

Step 9: Create mongodb.js where we will define MongoClient and export the connection to database as well as database object to utilize in a non redundant fashion while defining at one place, as follows:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ touch mongodb.js

Copy the below content in mongodb.js:


var MongoClient = require( 'mongodb' ).MongoClient;
var util = require('util');
var config = require('./config');
var _db;

var uri = util.format('mongodb://%s:%s@%s:%d/%s',
    config.mongodb.username, config.mongodb.password, config.mongodb.host, config.mongodb.port, config.mongodb.databaseName);

module.exports = {
  connectToServer: function( callback ) {
    /** Connect to the Mongo database at the URI using the client **/
    MongoClient.connect( uri, { auto_reconnect: true }, function( err, db ) {
      if (err) throw err;
      else if (!db) console.log('Unknown error connecting to database');
      else {
        console.log('Connected to MongoDB database server at:');
        console.log('\n\t%s\n', uri);
        _db = db;
      }
      return callback( err );
    } );
  },
  getDb: function() {
    return _db;
  }
};

require( ‘mongodb’ ).MongoClient specified above create a new MongoClient instance.

var config = require(‘./config’) – Imports config.js into the current file.

Step 10: Now let’s modify server.js to use the above two files we just created and setting up the server to listen on specified port only when MongoDB connection is successful:

const mongodb = require('./mongodb.js');
const config = require('./config.js');
var db

mongodb.connectToServer( function( err ) {
  app.listen(config.server.port, function() {
     console.log('Node server listening on ' + config.server.port);
     db = mongodb.getDb();
  })
});

Step 11: Next, modify GET request in server.js to get the list of employees from database using the MongoClient we just created, as follows:

app.get('/employee/get', function(req, res){
  db.collection('employees').find().toArray(function(err, results) {
  res.send(results);
  })
  res.set({
    'Cache-Control': 'no-cache'
  });
});

Executing the below command will list all the employees we create:

$ curl http://localhost:9999/employee/get

Step 12: Moving further to extend our app to create employee in a database by adding POST request. But, before introducing POST request we have to install body-parser as a node dependency which will help in parsing incoming request bodies in a middleware, as follows:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ npm install --save body-parser

Same like other dependencies we have to import it using require keyword to leverage the capability for parsing the JSON request, as follows:

const bodyParser= require('body-parser')

app.use(bodyParser.json())

app.post('/employee/create', (req, res) => {
  db.collection('employees').save(req.body, (err, result) => {
    if (err) return console.log(err)
    res.send('Employee created!');
  })
})

Now everything is in place for POST, let’s create an employee:

$ curl -H "Content-Type: application/json" -X POST -d '{"name": "Arpit Aggarwal","email":"aggarwalarpit.89@gmail.com"}' http://localhost:9999/employee/create

Next, we will extend our app to delete an employee and to that we will define the DELETE mapping in server.js, as follows:

app.delete('/employee/delete', (req, res) => {
  db.collection('employees').findOneAndDelete({name: req.body.name},
  (err, result) => {
    if (err) return res.send(500, err)
    res.send('Employee deleted!')
  })
})

Let’s verify DELETE call:

curl -H "Content-Type: application/json" -X DELETE -d '{"name": "Arpit Aggarwal"}' http://localhost:9999/employee/delete

Tired of restarting the Node.js server everytime you make changes to app?

If yes, we can use Nodemon, a utility that will monitor for any changes in our source and automatically restart our server. Let’s install it globally:

$ npm install -g nodemon

Once installed, we have to start our app with nodemon instead of node, for example:

$ cd /Users/ArpitAggarwal/nodejs-rest-api/
$ nodemon server.js

Complete source code is hosted on github.

Reference: http://expressjs.com/en/api.html#express