Working with Glide – Vendor Package Management for Go

In this post, we will use Glide to manage the dependencies of a Go project. Before starting, let me give you a brief introduction about Glide.

What is Glide?
Glide is a package manager for Go that is conceptually similar to package managers for other languages such as NPM for Node.js, Pip for Python, and so forth which records dependency information in a glide.yaml file and utilizes vendor/ directories, so that different projects can have different versions of the same dependencies.

Now, lets’ start with creating a small project with a single Go file which will give colourized outputs, following steps:

Step 1: Install Glide executing the following command:

$ curl https://glide.sh/get | sh

Above command will add glide binary in $GOPATH/bin.

Step 2: Move to $GOPATH/src to create a directory – golang-glide-example executing the following command:

$ cd $GOPATH/src && mkdir golang-glide-example

Step 3: Move to $GOPATH/src/golang-glide-example to create a Go file which will give colourized outputs executing the following command:

$ cd $GOPATH/src/golang-glide-example && touch color.go

Step 4: Copy the following code to color.go:

package main

import (
	"github.com/fatih/color"
)

func main() {
	color.Red("We have red")
	color.Blue("Prints %s in blue.", "text")
}

Step 5: Move to $GOPATH/src/golang-glide-example and execute the following command:

$ cd $GOPATH/src/golang-glide-example && glide init --non-interactive

Above command will create glide.yaml which contain all the dependencies to run the project.

Step 6: Move to $GOPATH/src/golang-glide-example, install all dependencies and build the project executing the following command:

$ cd $GOPATH/src/golang-glide-example && glide install && go build

Above command will download and export the dependencies and the output logs will look like as shown in a screenshot below:

Screen Shot 2018-04-14 at 12.20.06 PM

Step 7: Move to $GOPATH/src/golang-glide-example and Run golang-glide-example executing following command:

$ cd $GOPATH/src/golang-glide-example && ./golang-glide-example

which will give us the colourized output same as shown in a screenshot below:

Screen Shot 2018-04-14 at 12.20.24 PM

The complete source code is hosted on GitHub.

Advertisements

Optimizing Angular Application Using Template Cache

In one of my projects we were experiencing lot of server calls for the HTML files which we optimized it using Angular Template Cache and in this post I will showcase a thin example of our problem and how Angular Template Cache solved it using a Cache object.

As creating an application is not a part of this post, I already created angular-grunt-example which is an angular application with Grunt as a JavaScript task runner and available for clone from GitHub.

In an image below, one can see a server call each time we access .jsp file:

all-calls

to avoid all these server calls we will be implementing angular template cache, following below steps:

Step 1: Install grunt-html2js Grunt plugin with npm, as follows:

npm install grunt-html2js --save-dev

Above command will add grunt-html2js as devDependencies in package.json.

Step 2: Create html2js Grunt task in Gruntfile:

html2js: {
	  options: {
		base: 'src',
		module: 'myapp.template',
		singleModule: true,
		useStrict: true,
		htmlmin: {
		  collapseBooleanAttributes: true,
		  collapseWhitespace: true,
		  removeAttributeQuotes: true,
		  removeComments: true,
		  removeEmptyAttributes: true,
		  removeRedundantAttributes: true,
		  removeScriptTypeAttributes: true,
		  removeStyleLinkTypeAttributes: true
		},
		rename: function (url) {
				return url.replace('main/webapp/', '');
		}
	  },
	  main: {
		src: ['/pages/**/*.jsp'],
		dest: '/scripts/template.js'
	  }
}

module option specified above is the angular module name.

rename option specified above is used to make file paths match up exactly same as called from application, for example in application we are accessing .jsp pages as ‘pages/page-posts.jsp’, ‘pages/page-posts-list.jsp’ and ‘pages/page-authors.jsp’, etc and the file path generated is ​main/webapp/pages/angular-material.jsp`, so replace function strip ​main/webapp/ from each template path to produce a module identifier for the template.

Step 3: Load html2js task from the grunt-html2js Grunt plugin:

grunt.loadNpmTasks('grunt-html2js');

Step 4: Register html2js task with default task of grunt, so that when we run “default” task “uglify:dist” and “html2js” tasks run automatically, as follows:

grunt.registerTask('default', ['uglify:dist', 'html2js']);

On running the default task, template.js will be created in /webapp/scripts/ directory as a result of html2js task execution.

Step 5: Load tempalte.js generated by the html2js task as the first script after Angular and all related dependencies are loaded, as follows:

<spring:url value="/js/jquery-2.2.3.min.js" var="jQueryUrl"
<spring:url value="/js/angular.min.js" var="angularJsUrl"
<spring:url value="/scripts/template.js" var="templateJsUrl"
<spring:url value="/js/application.js" var="applicationJsUrl"

Step 6: Make myapp.template as the first dependency for the application module:

var myapp = angular.module('myapp', [ 'myapp.template', 'ui.router' ]);

Once done, browsing the application will not make call to the server for the .jsp files instead it fetch it from $templateCache which is a Cache object defined in template.js, as shown in the below image:

cache-calls.png

The complete source code is hosted on github.

Creating Your Own Package in Go

A package in Go is a namespace that organizes a set of related files. We can think of packages as being similar to different folders. Go standard library comes with a number of packages which can be used for building real-world applications. Moreover Go supports writing our own packages promoting code modularization and better composability of applications following certain rules, like all source files within the package must declare the same package-name. Identifiers, Function and Types will be exported to other packages if the first letter of the identifier name starts with an uppercase letter.

In post Creating HTTP Server in Go we already updated Path with GOROOT, GOPATH and GOBIN, so as part of this post we will simply start with creating a package in Go following below steps:

Step 1: Create a directory inside your workspace to keep source code of a package, for me it’s numberutil under $GOPATH/github.com/arpitaggarwal:

$ mkdir -p $GOPATH/github.com/arpitaggarwal/numberutil

Step 2: Move to directory we created in previous step and create a file named decimalTobinary.go inside it, as follows:

$ cd $GOPATH/github.com/arpitaggarwal/numberutil
$ touch decimalTobinary.go

Copy following Go code:

package numberutil

import "strconv"

//Convert Decimal to Binary
func DecimalToBinary(decimal int64) string {
	binary := strconv.FormatInt(decimal, 2)
	return binary
}

Above code contains a single go function which takes Decimal number as input and convert it to Binary using strconv.FormatInt function.

Step 3: Build numberutil package using go tool, as follows:

$ cd $GOPATH
$ go build github.com/arpitaggarwal/numberutil

Step 4: Next we will create number-util.go with a main() method to use DecimalToBinary function from numberutil package we created, as follows:

$ cd $GOPATH/github.com/arpitaggarwal/
$ touch number-util.go

Copy following Go code:

package main

import (
	"fmt"
	"github.com/arpitaggarwal/numberutil"
)

func main() {
	i23 := int64(23)
	fmt.Printf("Decimal to Binary for 23 is %s \n", numberutil.DecimalToBinary(i23))
}

Step 5: Install number-util.go using go tool:

$ go install $GOPATH/github.com/arpitaggarwal/number-util.go

Above command compile number-util.go and generate executable binary file of it in $GOROOT/bin or $GOBIN directory.

Step 6: Run number-util.go moving to golang directory:

$ cd golang
$ ./go/bin/number-util

Step 7: Now we will generate documentation of numberutil package we created which is as simple as running godoc tool with -http flag for the port number from the terminal, as follows:

godoc -http=:9999

Now, opening http://localhost:9999/pkg/github.com/arpitaggarwal/numberutil/ in browser will show us the documentation of numberutil package we have created.

Go also support using third party libraries or packages by installing them using go get command or copying it manually to $GOPATH/src or $GOPATH for example, If we want to use Reverse function from “github.com/golang/example/stringutil” package which is not available by default in go standard library then we can install it as:

$ cd golang
$ go get github.com/golang/example/stringutil

Or clone the package and copy it to $GOPATH/src or $GOPATH directory, then use it as:


package main

import (
	"fmt"
	"github.com/golang/example/stringutil"
)
func main() {
	fmt.Println(stringutil.Reverse("!olleh"))
}

Complete source code is hosted on github.

Creating HTTP Server in Go

In this post, we will create a simple HTTP server in Go following some simple steps. Before starting, let me give you a brief introduction about Go.

What is Go?

Go is an open source programming language which is case sensitive and statically-typed with syntax similar to that of C language created at Google in 2007. It compiles very quickly, supports concurrency at the language level and provides garbage collection, type safety, dynamic-typing capability, many advanced built-in types such as variable length arrays and key-value maps though there is no support for generic programming, type inheritance, method or operator overloading, pointer arithmetic and assertions.

Now, let’s start with setting up the environment to run our first HTTP Server, following below steps:

Step 1: Download and extract Go in any directory, for me it’s golang under directory /Users/ArpitAggarwal/ as follows:


$ mkdir golang
$ cd golang
$ wget https://storage.googleapis.com/golang/go1.8.3.darwin-amd64.tar.gz
$ tar -xvzf go1.8.3.darwin-amd64.tar.gz

Step 2: Update Path with GOROOT, GOPATH and GOBIN, as follows:

$ export GOROOT=/Users/ArpitAggarwal/golang/go
$ export GOPATH=$GOROOT/src
$ export GOBIN=$GOROOT/bin
$ export PATH=$PATH:$GOPATH:$GOBIN

GOROOT refers to the go installation directory.
GOPATH refers to our go workspace directory or where go build is done.
GOBIN refers to the directory where go generate executable binaries.

Step 3: Verify if GOPATH is set properly on Bash, executing following command:

$ env | grep -E "^GO"

Step 4: Create a directory inside workspace to keep source code:

$ mkdir -p $GOPATH/github.com/arpitaggarwal

Step 5: Move to directory we created in previous step and create a file named webserver.go inside it, as follows:

$ cd $GOPATH/github.com/arpitaggarwal
$ touch webserver.go

Copy following Go code:

package main

import (
	"fmt"
	"log"
	"net/http"
)

func helloWorld(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello World!") // send data to client side
}

func main() {
	http.HandleFunc("/", helloWorld)         // set router
	err := http.ListenAndServe(":8080", nil) // set listen port
	if err != nil {
		log.Fatal("Error while starting GO http server on port - 8080 : ", err) //log error and exit in case of error at server boot up
	}
}

Let’s understand what does each line mean.

package main – It defines the package name of the program.

import (“fmt”, “log” “net/http”) – is a preprocessor command which tells the Go compiler to include all files from fmt, log and net/http package.

func helloWorld(w http.ResponseWriter, r *http.Request) – is a go function.

main() – is the main function where the program execution begins.

Step 6: Install webserver.go using go tool, as follows:


$ go install $GOPATH/github.com/arpitaggarwal/webserver.go

Above command compile webserver.go and generate executable binary file of it in $GOROOT/bin or GOBIN.

Step 7: Run webserver moving to golang directory, as follows:


$ cd golang
$ ./go/bin/webserver

Now, opening http://localhost:8080/ in browser or executing command:

curl http://localhost:8080

will show us “Hello World!” as response.

Step 8: Optionally we can run binary file in background using nohup command, as follows:


$ cd golang
$ nohup ./go/bin/webserver &;

Complete source code is hosted on github.

Routing in React

In post Handling Events in React we extended react-app which we built as part of Rendering RESTful service with React to support add and delete employees operation.

Now as part of this post we will add Routing capabilities to it using react-router, by defining an additional column Details as a Link, click of which take you to the details of the employee selected, following below steps:

Step 1. Install react-router-dom as a dependency:

npm install react-router-dom --save

Step 2: Update react-handling-events/app/main.js to import HashRouter, Route and Switch from react-router-dom along with EmployeeDetail component which we will define later in the post.

import { HashRouter, Route, Switch } from 'react-router-dom';
import EmployeeDetail from './components/employee-detail.jsx'

Step 3: Instead of rendering the ReactApp component directly, we will use HashRouter and Route to render it:

ReactDOM.render(
	<HashRouter>
	 <Switch>
	    <Route exact path="/" component={ReactApp}/>
        <Route exact path="/employee/:id" component={EmployeeDetail}/>
     </Switch>
    </HashRouter>,
   document.getElementById('react')
)

Eventually, react-handling-events/app/main.js should look like:

'use strict';
const React = require('react');
const ReactDOM = require('react-dom')

import { HashRouter, Route, Switch } from 'react-router-dom';
import ReactApp from './components/react-app.jsx'
import EmployeeDetail from './components/employee-detail.jsx'

ReactDOM.render(
    <HashRouter>
      <Switch>
        <Route exact path="/" component={ReactApp}/>
        <Route exact path="/employee/:id" component={EmployeeDetail}/>
      </Switch>
    </HashRouter>,
  document.getElementById('react')
)

HashRouter tag specified above puts route information into the hash of the URL (after the #). I preferred HashRouter over BrowserRouter because to use BrowserRouter web server must be ready to handle real URLs. When the app first loads at / it will probably work, but as the user navigates around and then hits refresh at /employee/1 web server will get a request to /employee/1 then I have to handle that URL and include JavaScript application in the response.

Switch tag specified above renders the first child Route that matches the location.

As we defined two Route tag above, let’s understand what each one of them signify.

<Route exact path=/ component={ReactApp}/renders ReactApp component when the app loads at / .

<Route exact path=/employee/:id component={EmployeeDetail}/renders EmployeeDetail component when the app loads at /employee/{id} where id is a dynamic path variable.

Step 4: Create employee-detail.jsx component inside component directory, as follows:


cd react-handling-events/app/components/
touch employee-detail.jsx

And copy the below content:

import React, { Component } from 'react'

export default class EmployeeDetail extends Component {
  render(){
    if(this.props.match !== undefined && this.props.match.params !== undefined ){
      return(
        <div>
          <table style={{border: '1px solid black'}}>
            <tbody><tr>
              <td>Id</td>
              <td>Name</td>
              <td>Department</td>
            </tr>
            <tr>
              <td>{this.props.match.params.id}</td>
              <td>{this.props.location.query.employee.name}</td>
              <td>{this.props.location.query.employee.department}</td>
            </tr>
          </tbody>
        </table>
      </div>
    );
  }
  return (null);
}
}

{this.props.match.params.id} specified above is to access route params in the component.

{this.props.location.query.employee.name} specified above is to access query params in the component which React-router injects as a location property.

Step 5: Update EmployeeList component render method to have an additional Details column:

const React = require('react');
import Employee from './employee.jsx'

export default class EmployeeList extends React.Component{
    
    render() {
		var employees = this.props.employees.map((employee, i) =>
			<Employee key={i} employee={employee} deleteEmployee={() => this.props.deleteEmployee(employee.name)}/>
		);
		
		return (
			<table>
				<tbody>
					<tr>
						<th>ID</th>
						<th>Name</th>
						<th>Department</th>
						<th>Delete</th>
						<th>Details</th>
					</tr>
					{employees}
				</tbody>
			</table>
		)
	}
}

Step 6: Update Employee component to import Link from react-router-dom along with render method to have an additional column Details as a Link, click on which take you to the details of the employee:

const React = require('react');
import DeleteEmployee from './delete-employee.jsx'
import { Link } from 'react-router-dom';

export default class Employee extends React.Component{
	render() {
		return (
			<tr>
				<td>{this.props.employee.id}</td>
				<td>{this.props.employee.name}</td>
				<td>{this.props.employee.department}</td>
				<td><DeleteEmployee deleteEmployee={this.props.deleteEmployee}/></td>
				<td><Link to={{ pathname:'/employee/'+this.props.employee.id, query: {employee: this.props.employee } }}>Details</Link></td>
			</tr>
		)
	}
}

With all in place directory structure should look like:

Screen Shot 2017-05-28 at 12.27.17 am

Now re-run the application and visit http://localhost:8080it should look like as shown in below screenshot, once you add few employee.

Screen Shot 2017-05-28 at 12.30.17 am
Complete source code is hosted on github.

Disabling Caching at Runtime if Couchbase Connection Failed

In our application we have number of REST API calls out of which some of them are flagged with @Cacheable to cache the response. Cache is backed by Couchbase and whenever application fails to connect Couchbase node then application goes down.

Which is what we are not expecting, we expect data should be served from the source system whenever connection failed, basically bypassing the cache and as soon as the Couchbase node is up or connection established response should be served from cache.

To achieve the same mechanism, I returned “null” whenever connection fails or there is a Cache miss overriding CouchbaseCacheManager, as follows:


import java.util.Map;
import org.springframework.cache.Cache;
import com.couchbase.client.spring.cache.CacheBuilder;
import com.couchbase.client.spring.cache.CouchbaseCacheManager;

public class CouchbaseCustomCacheManager extends CouchbaseCacheManager {

	public CouchbaseCustomCacheManager(final Map<String, CacheBuilder> initialCaches) {
		super(initialCaches);
	}

	@Override
	public Cache getCache(String name) {
		return new CouchbaseCacheWrapper(super.getCache(name));
	}

	protected static class CouchbaseCacheWrapper implements Cache {

		private final Cache delegate;

		public CouchbaseCacheWrapper(Cache couchbaseCache) {
			this.delegate = couchbaseCache;
		}

		@Override
		public String getName() {
			try {
				return delegate.getName();
			} catch (Exception e) {
				return null;
			}
		}

		@Override
		public Object getNativeCache() {
			try {
				return delegate.getNativeCache();
			} catch (Exception e) {
				return null;
			}
		}

		@Override
		public ValueWrapper get(Object key) {
			try {
				return delegate.get(key);
			} catch (Exception e) {
				return null;
			}
		}

		@Override
		public <T> T get(Object key, Class<T> type) {
			try {
				return delegate.get(key, type);
			} catch (Exception e) {
				return null;
			}
		}

		@Override
		public void put(Object key, Object value) {
			try {
				delegate.put(key, value);
			} catch (Exception e) {
				try {
					handleErrors(e);
				} catch (Exception e1) {
				}
			}
		}

		@Override
		public ValueWrapper putIfAbsent(Object key, Object value) {
			try {
				return delegate.putIfAbsent(key, value);
			} catch (Exception e) {
				return null;
			}
		}

		@Override
		public void evict(Object key) {
			try {
				delegate.evict(key);
			} catch (Exception e) {
				try {
					handleErrors(e);
				} catch (Exception e1) {
				}
			}
		}

		@Override
		public void clear() {
			try {
				delegate.clear();
			} catch (Exception e) {
				try {
					handleErrors(e);
				} catch (Exception e1) {
				}
			}
		}

		protected <T> T handleErrors(Exception e) throws Exception {
			if (e instanceof Exception) {
				return null;
			} else {
				throw e;
			}
		}
	}
}

And used it as CacheManager, as follows:

@Bean
public CacheManager cacheManager() {
	final Map<String, CacheBuilder> cache = new HashMap<>();
	for (final String appCache : "127.0.0.1,127.0.0.2,127.0.0.3".split(",")) {
			cache.put(appCache, CacheBuilder.newInstance(CouchbaseCluster
					.create().openBucket("default", "")));
	}
	return new CouchbaseCustomCacheManager(cache);
}

Complete source is available on github.

Hosting WordPress Application on an EC2 Instance – AWS

In this post, we will deploy WordPress Application on an EC2 Amazon Linux AMI t2.micro instance following below steps:

Step 1: Set up Amazon EC2 instance following set-up-amazon-ec2-instance.

Step 2: Launch an EC2 instance following ec2-launch-linux-instance.

Step 3: As creating a wordpress application is not a part of this post, I already created one and zipped it as wordpress-app.zip which we will securely copy from local machine to an EC2 instance home directory (/home/ec2-user) using ec2-user as follows:

scp -i /Users/ArpitAggarwal/arpitaggarwal-key-pair.pem /Users/ArpitAggarwal/wordpress-app.zip ec2-user@ec2-54-218-30-7.us-west-2.compute.amazonaws.com:/home/ec2-user

arpitaggarwal-key-pair.pem refers to private key file.
ec2-54-218-30-7.us-west-2.compute.amazonaws.com refers to Public DNS name of EC2 instance.

Step 4: Export wordpress-app database, as follows:

cd /applications/MAMP/library/bin
./mysqldump -u root -p**** wordpress > /Users/ArpitAggarwal/export-wordpress-data.sql

/applications/MAMP/library/bin refers to MAMP local database store.
./mysqldump referes to command to get mysqldump.

Step 5: Copy export-wordpress-data.sql we created from local machine directory to EC2 instance home (/home/ec2-user) directory:

scp -i /Users/ArpitAggarwal/arpitaggarwal-key-pair.pem /Users/ArpitAggarwal/export-wordpress-data.sql ec2-user@ec2-54-218-30-7.us-west-2.compute.amazonaws.com:/home/ec2-user

Step 6: Login to your EC2 instance with private key file and Public DNS name using ssh:

ssh -i /Users/ArpitAggarwal/arpitaggarwal-key-pair.pem ec2-user@ec2-54-218-30-7.us-west-2.compute.amazonaws.com

Step 7: Change collation of your database by executing following commands in /home/ec2-user after login to an EC2 instance:

sed -i 's/utf8mb4/utf8/g' export-wordpress-data.sql
sed -i 's/utf8_unicode_ci/utf8_general_ci/g' export-wordpress-data.sql
sed -i 's/utf8_unicode_520_ci/utf8_general_ci/g' export-wordpress-data.sql

Step 8: Set up Linux, Apache, MySQL, PHP (LAMP) stack on an EC2 CentOS 6 instance and set the processes to run automatically when the server boots, executing below commands:

sudo yum install httpd
sudo yum install mysql-server
sudo yum install php php-mysql
sudo service mysqld start
sudo chkconfig httpd on
sudo chkconfig mysqld on

Step 9: Set a root MySQL password same as you have provided in your wordpess-app, executing below command and choosing specific option for all the prompt:

sudo /usr/bin/mysql_secure_installation

Step 10: Login to MySQL database on an EC2 instance and create DATABASE same as you have provided in your wordpess-app, for me it’s wordpress:

mysql -u root -p****
mysql> CREATE DATABASE IF NOT EXISTS wordpress;

Step 11: Import export-wordpress-data.sql to newly created database, as follows:

mysql -uroot -p**** wordpress < export-wordpress-data.sql

Step 12: Inflate wordpress-app.zip, Copy all the files to /var/www/html directory and create .htaccess file inside the same directory:

unzip wordpress-app.zip
sudo cp -R wordpress-app/* /var/www/html
cd /var/www/html
sudo touch .htaccess

Replace the content of .htaccess file with below:

# BEGIN WordPress
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /
  RewriteRule ^index\.php$ - [L]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule . /index.php [L]
</IfModule>
# END WordPress

Step 13: Edit httpd.conf placed in /etc/httpd/conf to set value of AllowOverride directive to All for the /var/www/html directory, as below:

<Directory "/var/www/html">
  Options Indexes FollowSymLinks
  AllowOverride All
  Order allow,deny
  Allow from all
</Directory>

Step 13: Restart apache to reflect all of the changes we did:

sudo service httpd restart

Now, access the wordpress-app from your browser using Public DNS name or Public IP of your EC2 instance as: http://ec2-54-218-30-7.us-west-2.compute.amazonaws.com/

Need to move WordPress site to a new Host?

It can be easily done by updating the option_value, post_content and guid of the application directly in the MySQL database executing below scripts:

UPDATE wp_options SET option_value = 'http://new-host/' WHERE option_name = 'home';
UPDATE wp_options SET option_value = 'http://new-host/' WHERE option_name = 'siteurl';
UPDATE wp_posts SET post_content = REPLACE(post_content,'http://old-host/','http://new-host/');
UPDATE wp_posts SET guid = REPLACE(guid,'http://old-host/','http://new-host/');