Using Ansible to Deploy Neo4j HA Cluster on AWS/Eucalyptus

As a follow-up to my last Neo4j, AWS/Eucalyptus blog, this entry demonstrates another great example of AWS/Eucalyptus fidelity by using Ansible to deploy a Neo4j High Available cluster.

Neo4j - Graph Database

Neo4j – Graph Database

Amazon AWS EC2

Amazon AWS EC2

Eucalyptus Systems Inc.

Eucalyptus Systems Inc.

Pre-requisites

In order to use this Ansible playbook on AWS/Eucalyptus, the following is needed:

Before deploying the cluster, a security group needs to be created that the cluster will use.  The security group must allow the following:

  • port 22 (SSH)
  • all instances part of the security group allowed to community with each other (ports 0 - 65535)

To create the security group and authorize the ports, make sure the user’s access key, secret access key, and EC2 URL are noted, and do the following:

  1. Create the security group

    ec2-create-group --aws-access-key <EC2_ACCESS_KEY> 
    --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -g neo4j-cluster -d "Neo4j HA Cluster"

  2. Authorize port for SSH in neo4j-cluster security group

    ec2-authorize 
    --aws-access-key <EC2_ACCESS_KEY> 
    --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -P tcp -p 22 -s 0.0.0.0/0 neo4j-cluster

  3. Authorize all port communication between cluster members 

    ec2-authorize 
    --aws-access-key <EC2_ACCESS_KEY> --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -P tcp -o neo4j-cluster -p -1 neo4j-cluster

After completing these steps, use

ec2-describe-group

to view the security group:

ec2-describe-group --aws-access-key <EC2_ACCESS_KEY> 
--aws-secret-key <EC2_SECRET_KEY> --url <EC2_URL> neo4j-cluster

GROUP sg-1cbc5777 986451091583 neo4j-cluster Neo4j HA Cluster
PERMISSION 986451091583 neo4j-cluster ALLOWS tcp 0 65535 FROM 
USER 986451091583 NAME neo4j-cluster ID sg-1cbc5777 ingress
PERMISSION 986451091583 neo4j-cluster 
ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 ingress

Neo4j HA Cluster Deployment

Once the security group is created with the correct ports authorized, the cluster can be deployed.  To deploy the cluster, do the following:

  1. Obtain Ansible from git and setup the environment by following the instructions mentioned here - http://ansible.cc/docs/gettingstarted.html#getting-ansible
  2. Obtain the Ansible Playbook for Neo4j HA Cluster using git

    git clone https://github.com/hspencer77/ansible-neo4j-cluster.git

  3. Change directory into ansible-neo4j-cluster  

    cd ansible-neo4j-cluster

  4. Set up /etc/ansible/hosts with the following information:
    [local]
    127.0.0.1
  5. Populate vars/ec2-config with either Eucalyptus/AWS information. vars/ec2-config contains the following variables:
    keypair: <EC2/Eucalyptus Keypair>
    ec2_access_key: <EC2_ACCESS_KEY>
    ec2_secret_key: <EC2_SECRET_KEY>
    ec2_url: <EC2_URL>
    instance_type: m1.small
    security_group: <AWS/Eucalyptus Security Group>
    image: <AMI/EMI>
  6. 
    

    Execute the following command:

    ansible-playbook neo4j-cluster.yml \
     --private-key=<AWS/Eucalyptus Private Key file> --extra-vars "node_count=3"
  7. After the playbook finishes, there will be an URL provided to access the cluster – similar to the example below:
    TASK: [Display HAProxy URL] *********************
    changed: [23.22.248.75] => {"changed": true, "cmd": 
    "echo \"HAProxy URL for Neo4j -
     http://ec2-23-22-248-75.compute-1.amazonaws.com/webadmin/#/info/org.neo4j/High%20Availability/\" ",
     "delta": "0:00:00.006835", "end": "2013-03-30 19:54:31.104320", 
    "rc": 0, "start": "2013-03-30 19:54:31.097485", "stderr": "", 
    "stdout": 
    "HAProxy URL for Neo4j - 
    http://ec2-23-22-248-75.compute-1.amazonaws.com/webadmin/#/info/org.neo4j/High%20Availability/"}

    To view the status of cluster in the browser, open up http://ec2-23-22-248-75.compute-1.amazonaws.com/webadmin/#/info/org.neo4j/High%20Availability/.

  8. To get the status of the cluster, use curl:
    curl -H "Content-Type:application/json" -d '["org.neo4j:*"]' 
    
    http://ec2-23-22-248-75.compute-1.amazonaws.com/db/manage/server/jmx/query

Thats it!  A Neo4j HA cluster with an HA Proxy server serving as an endpoint is available to be used.   If a bigger cluster is desired, just change the

node_count

value.   For additional information regarding this playbook, and how it handles the cluster membership, please refer to the following URL - https://github.com/hspencer77/ansible-neo4j-cluster/blob/master/README.md.

Hope you enjoy!  As always, questions/comments/suggestions are always welcome.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s