Using Ansible to Deploy Neo4j HA Cluster on AWS/Eucalyptus

As a follow-up to my last Neo4j, AWS/Eucalyptus blog, this entry demonstrates another great example of AWS/Eucalyptus fidelity by using Ansible to deploy a Neo4j High Available cluster.

Neo4j - Graph Database
Neo4j – Graph Database
Amazon AWS EC2
Amazon AWS EC2
Eucalyptus Systems Inc.
Eucalyptus Systems Inc.


In order to use this Ansible playbook on AWS/Eucalyptus, the following is needed:

Before deploying the cluster, a security group needs to be created that the cluster will use.  The security group must allow the following:

  • port 22 (SSH)
  • all instances part of the security group allowed to community with each other (ports 0 – 65535)

To create the security group and authorize the ports, make sure the user’s access key, secret access key, and EC2 URL are noted, and do the following:

  1. Create the security group

    ec2-create-group --aws-access-key <EC2_ACCESS_KEY> 
    --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -g neo4j-cluster -d "Neo4j HA Cluster"

  2. Authorize port for SSH in neo4j-cluster security group

    --aws-access-key <EC2_ACCESS_KEY> 
    --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -P tcp -p 22 -s neo4j-cluster

  3. Authorize all port communication between cluster members 

    --aws-access-key <EC2_ACCESS_KEY> --aws-secret-key <EC2_SECRET_KEY> 
    --url <EC2_URL> -P tcp -o neo4j-cluster -p -1 neo4j-cluster

After completing these steps, use


to view the security group:

ec2-describe-group --aws-access-key <EC2_ACCESS_KEY> 
--aws-secret-key <EC2_SECRET_KEY> --url <EC2_URL> neo4j-cluster

GROUP sg-1cbc5777 986451091583 neo4j-cluster Neo4j HA Cluster
PERMISSION 986451091583 neo4j-cluster ALLOWS tcp 0 65535 FROM 
USER 986451091583 NAME neo4j-cluster ID sg-1cbc5777 ingress
PERMISSION 986451091583 neo4j-cluster 
ALLOWS tcp 22 22 FROM CIDR ingress

Neo4j HA Cluster Deployment

Once the security group is created with the correct ports authorized, the cluster can be deployed.  To deploy the cluster, do the following:

  1. Obtain Ansible from git and setup the environment by following the instructions mentioned here –
  2. Obtain the Ansible Playbook for Neo4j HA Cluster using git

    git clone

  3. Change directory into ansible-neo4j-cluster  

    cd ansible-neo4j-cluster

  4. Set up /etc/ansible/hosts with the following information:
  5. Populate vars/ec2-config with either Eucalyptus/AWS information. vars/ec2-config contains the following variables:
    keypair: <EC2/Eucalyptus Keypair>
    ec2_access_key: <EC2_ACCESS_KEY>
    ec2_secret_key: <EC2_SECRET_KEY>
    ec2_url: <EC2_URL>
    instance_type: m1.small
    security_group: <AWS/Eucalyptus Security Group>
    image: <AMI/EMI>

    Execute the following command:

    ansible-playbook neo4j-cluster.yml \
     --private-key=<AWS/Eucalyptus Private Key file> --extra-vars "node_count=3"
  7. After the playbook finishes, there will be an URL provided to access the cluster – similar to the example below:
    TASK: [Display HAProxy URL] *********************
    changed: [] => {"changed": true, "cmd": 
    "echo \"HAProxy URL for Neo4j -\" ",
     "delta": "0:00:00.006835", "end": "2013-03-30 19:54:31.104320", 
    "rc": 0, "start": "2013-03-30 19:54:31.097485", "stderr": "", 
    "HAProxy URL for Neo4j -"}

    To view the status of cluster in the browser, open up

  8. To get the status of the cluster, use curl:
    curl -H "Content-Type:application/json" -d '["org.neo4j:*"]'

Thats it!  A Neo4j HA cluster with an HA Proxy server serving as an endpoint is available to be used.   If a bigger cluster is desired, just change the


value.   For additional information regarding this playbook, and how it handles the cluster membership, please refer to the following URL –

Hope you enjoy!  As always, questions/comments/suggestions are always welcome.

Using Ansible to Deploy Neo4j HA Cluster on AWS/Eucalyptus

On Cloud 10: Eutester, MacPorts and Mac OS X Lion

The Testing Clouds at 128bpm blog – authored by my esteemed colleague Vic Iglesias – is dedicated on providing information about implementing an automated, robust test framework for Eucalyptus cloud deployments with Eutester. Today’s post was inspired by the Eutester project and the blog post Eutester Basics Part 2: Setting Up a Development Environment.

In the Eutester Basics Part 2 blog entry, the development environment is focused on being set up on an Ubuntu system.  To share the Eutester wealth, the steps documented in the blog have been modified to address setting up a development environment on Mac OS X Lion.


This blog assumes the following has been installed:

Once the prereqs are set, its on to setting up the environment.


As stated in the Eutester Basics Part 2 blog, Python 2.6 is required. Python 2.7 will be used in this setup.  In addition to Python 2.7, we need to make sure that py27-distribute (Replacement for setuptools) and  py27-virtualenv (Virtual Python Environment builder) are installed as well.  Run the following command to install these packages via MacPorts:

sudo port install python27 py27-distribute py27-virtualenv

Setting Up Your Virtual Env and Installing Required Modules

Now we are ready to set up things up for Eutester.

  1. Create the virtual Python environment:  

    virtualenv eutester-dev

  2. Enter the environment that was just created:  

    source eutester-dev/bin/activate

  3. Install required modules.  Here I will deviate a bit and install ipython as well:  

    easy_install ipython readline boto paramiko

  4. Since Mac OS X Lion includes git, we can go ahead and download eutester:  

    git clone git://

  5. Navigate to the eutester directory, and install eutester using ipython:

    cd eutester; ipython install

From here on out, just follow the instructions under the sections “Setting up your Eucarc file” and “Configuring Python shell for tab complete and history” in the Eutester Basics Part 2 blog.

When you are ready to start the virtual Python environment, instead of using python, use ipython instead:


In my opinion, ipython makes it a bit easier to develop eutester scripts in an interactive environment.  The real power is with the “?” feature.  If you aren’t sure what arguments a given method accepts, just use the “?”:

(eutester-dev)Nodachi-2:~ hspencer$ ./Desktop/eutester-dev/bin/ipython
Python 2.7.3 (default, Apr 14 2012, 19:55:34)
Type "copyright", "credits" or "license" for more information.

IPython 0.12 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: from eucaops import Eucaops

In [2]: tester = Eucaops(credpath="/Users/hspencer/Desktop/eutester-dev")
[2012-04-15 16:51:10,392] [EUTESTER] [DEBUG]: Extracting keys from /Users/hspencer/Desktop/eutester-dev

In [3]: tester.get_current_resources(?
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form:<bound method Eucaops.get_current_resources of <eucaops.Eucaops object at 0x107b99b50>>
Namespace: Interactive
File: /Users/hspencer/Desktop/eutester-dev/lib/python2.7/site-packages/eutester-0.0.1-py2.7.egg/eucaops/
Definition: tester.get_current_resources(self, verbose=False)
Return a dictionary with all known resources the system has. Optional pass the verbose=True flag to print this info to the logs
Included resources are: addresses, images, instances, key_pairs, security_groups, snapshots, volumes, zones

In [4]:

On to writing more eutester test cases….Enjoy!

On Cloud 10: Eutester, MacPorts and Mac OS X Lion