I recently ran across a blog entry entitled Neo4j 1.9.M01 – Self-managed HA. I found the concept of graph databases storing data really interesting and reached out to the guys at Neo4j to get some insight on how to deploy their HA solution on Eucalyptus. Amongst the resources that they provided, they shared this little gem – how to deploy Neo4j on EC2. In order to run first, you need to know how to walk – so before going down the path of standing up HA Neo4j, I decided to be influenced by the DIY on EC2 article provided by Neo4j and deploy Neo4j on Eucalyptus – with a little help from Cloud-Init. The follow-up blog will show how to use the same setup, and deploy an HA Neo4j environment.
Ubuntu cloud images come pre-packaged with cloud-init, which helps with bootstrapping the instance.
I wanted to have the solution work on AWS EC2 and Eucalyptus; since Ubuntu cloud images work on both, its a great choice.
Registering the Ubuntu Cloud Image with Eucalyptus
In order for us to get started, we need to get the Ubuntu Cloud image into Eucalyptus so that we can use it for our instance. To upload, bundle and register the Ubuntu Cloud image, ramdisk and kernel, do the following:
$ tar -zxvf precise-server-cloudimg-amd64.tar.gz
We need to bundle, upload, and register precise-server-cloudimg-amd64-loader (ERI), precise-server-cloudimg-amd64-vmlinuz-virtual (EKI), and precise-server-cloudimg-amd64.img (EMI). For more information regarding this, please refer to the “Image Overview” section of the Eucalyptus 3.1 User Guide.
After bundling, uploading and registering the ramdisk, kernel and image, the latest-ubuntu-precise bucket in Walrus should have the following images:
$ euca-describe-images | grep latest-ubuntu-precise
IMAGE eki-0F3937E9 latest-ubuntu-precise/precise-server-cloudimg-amd64-vmlinuz-virtual.manifest.xml 345590850920 available public x86_64 kernel instance-store
IMAGE emi-C1613E67 latest-ubuntu-precise/precise-server-cloudimg-amd64.img.manifest.xml 345590850920 available public x86_64 machine instance-store
IMAGE eri-0BE53BFD latest-ubuntu-precise/precise-server-cloudimg-amd64-loader.manifest.xml 345590850920 available public x86_64 ramdisk instance-store
Cloud-init Config File
Now that we have the image ready to go, we need to create a cloud-init config file to pass in using the –user-data-file option that is part of euca-run-instances. For more examples of different cloud-init files, please refer to the cloud-init-dev/cloud-init repository on bazaar.launchpad.net. Below is the cloud-init.config file I created for bootstrapping the instance with an install of Neo4j, using ephemeral disk for the application storage, and installing some other packages (i.e. latest euca2ools, mlocate, less, etc.). The script can be also accessed from github as well – under the eucalptus/recipes repo.
Next, we need to create a security group for our instance. To create a security group, use euca-create-group. To open any ports you need for the application, use euca-authorize. The ports we will open up for the Neo4j application are SSH (22), ICMP, HTTP( 7474), and HTTPS (7473).
Create security group:
# euca-create-group neo4j-test -d "Security for Neo4j Instances"
Thats it! The cool thing about this too, is that you can find an Ubuntu Precise AMI on AWS EC2, use the same cloud-init script, use euca2ools, and follow these instructions to get the same deployment on AWS EC2.
As mentioned before, the follow-up blog will be how to deploy the HA solution of Neo4j on Eucalyptus. Enjoy!