In 2014, I created a series of blog posts that have discussed using CoreOS on Eucalyptus cloud infrastructures.  This blog post is an updated version of the entry which discussed how to deploy a CoreOS cluster using a cloudformation template on Eucalyptus 4.0.1.  It will cover how to deploy a CoreOS cluster using Cloudformation on a HPE Helion Eucalyptus 4.2 VPC environment.

In HPE Helion Eucalyptus 4.1, VPC (Virtual Private Cloud) was in technical preview state.  With the release of Eucalyptus 4.2, VPC was upgraded to stable release.  HPE Helion Eucalyptus VPC provides similar features as AWS VPC.  For more information about what is currently supported in Eucalyptus VPC, please refer to the online documentation.

Prerequisites

Prerequisites for this blog entry are listed in the following previous blogs:

• Using Eucalyptus 4.0.1 Cloudformation To Deploy a CoreOS Cluster
• CoreOS CloudInit Config For Docker Storage Management

Please note the information regarding HPE Helion Eucalyptus IAM and how to obtain the CoreOS Beta AMI image in the previous listed blog entries.

CoreOS ETCD Discovery Service Token

When setting up the CoreOS cluster, the method used to handle cluster membership is using etcd Discovery.  This provides a unique discovery URL that will show all the members of the cluster.  To obtain a token for the size of the cluster you desire, use the following URL and add the value for the size of the cluster.  For example, if the cluster will have five members, using curl – the request URL will look like the following:

curl https://discovery.etcd.io/new?size=5

The value returned will look similar to the following:

https://discovery.etcd.io/fdd7d8ac203d2cac0c27ead148ad83ed

This URL can be referenced to see if all the members of the cluster registered successfully.

Deploying the Cluster on HPE Helion Eucalyptus VPC

When deploying the cluster on a Eucalyptus VPC environment, there are additional variables that have to be taken into account.  To download the example template, use the following URL:

https://s3-us-west-1.amazonaws.com/cfn-coreos-deployment/cfn-coreos-as-vpc.json

After downloading the template, use either euca2ools or AWS CLI to validate the template.  This will display the arguments that need to be passed when creating the cloudformation stack on Eucalyptus.  For example:

# euform-validate-template --template-file cfn-coreos-as.json 
DESCRIPTION Deploy CoreOS Cluster on Eucalyptus VPC
PARAMETER VpcId false VpcId of your existing Virtual Private Cloud (VPC)
PARAMETER Subnets false The list of SubnetIds in your Virtual Private Cloud (VPC)
PARAMETER AZs false The list of AvailabilityZones for your Virtual Private Cloud (VPC)
PARAMETER CoreOSImageId false CoreOS Image Id
PARAMETER UserKeyPair true User Key Pair
PARAMETER ClusterSize false Desired CoreOS Cluster Size
PARAMETER VmType false Desired VM Type for Instances

Notice the template requires unique variables associated with HPE Helion Eucalyptus VPC.

Now that the template has been downloaded, create the CoreOS stack using euca2ools.  For example:

# euform-create-stack CoreOSCluster --template-file cfn-coreos-as.json --parameter Subnets=subnet-0814e7aa,subnet-5d816215,subnet-c3755d6c --parameter AZs=euca-east-1c,euca-east-1b,euca-east-1a --parameter CoreOSImageId=emi-dfa27782 --parameter UserKeyPair=devops-admin --parameter ClusterSize=5 --parameter VmType=m1.large --parameter VpcId=vpc-d7fcff27

Once the cluster has been deployed, confirm that the cloudformation stack deployed successfully:

# euform-describe-stacks
STACK CoreOSCluster CREATE_COMPLETE Complete! Deploy CoreOS Cluster on Eucalyptus VPC 2016-01-01T21:09:10.965Z
PARAMETER VpcId vpc-d7fcff27
PARAMETER Subnets subnet-0814e7aa,subnet-5d816215,subnet-c3755d6c
PARAMETER AZs euca-east-1c,euca-east-1b,euca-east-1a
PARAMETER CoreOSImageId emi-dfa27782
PARAMETER UserKeyPair ****
PARAMETER ClusterSize 5
PARAMETER VmType m1.large
OUTPUT AutoScalingGroup CoreOSCluster-CoreOsGroup-JTKMRINKKMYDI

Check the discovery URL using curl, wget or any browser to confirm that the cluster membership completed:

# curl https://discovery.etcd.io/fdd7d8ac203d2cac0c27ead148ad83ed
{"action":"get","node":{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed","dir":true,"nodes":[{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed/d0a4c6d73d0d8d17","value":"8981923b54d7d7f46fabc527936a7dcf=http://172.31.4.17:2380","modifiedIndex":953833155,"createdIndex":953833155},{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed/12b6e6e78c9cb70c","value":"33a3209006d2be1d5be0da6eaea007c5=http://172.31.19.215:2380","modifiedIndex":953833156,"createdIndex":953833156},{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed/d5c5d93e360ba87","value":"e71b1fefcd65c43a0fbacc7103efbc2b=http://172.31.22.157:2380","modifiedIndex":953833162,"createdIndex":953833162},{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed/cffd4985c990f872","value":"f047b9ff24f3d0c4e74c660709103b36=http://172.31.6.166:2380","modifiedIndex":953833167,"createdIndex":953833167},{"key":"/_etcd/registry/fdd7d8ac203d2cac0c27ead148ad83ed/8e6ccfef42f98260","value":"c48b163558b61733c1aa44dccb712406=http://172.31.47.175:2380","modifiedIndex":953833339,"createdIndex":953833339}],"modifiedIndex":953831075,"createdIndex":953831075}}

To confirm the health of the cluster, SSH into one of the cluster nodes, and use fleetctl and etcdctl:

# ssh -i devops-admin-key core@euca-10-116-131-230.eucalyptus.c-05.autoqa.qa1.eucalyptus-systems.com
Last login: Sat Jan 2 23:53:25 2016 from 10.111.1.71
CoreOS beta (877.1.0)
core@euca-172-31-22-157 ~ $ fleetctl list-machines
MACHINE IP METADATA
33a32090... 10.116.131.107 purpose=coreos-cluster,region=euca-us-east-1
8981923b... 10.116.131.121 purpose=coreos-cluster,region=euca-us-east-1
c48b1635... 10.116.131.213 purpose=coreos-cluster,region=euca-us-east-1
e71b1fef... 10.116.131.230 purpose=coreos-cluster,region=euca-us-east-1
f047b9ff... 10.116.131.197 purpose=coreos-cluster,region=euca-us-east-1
core@euca-172-31-22-157 ~ $ etcd
etcd etcd2 etcdctl
core@euca-172-31-22-157 ~ $ etcdctl cluster-health
member d5c5d93e360ba87 is healthy: got healthy result from http://10.116.131.230:2379
member 12b6e6e78c9cb70c is healthy: got healthy result from http://10.116.131.107:2379
member 8e6ccfef42f98260 is healthy: got healthy result from http://10.116.131.213:2379
member cffd4985c990f872 is healthy: got healthy result from http://10.116.131.197:2379
member d0a4c6d73d0d8d17 is healthy: got healthy result from http://10.116.131.121:2379
cluster is healthy
core@euca-172-31-22-157 ~ $ etcdctl member list
d5c5d93e360ba87: name=e71b1fefcd65c43a0fbacc7103efbc2b peerURLs=http://172.31.22.157:2380 clientURLs=http://10.116.131.230:2379
12b6e6e78c9cb70c: name=33a3209006d2be1d5be0da6eaea007c5 peerURLs=http://172.31.19.215:2380 clientURLs=http://10.116.131.107:2379
8e6ccfef42f98260: name=c48b163558b61733c1aa44dccb712406 peerURLs=http://172.31.47.175:2380 clientURLs=http://10.116.131.213:2379
cffd4985c990f872: name=f047b9ff24f3d0c4e74c660709103b36 peerURLs=http://172.31.6.166:2380 clientURLs=http://10.116.131.197:2379
d0a4c6d73d0d8d17: name=8981923b54d7d7f46fabc527936a7dcf peerURLs=http://172.31.4.17:2380 clientURLs=http://10.116.131.121:2379

Thats it! The CoreOS cluster has been successfully deployed.  Given HPE Helion Eucalyptus’s AWS compatibility, this template can be used on AWS as well.

As always, please let me know if there are any questions.  Enjoy!