What’s new in Ansible 1.1 for AWS and Eucalyptus users?

hspencer77:

Ansible providing more AWS/Eucalyptus support!

Originally posted on Take that to the bank and cash it!:

I thought the Ansible 1.0 development cycle was busy but 1.1 is crammed full of orchestration goodness.  On Tuesday, 1.1 was released and you can read more about it here: http://blog.ansibleworks.com/2013/04/02/ansible-1-1-released/

For those working on AWS and Eucalyptus, 1.1 brings some nice module improvements as well as a new cloudformation and s3 module.  It’s great to see the AWS-related modules becoming so popular so quickly.  Here are some more details about the changes but you can find info in the changelog here: https://github.com/ansible/ansible/blob/devel/CHANGELOG.md

Security group ID support

It’s now possible to specify the security group by its ID.  This is quite typically behaviour in EC2 and Eucalyptus will support this with the pending 3.3 release.  The parameter is optional.

VPC subnet ID

VPC users can now specify a subnet ID associated with their instance.

Instance state wait timeout

In 1.0 there was no way to specify how long to wait…

View original 300 more words

UCSB Sweethearts of Cloud Computing: AppScale and Eucalyptus

hspencer77:

AppScale and Eucalyptus: Great Tag Team for On-Premise Cloud Computing

Originally posted on shaon's Blog:

Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). – Wikipedia

According to Wikipedia currently there are few popular service models exist.

1. Infrastructure as a service (IaaS)
2. Platform as a service (PaaS)
3. Software as a service (SaaS)

So, I have an Eucalyptus cloud, which is great, serves as AWS-like IaaS platform. But now I want PaaS. And right here Appscale comes into play with full compatibility of Google App Engine (GAE) applications. In this post, we will install the popular open source PaaS framework Appscale on Eucalyptus, the AWS compatible open source IaaS platform.

Agenda
0. Introduction
1. Resize Lucid image
2. Install Appscale from source
3. Install Appscale Tool
4. Bundle Appscale image
5. Run Appscale
6. Run an application on Appscale

Eucalyptus
Eucalyptus Cloud platform is open source software for building…

View original 662 more words

Jenkins, Stackato, Cloud-Init and Eucalyptus == Potent Combination for an On-Premise Continuous Integration Environment

The Ingredients

Jenkins

An extendable open source continuous integration server.

Stackato

The Enterprise Private PaaS that makes it easy to deploy, manage, and monitor applications on any cloud.

Cloud-init

The Ubuntu package that handles early initialization of a cloud instance. It is installed in the Ubuntu Cloud Images and also in the official Ubuntu images available on EC2.

Eucalyptus

Allows you to build production-ready, AWS-compatible private and hybrid clouds by leveraging your existing virtualized infrastructure to create on-demand cloud resource pools.

What happens when you combine all three of these tools?  A potent combination for continuous integration on a easy-to-configure PaaS and an on-premise, AWS-compatible IaaS.  With this combination, developers can take advantage of easy configuration that Stackato brings to the table, running on top of Eucalyptus – bringing an AWS-like cloud environment into your datacenter.

This blog entry will discuss the steps that I took to get Jenkins installed on a Stackato instance store-backed instance running on Eucalyptus.  But before I get started, I would like to thank the folks from ActiveState for all their guidance.  Their support staff is really top notch, and very helpful.  Check them out in #stackato on freenode.net.  They can also be checked out on Twitter at @ActiveState. Now on to the dirty work…..

The Recipe for Success

The Stackato Microcloud Image and Cloud-Init

To begin, the following is needed:

After downloading the Stackato VM for KVM and unzipping the file, we will need to pull out the root file system, the kernel and ramdisk.  These will be uploaded, bundled and registered as the EMI, EKI, and ERI.  To extract the root filesystem, do the following:

  1. Use parted to locate the root filesystem as follows:    
    # parted stackato-img-kvm-v2.6.6.img
    GNU Parted 2.1
    Using /root/images/Stackato-VM/stackato-img-kvm-v2.6.6.img
    Welcome to GNU Parted! Type 'help' to view a list of commands.
    (parted) U
    Unit? [compact]? b
    (parted) p
    Model: (file)
    Disk /root/images/Stackato-VM/stackato-img-kvm-v2.6.6.img: 10737418240B
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    
    Number Start End Size Type File system Flags
    1 1048576B 200278015B 199229440B primary ext3 boot
    3 200278016B 1237319679B 1037041664B primary linux-swap(v1)
    2 1237319680B 10736369663B 9499049984B primary ext4
    
    (parted) quit
  2. In this example, the root filesystem is partition 2.  The value for “Start” and “Size” will need to be used.  Next, run dd to extract the root filesystem:
    dd if=stackato-img-kvm-v2.6.6.img of=stackato-rootfs.img
     bs=1 skip=1237319680 count=9499049984
    
  3. Once it has completed, mount  stackato-rootfs.img to the loopback device:
    mount -o loop stackato-rootfs.img /mnt/
  4. Copy out initrd.img-3.2.0-27-virtual  and vmlinuz-3.2.0-27-virtual from /mnt/boot.
  5. In /mnt/etc/fstab, replace the UUID entry with LABEL.  The LABEL will look simliar to the following:  
    LABEL=cloudimg-rootfs    /               ext4   defaults     1       1
  6. Chroot to /mnt – there may be a need to do a mount -o bind for sys, dev, and proc.
  7. Run “dpkg-reconfigure cloud-init”, and make sure that the EC2 Data Source is selected.
  8. Unmount stackato-rootfs.img (if sys, dev, and proc were mounted, unmount them before unmounting stackato-rootfs.img).  After it has been unmounted, run tune2fs to change the label of the image:
    tune2fs -L cloudimg-rootfs stackato-rootfs.img

After following these steps, the following should be available:

  • initrd.img-3.2.0-27-virtual – to be bundled, uploaded and registered as the ERI
  • vmlinuz-3.2.0-27-virtual – to be bundled, uploaded and registered as the EKI
  • stackato-rootfs.img – to be bundled, uploaded and registered as the EMI

Go through the steps of bundling, uploading and registering the ERI, EKI, and EMI.  For additional information, please refer to the Add an Image section of the Eucalyptus 3.2 User Guide.

Launching the Stackato Image

Now its time to launch the Stackato image on Eucalyptus.  Since cloud-init has the enabled EC2 data source now, when the image is launched, the instance will grab ssh keys, and mount the ephemeral storage.  Also, additional configuration can be passed using the user-data file option.  More information regarding this can be found on Stackato’s documentation in reference to using cloud-init.  Key thing to remember here is that the minimum RAM requirement for the Stackato image is 2 gigs.  Make sure the VM type used for launching the Stackato image has at least 2 gigs of RAM or more.  In this example, the image ID is emi-DAB23A8A.  The ramdisk and kernel are registered as eri-9B453C09 and eki-ADF640B0.   The VM type c1.xlarge is used, which has 4 CPU,  4096 MB of RAM,  and    50 Gigs of disk space.

euca-run-instances -k admin emi-DAB23A8A -t c1.xlarge 
--kernel eki-ADF640B0 --ramdisk eri-9B453C09

Use euca-describe-instances to check to see when the instance reaches a running state:

euca-describe-instances i-100444EF
RESERVATION r-CC69438B 345590850920 default
INSTANCE i-100444EF emi-DAB23A8A euca-192-168-55-104.wu-tang.euca-hasp.eucalyptus-systems.com 
euca-10-106-101-17.wu-tang.internal running admin 0 c1.xlarge 2013-02-23T00:34:07.436Z enter-the-wu eki-ADF640B0 
eri-9B453C09 monitoring-disabled 192.168.55.104 10.106.101.17 instance-store

The key thing for running a Stackato instance is setting up the correct DNS entries.  For more information regarding setting up DNS with regards to a Stackato instance, please read the Detail Configuration section on DNS in the Stackato online documentation.  For this example, instead of using an external DNS service using a tool like nsupdate, to configure the A record and CNAME records, we will use xip.io. xip.io is a magic domain name that provides wildcard DNS for any IP address.  Next, its time to configure the Stackato instance.

Configuration of the Stackato Instance

To configure the Stackato instance, do the following:

  1. SSH into the instance.  

    ssh -i creds/admin.priv 
    stackato@euca-192-168-55-104.wu-tang.euca-hasp.eucalyptus-systems.com
  2. Make note of the ip address associated with eth0 and the netmask using ifconfig.  Also note the gateway IP by using the route command.
  3. Run “kato op static_ip” to configure the static IP address for eth0.  Make sure and add 127.0.0.1 as the first entry as part of the nameservers, and add “containers.” as the first entry under the search domains.
  4. Run “kato rename public DNS name “, where public DNS name includes the public IP of the instance, using xip.io (e.g. 192.168.55.104.xip.io)
  5. Run “kato disable mdns”, then run “sudo reboot” to reboot the instance.
  6. Once the instance has come back up, ssh into the instance, and run the following command  “kato setup core api.public DNS name” where public DNS name is the same value used for the “kato rename” step (e.g.  192.168.55.104.xip.io).
  7. Next, edit /etc/resolv.conf and make sure that the value for the search option is “containers.”, and the first entry for the nameservers is 127.0.0.1.
  8. Finally, run “kato enable –all-but mdns”

Thats it!  Now go to the public DNS name that was used in your favorite browser.  For this example, 192.168.55.104.xip.io was used.  The following landing page should look similar to what you see here in the Stackato documentation regarding accessing the instance through the management console.

Setting Up Jenkins

After setting up the admin account, navigate to the “App Store” on the lefthand menu.  Once selected, navigate to find the Jenkins application:

 

Jenkins Application

Jenkins Application

 

After selecting to install Jenkins, select “Yes” to install.  After the installation takes place,  select “Applications” in the left hand menu.  From there, select the Jenkins application, and select “Start” (its the green arrow to the right of the application window).   Once it has started, you will see the following:

Jenkins Running in Stackato

Jenkins Running in Stackato

 

Now Jenkins is ready to be used.

If anyone wants to test this out on Eucalyptus but doesn’t have access to their own Eucalyptus cloud, fear not, the Eucalyptus Community Cloud has the Stackato image available.  After applying to get access to the Community Cloud, follow the steps above.  The image for Stackato is as follows:

IMAGE emi-859B3D5C stackato_v2.6.6/stackato-cloudinit.manifest.xml 150820662310 available public x86_64 machine eki-6FBE3D2D eri-67463B77 instance-store

And as always, this image and steps can be used on AWS EC2 as well. :-)

Let me know if there are any questions.  Feedback is always welcome.  Enjoy!

Ansible Module for EC2

hspencer77:

Nice write-up on first time experience writing an Ansible module. Good stuff!

Originally posted on Take that to the bank and cash it!:

As is probably quite evident, I’ve recently been using Ansible to deploy workloads into EC2 and Eucalyptus.  One of the ideas behind this is the convenience of being able to leverage the common API to achieve a hybrid deployment scenario.  Thanks to various folk (names mentioned in previous posts) we have a solid ec2 boto-based Python module for instance launching.   One thing I wanted to do when spinning up instances and configuring a workload was to add some persistent storage for the application.  To do this I had to create and attach a volume as a manual step or run a local_action against euca2ools.  I figured I could try to write my own module to practice a bit more python (specifically boto).  The result is something terribly (perhaps in more ways that one?) simple but I think this is a testament to just how easy it is to write…

View original 253 more words

hspencer77:

Nilgiri: making cloud management a breeze..

Originally posted on shaon's Blog:

I am working on this Nilgiri project lately aiming to provide a flexible environment of managing private cloud or hybrid cloud to the end users as well as for the sysadmins to make managing cloud simpler than ever. So, someone doesn’t need a tech-expert just to run an instance or to take a snapshot for later use or may be an user want to change his server’s security group at this very moment and he doesn’t have to wait for the system admin to response.

My aim is to make a full featured web console for Eucalyptus cloud ecosystem and as Eucalyptus follows the AWS standard so eventually it will work on AWS too. Why choose Eucalyptus as a platform? Well, currently Amazon is the most widely used cloud service provider in the world and also AWS has become the De-facto standard for cloud APIs. Though Openstack was following the…

View original 323 more words

hspencer77:

Great blog about euca2ools. Provides really good insight on the future plans of euca2ools.

Originally posted on /dev/zero:

For those who don’t know, I work on the euca2ools suite of command line tools for interacting with Eucalyptus and Amazon Web Services clouds on Launchpad. As of late the project has stagnated somewhat, due in part to the sheer number of different tools it includes. Nearly every command one can send to a server that uses Amazon’s APIs should have at least one corresponding command line tool, making development of euca2ools’ code repetitive and error-prone.

Today this is going to end.

But before we get to that part, let’s chronicle how euca2ools got to where they are today.

The Past

Early euca2ools versions employed the popular boto Python library to do their heavy lifting. Each tool of this sort triggers a long chain of events:

  • The tool translates data from the command line into its internal data structures.
  • The tool translates its internal data into the form that…

View original 1,451 more words

hspencer77:

Pretty sweet….using Twitter with Eucalyptus for cloud monitoring. Good stuff!

Originally posted on Kyo Lee:

Being a system administrator is the easiest job in the building when the system is working; no one questions your presence nor existence. Your tasks are highly under-appreciated during the time of peace, yet you do not mind for such vanity since you’d rather be reading blogs and watching youTubes in serenity. Every once in a while, an idiot cracks an ancient joke, “hey, aren’t you supposed be working?”. But, I’m working, you imbecile employ-of-the-month.

However, the curse begins once you step out of the building, entering the realm of unknowns, far-disconnected from the comfort of your Macbook and Wi-Fi. While staring at endless tail-lights on a freeway, having a long walk on the beach, or being queued behind 7 shopping carts at a grocery market, your mind begins to wonder, “how’s my system doing?”

Since the day 1, you have set up numerous layers of email-notification alarms, but it’s…

View original 1,156 more words