Our demo day for milestone 6 was yesterday, and it was choice. We're at feature completeness at this point, and we're now on final approach for release sometime Soon-ish, as soon as we shake out all the code nasties. We've got some good stuff to show off on Vimeo. The basic transcript:
- 0:00 Eric Choi, Product Mktg Manager, with agenda/housekeeping.
I thought the Ansible 1.0 development cycle was busy but 1.1 is crammed full of orchestration goodness. On Tuesday, 1.1 was released and you can read more about it here: http://blog.ansibleworks.com/2013/04/02/ansible-1-1-released/
For those working on AWS and Eucalyptus, 1.1 brings some nice module improvements as well as a new cloudformation and s3 module. It's great to see the AWS-related modules becoming so popular so quickly.
Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). - Wikipedia
According to Wikipedia currently there are few popular service models exist.
1. Infrastructure as a service (IaaS)
2. Platform as a service (PaaS)
3. Software as a service (SaaS)
So, I have an Eucalyptus cloud, which is great, serves as AWS-like IaaS platform.
Jenkins, Stackato, Cloud-Init and Eucalyptus == Potent Combination for an On-Premise Continuous Integration Environment
An extendable open source continuous integration server.
The Enterprise Private PaaS that makes it easy to deploy, manage, and monitor applications on any cloud.
The Ubuntu package that handles early initialization of a cloud instance. It is installed in the Ubuntu Cloud Images and also in the official Ubuntu images available on EC2.
Allows you to build production-ready, AWS-compatible private and hybrid clouds by leveraging your existing virtualized infrastructure to create on-demand cloud resource pools.
What happens when you combine all three of these tools? A potent combination for continuous integration on a easy-to-configure PaaS and an on-premise, AWS-compatible IaaS. With this combination, developers can take advantage of easy configuration that Stackato brings to the table, running on top of Eucalyptus – bringing an AWS-like cloud environment into your datacenter.
This blog entry will discuss the steps that I took to get Jenkins installed on a Stackato instance store-backed instance running on Eucalyptus. But before I get started, I would like to thank the folks from ActiveState for all their guidance. Their support staff is really top notch, and very helpful. Check them out in #stackato on freenode.net. They can also be checked out on Twitter at @ActiveState. Now on to the dirty work…..
The Recipe for Success
The Stackato Microcloud Image and Cloud-Init
To begin, the following is needed:
- A running Eucalyptus cloud
- User credentials and proper Eucalyptus IAM policies to allow uploading of images and launching instances. (If there is more information needed here, please check out the Managing Access section in Eucalyptus 3.2 Administrator’s Guide.)
- the Stackato Mircocloud VM for KVM
- A Linux desktop with euca2ools installed.
After downloading the Stackato VM for KVM and unzipping the file, we will need to pull out the root file system, the kernel and ramdisk. These will be uploaded, bundled and registered as the EMI, EKI, and ERI. To extract the root filesystem, do the following:
- Use parted to locate the root filesystem as follows:
# parted stackato-img-kvm-v2.6.6.img GNU Parted 2.1 Using /root/images/Stackato-VM/stackato-img-kvm-v2.6.6.img Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) U Unit? [compact]? b (parted) p Model: (file) Disk /root/images/Stackato-VM/stackato-img-kvm-v2.6.6.img: 10737418240B Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 1048576B 200278015B 199229440B primary ext3 boot 3 200278016B 1237319679B 1037041664B primary linux-swap(v1) 2 1237319680B 10736369663B 9499049984B primary ext4 (parted) quit
- In this example, the root filesystem is partition 2. The value for “Start” and “Size” will need to be used. Next, run dd to extract the root filesystem:
dd if=stackato-img-kvm-v2.6.6.img of=stackato-rootfs.img bs=1 skip=1237319680 count=9499049984
- Once it has completed, mount stackato-rootfs.img to the loopback device:
mount -o loop stackato-rootfs.img /mnt/
- Copy out initrd.img-3.2.0-27-virtual and vmlinuz-3.2.0-27-virtual from /mnt/boot.
- In /mnt/etc/fstab, replace the UUID entry with LABEL. The LABEL will look simliar to the following:
LABEL=cloudimg-rootfs / ext4 defaults 1 1
- Chroot to /mnt – there may be a need to do a mount -o bind for sys, dev, and proc.
- Run “dpkg-reconfigure cloud-init”, and make sure that the EC2 Data Source is selected.
- Unmount stackato-rootfs.img (if sys, dev, and proc were mounted, unmount them before unmounting stackato-rootfs.img). After it has been unmounted, run tune2fs to change the label of the image:
tune2fs -L cloudimg-rootfs stackato-rootfs.img
After following these steps, the following should be available:
- initrd.img-3.2.0-27-virtual – to be bundled, uploaded and registered as the ERI
- vmlinuz-3.2.0-27-virtual – to be bundled, uploaded and registered as the EKI
- stackato-rootfs.img – to be bundled, uploaded and registered as the EMI
Launching the Stackato Image
Now its time to launch the Stackato image on Eucalyptus. Since cloud-init has the enabled EC2 data source now, when the image is launched, the instance will grab ssh keys, and mount the ephemeral storage. Also, additional configuration can be passed using the user-data file option. More information regarding this can be found on Stackato’s documentation in reference to using cloud-init. Key thing to remember here is that the minimum RAM requirement for the Stackato image is 2 gigs. Make sure the VM type used for launching the Stackato image has at least 2 gigs of RAM or more. In this example, the image ID is emi-DAB23A8A. The ramdisk and kernel are registered as eri-9B453C09 and eki-ADF640B0. The VM type c1.xlarge is used, which has 4 CPU, 4096 MB of RAM, and 50 Gigs of disk space.
euca-run-instances -k admin emi-DAB23A8A -t c1.xlarge --kernel eki-ADF640B0 --ramdisk eri-9B453C09
Use euca-describe-instances to check to see when the instance reaches a running state:
euca-describe-instances i-100444EF RESERVATION r-CC69438B 345590850920 default INSTANCE i-100444EF emi-DAB23A8A euca-192-168-55-104.wu-tang.euca-hasp.eucalyptus-systems.com euca-10-106-101-17.wu-tang.internal running admin 0 c1.xlarge 2013-02-23T00:34:07.436Z enter-the-wu eki-ADF640B0 eri-9B453C09 monitoring-disabled 192.168.55.104 10.106.101.17 instance-store
The key thing for running a Stackato instance is setting up the correct DNS entries. For more information regarding setting up DNS with regards to a Stackato instance, please read the Detail Configuration section on DNS in the Stackato online documentation. For this example, instead of using an external DNS service using a tool like nsupdate, to configure the A record and CNAME records, we will use xip.io. xip.io is a magic domain name that provides wildcard DNS for any IP address. Next, its time to configure the Stackato instance.
Configuration of the Stackato Instance
To configure the Stackato instance, do the following:
- SSH into the instance.
ssh -i creds/admin.priv firstname.lastname@example.org
- Make note of the ip address associated with eth0 and the netmask using ifconfig. Also note the gateway IP by using the route command.
- Run “kato op static_ip” to configure the static IP address for eth0. Make sure and add 127.0.0.1 as the first entry as part of the nameservers, and add “containers.” as the first entry under the search domains.
- Run “kato rename public DNS name “, where public DNS name includes the public IP of the instance, using xip.io (e.g. 192.168.55.104.xip.io)
- Run “kato disable mdns”, then run “sudo reboot” to reboot the instance.
- Once the instance has come back up, ssh into the instance, and run the following command ”kato setup core api.public DNS name” where public DNS name is the same value used for the “kato rename” step (e.g. 192.168.55.104.xip.io).
- Next, edit /etc/resolv.conf and make sure that the value for the search option is “containers.”, and the first entry for the nameservers is 127.0.0.1.
- Finally, run “kato enable –all-but mdns”
Thats it! Now go to the public DNS name that was used in your favorite browser. For this example, 192.168.55.104.xip.io was used. The following landing page should look similar to what you see here in the Stackato documentation regarding accessing the instance through the management console.
Setting Up Jenkins
After setting up the admin account, navigate to the “App Store” on the lefthand menu. Once selected, navigate to find the Jenkins application:
After selecting to install Jenkins, select “Yes” to install. After the installation takes place, select “Applications” in the left hand menu. From there, select the Jenkins application, and select “Start” (its the green arrow to the right of the application window). Once it has started, you will see the following:
Now Jenkins is ready to be used.
If anyone wants to test this out on Eucalyptus but doesn’t have access to their own Eucalyptus cloud, fear not, the Eucalyptus Community Cloud has the Stackato image available. After applying to get access to the Community Cloud, follow the steps above. The image for Stackato is as follows:
IMAGE emi-859B3D5C stackato_v2.6.6/stackato-cloudinit.manifest.xml 150820662310 available public x86_64 machine eki-6FBE3D2D eri-67463B77 instance-store
And as always, this image and steps can be used on AWS EC2 as well.
Let me know if there are any questions. Feedback is always welcome. Enjoy!
Version 3 of euca2ools, slated for release in just a couple months, gives the command line suite a much-needed refresh that makes it both easier to write and easier to use. Most of the innovation here involves changes to the platform upon which it is built. I will cover those changes from a developer's perspective in future blog posts, but today I'm going to focus on what euca2ools 3 brings to the table for developers and other users alike.
My previous post talked a little bit about new functionality from (new and updated) ec2-related modules in the Ansible 1.0 release. In this post I'll go through the practical example of launching multiple instances with the ec2 module and then configuring them with a workload. In this case the workload will be the Eucalyptus User Console :)
For those who are unfamiliar with ansible, check out the online documentation…
You may be forgiven for thinking that a version 1.0 software release indicates some sort of significant milestone for the lifecycle of a project. Perhaps in many cases it does but with Ansible, not so much. Michael DeHaan articulates it much better than I could in this post to the project mailing list. My personal experience from using Ansible since v0.8 is that each release delivers consistency, quality and increased flexibility.
At the end of last year, we (Eucalyptus) released version 3.2 which included our user console. This feature finally allowed regular users to login to a web UI to manage their resources. Because this was our first release, we had a lot of catching up to do. I would say that is still the case, but the point here is that we were able to test all of our features against Eucalyptus.
A few weeks back, I was doing some testing with the guys from AppScale to get an Eucalyptus Machine Image (EMI) to run on Eucalyptus. The image that was provided to me was an EBS-backed Amazon Machine Image (AMI), using a published EC2 Lucid Ubuntu Cloud image. This blog entry describes the procedure to convert an EBS-backed AMI to an Walrus-backed EMI. The goal here is to demonstrate how easy it is to use Ubuntu Cloud images to set up AppScale on both AWS and Eucalyptus as a hybrid cloud use case. There are many other hybrid cloud use cases that can be done with this setup, but this blog entry will focus on the migration of AMI images to EMI images.
*NOTE* This entry assumes that a user is experienced with both Amazon Web Services and Eucalyptus. For additional information, please refer to the following resources:
- Amazon Elastic Compute Cloud User Guide
- AWS Identity and Access Management – Using IAM Guide
- Amazon Elastic Compute Cloud CLI Guide
- Eucalyptus 3.2 Administrator’s Guide
- Eucalyptus 3.2 User Guide
Before getting started, the following is needed:
- For Amazon Web Services
- For Eucalyptus
*NOTE* Make sure there is an understanding of the IAM policies on AWS and Eucalyptus. These are key in making sure that the user on both AWS and Eucalyptus can perform all the steps covered in this topic.
Work in AWS…
After setting up the command-line tools for AWS EC2, and adding in the necessary EC2 and S3 IAM policies, everything is in place to get started with working with the AWS instances and images. *NOTE* To get help with setting up the IAM policies, check out the AWS Policy Generator. To make sure things look good, I tested out my EC2 access by running ec2-describe-availability-zones:
$ ec2-describe-availability-zones AVAILABILITYZONE us-east-1a available us-east-1 AVAILABILITYZONE us-east-1b available us-east-1 AVAILABILITYZONE us-east-1c available us-east-1 AVAILABILITYZONE us-east-1d available us-east-1
After that, I set up a keypair and SSH access for any instance that is launched within the default security group:
$ ec2-create-keypair hspencer-appscale –region ap-northeast-1 > hspencer-appscale.pem
$ ec2-authorize -P tcp -p 22 -s 0.0.0.0/0 default –region ap-northeast-1
With everything looking good, I went ahead and checked out the AMI that I was asked to test. Below is the AMI that was given to me:
$ ec2-describe-images ami-2e4bf22f --region ap-northeast-1 IMAGE ami-2e4bf22f 839953741869/appscale-lite-1.6.3-testing 839953741869 available public x86_64 machine aki-d409a2d5 ebs paravirtual xen BLOCKDEVICEMAPPING EBS /dev/sda1 snap-7953a059 8 true standard BLOCKDEVICEMAPPING EPHEMERAL /dev/sdb ephemeral0
As you can see, the AMI given to me is an EBS-backed image, and it is in a different region (ap-northeast-1). I could have done all my work in the ap-northeast-1 region, but I wanted to test out region-to-region migration of images on AWS S3 using ec2-migrate-manifest. In order to access the EBS-backed instance that is launched, I set up a keypair and SSH access for any instance that is launched within the default security group:
$ ec2-create-keypair hspencer-appscale --region ap-northeast-1 > hspencer-appscale.pem $ ec2-authorize -P tcp -p 22 -s 0.0.0.0/0 default --region ap-northeast-1
Now that I have my image, keypair and security group access, I am ready to launch an instance, so I can use the ec2-bundle-vol command to create an image of the instance. To launch the instance, I ran the following:
$ ec2-run-instances -k hspencer-appscale ami-2e4bf22f –region ap-northeast-1
After the instance is up and running, I scp’d my EC2_PRIVATE_KEY and EC2_CERT to the instance using the keypair created (hspencer-appscale.pem). The instance already had the latest version of ec2-api-tools and ec2-ami-tools as part of the installation of AppScale. Similar to the instructions provided by AWS for creating an instance-store backed AMI from an existing AMI, I used ec2-bundle-vol to bundle a new image and used /mnt/ (which is ephemeral storage) to store the manifest information.
root@ip-10-156-123-126:~# ec2-bundle-vol -u 9xxxxxxx3 -k pk-XXXXXXXXXXXXXXXX.pem -c cert-XXXXXXXXXXXXXXXXX.pem -d /mnt/ -e /mnt/ Please specify a value for arch [x86_64]: x86_64 Copying / into the image file /mnt/image... Excluding: /dev /sys /sys/kernel/security /sys/kernel/debug /proc /dev/pts /dev /media /mnt /proc /sys /mnt/ /mnt/image /mnt/img-mnt 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990555 s, 106 MB/s mke2fs 1.41.11 (14-Mar-2010) Bundling image file... Splitting /mnt/image.tar.gz.enc... Created image.part.000 ……………..
Next, I need to inform the manifest to use us-west-1 as the region to store the image, and not ap-northeast-1. To do this, I used ec2-migrate-manifest. *NOTE* This tool can only be used in the following regions: EU,US,us-gov-west-1,us-west-1,us-west-2,ap-southeast-1,ap-southeast-2,ap-northeast-1,sa-east-1.
root@ip-10-156-123-126:~# ec2-migrate-manifest -m /mnt/image.manifest.xml -c cert-XXXXXXXXX.pem -k pk-XXXXXXXXXXXX.pem -a XXXXXXXXXX -s XXXXXXXXX --region us-west-1 Backing up manifest... warning: peer certificate won't be verified in this SSL session warning: peer certificate won't be verified in this SSL session warning: peer certificate won't be verified in this SSL session warning: peer certificate won't be verified in this SSL session Successfully migrated /mnt/image.manifest.xml It is now suitable for use in us-west-1.
Time to upload the bundle to S3 using ec2-upload-bundle:
root@ip-10-156-123-126:~# ec2-upload-bundle -b appscale-lite-1.6.3-testing -m /mnt/image.manifest.xml -a XXXXXXXXXX -s XXXXXXXXXX --location us-west-1 You are bundling in one region, but uploading to another. If the kernel or ramdisk associated with this AMI are not in the target region, AMI registration will fail. You can use the ec2-migrate-manifest tool to update your manifest file with a kernel and ramdisk that exist in the target region. Are you sure you want to continue? [y/N]y Creating bucket... Uploading bundled image parts to the S3 bucket appscale-lite-1.6.3-testing ... Uploaded image.part.000 Uploaded image.part.001 Uploaded image.part.002 Uploaded image.part.003 ………….
After the image has been uploaded successfully, all that is left to do is register the image.
root@ip-10-156-123-126:~# export JAVA_HOME=/usr root@ip-10-156-123-126:~# ec2-register -K pk-XXXXXXXXXXXX.pem -C cert-XXXXXXXXXX.pem --region us-west-1 appscale-lite-1.6.3-testing/image.manifest.xml --name appscale1.6.3-testing IMAGE ami-705d7c35 $ ec2-describe-images ami-705d7c35 --region us-west-1 IMAGE ami-705d7c35 986451091583/appscale1.6.3-testing 986451091583 available private x86_64 machine aki-9ba0f1de instance-store paravirtual xen BLOCKDEVICEMAPPING EPHEMERAL /dev/sdb ephemeral0
Work in Eucalyptus…
Now that we have the image registered, we can use ec2-download-bundle and ec2-unbundle to get the machine image to an instance running on Eucalyptus, so that we can bundle, upload and register the image to Eucalyptus.
- lucid-server-cloudimg-amd64-loader (ramdisk)
- lucid-server-cloudimg-amd64-vmlinuz-virtual (kernel)
- lucid-server-cloudimg-amd64.img (root image)
After bundling, uploading and registering those images, I created a keypair, and SSH access for the instance that is launched within the default security group:
euca-add-keypair hspencer-euca > hspencer-euca.pem euca-authorize -P tcp -p 22 -s 0.0.0.0/0 default
Now, I run the EMI for the Lucid image that was registered:
euca-run-instance -k hspencer-euca --user-data-file cloud-init.config -t m1.large emi-29433329
I used vm.type m1.large so that I can use the space on ephemeral to store the image that I will pull from AWS.
Once the instance is running, I scp’d my EC2_PRIVATE_KEY and EC2_CERT to the instance using the keypair created (hspencer-euca.pem). After installing the ec2-ami-tools on the instance, I used ec2-download-bundle to download the bundle to /media/ephemeral0, and ec2-unbundle the image:
# ec2-download-bundle -b appscale-lite-1.6.3-testing -d /media/ephemeral0/ -a XXXXXXXXXXX -s XXXXXXXXXXXX -k pk-XXXXXXXXX.pem --url http://s3-us-west-1.amazonaws.com # ec2-unbundle -m /media/ephemeral0/image.manifest.xml -s /media/ephemeral0/ -d /media/ephemeral0/ -k pk-XXXXXXXXXX.pem
Now that I have the root image from AWS, I just need to bundle, upload and register the root image to Eucalyptus. To do so, I scp’d my Eucalyptus user credentials to the instance. After copying the Eucalyptus credentials to the instance, I ssh’ed into the instance and source the Eucalyptus credentials.
Since I have already bundled the kernel and ramdisk for the Ubuntu Cloud Lucid image before, I just need to upload, bundle and register the image I unbundled from AWS. To do so, I did the following:
euca-bundle-image -i image euca-upload-bundle -b appscale-1.6.3-x86_64 -m /tmp/image.manifest.xml euca-register -a x86_64 appscale-1.6.3-x86_64/image.manifest.xml
Now the image is ready to be launched on Eucalyptus.
As demonstrated above, because of the AWS fidelity that Eucalyptus provides, it enables setting up hybrid cloud environments with Eucalyptus and AWS that can be leveraged by applications, like AppScale.
Other examples of AMI to EMI conversions can be found here: