At AWS re:Invent, Amazon introduced its new EC2 Container Service (ECS). Although not available yet, it promises to be a vital part of the future of the AWS ecosystem. ECS is touted to be a high performance, highly scalable service that allows you to run distributed applications (in the form of Docker containers) on a fully managed cluster of EC2 instances. The main benefits of ECS as described by Amazon are: Easy Cluster Management, High Scale Performance, Flexible Scheduling, Extensible & Portable, Resource Efficiency, AWS Integration, and Security. All of these benefits help you easily build, run, and scale Docker containers in the cloud.
Is the concept of containers new to you? Let’s take a step back and talk about virtualization and the benefits of containers in terms of running web applications.
In its simplest most well-known form, classic computer virtualization is the process of separating the software layer (guest OS) with the hardware layer (physical server). The separation is facilitated by other layers of software (host OS and hypervisor) that act as the go-between. This gives you the ability to run multiple virtual machines on a single piece of physical hardware. This simple explanation is the basis for virtualization technologies including Amazon’s EC2 service.
Now let’s say you want to use the virtual infrastructure to run a web application. In a classic VM you are in charge of installing the OS. EC2 goes one step further than a classic VM as it provides you the virtual infrastructure with a vanilla OS. With EC2, when you fire up an instance you are given the choice of which operating system to run – Amazon Linux, Red Hat, Windows, etc. From there, the common steps needed to run a web application would be to build the application, install the needed binaries and libraries, and start the appropriate services. With a few changes to firewall rules or Security Groups, your application would be online. Congratulations you now have your application running!
So how does containerization help? I like to think of it as containerization takes virtualization one step further. Having the ability to run applications on individual virtual machines or instances is great but can become bulky and difficult to manage. An application that may be only 10-50 MBs still requires all of the binaries, libraries, and the entire guest operating system to function. This can easily require an additional 10-15 GBs, yes gigabytes, not megabytes, for the application to run on its own VM. If you want to run several applications, VM resources and administration overhead multiplies quickly. Containerization technologies like Docker have gained industry popularity for the ability to build, transport and run distributable applications in these smaller self-contained packages. A container includes just the application and needed dependencies. It runs as a separate isolated process on the host operating system and shares the kernel with other containers. This allows it to be highly portable and much more efficient by allowing multiple containerized applications to run on the same system. The beauty of it is that a Docker container is completely portable, so you can run it anywhere – like on a desktop computer, a physical server, VM, or EC2 instance – effectively facilitating faster deployments for development, QA, and production environments.
With ECS, Amazon aims to simplify managing containers even more by allowing you to run distributed applications on a managed cluster of EC2 instances. By having a managed cluster you can concentrate on your containerized applications and not cluster software or a configuration management systems to manage the infrastructure. This would be similar to how RDS is a fully managed database service that allows you to concentrate on your data and not the management and administration of the infrastructure that runs it. The light weight footprint of a container allows the environment to scale up and down quickly with demand, making it a perfect match for the elasticity of EC2. Additionally, AWS provides a set of simple APIs, so you have complete control of the cluster running your containers and the ability to extend and integrate with your current environment.
The initial announcement is definitely intriguing and something to watch closely. The service is currently in preview, but you can sign up for the waitlist here.
-Derek Baltazar – 2nd Watch Senior Cloud Engineer
Learn how you can shut down your data center and migrate to AWS in only 2 months. Alvaro Echeverri, Senior Cloud Engineer at 2nd Watch, discusses how to move data out of your data center quickly in the final part of our 5-part video series.
Learn how you can shut down your data center and migrate to AWS in only 2 months. Imran Ahmed, Practice Director at 2nd Watch, discusses key factors in ensuring a smooth transition of your data center to the cloud in part 4 of our 5-part video series.
Learn how you can shut down your data center and migrate to AWS in only 2 months. Randall Barnes, Principal Architect at 2nd Watch, discusses best practices in migrating large transactional datasets and scripting or coding recommended to orchestrate a clean cutover in part 3 of our 5-part video series.
2nd Watch has helped hundreds of companies shut down their data centers and migrate to AWS, taking advantage of the cost savings, elasticity and on-demand benefits of the cloud. Many of those companies required tight timeframes of only 1-3 months!
Learn how you can shut down your data center and migrate to AWS in only 2 months. Chris Nolan, Director of Engineering at 2nd Watch, discusses the type of technologies enterprises are using today to help make migrations go faster and what an enterprise should have ready prior to engaging in a migration to ensure it goes quickly and smoothly.
Check back here on our blog or on our YouTube channel for the next video in this 5-part series.
One of the things “everyone knows” about migrating to the Cloud is that it saves companies money. Now you don’t need all those expensive data centers with the very physical costs associated with them. So companies migrate to the Cloud and are so sure they will see their costs plummet… then they get their bill for Cloud usage and experience sticker shock. Typically, this is when our customers reengage 2nd Watch – they ask us why it costs so much, what they can do to decrease their costs, and of course everyone’s favorite – why didn’t you tell me it would be so much?
First, in order to know why you are spending so much you need to analyze your environment. I’m not going to go into how Amazon bills and walk you through your entire bill in this blog post. That’s something for another day perhaps. What I do want to look into is how to enable you to see what you have in your Cloud.
Step one: tag it! Amazon gives you the ability to tag almost everything in your environment, including ELB’s, which was most recently added. I always highly recommend to my customers to make use of this feature. Personally, whenever I create something manually or programmatically I add tags to identify what it is, why it’s there, and of course who is paying for it. Even in my sandbox environment, it’s a way to tell colleagues “Don’t delete my stuff!” Programmatically, tags can be added through CloudFormation, Elastic Beanstalk, auto scaling, CLI, as well as third party tools like Puppet and Chef. From a feature perspective, there are very few AWS components that don’t support tags, and more are constantly being added.
That’s all well and good, but how does this help analytics? Tagging is actually is the basis for pretty much all analytics, and without it you have to work much harder for far less information. For example, I can tag EC2 instances to indicate applications, projects, or environments. I can then run reports that look for specific tags – how many EC2 instances are associated with Project X and what are the instance types? What are the business applications using my various RDS instances? – and suddenly when you get your bill, you have the ability to determine who is spending money in your organization and work with them on spending it smartly.
Let’s take it a step further and talk about automation and intelligent Cloud management. If I tag instances properly I can automate tasks to control my Cloud based on those tags. For example, maybe I’m a nice guy and don’t make my development team work weekends. I can set up a task to shutdown any instance with “Environment = Development” tag every Friday evening and start again Monday morning. Maybe I want to have an application only online at month end. I can set up another task to schedule when it is online and offline. Tags give us the ability to see what we are paying for and the hooks to control that cost with automation.
I would be remiss if I didn’t point out that tags are an important part of using some great 2nd Watch offerings that help manage your AWS spend. Please check out 2W Insight for more information and how to gain control over and visibility into your cloud spend.
-Keith Homewood, Cloud Architect