1-888-317-7920 info@2ndwatch.com

Optimizing your AWS environment using Trusted Advisor (Part 2)

AWS provides an oft overlooked tool available to accounts with “Business” or “Enterprise” level support called Trusted Advisor (TA). Trusted Advisor is a tool that analyzes your current AWS resources for ways to improve your environment in the following categories:

  • Cost Optimization
  • Security
  • Performance
  • Fault Tolerance

It rigorously scours your AWS resources for inefficiencies, waste, potential capacity issues, best practices, security holes and much, much more. It provides a very straightforward and easy to use interface for viewing the identified issues.

Trusted Advisor will do everything from detecting EC2 instances that are under-utilized (e.g. using an m3.xlarge for a low traffic NAT instance), to detecting S3 buckets that are good candidates for fronting with a CloudFront distribution, to identifying Security Groups with wide open access to a port(s), and everything in between.

In Amazon’s own words…

[blockquote]AWS Trusted Advisor inspects your AWS environment and makes recommendations for saving money, improving system performance and reliability, or closing security gaps. Since 2013, customers have viewed over 1.7 million best-practice recommendations in AWS Trusted Advisor in the categories of cost optimization, performance improvement, security, and fault tolerance, and they have realized over $300 million in estimated cost reductions. Currently, Trusted Advisor provides 37 checks; the most popular ones are Low Utilization Amazon EC2 Instances, Amazon EC2 Reserved Instances Optimization, AWS CloudTrail Logging, Amazon EBS Snapshots, and two security group configuration checks.[/blockquote]

This week (7/23/2014) AWS just announced the release of the new Trusted Advisor Console.

Two new features of the TA console I found particularly noteworthy and useful are the Action Links and Access Management.

Action Links allow you to click a hyperlink next to an issue in the TA Console that redirects you to the appropriate place to take action on the issue. Pretty slick… saves you time jumping around tabs in your browser or navigate to the correct Console and menus. Action Links will also take the guess work out of hunting down the correct place if you aren’t that familiar with the AWS Console.

Access Management allows you to use AWS IAM (Identity and Access Management) credentials to control access to specific categories and checks within Trusted Advisor. This gives you the ability to have granular access control over which people in your organization can view and act on specific checks.

In addition to the console, Trusted Advisor also supports API access. And this wouldn’t be my AWS blog post without some kind of coding example using Python and the boto library. The following example code will print out a nicely formatted list of all the Trusted Advisory categories and each of the checks underneath them in alphabetical order.

#!/usr/bin/python
from boto import connect_support
conn = connect_support()
ta_checks = sorted(conn.describe_trusted_advisor_checks('en')['checks'],
                   key=lambda check: check['category'])
for cat in sorted(set([ x['category'] for x in ta_checks ])):
    print "\n%s\n%s" % (cat, '-' * len(cat))
    for check in sorted(ta_checks, key=lambda check: check['name']):
        if check['category'] == cat:
            print "  %s" % check['name']

Here is the resulting output (notice all 37 checks are accounted for):

cost_optimizing
---------------
Amazon EC2 Reserved Instances Optimization
Amazon RDS Idle DB Instances
Amazon Route 53 Latency Resource Record Sets
Idle Load Balancers
Low Utilization Amazon EC2 Instances
Unassociated Elastic IP Addresses
Underutilized Amazon EBS Volumes

fault_tolerance
---------------
Amazon EBS Snapshots
Amazon EC2 Availability Zone Balance
Amazon RDS Backups
Amazon RDS Multi-AZ
Amazon Route 53 Deleted Health Checks
Amazon Route 53 Failover Resource Record Sets
Amazon Route 53 High TTL Resource Record Sets
Amazon Route 53 Name Server Delegations
Amazon S3 Bucket Logging
Auto Scaling Group Health Check
Auto Scaling Group Resources
Load Balancer Optimization
VPN Tunnel Redundancy

performance
-----------
Amazon EBS Provisioned IOPS (SSD) Volume Attachment Configuration
Amazon Route 53 Alias Resource Record Sets
CloudFront Content Delivery Optimization
High Utilization Amazon EC2 Instances
Large Number of EC2 Security Group Rules Applied to an Instance
Large Number of Rules in an EC2 Security Group
Overutilized Amazon EBS Magnetic Volumes
Service Limits

security
--------
AWS CloudTrail Logging
Amazon RDS Security Group Access Risk
Amazon Route 53 MX and SPF Resource Record Sets
Amazon S3 Bucket Permissions
IAM Password Policy
IAM Use
MFA on Root Account
Security Groups - Specific Ports Unrestricted
Security Groups - Unrestricted Access

In addition to the meta-data about categories and checks, actual TA check results and recommendations can also be pulled and refreshed using the API.

While Trusted Advisor is a great tool to quickly scan your AWS environment for inefficiencies, waste, potential cost savings, basic security issues, and best practices, it isn’t a “silver bullet” solution. It takes a specific set of AWS architectural understanding, skills, and experience to look at an entire application stack or ecosystem and ensure it is properly designed, built, and/or tuned to best utilize AWS and its array of complex and powerful building blocks. This where a company like 2nd Watch can add immense value in a providing a true “top down” cloud optimization. Our architects and engineers are the best in the business at ensuring applications and infrastructure are designed and implemented using AWS and cloud computing best practices with a fierce attention to detail and focus on our customers’ success in their business and cloud initiatives.

-Ryan Kennedy, Senior Cloud Architect

Facebooktwittergoogle_pluslinkedinmailrss

AWS Identity and Access Management (IAM)

Dealing with organizational change is a challenge in today’s fast-paced business environment.  Long gone are the days when employees stayed with companies until retirement.  The mindset of many employees is to move around to different companies for a promotion, a better salary, or new challenging opportunities.  Managing organizational change in terms of user access is becoming more and more complex due to the changing technology landscape.  With systems being accessible over the network, IT shops can’t just deny ex-employees physical access to the building, but have to cut their credentials to the network as well. With the proliferation of cloud technologies this can become even more of a challenge because your digital assets are accessible over the internet from anywhere in the world. In many technology centric companies managing login credentials and access are paramount for securing the assets of the business and coping with organizational change.

IAM 1To solve this problem AWS has a service called Identity and Access Management (IAM).  IAM is an AWS feature that allows you to regulate use and access to AWS resources.  With IAM you can create and manage users and groups for access to your AWS environment.  IAM also gives you the ability to assign permissions to the users and groups to allow or deny access.  With IAM you can assign users access keys, passwords and even Multi Factor Authentication devices to access your AWS environment.  IAM on AWS even allows you to manage access with federated users, a way to configure access using credentials that expire and are manageable through traditional corporate directories like Microsoft Active Directory.

With IAM you can set permissions based on AWS provided policy templates like “Administrator Access” which allows full access to all AWS resources and services, “Power User Access” which provides full access to all AWS resources and services but does not allow access to managing users and groups, or even “Read Only Access”.  These policies can be applied to users and groups.  Some policy templates provided can limit users to use certain services like the policy template, “Amazon EC2 Full Access” or “Amazon EC2 Read Only Access”, which gives a user full access to EC2 via the AWS management console and read only access to EC2 via the AWS management console respectively.

User Permissions

IAM also allows you to set your own policies to manage permissions.  Say you wanted an employee to be able to just start and stop instances you can use the IAM Policy Generator to create a custom policy to do just that.  You would select the effect, Allow or Deny, the specific service, and the action.  IAM also gives you the ability to layer the permissions on top of each other by adding additional statements to the policy.

Edit Permissions

Once you create a policy you can apply it to any user or group and it automatically takes effect.  When something changes in the organization, like an employee leaving, AWS IAM simplifies management of access and identity by allowing you to just delete the user or policy attached to that user. If an employee moves from one group to another it is easy to reassign the user to a different group with the appropriate access level.  As you can see the variety of policy rules is extensive, allowing you to create very fine grained permissions around your AWS resources and services.

Another great thing about IAM is that it’s a free service that comes with every AWS account, it is surprising to see how many people overlook this powerful tool.  It is highly recommended to always use IAM with any AWS account.  It gives you the ability to have an organized way to manage users and access to your AWS account and simplifies the management nightmare of maintaining access controls as the environment grows.

-Derek Baltazar

Senior Cloud Engineer

Facebooktwittergoogle_pluslinkedinmailrss