1-888-317-7920 info@2ndwatch.com

Optimizing your AWS environment using Trusted Advisor (Part 2)

AWS provides an oft overlooked tool available to accounts with “Business” or “Enterprise” level support called Trusted Advisor (TA). Trusted Advisor is a tool that analyzes your current AWS resources for ways to improve your environment in the following categories:

  • Cost Optimization
  • Security
  • Performance
  • Fault Tolerance

It rigorously scours your AWS resources for inefficiencies, waste, potential capacity issues, best practices, security holes and much, much more. It provides a very straightforward and easy to use interface for viewing the identified issues.

Trusted Advisor will do everything from detecting EC2 instances that are under-utilized (e.g. using an m3.xlarge for a low traffic NAT instance), to detecting S3 buckets that are good candidates for fronting with a CloudFront distribution, to identifying Security Groups with wide open access to a port(s), and everything in between.

In Amazon’s own words…

[blockquote]AWS Trusted Advisor inspects your AWS environment and makes recommendations for saving money, improving system performance and reliability, or closing security gaps. Since 2013, customers have viewed over 1.7 million best-practice recommendations in AWS Trusted Advisor in the categories of cost optimization, performance improvement, security, and fault tolerance, and they have realized over $300 million in estimated cost reductions. Currently, Trusted Advisor provides 37 checks; the most popular ones are Low Utilization Amazon EC2 Instances, Amazon EC2 Reserved Instances Optimization, AWS CloudTrail Logging, Amazon EBS Snapshots, and two security group configuration checks.[/blockquote]

This week (7/23/2014) AWS just announced the release of the new Trusted Advisor Console.

Two new features of the TA console I found particularly noteworthy and useful are the Action Links and Access Management.

Action Links allow you to click a hyperlink next to an issue in the TA Console that redirects you to the appropriate place to take action on the issue. Pretty slick… saves you time jumping around tabs in your browser or navigate to the correct Console and menus. Action Links will also take the guess work out of hunting down the correct place if you aren’t that familiar with the AWS Console.

Access Management allows you to use AWS IAM (Identity and Access Management) credentials to control access to specific categories and checks within Trusted Advisor. This gives you the ability to have granular access control over which people in your organization can view and act on specific checks.

In addition to the console, Trusted Advisor also supports API access. And this wouldn’t be my AWS blog post without some kind of coding example using Python and the boto library. The following example code will print out a nicely formatted list of all the Trusted Advisory categories and each of the checks underneath them in alphabetical order.

from boto import connect_support
conn = connect_support()
ta_checks = sorted(conn.describe_trusted_advisor_checks('en')['checks'],
                   key=lambda check: check['category'])
for cat in sorted(set([ x['category'] for x in ta_checks ])):
    print "\n%s\n%s" % (cat, '-' * len(cat))
    for check in sorted(ta_checks, key=lambda check: check['name']):
        if check['category'] == cat:
            print "  %s" % check['name']

Here is the resulting output (notice all 37 checks are accounted for):

Amazon EC2 Reserved Instances Optimization
Amazon RDS Idle DB Instances
Amazon Route 53 Latency Resource Record Sets
Idle Load Balancers
Low Utilization Amazon EC2 Instances
Unassociated Elastic IP Addresses
Underutilized Amazon EBS Volumes

Amazon EBS Snapshots
Amazon EC2 Availability Zone Balance
Amazon RDS Backups
Amazon RDS Multi-AZ
Amazon Route 53 Deleted Health Checks
Amazon Route 53 Failover Resource Record Sets
Amazon Route 53 High TTL Resource Record Sets
Amazon Route 53 Name Server Delegations
Amazon S3 Bucket Logging
Auto Scaling Group Health Check
Auto Scaling Group Resources
Load Balancer Optimization
VPN Tunnel Redundancy

Amazon EBS Provisioned IOPS (SSD) Volume Attachment Configuration
Amazon Route 53 Alias Resource Record Sets
CloudFront Content Delivery Optimization
High Utilization Amazon EC2 Instances
Large Number of EC2 Security Group Rules Applied to an Instance
Large Number of Rules in an EC2 Security Group
Overutilized Amazon EBS Magnetic Volumes
Service Limits

AWS CloudTrail Logging
Amazon RDS Security Group Access Risk
Amazon Route 53 MX and SPF Resource Record Sets
Amazon S3 Bucket Permissions
IAM Password Policy
MFA on Root Account
Security Groups - Specific Ports Unrestricted
Security Groups - Unrestricted Access

In addition to the meta-data about categories and checks, actual TA check results and recommendations can also be pulled and refreshed using the API.

While Trusted Advisor is a great tool to quickly scan your AWS environment for inefficiencies, waste, potential cost savings, basic security issues, and best practices, it isn’t a “silver bullet” solution. It takes a specific set of AWS architectural understanding, skills, and experience to look at an entire application stack or ecosystem and ensure it is properly designed, built, and/or tuned to best utilize AWS and its array of complex and powerful building blocks. This where a company like 2nd Watch can add immense value in a providing a true “top down” cloud optimization. Our architects and engineers are the best in the business at ensuring applications and infrastructure are designed and implemented using AWS and cloud computing best practices with a fierce attention to detail and focus on our customers’ success in their business and cloud initiatives.

-Ryan Kennedy, Senior Cloud Architect


Optimizing AWS Costs with Trusted Advisor

Moving to the cloud is one of the best decisions you can make for your business.  The low startup costs, instant elasticity, and near endless scalability have lured many organizations from traditional datacenters to the cloud.  Although cloud startup costs are extremely low, over time the burgeoning use of resources within an AWS account can slowly increase the cost of operating in the cloud.

One service AWS provides to help with watching the costs of an AWS environment is AWS Trusted Advisor.  AWS touts the service as “your customized cloud expert” that helps you monitor resources according to best practices.  Trusted Advisor is a service that runs in the background of your AWS account and gathers information regarding cost optimization, security, fault tolerance, and performance.  Trusted Advisor can be accessed proactively through the support console or can be setup to notify you via weekly email.

The types of Trusted Advisor notifications available for Cost Optimization are Amazon EC2 Reserved Instance Optimization, Low Utilization Amazon EC2 Instances, Idle Load Balancers, Underutilized Amazon EBS Volumes, Unassociated Elastic IP Addresses, and RDS Idle DB Instances.  Within these service types, Trusted Advisor gives you four types of possible statuses; “No problems detected,” “Investigation recommended,” “Action recommended,” and “Not available.”  Each one of these status types give insight to how effectively you are running your account based on the best-practice algorithm the service uses.  In the below example, AWS Trusted Advisor points out $1,892 of potential savings for this account.

Trusted Advisor

Each one of these notifications adds up to the total potential monthly savings.  Here is one “Investigation recommended” notification from the same account. It says “3 of 4 DB instances appear to be Idle. Monthly savings of up to $101 are available by minimizing idle DB Instances.”

Drop Down

Clicking the drop down button reveals more:

Amazon RDS

The full display tells you exactly what resource in your account is causing the alert and even gives you the estimated monthly savings if you were to make changes to the resource.   In this case the three RDS instances are running in Oregon and Ireland.  This particular service is basing the alert on the “Days since last connection,” which is extremely helpful because if there have been no connections to the database in 14+ days, there’s a good chance it’s not even being used.  One of the best things about Trusted Advisor is that it gives the overview broken down by service type and gives just enough information to be simple and useful.  We didn’t have to login to RDS and navigate to the Oregon region or the Ireland region to find this information. It was all gathered by Trusteed Advisor and presented in an easy to read format.  Remember, not all of the information provided may need immediate attention, but it’s nice to have it readily available.   Another great feature is each notification can be downloaded as a Microsoft Excel spreadsheet that allows you to have even more control over the data the service provides.

Armed with the Trusted Advisor tool you can keep a closer eye on your AWS resources and gain insight to optimizing costs on a regular basis.   The Trusted Advisor covers the major AWS services but is only available to accounts with Business or Enterprise-level support.  Overall, it’s a very useful service for watching account costs and keeping an eye on possible red flags on an account.  It definitely doesn’t take the place of diligent implementation and monitoring of resources by a cloud engineer but can help with the process.

– Derek Baltazar, Senior Cloud Engineer