“Whatever you do in life, surround yourself with smart people who argue with you.” – John Wooden
Many AWS customers and practitioners have leveraged the Well-Architected Framework methodology in building new applications or migrating existing applications. Once a build or migration is complete, how many companies implement Well-Architected Framework reviews and perform those reviews regularly? We have found that many companies today do not conduct regular Well Architected Framework reviews and as a result, potentially face a multitude of risks.
What is a Well-Architected Framework?
The Well-Architected Framework is a methodology designed to provide high-level guidance on best practices when using AWS products and services. Whether building new or migrating existing workloads, security, reliability, performance, cost optimization, and operational excellence are vital to the integrity of the workload and can even be critical to the success of the company. A review of your architecture is especially critical when the rate of innovation of new products and services are being created and implemented by Cloud Service Providers (CSP).
2nd Watch Well-Architected Framework Reviews
At 2nd Watch, we provide Well-Architected Framework reviews for our existing and prospective clients. The review process allows customers to make informed decisions about architecture decisions, the potential impact those decisions have on their business, and tradeoffs they are making. 2nd Watch offers its clients free Well-Architected Framework reviews—conducted on a regular basis—for mission-critical workloads that could have a negative business impact upon failure.
Examples of issues we have uncovered and remediated through Well-Architected Reviews:
- Security: Not protecting data in transit and at rest through encryption
- Cost: Low utilization and inability to map cost to business units
- Reliability: Single points of failure where recovery processes have not been tested
- Performance: A lack of benchmarking or proactive selection of services and sizing
- Operations: Not tracking changes to configuration management on your workload
Using a standard based methodology, 2nd Watch will work closely with your team to thoroughly review the workload and will produce a detailed report outlining actionable items, timeframes, as well as provide prescriptive guidance in each of the key architectural pillars.
In reviewing your workload and architecture, 2nd Watch will identify areas of improvement, along with a detailed report of our findings. A separate paid engagement will be available to clients and prospects who want our AWS Certified Solutions Architects and AWS Certified DevOps Engineer Professionals to remediate our findings. To schedule your free Well-Architected Framework review, contact 2nd Watch today.
— Chris Resch, EVP Cloud Solutions, 2nd Watch
Controlling costs is one of the grea challenges facing IT and Finance managers today. The cloud, by nature, makes it easy to spin up new environments and resources that can cost thousands of dollars each month. And, while there are many ways to help control costs, one of the simplest and most effective methods is to set and manage cloud spend-to-budget. While most enterprise budgets are set at a business unit or department, for cloud spend, mapping that budget down to the workload can establish strong accountability within the organization.
One popular method that workload owners use to manage spend is to track month-over-month cost variances. However, if costs do not drastically increase from one month to another, this method does very little to control spend. It is only until a department is faced with budget issues that workload owners work diligently to reduce costs. That’s because, when budgets are set for each workload, owners become more aware of how their cloud spend impacts the company financials and tend to more carefully manage their costs.
In this post, we provide four easy steps to help you manage workload spend-to-budget effectively.
Step 1: Group Your Cloud Resources by Workload and Environment
Use a financial management tool such as 2nd Watch CMP Finance Manager to group your cloud resources by workload and its environment (Test, Dev, Prod). This can easily be accomplished by creating a standard where each workload/environment has its own cloud account, or by using tags to identify the resources associated with each workload. If using tags, use a tag for the workload name such as workload_name: and a tag for the environment such as environment:. More tagging best practices can be found here.
Step 2: Group Your Workloads and Environments by Business Group
Once your resources are grouped by workload/environment, CMP Finance Manager will allow you to organize your workload/environments into business groups. For example:
a. Business Group 1
i. Workload A
1. Workload A Dev
2. Workload A Test
3. Workload A Prod
ii. Workload B
1. Workload B Dev
2. Workload B Test
3. Workload B Prod
b. Business Group 2
i. Workload C
1. Workload C Dev
2. Workload C Test
3. Workload C Prod
ii. Workload D
1. Workload D Dev
2. Workload D Test
3. Workload D Prod
Step 3: Set Budgets
At this point, you are ready to set up budgets for each of your workloads (each workload/environment and the total workload as you may have different owners). We suggest you set annual budgets aligned to your fiscal year and have the tool you use programmatically recalculate the budget at the end of each month with the amount remaining in your annual budget.
Step 4: Create Alerts
The final step is to create alerts to notify owners and yourself when workloads either have exceeded or are on track to exceed the current month or annual budget amount. Here are some budget notifications we recommend:
- ME forecast exceeds month budget
- MTD spend exceeds MTD budget
- MTD spend exceeds month budget
- Daily spend exceed daily budget
- YE forecast exceeds year budget
- YTD spend exceeds YE budget
Once alerts are set, owners can make timely decisions regarding spend. The owner can now proactively shift to spot instances, purchase reserved instances, change instance sizes, park the environment when not in use, or even refactor the application to take advantage of cloud native services like AWS Lambda.
Our experience has shown that enterprises that diligently set up and manage spend-to-budget by workload have more control of their costs and ultimately, spend less on their cloud environments without sacrificing user experience.
–Timothy Hill, Senior Product Manager, 2nd Watch
AWS regularly cuts customer cost by reducing the price of their services. This happened most recently with the price reduction of C4, M4 and R3 instances. These instances saw a 5% price cut when running on Linux. This was their 51st price reduction. Customers are clearly benefiting from the scale that AWS can bring to the market. Spot Instances and Reserved Instances are another way customers can significantly reduce the cost to run their workloads in the cloud.
Sometimes these cost savings are not as obvious, but they need to be understood and measured when doing a TCO calculation. AWS recently announced Certificate Manager. Certificate Manager allows you to request new SSL/TLS certificates and then manage them with automated renewals. The best part is that the service is free! Many vendors charge hundreds of dollars for new certificates, and AWS is now offering it for free. The automated renewal could also save you time and money while preventing costly outages. Just ask the folks over at Microsoft how costly a certificate expiring can be.
Another way AWS reduces the cost to manage workloads is by offering new features in an existing service. S3 Standard – Infrequent Access is an example of this. AWS offered the same eleven 9s of durability while reducing availability from four 9s to three. Customers who are comfortable going from 52 minutes of downtime a year to 8.5 hours of downtime per year for objects that don’t need the same level of availability can save well over 50%, even at the highest usage levels. When you add features like encryption, versioning, cross-region replications and others, you start to see the true value. Building and configuring these features yourself in a private cloud or in your own infrastructure can be costly add-ons. AWS often offers these add-ons for free or only charges for the associated use, like the storage cost for cross-region replication.
Look beyond CPUs, memory, and bytes on disk when calculating the savings you will get with a move to AWS. Explore the features and services you cannot offer your business from within your own datacenter or colocation facility. Find a partner like 2nd Watch to help you manage and optimize your cloud infrastructure for long-term savings.
-Chris Nolan, Director of Product
AWS enables enterprises to trade capital expense for variable expense, lower operating costs and increase speed and agility. As enterprises begin to deploy cloud services across their business, it is critical to have a standardized approach to allocate usage costs to the appropriate department or cost center. By tracking costs at the cost center level, Enterprises gain visibility throughout their organization – and specifically who is spending precious IT funds.
To allocate costs, usage must first be grouped. AWS provides two methods to group usage; Resources Tags and AWS accounts. Each method is useful but also comes with downsides.
Using AWS Tagging to group usage
- Grouping by tag enables enterprises to run all of their workloads (applications) in a single AWS account, simplifying management within the AWS console.
- A tagging schema needs to be created, universally deployed and tightly controlled.
- Care has to be taken to ensure all individual AWS resources are tagged properly as any mistake in tagging will cause a resource to be left out of the group and not reported properly.
- Many AWS resources are un-tagable, which will require the creation and maintenance of a separate cost distribution scheme to allocate those costs across the enterprise.
- Reserved Instance (RI) discounted usage pricing cannot be linked to a single tag group and can result in significant costing inaccuracies.
Using Multiple AWS Accounts to group usage
- Using individual AWS accounts for each workload provides the most accurate and detailed reporting of costs and usage.
- By creating a separate AWS account for each workload, enterprises can track all associated costs (including RIs) and allocate them to cost centers, departments and/or business units.
- When using AWS accounts to group usage, each account must be manually set up.
- There is no method of sharing resources, such as databases, with multiple workloads as each workload is located in separate AWS accounts.
Given the challenges of both “account based” and “tag based” grouping, we have found that the tracking methodology needs be aligned to the applications or workloads. For deployments where the resources are 100% dedicated to a specific workload, grouping by AWS accounts is ideal as it is the only way to ensure fully accurate costing. Using AWS tagging should be used when you need to share resources across multiple workloads, however enterprises must note that the costing will not be 100% accurate when using tag groups.
Tracking and Allocating Costs for Workloads with Dedicated Resources
As stated above, workloads that do not need to share resources should be set up in unique AWS accounts. This can be accomplished by establishing individual AWS accounts for each workload and mapping them directly to your enterprise organizational structure. The example below illustrates how a complex enterprise can organize its cloud expenses and provide showback or chargeback reports across the enterprise.
In this example, the enterprise would receive two bills for their cloud usage – Business Unit 1 and Business Unit 2. Under each business unit there are multiple levels of cost centers that roll up to each subsequent higher level – which is typical with many Enterprise organizations. In this example, AWS accounts are created for each project/workload then rolled up to provide consolidated usage information by department and business unit. This type of structure enables:
- The owners at the “resources and workload cost accrual and tracking” levels to track their individual usage by AWS accounts, which captures 100% of the cost associated with each AWS account
- The management of department level to view the consolidated usage for their respective cost centers and workloads
- The management of each business unit to view usage by department and AWS account and receive a bill for its entire consolidated usage
This provides a reliable and accurate methodology to track and allocate cloud usage based on your distinct enterprise organizational structure. It does, however, require a disciplined approach to creating new projects and updating your expense management platform to provide executive-level dashboards and the ability to drill-down to detailed consumption reports by cost center. This enables Enterprise IT to provide executive-level transparency while keeping excessive resource consumption under control and reduce IT costs.
Tracking and Allocating Costs for Workloads with Shared Resources
In many organizations there is a need to share key resources, such as databases, across multiple workloads. In these cases it is a best practice to use AWS tags to group your expenses. This method requires careful set up of resources and the creation of a schema to allocate shared resources and resources that cannot be tagged across the enterprise.
Tagging allows enterprises to assign its own metadata to each tag-able resource. Tags do not have any semantic meaning to AWS resources and are interpreted strictly as a string of characters. Tags are made up of both a “Key” and a “Value”. AWS allows up to 10 Keys for each resource, and each Key can have can have unlimited values enabling very detailed grouping of resources. Tagging should be set up based on the needs of the organization and the AWS architecture design. The image below illustrates how to establish a tagging scheme for a 2-Tier Auto-scalable Web Application.
As the project moves from Web Sandbox to Web Staging to Web Production, you can use tags to track usage. When the application is in the Sandbox all resources are tagged with the key “Web Sandbox” and the appropriate value (Environment, Owner, App and/or IT Tower). When the project moves to “Web Staging” you simply replace the original key and values with the ones associated with the next step in development.
While there is no one-size-fits-all solution to AWS expense management, deploying one or both of these methods can provide you the visibility necessary to successfully meet the tracking and analytical needs of your enterprise.
-Tim Hill, Product Manager
As firms progress through the transition from traditional IT to the AWS Cloud, there is often a moment of fear and anxiety related to managing cost. The integration and planning group has done an excellent job of selecting the right consulting partner. Contracts have been negotiated by legal and procurement. Capital funding has been allocated to cover the cost of application migrations. Designs are accepted and the project manager has laid out the critical path to success. Then at the last hour, just before production launch, the finance team chimes in – “How are we going to account for each application’s monthly usage?”
So much planning and preparation is put into integration, because we’ve gone through this process with each new application. But moving to the public cloud presents a new challenge, one that’s easily tackled with a well-developed model for managing cost in a usage-based environment.
AWS allows us to deploy IaaS (Infrastructure as a Service), and that infrastructure is shared across all of our applications in the cloud. With the proper implementation of AWS Resource Tags, cloud resources can be associated with unique applications, departments, environments, locations and any other category for which cost-reporting is essential.
Firms must have the right dialog in the design process with their cloud technology partner. Here’s an outline of the five phases of the 2nd Watch AWS Tagging Methodology, which has been used to help companies plan for cloud-based financial management:
Phase 1: Ask Critical Questions – Begin by asking Critical Questions that you want answered about utilization, spending and resource management. Consider ongoing projects, production applications, and development environments. Critical Questions can include: Which AWS resources are affecting my overall monthly bill? What is the running cost of my high availability features like warm standby, pilot light or geo-redundancy? How do architectural changes or application upgrades change my monthly usage?
Phase 2: Develop a Tagging Strategy – The Cloud Architect will interpret these questions and develop a tagging strategy to meet your needs. The strategy then turns into a component of the Detailed Design and later implemented during the build project. During this stage it’s important to consider the enforcement of standards within the organization. Configuration drift is when other groups don’t use standardized AWS Resource Tags, or those defined in a naming convention. Later when it’s time for reporting, this will create problems for accounting and finance.
Phase 3: Determine Which AWS Resources Are In Scope – Solicit feedback from your internal accounting department and application owners. Create a list of AWS Resources and applications that need to be accounted for. Refer frequently to AWS online documentation because the list of taggable resource types is updated often.
Phase 4: Define How Chargebacks and Showbacks Will Be Used – Determine who will receive usage-based reports for each application, environment or location. Some firms have adopted a Chargeback model in which the accounting team bills the internal groups who have contributed to the month’s AWS usage. Others have used these reports for Showback only, where the usage & cost data is used for planning, forecasting and event correlation. 2W Insight offers a robust reporting engine to allow 2nd Watch customers the ability to create, schedule and send reports to stakeholders.
Phase 5: Make Regular Adjustments For Optimization – Talk to your Cloud Architect about automation to eliminate configuration drift. Incorporate AWS tagging standards into your cloud formation templates. Regularly review tag keys and values to identify non-standard use cases. And solicit the feedback of your accounting team to ensure the reports are useful and accurate.
Working with an AWS Premier Consulting Partner is critical to designing for best practices like cost management. Challenge your partner and ask for real-world examples of AWS Resource Tagging strategies and cost reports. Planning to manage costs in the cloud is not a step that should be skipped. It’s critical to incorporate your financial reporting objectives into the technical design early, so that they can become established, standardized processes for each new application in the cloud.
For more information, please reach out to Zachary Bonugli firstname.lastname@example.org.
– Zachary Bonugli, Global Account Manager