Tag Strategy is key to Cost Allocation for Cloud Applications.
Have you ever been out to dinner with a group of friends and at the end of the dinner the waiter comes back with one bill? Most of us have experienced this. Depending on the group of friends it’s not a big deal and everyone drops in a credit card so the bill can be split evenly. Other times, someone invites Harry and Sally, and they scrutinize the bill line by line. Inevitably they protest that they only had one glass of wine and Sally only had the salad. You recall that Sally was a little ‘handsy’ with the sampler platter, but you sit quietly. It’s in that moment you remember, that’s why the group didn’t include Harry and Sally to last year’s New Year’s dinner. No need to start the new year with an audit, am I right?
This situation can be eerily similar in many ways to cloud billing in a large enterprise. The fact that Amazon Web Services (AWS) has changed the way that an organization uses computing resources is evident. However, AWS has also delivered on the promise of truly enabling ‘chargeback’ or ‘showback’ in the enterprise so that the business units themselves are stakeholders in what was traditionally silo’d in an IT Department budget.
Now multiple stake holders from many organizations have a stake in the cost and usage of an app that resides in AWS. Luckily there are tools like 2nd Watch’s Cloud Management Platform (CMP) that can easily provide visibility to the cost of their app, or even what their entire infrastructure is costing them at the click of a button.
2nd Watch’s CMP tools are great for showing an organization’s costs and can even be used to set budget notifications so that the business unit doesn’t inadvertently spend more than is budgeted on an environment. CMP is a powerful tool that can deliver powerful insights to your business and can be made more powerful by implementing a thorough tagging strategy.
Tag your it…
We live in a world of tags and hashtags. Seemingly overnight tags have made their way into everyday language. This is not by accident as cloud interactions with Facebook and Twitter have become so commonplace, they have altered the world’s language.
Beyond their emergence in our everyday vernacular, they have a key function. In AWS, applying tags to various cloud resources like EC2 and RDS is key to having quality accounting for allocating charges. Our team of experts at 2nd Watch can work with you to ensure that your tagging strategy is implemented in the most effective manner for your organization. After all, a tagging strategy can and will vary by organization. It depends on you and how you want to be able to report on your resources. Do you want to be able to report on your resources used by cost center, application, environment type (like dev or prod), owner, department, geographic area, or if this resource was managed by a managed service provider like 2nd Watch?
Without having a well thought out tagging strategy your invoicing discussions will sound much like the fictional dinner described above. Who pays for what and why?
Tag Strategy and Hygiene…
Implementing a sound tagging strategy at the outset when a resource or environment is deployed is the first step. At the inception it’s important to know some “gotchas” that can derail a tagging implementation. One of these is that tags are case sensitive. For example, mktg will report separately from Mktg. Also keep in mind, that in today’s ever changing business environment organizations are forced to adjust and reorganize themselves to stay competitive.
Revisiting your tagged resource strategy will need to be done from time to time to ensure tag relevance. If a stake holder moves out of a role, gets promoted, or retires from the organization altogether, you will need to stay on top of the tagging for their environment to be sure that it is still relevant to the new organization.
What about the un-taggables?
Having standardization and a tag plan works great for AWS resources like EC2 and RDS as explained before. What about untaggable resources, Network transfer charges, and items like a NAT gateway or a VPC Endpoint? There will be shared resources like these in your applications environment. It is best to review these shared untagged resources early on, and decide where to best allocate that cost.
At 2nd Watch, we have these very discussions with our clients on a regular basis. We can easily guide them through the resources associated with the app and where to allocate each cost. With a tool like CMP we can configure a client’s cost allocation hierarchy so they can view their ongoing costs in real time.
For it’s part, Amazon does a great job providing an up-to-date user guide for what resources can be tagged. Click here for great reference documentation to help while you develop your tag strategy.
Rinse and repeat as necessary
Your tagging strategy can’t be a ‘fire and forget’ pronouncement. To be effective your organization will need to enforce it on a consistent basis. For instance, as new devops personnel are brought into an organization, it will be key to insuring it stays under control.
These are the types of discussions that 2nd Watch adds a lot of value to. Our cloud expertise in AWS for large enterprises will insure that you are able to precisely account for your cloud infrastructure spend at the click of a button through CMP.
After all, we all want to enjoy our meal and move on with the next activity. Stop by and visit us at re:Invent booth #1104 for more help.
— Paul Wells, Account Manager, 2nd Watch
Without a doubt, AWS has fundamentally changed how modern enterprises deploy IT infrastructure. Their services are flexible, cost effective, scalable, secure and reliable. And while moving from on-premise data centers to the cloud is, in most cases, the smart move; once there managing your costs becomes much more complex.
On-premise costs are straight forward, enterprises purchase servers and amortize their costs over the expected life. Shared services such as internet access, racks, power and cooling are proportionally allocated to the cost of each server. AWS on the other hand, invoices each usage type separately. For example, if you are running a basic EC2 instance, you will not only be charged for the EC2 box usage but also the data transfer, EBS Storage and associated snapshots. You could end up with as many as 13 line items of cost for a single EC2.
Example: Pricing line items for a single c4.xlarge Linux virtual machine running in the US East Region (Click on image to view larger)
When examining the composition of various workload types the numbers of line items to manage will vary. A traditional VM-based workload may have 50 cost line items for every $1,000 of spend while an agile, cloud-native workload may have as many as 500 per $1,000 and a dynamic workload leveraging spot instances may have upwards of 1,200 per $1,000. This “parts bin” approach to pricing makes the job of cost account challenging.
To address this complexity and enable accurate cost accounting of your cloud costs; we recommend creating a business-relevant financial tagging schema to organize your resources and associated cost line items based on your specific financial accounting structure.
Here are some recommended financial management tags you should consider (Click on image to view larger):
AWS Tagging data integrity is extremely important in ensuring the quality of the information it provides and is directly dependent upon the rigor applied in adopting a systematic and disciplined approach to AWS Tagging.
Financial Management Tagging – Best Practices
- Create a framework or standard for your enterprise that outlines required tag names, tag formatting rules, and governance of tags.
- Tags should be enforced and automated at startup of the resource via Cloud Formation templates or other infrastructure as code tools, such as Terraform, to ensure cost accounting details are captures from time of launch.
- NOTE: Tags are point in time based. If a resource is launched without being tagged and then tagged sometime in the future, all hours the resource ran prior to being tagged will not be included in tag reports in the AWS console.
- Manually creating tags and associated values is strongly discouraged as it leads to miss-tagged and untagged resources and in-accurate cost accounting
- Select all upper case or all lower-case keys and values to avoid discrepancies with capitalization.
- NOTE: “Production” and “production” are considered two different tag names or values.
- Monitor resources with AWS Config Rules and alert for newly created resources that are not tagged
Once your tagging schema is created, automation is in place to tag resources during startup and alerts are set up to ensure tagging is managed, you can accurately to view, track and report your cost and usage using any of your tagging dimensions.
Financial Management Reporting – Best Practices
- Using your tagging schema, group your resources by workload.
- Apply Reserved Instance discounts to the workloads you intended them to be for.
- NOTE: 2nd Watch’s CMP Finance Manager tool converts reserved instances into resources so that you can add them to the workload they were intended for.
- Organize your groups to match your specific multi-level financial reporting structure.
- Managed shared resources
- Create groups for shared resources. If you have resources that are shared across multiple workloads such as a database used my multiple applications or virtual machines with more than one applications running on it, create groups to capture these costs and allocate them proportionally to the applications using them.
- Manage un-taggable resources
- Create a group for un-taggable resources. Some AWS resources are not taggable and should be grouped together and their associated costs proportionally allocated to all applications.
- Manage spend to budget
- Create budgets and budget alerts for each group to ensure you stay in budget throughout the year.
- Key alerts
- Forecasted month end cost exceeds alert threshold
- MTD cost is over alert threshold
- Forecasted year end cost exceeds alert threshold
- YTD cost is over alert threshold
- Sign up to receive monthly cost and usage reports for integration into your internal cost accounting system.
- Cost by application, environment, business unit etc.
Even though AWS’ “parts bin” approach to pricing is complicated, following these guidelines will help ensure accurate cost accounting of your cloud spend.
–Timothy Hill, Senior Product Manager, 2nd Watch
Cloud billing is often complex and one-dimensional, and allocating costs across your organization – to the right departments and projects – can be difficult and time-consuming. With over 28,000 different ways to buy products and services from AWS, enterprises need sophisticated software and expertise to ensure they are maximizing the use of their AWS resources while optimizing their cloud spend and controlling cloud sprawl.
2nd Watch Managed Billing can help simplify your cloud billing. 2W Managed Billing provides a concierge-level billing service and online billing portal that simplifies analyzing, budgeting, tracking, forecasting and invoicing the cost of the public cloud, giving you an easy-to-understand view into your cloud costs.
Download the 2nd Watch Managed Billing datasheet to learn more about how managed billing can help you gain visibility into and understand your cloud bill. Or sign up for a free trial of 2W Managed Billing Service to start effectively managing your cloud usage and costs across your organization right away.
-Nicole Maus, Marketing Manager
When it comes to managing and monitoring IT spending, the cloud has created a new layer of complexity. Consider the fact that AWS provides as many as 28,000 service offerings, generating up to millions of billing line items each month. This creates budgeting and planning problems for CIOs because there’s no easy way to interpret what percentage of cloud spending is going toward storage, compute, or network, along with specific applications, projects and services. IT departments also need a way to merge cloud costs with on-premise IT costs to see the full picture of infrastructure spending across key categories. As well, with many individuals from different departments procuring their own AWS resources, a company can have dozens of unmanaged and unlinked accounts. This creates gaps in financial tracking and spend management and prevents a company from taking advantage of volume discounts.
IT needs a unified model to categorize cloud and non-cloud costs together, and automation to map line items into the IT cost model each month. To automate the mapping process, 2nd Watch and Apptio have worked together on a mapping table that specifies where each Amazon product fits within a standard cost model. This mapping is now embedded in the Apptio Cost Transparency application, a solution for integrating AWS usage with billing, cost categorization, modeling of total costs including internal labor and self-service analytics. This allows IT organizations to categorize cloud costs into trackable categories such as Cloud Windows in Compute or Cloud Archive in Storage.
Determining the run costs of an application is another goal, and requires mapping cloud resources, such as servers and storage, to individual applications. Many IT organizations have not yet adapted their management processes to track application relationship data for cloud infrastructure. Linked accounts and tagging are two ways to get around these hurdles on AWS.
Many enterprises have several AWS accounts at the team and departmental levels in order to encourage agility, but these unlinked accounts create gaps in cost and operational management. To unify unlinked accounts across an organization, companies can use the Apptio application to link individual accounts into one “master account” paid through an IT cost center. This provides visibility into enterprise spend on AWS yet still maintains business-unit level tracking. It also enables savings with volume discounts, that’s not possible when spending goes across several individual AWS accounts.
AWS tags help group usage and expenses across shared key resources like databases. Tagging helps accomplish the problem of mapping AWS resources back to specific business projects, such as “Marketing Web Staging” and “Marketing Web Production.” Detailed tagging can help answer questions such as, how much of an entire application portfolio is comprised of AWS services or what percentage of which projects are using cloud resources? One thing to keep in mind is that AWS tags are applied only to individual accounts. AWS tagging is ideal for environments where you need to share resources across multiple workloads.
There are some limitations to this manual approach for managing individual accounts and tagging, however – managing numerous logins and passwords, going through the AWS setup process for each individual account, creating and controlling a tagging schema, etc. For a more scalable approach to managing AWS accounts and tagging, consider solutions like our 2W Insight billing application, which enables grouping of tags across AWS accounts and provides tools to track and analyze cloud costs by cost center, business unit department, etc. For more information on 2W Insight, contact us.
To learn more about best practices for managing and tracking cloud spending, download our Analyzing Cloud Costs white paper.
-Jeff Aden, EVP, Marketing & Strategic Business Development