Retrieve Azure Billing Data

You can configure Commander to retrieve Azure billing data. Retrieving billing data improves the accuracy of costing reports and Cost Analytics, because these reports include metered usage costs such as Storage and Network I/O. Billing data is retrieved immediately after you configure the billing data location. During a nightly scheduled task, the billing data is merged with the existing billing data in the VM billing records.

For more information on how Commander handles services it has not managed, see Billing data for VMs not managed by Commander.

Microsoft has deprecated support for Azure Enterprise Agreement Reporting APIs, which Commander use to retrieve billing data from Azure. Commander has introduced a new method for retrieving billing data for Azure Enterprise Agreement, Azure Plan, and Pay-As-You-Go accounts, that offers increased accuracy of Azure billing data. For more information, see Retrieve Azure billing data with .csv exports.

For information on the legacy retrieval method, see Retrieve Azure billing data (legacy method). The legacy method is only supported if you have upgraded to Commander 9.6 from a previous version.

Commander billing data are stored and queried in a universal date without time or time zone. For example, if you are retrieving or injecting billing data for November 30, 2021, Commander interprets this date without any time zone shift, regardless of where you are located. Cost information retrieved from this public cloud provider is split at midnight in the GMT time zone and then stored in Commander with a universal date without time or time zone.

Retrieve Azure billing data with .csv exports

You can retrieve Azure billing data for Enterprise Agreement, Azure Plan, or Pay-As-You-Go accounts using .csv billing data exports. To use this retrieval method, you must set up a number of prerequisites in Azure.

Switching to this retrieval method requires a billing reset on your Azure cloud accounts. This will erase historical billing data for your Azure subscriptions, however there are measures you can take to maintain historical records. Follow the steps for one of the options below.

• Historical billing data is not maintained

Follow these steps to switch to the new retrieval method without maintaining historical billing records for your Azure subscriptions.

1. Complete the Azure configurations for each of your Azure subscriptions. For more information, see Create billing exports in Azure and Generate blob SAS tokens and blob SAS URLs in Azure.
2. Upgrade to Commander 9.6.0.
3. Configure Commander to use the new billing retrieval method. For more information, see Configure billing data retrieval in Commander.
4. Reset Azure billing data for each of your Azure cloud accounts using the Commander REST API. For more information, see Manual Billing Reset through Commander API.

• Historical billing data is maintained

Follow these steps to maintain your historical billing records when switching to the new retrieval method.

It is recommended you wait five weeks between creating billing exports in your Azure subscriptions and configuring Commander to use the new retrieval method, since Azure can make retroactive changes to the exported billing files in the month after they have generated. For more information on how Commander accounts for these retroactive changes, see the note below in this section.

1. Complete the Azure configurations for each of your Azure subscriptions. For more information, see Create billing exports in Azure and Generate blob SAS tokens and blob SAS URLs in Azure.
2. Upgrade to Commander 9.6.0.
3. Wait five weeks for the exported .csv billing data to accumulate in Azure, and then configure Commander to use the new billing retrieval method. For more information, see Configure billing data retrieval in Commander.
4. Reset Azure billing data for each of your Azure cloud accounts using the Commander REST API, and enter a start date of four weeks prior to the current date. This will maintain Azure billing records that were retrieved prior to the creation of the .csv billing exports noted in step 1. For more information on performing billing resets, see Manual Billing Reset through Commander API.

Occasionally, Azure will add retroactive charges or modify existing charges in exported billing files, which results in discrepancies between the billing data that was ingested by Commander and the costs that are presently shown in the .csv files in Azure. To account for these discrepancies, for the first four days of each month, Commander inspects all .csv billing files from the previous calendar month. If billing changes are detected in any of the .csv files, the Commander billing data is updated accordingly.

Create billing exports in Azure

This procedure applies to Azure's Improved exports experience. For more information, see Create exports.

If you have already created billing exports for your Azure subscriptions, review the details to ensure the export settings are compatible with Commander's requirements.

For information on the legacy process for creating billing exports in Azure, see Create a daily export. If using the legacy process, select Actual cost as the Metric, Daily export of month-to-date costs as the Export type, and turn on File partitioning to enable Commander to retrieve the billing data.

1. Sign in to the Azure portal.
2. Go to Cost Management + Billing > Exports.
3. On the Exports page, select the billing scope you want to capture data for. It is recommended that you configure a separate export for each of your Azure subscriptions.
4. Select Create.
5. On the Basics tab, complete the fields in the Add export dialog:
• Type of data — Select Cost and usage details (actual).
• Export name — Enter a custom name for your export.
• Dataset version— Select 2021-01-01.
• Frequency — Select Daily export of month-to-date costs.
• Export description — Optional: enter a description for the export.
6. Select Add.
7. Go to the Destination tab.
8. For Storage type, select Azure blob storage.
9. For Destination and storage, select Use existing to export your billing data to an existing storage account, or Create new to export billing data to a new storage account.

   If you select Use existing, complete the following fields:

• Subscription — Choose the subscription your .csv billing data files will be exported to.

• Storage account — Choose the storage account your .csv billing data files will be exported to.

• Container — Enter the name of the storage container your .csv files will be exported to.

• Directory — Enter the directory path for the container your .csv files will be exported to.

If you select Create new, complete the following fields:

• Subscription — Choose the subscription you want to deploy your new storage account in.
• Resource group — Choose the resource group you want your new storage account to be associated with.
• Account name — Enter a name for the storage account you are creating. This is the storage account your .csv billing data files will be exported to.

• Location — Choose the Azure region where your new storage account will be deployed.

• Container — Enter the name of the storage container your .csv files will be exported to.

• Directory — Enter the directory path for the container your .csv files will be exported to.

10. Enable File partitioning.

    If File partitioning is disabled, billing retrieval will fail.

11. Disable Overwrite data.

    If Overwrite data is enabled, billing retrieval will fail.

12. On the Review + create tab, review the details of your export and select Create.

Generate blob SAS tokens and blob SAS URLs in Azure

Follow these steps for each billing export you create at the subscription level.

A shared access signature (SAS) is a URI that grants restricted access to storage container resources for a specified time range. For more information on generating SAS tokens in Azure, see Generate a SAS token in the Microsoft documentation.

1. In the Azure portal, go to the storage container that has the billing export you want Commander to retrieve.
2. Select Shared access tokens on the sidebar menu.
3. Complete the following settings:
• For Signing method, select Account key.
• For Signing key, choose a key to associate with the SAS.
• For Stored access policy, select None. Optionally, to create a stored access policy that manages the lifecyle of your SAS, see Define a stored access policy.
• For Permissions, select Read and List only.
4. Choose a start and expiry date and time for you SAS.

   Once the SAS expires, Commander will no longer be able to retrieve exported billing data for your Azure subscription. To manage the lifecycle of your SAS, you can configure a stored access policy. For more information, see Define a stored access policy. If your SAS expires and you do not have a stored access policy, you must manually re-generate a blob SAS token and blob SAS URL in Azure, and then update the values in Commander.

5. For Allowed protocals, select HTTPS.
6. Select Generate SAS and connection string.

   The blob SAS token and blob SAS URL will generate and display below. Copy and save these values before leaving the page.

Configure billing data retrieval in Commander

1. From the Inventory tree, select an Azure cloud account.
2. Go to Actions > Retrieve Billing Data.
3. In the Retrieve Azure Usage and Cost Data dialog, select Enable billing data retrieval.
4. Enter a blob SAS token, blob SAS URL, blob root directory, then click OK.

The blob root directory is the navigational path to the .csv billing files, excluding the storage account and container name. To find this path in the Azure portal, go to Cost Management + Billing > Exports, and select your billing export. From the Export details, copy the Storage directory value.

Switching to this retrieval method from the legacy method requires a billing data reset using REST API. For more information, see Manual Billing Reset through Commander API.

Retrieve Azure billing data (legacy method)

The legacy methods for billing retrieval are only supported for Commander versions prior to 9.6, or if you have upgraded to Commander 9.6 from a previous version. It is not available in new installations of version 9.6.

Using these legacy methods, Commander can retrieve a maximum of 60 days worth of billing data, including data from before the Azure account was added as a cloud account. Commander also retrieves Azure billing data for VMs that have never been managed by Commander. For example, if an Azure account was added as a cloud account 90 days ago, and you configure Commander to retrieve billing data today, Commander retrieves the last 60 days worth of billing data for all Azure VMs that have existed at any point over the last 60 days. As another example, if you add an Azure account as a cloud account today, and you configure Commander to retrieve billing data today, Commander retrieves the last 60 days worth of billing data for all Azure VMs that have existed at any point over the last 60 days, including VMs that were never managed by Commander.

Retrieve Azure billing data for Enterprise Agreement accounts (legacy)

1. From the Inventory tree, select an Azure cloud account.
2. Select Actions > Retrieve Billing Data.
3. In the Retrieve Azure Usage and Cost Data dialog, select Enable Azure Enterprise Agreement billing data retrieval (legacy).
4. Enter the Enrollment Number that you received when you signed the Azure Enterprise Agreement contract.
5. Enter the Billing API Access Key.

For instructions on how to generate the billing API access key, go to Overview of the Azure Enterprise Reporting APIs.

6. Specify the Currency of the billing data being retrieved from Azure.
7. Click Test to validate the enrollment number and check that the billing API access key is valid and hasn't expired.
8. Click OK.

   Sixty days of billing data is retrieved immediately, and a nightly task will update future records on a daily basis.

After you have chosen an offer type for Azure Billing Retrieval, changing to a different offer type requires a billing data reset using REST API. For more information, see Manual Billing Reset through Commander API.

Retrieve Azure billing data for Pay-As-You-Go accounts (legacy)

If you set up a Pay-As-You-Go account package when you registered with Azure, use these steps to retrieve Azure billing data.

1. From the Inventory tree, select an Azure cloud account.
2. Select Actions > Retrieve Billing Data.
3. In the Retrieve Azure Usage and Cost Data dialog, select Enable Azure billing data retrieval (legacy).
4. For Offer Type, select Pay-As-You-Go.
5. Click OK.

   Sixty days of billing data is retrieved immediately, and a nightly task will update future records.

After you have chosen an offer type for Azure Billing Retrieval, changing to a different offer type requires a billing data reset using REST API. For more information, see Manual Billing Reset through Commander API.

Retrieve Azure billing data for Azure Plan accounts (legacy)

Before you can retrieve Azure billing data for an account under the Azure Plan, you must set up a number of prerequisites.

Prerequisites for retrieving Azure Plan billing data

Follow these steps for each new subscription under the Azure Plan:

1. Using an automation tool, such as Azure Lighthouse, create a subscription for each of the customers you wish to onboard.
2. For each of these subscriptions, create a dedicated service principle (an app) with an API key.

This app needs to have an IAM role on the subscription, with read permission on Microsoft.Commerce, specifically the "Read Rate Card" and "Read Usage Aggregates" permissions.

3. Using the subscription ID and API key, add this subscription as a cloud account in Commander.

Retrieve billing data for Azure Plan

1. From the Inventory tree, select an Azure cloud account.
2. Select Actions > Retrieve Billing Data.
3. In the Retrieve Azure Usage and Cost Data dialog, select Enable Azure billing data retrieval.
4. For Offer Type, select Azure Plan.
5. Click OK.

   Sixty days of billing data is retrieved immediately and a nightly task will update future records.

After you've chosen an offer type for Azure Billing Retrieval, changing to a different offer type requires a billing data reset using REST API. For more information, see Manual Billing Reset through Commander API.

Disable the retrieval of Azure billing data

1. From the Inventory tree, select an Azure cloud account.
2. Select Actions > Retrieve Billing Data.
3. In the Retrieve Azure Usage and Cost Data dialog, select Not Configured and click OK.

Commander will no longer retrieve billing data for this cloud account.

Troubleshooting

The following error message may show up on the Retrieve Azure Usage and Cost Data dialog after you enter your billing API access key: "Unable to access the Billing API service. The enrollment number might be incorrect, or the API access key is invalid or has expired." If your key has expired, you'll need to regenerate it. For instructions on how to generate the Billing API Access key, go to Overview of Reporting APIs for Enterprise Customers .

Billing data for VMs not managed by Commander

Commander also retrieves Azure billing data for VMs that have never been managed by Commander, including charges for:

• VMs that exist only between Commander synchronizations with Azure, such as VMs spawned and torn down by an Auto Scaling Group
• VMs that existed only before the Azure account was added as a Commander cloud account
• VMs that existed only when the Commander service was down
• existing VMs, including for periods before the Azure account was added as a cloud account

When a VM not managed by Commander is discovered in the Azure billing report, the following occurs:

• Cost Models are applied to VMs, based on their location in the Infrastructure or Applications tree
• Ownership information is applied by the Default Ownership policy, or is inherited from the parent Resource Group.

The Default Attributes policy doesn't affect VMs discovered in the Azure billing report.

Billing data for all Azure services

When billing data retrieval is configured, Commander's cost analytics data accurately reflects the total cost you'll have to pay on your Azure bill and includes costs for services that Commander doesn't manage. Examples of such services include but are not limited to:

• data management
• security center
• business analytics
• data services

On your Azure bill, some of the costs for services may be divided into sub-categories. These sub-categories don't show up in Commander's cost analytics data. Only the total cost for a service is displayed.

If you have an Enterprise Agreement account, but configure the system for standard billing data retrieval, the resulting reports may show values that differ from your actual payments.

Ownership of each service will reflect the ownership policy that you set up on the cloud account, region, or resource group level.

Tax and Azure support plan fees are not included in the list of services because they are charged on a monthly basis.

Azure Billing Tags

Billing Tags allow you to group and filter your expenses in more detail. You can use these tags to filter expenses in Cost Analytics, the Service Portal Cost Dashboard, and the Cloud Billing Report. Billing Tag information is part of your Azure account billing data and is automatically imported during billing data retrieval. For information on using Billing Tags in formulas for Additional Line Items, see Additional Line Items page.

• Before upgrade to 8.1.0, filtering Cost Analytics and the Service Portal Cost Dashboard by Custom Attributes affected the Recommendations. After upgrade, filtering by Billing Tags won't affect the Recommendations.
• Billing Tags and SKUs will only be retrieved from Version 8.1.0 and forward, unless billing records are reset. Create a case through the Snow Support Portal if you want to perform this activity.

Snow Software does not own the third party trademarks, software, products, or tools (collectively, the “Third Party Products”) referenced herein. Third Party Product updates, including user interface updates, may not be reflected in this content.