Azure Data Factory: Reduce Your ETL Service Costs With Microsoft’s Data Transformation Solution

We recently had the opportunity to help a not-for-profit client reduce their extract, transform and load (ETL service) costs by 95% by moving from a subscription-based cloud platform model to Azure Data Factory.

The client had a looming contract deadline and was under an expensive agreement that was about to increase even more. With Azure’s pay-as-you-go model, the client could leverage the ability to “right-size” their usage to ensure that costs scale appropriately with the volume of their business and their data transformation needs.

You can read about it in our recent case study detailing the project.

Azure Data Factory is a tool that orchestrates a wide range of data processes for you without installing anything on a server. It can seamlessly schedule the movement and transformation of data in Salesforce, SharePoint, accounting software or other on-premises solutions (to name only a few). You could also use code to access data via API using any other system you have permission to access.

If you need more advanced transformations with your data, Azure offers integrated access to compute services. You can set up a Spark cluster with Azure HDInsight, utilize Azure Data Lake Analytics, Azure Batch, ML Studio and an array of Stored Procedure Activity options using SQL services.

Azure Data Factory runs on a pay-as-you-go model. In the case study we mentioned above, the client was trapped in a situation where they were paying a flat rate regardless of usage. This meant that they were vastly overpaying even when no data was flowing. Even worse, the client was quoted a 17% cost increase for the same usage when their contract was up for renewal for the next year.

Data Factory breaks down the cost into components you may or may not use to execute your data transformation pipelines. It also offers a cost calculator and usage metrics in debugging sessions, allowing you to estimate and modify your approach while developing to minimize cost. There are a variety of factors that can be adjusted to tightly control costs and flexible options for frequency of data orchestration in the case of a cyclical or seasonal business. These factors include but are not limited to the location of data, transformations required, ingestion method, storage needs and volume. Azure Data Factory and the whole list of Azure resources that are easily integrated can help you impact those factors that determine the cost, which is what achieves the most seamless experience with your ETL service.