Today we’ll begin our dive into building a pipeline in Azure Data Factory. Initially, I planned to have a single post explaining how to copy data from an Azure SQL Database to an Azure Data Lake Gen2 storage container, however, covering all that information would’ve extended over ten pages! This post will explore the first step in building out our copy data functionality: Creating Linked Service to an Azure SQL Database.
Azure Data Factory relies on Linked Services as the backbone of the platform.
What is a Linked Service, and Why Does it Matter?
At its core, a Linked Service is a connection to a data source and/or destination. Azure Data Factory relies on Linked Services as the backbone of the platform, enabling the user to “bring your own connection string.” The amount of options to select from is extensive. You can connect to virtually anywhere that holds data, and you can even copy a dataset straight from the URL! The Linked Service idea and support is really what makes Azure Data Factory such a powerful platform.
How to Create an Azure SQL Database Linked Service
Before we begin, make sure to have your Azure Data Factory instance open.
1) First, navigate to the Manage tab in Azure Data Factory.
2) Then, select the ‘Linked Services’ tab and click the ‘New’ button.
3) You will notice a plethora of options are available for a linked service. I encourage you to explore at your own leisure. For this example, we will use the Azure SQL Database option.
4) Name your linked service appropriately. I like to start every linked service name starting with ‘ls_’ to denote the type within the name. This is useful when searching for linked services within your pipelines.
5) When Azure SQL Database is selected, click the ‘enter manually’ radio button under ‘Account Selection Method.’ Enter the server name, database name and a username/password. Notice there is an option for Azure Key Vault. This is the recommended way to store connection string and credentials for Azure Data Factory, however, for the sake of demonstration, we will be using the connection string method.
6) After all information is entered, you can test the connection to the Azure SQL Database. If the test comes back with a green indicator, then click create. You have successfully created an Azure SQL Database Linked Service in Azure Data Factory!
Connect and Follow-up
If you would like to explore options in partnering with Tallan to help build out your businesses cloud data analytics platform, please reach out to me at Conner.Wulf@tallan.com or connect on LinkedIn.
Interested in learning about our DPi30 Data Platform in 30 Days? Click here to get started.