ADLS Gen1

The ADLS Gen1 destination writes files to Microsoft Azure Data Lake Storage Gen1. To write to Azure Data Lake Storage Gen2, use the ADLS Gen2 destination.

Note: When this stage is included in a pipeline that runs on an Azure HDInsight cluster, use an Azure HDInsight cluster version 4.0 or later.

The destination connects to Azure Data Lake Storage Gen1 using Azure Active Directory service principal authentication, also known as service-to-service authentication.

The destination writes data based on the specified data format and creates a separate file for every partition. Before you use the ADLS Gen1 destination, you must perform some prerequisite tasks.

When you configure the ADLS Gen1 destination, you specify the service name and Azure authentication information such as the application ID and key. Or, you can have the destination use Azure authentication information configured in the cluster where the pipeline runs.

You specify the output directory and write mode to use. When overwriting related partitions, first complete the overwrite partition requirement.

You select the data format to write and configure related properties. You can specify fields to use for partitioning files. You can also drop unrelated master records when using the destination as part of a slowly changing dimension pipeline.