Azure Blob Storage

Available when using an authoring Data Collector version 5.5.0 or later.

To create an Azure Blob Storage connection, the Azure stage library, streamsets-datacollector-azure-lib, must be installed on the selected authoring Data Collector.

For a description of the connection properties, see Azure Blob Storage Connection Properties.

After you create an Azure Blob Storage connection, you can use the connection in the following stages:


Engine	Stages
Data Collector 5.10.0 or later	Azure Blob Storage destination
Data Collector 5.5.0 or later	Azure Blob Storage origin

Prerequisites

Before configuring an Azure Blob Storage connection, you must ensure that the connection provides read and write access to the needed objects. The connection uses an authentication method to establish its identity with Azure. Azure authorizes an identity to read and write objects. Azure provides various authorization methods for specific types of objects. The methods offer different levels of security. You must choose compatible authentication and authorization methods for your objects. Consult the Microsoft Azure Blob Storage documentation for information about the authorization methods. You must complete appropriate steps for the chosen method, such as configuring an Azure Active Directory application, creating a shared key, or creating a shared access signature (SAS) token.

An Azure Blob Storage connection can use one of the following authentication methods:

OAuth with Service Principal

Connections made with OAuth with Service Principal authentication require the following information:

Application ID - Application ID for the Azure Active Directory Data Collector application. Also known as the client ID.
For information on accessing the application ID from the Azure portal, see the Azure documentation.
Tenant ID - Tenant ID for the Azure Active Directory Data Collector application. Also known as the directory ID.
For information on accessing the tenant ID from the Azure portal, see the Azure documentation.
Application Key - Authentication key or client secret for the Azure Active Directory application. Also known as the client secret.
For information on accessing the application key from the Azure portal, see the Azure documentation.

Azure Managed Identity

Connections made with Azure Managed Identity authentication require the following information:

Application ID - Application ID for the Azure Active Directory Data Collector application. Also known as the client ID.
For information on accessing the application ID from the Azure portal, see the Azure documentation.

Shared Key

Connections made with Shared Key authentication require the following information:

Account Shared Key - Shared access key that Azure generated for the storage account.
For more information on accessing the shared access key from the Azure portal, see the Azure documentation.

SAS Token

Connections made with SAS Token authentication require the following information:

Azure SAS Token - Shared access signature (SAS) token that provides secure access to the needed resources in Azure Blob Storage.
For more information on SAS tokens for storage containers, see the Azure documentation.

Azure Blob Storage Connection Properties

When creating an Azure Blob Storage connection, configure the following properties on the Azure tab:


Azure Property	Description
Account FQDN	The host name of the Blob Storage account. For example: `<storage account name>.blob.core.windows.net`
Storage Container / File System	Name of the storage container or file system that contains the data to be read or written.
Authentication Method	Authentication method used to connect to Azure: OAuth with Service Principal Azure Managed Identity Shared Key SAS Token
Application ID	Application ID for the Azure Active Directory Data Collector application. Also known as the client ID. For information on accessing the application ID from the Azure portal, see the Azure documentation. Available when using the OAuth with Service Principal or the Azure Managed Identity authentication method.
Endpoint Type	Method to provide endpoint details. Available when using the OAuth with Service Principal authentication method.
Tenant ID	Tenant ID for the Azure Active Directory Data Collector application. Also known as the directory ID. For information on accessing the tenant ID from the Azure portal, see the Azure documentation. Available when Endpoint Type is set to Tenant ID.
Endpoint URL	Endpoint URL for the Azure Active Directory Data Collector application. Default is `https://login.microsoftonline.com/<tenant-id>/oauth2/token`. In the URL, specify the tenant ID for the Azure Active Directory Data Collector application. For information on accessing the tenant ID from the Azure portal, see the Azure documentation. Available when Endpoint Type is set to Endpoint URL.
Application Key	Authentication key or client secret for the Azure Active Directory application. Also known as the client secret. For information on accessing the application key from the Azure portal, see the Azure documentation. Available when using the OAuth with Service Principal authentication method.
Account Shared Key	Shared access key that Azure generated for the storage account. For more information on accessing the shared access key from the Azure portal, see the Azure documentation. Available when using the Shared Key authentication method.
Azure SAS Token	Shared access signature (SAS) token that provides secure access to the needed resources in Azure Blob Storage. For more information on SAS tokens for storage containers, see the Azure documentation. Available when using the SAS Token authentication method.