Enabling Credential Stores

You can configure Transformer to use one or more credential stores. Each credential store is identified by a unique credential store ID.

You specify the credential stores that Transformer can use in the $TRANSFORMER_CONF/credential-stores.properties file. The file includes the following informationTransformer credential configuration properties. To enable a credential store, you configure the following information:

credentialStores property: This property defines the credential stores that Transformer can use.; By default, the property is commented out and includes a default credential store ID for each of the supported credential store types, such as aws for AWS Secrets Manager and azure for Azure Key Vault.; To enable using credential stores, you uncomment this property and enter a comma-separated list of the credential store IDs to use.; You can specify multiple credential stores of the same type or of different types, such as two AWS Secret Managers and one Java Keystore. You simply specify a unique ID for each credential store.
Sets of related properties: Each supported credential store type has a set of related properties. The property names include the default credential store IDs originally specified in the credentialStores property.; For example, the AWS Secrets Manager properties include aws, the default Secrets Manager ID, in each Secrets Manager property name, such as credentialStore.aws.config.region and credentialStore.aws.config.access.key.; When you use a custom credential store ID, you must update all related property names to match the custom ID. For example, if you want to use awsUS as a custom ID, you must update all Secrets Manager default property names for the awsUS credential store replacing aws with awsUS.; Note: When you want to use multiple credential stores of the same type, you must have a set of related store properties that are renamed and defined appropriately for each credential store.

For example, say you want to use two Azure credential stores, azureDev for development and azureProd for production. To do this, you specify the credential store IDs in the credentialStores property and make a copy of the related Azure credential store properties, so you have one set for each credential store.

Then, you rename and configure the properties for azureDev, and you do the same for azureProd. The resulting properties might look as follows, with important changes highlighted:

################################################
#        Transformer Credential Stores         #
################################################

credentialStores=azureDev,azureProd

############################################################
# azureDev: Azure Key Vault Credential Store Configuration #
############################################################

credentialStore.azureDev.def=streamsets-transformer-azure-keyvault-credentialstore-lib::com_streamsets_datacollector_credential_azure_keyvault_AzureKeyVaultCredentialStore
credentialStore.azureDev.config.credential.refresh.millis=30000
credentialStore.azureDev.config.credential.retry.millis=15000
credentialStore.azureDev.config.vault.url=https://development.vault.azure.net/
credentialStore.azureDev.config.client.id=devClientID
credentialStore.azureDev.config.client.key=devClientKey
credentialStore.azureDev.config.enforceEntryGroup=false

#############################################################
# azureProd: Azure Key Vault Credential Store Configuration #
#############################################################

credentialStore.azureProd.def=streamsets-transformer-azure-keyvault-credentialstore-lib::com_streamsets_datacollector_credential_azure_keyvault_AzureKeyVaultCredentialStore
credentialStore.azureProd.config.credential.refresh.millis=30000
credentialStore.azureProd.config.credential.retry.millis=15000
credentialStore.azureProd.config.vault.url=https://production.vault.azure.net/
credentialStore.azureProd.config.client.id=prodClientID
credentialStore.azureProd.config.client.key=prodClientKey
credentialStore.azureProd.config.enforceEntryGroup=false