Credential Stores

You can configure Transformer to access sensitive information that is secured in a credential store.

Transformer pipelines communicate with external systems to perform tasks such as launching a Spark application, or reading and writing data. Most of these external systems require sensitive information, such as user names or passwords, to access the system. When you configure pipeline stages for these external systems, you must specify the details that the stages need to connect to the system.

If you enter sensitive information directly in stage and pipeline properties, you expose those details to any user with access to the pipeline. To access external systems without exposing sensitive details, add them as secrets to a credential store and then use StreamSets credential functions in stage and pipeline properties to retrieve those values.

Defining secrets in a credential store can make it easier to migrate pipelines to another environment. For example, if you migrate multiple pipelines from a development to a production environment, you do not need to edit each pipeline with details for the production environment. You can simply replace the development credential store with the production version.

You can configure Transformer to use multiple credential stores. Each credential store is identified by a unique credential store ID.