Securing Connections

Connections require sensitive information, such as user names or passwords, to access data in external systems.

To ensure that this sensitive information is not compromised, use roles and permissions to secure connection objects and use credential stores or runtime resources to secure connection values.

Secure Connection Objects

To secure connection objects, use roles and permissions as follows:
  • Grant a small set of DevOps engineers the Connection Editor role and Write permission on connections. With this access, a user has full access to connection objects. The user can create and edit connections in the Connections view.
  • Grant data engineers who design pipelines the Connection User role and Read permission on connections. With this access, a user can select the connection name when configuring a pipeline or fragment, but cannot view the connection values.

For more information, see Roles and Permissions.

Secure Connection Values

To secure connection values, use one of the following methods when you define connection properties:
Credential stores
To use credential stores, you add the sensitive values as secrets in an external credential store system, such as AWS Secrets Manager or Azure Key Vault. Then you use credential functions in connection properties to retrieve those values.
You must enable all registered engines that access the connection to use the external credential store system. For example, you must enable the authoring Data Collector to use the credential store so that you can test a connection or preview a pipeline that uses credential functions. Similarly, you must enable the execution Data Collector to use the credential store so that you can run a pipeline that uses credential functions.
For more information, see credential stores for Data Collector or credential stores for Transformer.
Runtime resources
To use runtime resources, you define runtime resources in a file that is locally stored on Data Collector or Transformer, and then you call the resources from connection properties.
You must locally store the runtime resource file on all registered Data Collectors or Transformers that use the connection. For example, you must locally store the file on the authoring Data Collector so that you can test a connection or preview a pipeline that calls a resource from the file. Similarly, you must store the file on the execution Data Collector so that you can run a pipeline that calls a resource from the file.
Using runtime resources also allows you to change connection values by simply changing the values in the runtime resource files. For example, you might store the same resource file on two execution Data Collectors, but define a different resource value in each file. That way, a pipeline using the same connection can run on both execution Data Collectors, but the connection uses different values.
For more information, see runtime resources for Data Collector or runtime resources for Transformer.
Tip: To more securely define sensitive values, use credential stores.