Group Access to Secrets

As an additional layer of security, you can employ user groups to further limit access to the secrets defined in credential stores.

Transformer provides two methods to limit access with user groups:

Required group argument in credential functions

Credential functions include a group argument that defines the group that can access the secret. The group argument ensures that the user who attempts to preview, validate, or start a pipeline that includes a credential function belongs to the group specified in the function. The user must also have execute permission on the pipeline.

When working only with Transformer, simply specify the group name, such as devops. When working with Control Hub, sSpecify the group using the required naming convention:

<group
                            ID>@<organization ID>

. For example, devops@MyCompany.

If you do not want to restrict access to a secret, specify the default all group when working only with Transformer. When working with Control Hub and Transformer version 3.14.0 or later, you can specify the default group using all or

all@<organization
                            ID>

. StreamSets recommends using all so that you do not need to modify credential functions when migrating pipelines from Transformer to Control Hub.

Note: When working with Control Hub and a Transformer version earlier than 3.14.0, you must use the default all@<organization ID> group.

If Transformer shuts down while running a pipeline that uses a credential function, Transformer restarts the pipeline without checking the group access.

Optional group secrets in the credential store

In addition to using the group argument in credential functions, you can configure Transformer to require group secrets for a credential store.

To require the use of group secrets, in the $TRANSFORMER_CONF/credential-stores.properties fileTransformer credential store configuration properties, set the credentialStore.<cstore ID>.config.enforceEntryGroup property to true.

A group secret is a secret defined in the credential store that contains a comma-delimited list of Transformer user groups permitted to access the associated secret.

When the credential store ID requires group secrets, you must define a group secret for every secret that Transformer accesses in that credential store. The name of the group secret is based on the secret name, as follows:

<secret name>-groups

When you configure a credential function to call a secret, the user group specified in the credential function must be listed in the associated group secret that is defined in the credential store.

For example, say you enable Transformer to require group secrets for Azure Key Vault. Then, in a Azure Event Hubs origin, you use the following expression to retrieve a shared access key from the azure credential store:

${credential:get("azure", "production@MyCompany", sharedAccessKey)}

When you run the pipeline, Transformer validates all of the following:

The user who starts the pipeline is in the production user group.
The sharedAccessKey secret has an associated sharedAccessKey-groups secret defined in the credential store.
The sharedAccessKey-groups secret includes the production user group.

When Transformer is not configured to require group secrets, Transformer validates only the first point, verifying that the user belongs to the specified group.