Enabling LDAP Authentication

If your company uses Lightweight Directory Access Protocol (LDAP), you can use the LDAP provider to authenticate Control Hub users. LDAP authenticates a user using the credentials stored in the LDAP server.

Note: You can also use the built-in Control Hub authentication method or the SAML authentication method. These authentication methods are configured by the organization administrator for their organization. For more information, see Authentication.

LDAP authentication is configured by the default system administrator - the admin@admin user account - for the entire Control Hub system. After LDAP authentication is enabled, all organizations must use LDAP authentication. Users log in to Control Hub using their Control Hub user ID and their LDAP password.

To use LDAP authentication, the Control Hub system administrator configures LDAP connection information for Control Hub and then maps organization administrator accounts to LDAP users. Organization administrators then create Control Hub user accounts for their organization, mapping the Control Hub user accounts to LDAP users.

Control Hub can also retrieve group membership from the LDAP provider. To group users, organization administrators create Control Hub groups, and then map the Control Hub groups to LDAP groups.

You can enable LDAP authentication during the installation or after the installation.

Synchronization between Control Hub and LDAP

Once you enable LDAP authentication, Control Hub automatically adds and removes existing users from groups as changes occur in the LDAP provider. Control Hub takes up to a minute to reflect such changes in the LDAP provider.

With additional configuration, Control Hub can also automatically create and deactivate users to match users in LDAP groups. When configured, Control Hub periodically checks the LDAP provider and does the following:
  • Creates Control Hub users when necessary to match users in the LDAP groups.
  • Deactivates but does not delete users from Control Hub when those users are removed from all LDAP groups linked to Control Hub.

    If these same users are added back to one of the LDAP groups linked to Control Hub, the users remain deactivated in Control Hub. An organization administrator must activate the user accounts in Control Hub.

    Alternatively, you can manually delete deactivated users. If these same users are added back to one of the LDAP groups, Control Hub creates the users again with an active status.

    Note: Control Hub never deactivates an organization administrator.

Control Hub checks for users to create and deactivate at most every five minutes. You can configure a longer period between checks.

To accurately create and deactivate users, Control Hub requires that the Control Hub user IDs match the LDAP user names. Control Hub records created and deactivated users as actions, and administrators can see the changes from the Action Audit view.

System Administrator and LDAP Authentication

When LDAP authentication is enabled, the default system administrator - the admin@admin user account - continues to log in using the password for the account stored in the Control Hub relational database.

The system administrator account cannot be mapped to an LDAP user account. If LDAP authentication is incorrectly configured within Control Hub, users will not be able to log in using LDAP authentication. However, the system administrator can use Control Hub credentials to log in, troubleshoot LDAP-related issues, and then re-enable access to Control Hub.

Step 1. Configure LDAP Connection Information

To enable LDAP authentication, configure LDAP connection information in the Control Hub security configuration file, $DPM_CONF/security-app.properties.

If your organization has multiple LDAP servers, you can include multiple LDAP configurations within the file - so that Control Hub can connect to multiple LDAP servers. LDAP user accounts must be unique across the servers. A user account that exists in multiple LDAP servers might encounter unexpected results after logging in to Control Hub.
Important: Instead of entering sensitive data such as passwords in clear text in the security configuration file, you can protect the sensitive data by storing the data in an external location and then using functions to retrieve the data.
  1. In the security-app.properties file, enable LDAP authentication by completing the following steps:
    1. Comment out the userGroupProvider.id=embedded line, as follows: 
      #userGroupProvider.id=embedded

      When the property is enabled and set to embedded, Control Hub uses either the built-in Control Hub authentication method or the SAML authentication method. Organization administrators configure these authentication methods for each organization.

    2. Uncomment the second userGroupProvider.id property, as follows: 
      userGroupProvider.id=M

      Do not change the value of this property.

  2. Configure the following global properties used by all LDAP configurations included in the file:
    Global Property Description
    userGroupProvider.M.providerClass Do not change this value.
    userGroupProvider.M.multi.ids Unique ID for each LDAP configuration. The ID can be any string value. To include multiple LDAP configurations, use a comma-separated list.
    For example, if you want to connect to both a Microsoft Active Directory server and to an OpenLDAP server, you might define the property as follows:
    userGroupProvider.M.multi.ids=AD,OpenLDAP

    Default is L.
    userGroupProvider.M.multi.fetchGroups Retrieves group information from the provider. Must be true if you map Control Hub groups to the provider groups.

    Default is true.
    userGroupProvider.M.multi.allGroupsProviderId Set to the same value as the userGroupProvider.M.multi.ids property.
  3. Configure the following properties for each LDAP configuration.

    Each LDAP property includes the following prefix:

    userGroupProvider.M.multi.<ID>.ldap
    When you configure the properties, replace the <ID> variable in each property name with the unique ID that you defined for the LDAP configuration. For example, if you defined both AD and OpenLDAP as IDs to create multiple LDAP configurations, then you would replace the <ID> variable in the first three properties as follows:
    # Connection information for Active Directory
userGroupProvider.M.multi.AD.providerClass=
         com.streamsets.apps.security.authentication.ldap.LdapUserGroupProvider
userGroupProvider.M.multi.AD.ldap.poolMinConnections=3
userGroupProvider.M.multi.AD.ldap.poolMaxConnections=10

...

# Connection information for OpenLDAP
userGroupProvider.M.multi.OpenLDAP.providerClass=
        com.streamsets.apps.security.authentication.ldap.LdapUserGroupProvider
userGroupProvider.M.multi.OpenLDAP.ldap.poolMinConnections=3
userGroupProvider.M.multi.OpenLDAP.ldap.poolMaxConnections=10

...
    For simplicity, the table below has dropped the following prefix from each LDAP property:
    userGroupProvider.M.multi.<ID>.ldap
    Global Property Description
    providerClass Use the correct <ID> variable in the property prefix, but do not change the property value.
    poolMinConnections Minimum number of connections for the Bind DN connection pool. Default is 3.
    poolMaxConnections Maximum number of connections for the Bind DN connection pool. Default is 10.
    poolValidateConnections Validate the connections retrieved from the Bind DN connection pool. Default is true.
    connectionTimeoutMillis Connection timeout in milliseconds. Default is 5000.
    responseTimeoutMillis Response timeout in milliseconds. Default is 5000.
    hostname LDAP server host name.
    port LDAP server port.

    To use unencrypted connections or to use connections secured with StartTLS, enter the LDAP port number, typically 389. To use connections secured with LDAPS, enter the port number for secure connections, typically 636.
    ldaps Secure LDAP connections using the LDAPS (LDAP over SSL) protocol. Default is false.

    You must complete additional steps to use LDAPS, see Configuring Secure Connections to LDAP.
    startTLS Secure LDAP connections using the StartTLS protocol. Default is false.

    You must complete additional steps to use StartTLS, see Configuring Secure Connections to LDAP.

    Note: StartTLS and LDAPS cannot be used at the same time. If both startTLS and ldaps are set to true, startTLS takes precedence.
    userBaseDn Base DN under which user accounts are located.
    userObjectClass Name of the user object class. Default is inetOrgPerson.
    userNameAttribute Name of the user ID attribute. Default is uid.
    userEmailAttribute Name of the email address attribute. Default is mail.
    userFullNameAttribute Name of the user full name attribute. Default is cn.
    userFilter User attribute used to log in to Control Hub. For example, LDAP users can log in using a username, uid, or email address.

    Default is %s={user}, where %s is replaced with the value of the userNameAttribute property.
    bindDn Root distinguished name (DN) used to query the directory server. This user must have privileges to search the directory.
    bindPassword Password for the root distinguished name. To protect the password, store the password in an external location and then use a function to retrieve the password.
    fetchGroups Retrieves group information from the LDAP provider. Must be true if you map Control Hub groups to LDAP groups.

    Default is true.
    groupBaseDn Base DN to search for group membership.
    groupObjectClass Name of the group object class.

    Default is groupOfUniqueNames.
    groupMemberAttribute Name of the group attribute for user names. Default is uniqueMember.
    groupNameAttribute Name of the attribute for group names. Default is cn.
    groupFullNameAttribute Name of the group attribute included with the group display name. Default is description.
    groupFilter Group attribute used to look up the group memberships for a user. For example, you can search for a user's groups using a base DN, username, or uid.

    Default is %s={dn}, where %s is replaced with the value of the groupMemberAttribute property. The group object class is also automatically added to the group filter.
  4. To configure Control Hub to automatically create and deactivate users to match LDAP groups, edit the following properties:
    Property Description
    ldap.automaticResolutionEnabled Enables Control Hub to automatically create and deactivate users to match users in LDAP groups. Set to true.
    ldap.resolutionFrequencyMillis Number of milliseconds between checks for users to create or deactivate. Default value is 5 minutes. You can set to a larger value, if desired.
  5. Shut down and then launch Control Hub again to enable the changes.

Example for Active Directory

The following example shows a configuration to connect to a Microsoft Active Directory server and enable automatic creation and deactivation of users:
ldap.automaticResolutionEnabled=true
ldap.resolutionFrequencyMillis=300000
#userGroupProvider.id=embedded
userGroupProvider.externalProvider.principalCache.expiration.secs=60
userGroupProvider.id=M
userGroupProvider.M.providerClass=
        com.streamsets.apps.security.authentication.MultiUserGroupProvider
userGroupProvider.M.multi.ids=AD
userGroupProvider.M.multi.fetchGroups=true
userGroupProvider.M.multi.allGroupsProviderId=AD
userGroupProvider.M.multi.AD.providerClass=
        com.streamsets.apps.security.authentication.ldap.LdapUserGroupProvider
userGroupProvider.M.multi.AD.ldap.poolMinConnections=3
userGroupProvider.M.multi.AD.ldap.poolMaxConnections=10
userGroupProvider.M.multi.AD.ldap.poolValidateConnections=true
userGroupProvider.M.multi.AD.ldap.connectionTimeoutMillis=5000
userGroupProvider.M.multi.AD.ldap.responseTimeoutMillis=5000
userGroupProvider.M.multi.AD.ldap.hostname=abc01.mycompany.net
userGroupProvider.M.multi.AD.ldap.port=636
userGroupProvider.M.multi.AD.ldap.ldaps=true
userGroupProvider.M.multi.AD.ldap.startTLS=false
userGroupProvider.M.multi.AD.ldap.userBaseDn=OU=MyCompany,DC=mycompany,DC=net
userGroupProvider.M.multi.AD.ldap.userObjectClass=organizationalPerson
userGroupProvider.M.multi.AD.ldap.userNameAttribute=sAMAccountName
userGroupProvider.M.multi.AD.ldap.userEmailAttribute=mail
userGroupProvider.M.multi.AD.ldap.userFullNameAttribute=cn
userGroupProvider.M.multi.AD.ldap.userFilter=%s={user}
userGroupProvider.M.multi.AD.ldap.bindDn=admin@mycompany.net
userGroupProvider.M.multi.AD.ldap.bindPassword=******
userGroupProvider.M.multi.AD.ldap.fetchGroups=true
userGroupProvider.M.multi.AD.ldap.groupBaseDn=OU=MyCompany,DC=mycompany,DC=net
userGroupProvider.M.multi.AD.ldap.groupObjectClass=group
userGroupProvider.M.multi.AD.ldap.groupMemberAttribute=member
userGroupProvider.M.multi.AD.ldap.groupNameAttribute=cn
userGroupProvider.M.multi.AD.ldap.groupFullNameAttribute=description
userGroupProvider.M.multi.AD.ldap.groupFilter=%s={dn}

Example for OpenLDAP

The following example shows a configuration to connect to an OpenLDAP server and enable automatic creation and deactivation of users:
ldap.automaticResolutionEnabled=true
ldap.resolutionFrequencyMillis=300000
#userGroupProvider.id=embedded
userGroupProvider.externalProvider.principalCache.expiration.secs=60
userGroupProvider.id=M
userGroupProvider.M.providerClass=
        com.streamsets.apps.security.authentication.MultiUserGroupProvider
userGroupProvider.M.multi.ids=OpenLDAP
userGroupProvider.M.multi.fetchGroups=true
userGroupProvider.M.multi.allGroupsProviderId=OpenLDAP
userGroupProvider.M.multi.OpenLDAP.providerClass=
        com.streamsets.apps.security.authentication.ldap.LdapUserGroupProvider
userGroupProvider.M.multi.OpenLDAP.ldap.poolMinConnections=3
userGroupProvider.M.multi.OpenLDAP.ldap.poolMaxConnections=10
userGroupProvider.M.multi.OpenLDAP.ldap.poolValidateConnections=true
userGroupProvider.M.multi.OpenLDAP.ldap.connectionTimeoutMillis=5000
userGroupProvider.M.multi.OpenLDAP.ldap.responseTimeoutMillis=5000
userGroupProvider.M.multi.OpenLDAP.ldap.hostname=abc02.mycompany.net
userGroupProvider.M.multi.OpenLDAP.ldap.port=389
userGroupProvider.M.multi.OpenLDAP.ldap.ldaps=false
userGroupProvider.M.multi.OpenLDAP.ldap.startTLS=false
userGroupProvider.M.multi.OpenLDAP.ldap.userBaseDn=ou=employees,dc=example,dc=org
userGroupProvider.M.multi.OpenLDAP.ldap.userObjectClass=inetOrgPerson
userGroupProvider.M.multi.OpenLDAP.ldap.userNameAttribute=uid
userGroupProvider.M.multi.OpenLDAP.ldap.userEmailAttribute=mail
userGroupProvider.M.multi.OpenLDAP.ldap.userFullNameAttribute=cn
userGroupProvider.M.multi.OpenLDAP.ldap.userFilter=%s={user}
userGroupProvider.M.multi.OpenLDAP.ldap.bindDn=cn=admin,dc=example,dc=org
userGroupProvider.M.multi.OpenLDAP.ldap.bindPassword=******
userGroupProvider.M.multi.OpenLDAP.ldap.fetchGroups=true
userGroupProvider.M.multi.OpenLDAP.ldap.groupBaseDn=ou=departments,dc=example,dc=org
userGroupProvider.M.multi.OpenLDAP.ldap.groupObjectClass=groupOfNames
userGroupProvider.M.multi.OpenLDAP.ldap.groupMemberAttribute=member
userGroupProvider.M.multi.OpenLDAP.ldap.groupNameAttribute=cn
userGroupProvider.M.multi.OpenLDAP.ldap.groupFullNameAttribute=description
userGroupProvider.M.multi.OpenLDAP.ldap.groupFilter=%s={dn}

Step 2. Configure Secure Connections (Optional)

You can optionally configure Control Hub to use one of the following methods to make secure connections to the LDAP server:
LDAP over SSL (LDAPS)
LDAPS uses SSL to encrypt LDAP connections. LDAPS uses the ldaps:// scheme.
StartTLS
StartTLS can wrap an unencrypted connection with TLS during the connection process. This allows the same port to handle both unencrypted and encrypted connections. StartTLS uses the ldap:// scheme.

For either encryption method, if the LDAP server certificate is signed by a private Certificate Authority (CA) or not trusted by the default Java truststore, you must create a custom truststore file or modify the default Java truststore file to add the CA to the file. Then configure Control Hub to use the modified truststore file.

Use the same procedure to configure either secure method.

  1. In the $DPM_CONF/security-app.properties file, set one of the following properties to true:
    • userGroupProvider.M.multi.<ID>.ldap.ldaps
    • userGroupProvider.M.multi.<ID>.ldap.startTLS

    By default, both properties are false and so Control Hub makes unencrypted connections to the LDAP server. If you set both properties to true, StartTLS takes precedence.

  2. Set the port property in the security-app.properties file as required, based on the method that you enabled:
    • LDAPS - Use the port number for secure connections, typically 636.
    • StartTLS - Use the LDAP port number, typically 389.
  3. If the LDAP server certificate is signed by a private CA or not trusted by the default Java truststore, create a custom truststore file or modify the default Java truststore file to add the CA to the file. Then configure Control Hub to use the modified truststore file.

    By default, Control Hub uses the Java truststore file located in $JAVA_HOME/jre/lib/security/cacerts. If your certificate is signed by a trusted CA that is included in the default Java truststore file, you do not need to create a truststore file and can skip this step.

    In these steps, we show how to modify a copy of the default truststore file to add an additional CA to the list of trusted CAs. If you prefer to create a custom truststore file, see the keytool documentation.

    Note: If you've already configured Control Hub to use a custom truststore file to enable HTTPS, then simply add this additional CA to the same modified truststore file that you created in Step 2. Create a Truststore File.
    1. Use the following command to set the JAVA_HOME environment variable: 
      export JAVA_HOME=<Java home directory>
    2. Use the following command to set the DPM_CONF environment variable: 
      export DPM_CONF=<Control Hub configuration directory>

      For example, for an RPM installation use:

      export DPM_CONF=/etc/dpm
    3. Use the following command to copy the default Java truststore file to the Control Hub configuration directory: 
      cp "${JAVA_HOME}/jre/lib/security/cacerts" "${DPM_CONF}/truststore.jks"
    4. Use the following keytool command to import the CA certificate into the truststore file: 
      keytool -import -file <LDAP certificate> -trustcacerts -noprompt -alias <LDAP alias> -storepass <password> -keystore "${DPM_CONF}/truststore.jks"
      For example:
      keytool -import -file  myLDAPServer.pem -trustcacerts -noprompt -alias MyLDAPServer -storepass changeit -keystore "${DPM_CONF}/truststore.jks"
    5. Define the following options in the DPM_JAVA_OPTS environment variable:
      • javax.net.ssl.trustStore - Path to the truststore file on the Control Hub machine.
      • javax.net.ssl.trustStorePassword - Truststore password.

      Modify environment variables in the file required by your installation type.

      For example, define the options as follows:
      export DPM_JAVA_OPTS="${DPM_JAVA_OPTS} -Djavax.net.ssl.trustStore=/etc/dpm/truststore.jks -Djavax.net.ssl.trustStorePassword=mypassword 
-server -Xmx1024m -Xms1024m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./dpm-$$-outOfMem.hprof -XX:ErrorFile=./dpm-$$-error.log"

      Or to avoid saving the password in the export command, save the password in a text file and then define the truststore password option as follows: -Djavax.net.ssl.trustStorePassword=$(cat passwordfile.txt)

  4. Shut down and then launch Control Hub again to enable the changes.

Step 3. Map Organization Administrators to LDAP Users

As the system administrator, you must map each StreamSets Control Hub organization administrator to an LDAP user account.

The steps you use to map organization administrators depend on whether you are creating new organizations or editing existing organizations created before LDAP authentication was enabled:
New organizations
When you create new organizations, you simply enter the name of the LDAP user account to map to the organization administrator created with the organization, as described in Creating Organizations.
Existing organizations
If you have existing organizations created before LDAP authentication was enabled, edit the organization administrator user accounts as follows:
  1. In the Navigation panel, click Administration > Users.
  2. Click the Toggle Filter Column icon () to filter the users by organization.
  3. Select an organization and then click the existing organization administrator to display the account details.
  4. Enter the name of the LDAP user account to map to the organization administrator in the LDAP User Name property.
  5. Click Save.

Step 4. Map Users and Groups to LDAP Users and Groups

Organization administrators must map StreamSets Control Hub users and groups to LDAP users and groups for their organization.

When using LDAP authentication with Control Hub, we recommend that organization administrators follow these best practices:

  1. If you did not configure Control Hub to automatically create users, create Control Hub user accounts for the organization and map them to LDAP user accounts by entering the name of the LDAP user in the LDAP User Name property.
    Tip: Create Control Hub user IDs that match LDAP user names. For example, to add the LDAP user Amy to the Control Hub organization MyOrg, create a Control Hub user ID Amy@MyOrg. By doing so, you can later configure Control Hub to automatically create users.

    Assign the Organization User role to each Control Hub user account, and then clear all other roles.

  2. Create Control Hub groups and map them to LDAP groups by entering the name of the LDAP group in the LDAP Group Name property. LDAP group names are case sensitive.

    When LDAP authentication is enabled, you cannot assign Control Hub users to Control Hub groups. Control Hub retrieves the group memberships from the LDAP provider.

  3. Assign Control Hub roles to the Control Hub groups.

    To more efficiently manage user accounts, assign roles to groups rather than individually to each user account.