Security in Kafka Stages

Add the configuration properties required for Kafka clients based on your installation and authentication type:

• Tarball, RPM, or Cloudera Manager installation without LDAP authentication - If Data Collector does not use LDAP authentication, create a separate JAAS configuration file on the Data Collector machine. Add the following KafkaClient login section to the file:

  KafkaClient {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      keyTab="<keytab path>"
      principal="<principal name>/<host name>@<realm>";
  };

  For example:

  KafkaClient {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      keyTab="/etc/security/keytabs/kafka_client.keytab"
      principal="kafka/node-01.cluster@EXAMPLE.COM";
  };

  Then modify the SDC_JAVA_OPTS environment variable to include the following option that defines the path to the JAAS configuration file:

  -Djava.security.auth.login.config=<JAAS config path>/kafka_client_jaas.conf

  Modify environment variables using the method required by your installation type.

• Tarball or RPM installation with LDAP authentication - If LDAP authentication is enabled in a tarball or RPM installation, add the properties to the JAAS configuration file used by Data Collector - the $SDC_CONF/ldap-login.conf file. Add the following KafkaClient login section to the end of the ldap-login.conf file:

  KafkaClient {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      keyTab="<keytab path>"
      principal="<principal name>/<host name>@<realm>";
  };

  For example:

  KafkaClient {
      com.sun.security.auth.module.Krb5LoginModule required
      useKeyTab=true
      keyTab="/etc/security/keytabs/kafka_client.keytab"
      principal="kafka/node-01.cluster@EXAMPLE.COM";
  };

• Cloudera Manager installation with LDAP authentication - If LDAP authentication is enabled in a Cloudera Manager installation, enable the LDAP Config File Substitutions (ldap.login.file.allow.substitutions) property for the StreamSets service in Cloudera Manager.

  If the Use Safety Valve to Edit LDAP Information (use.ldap.login.file) property is enabled and LDAP authentication is configured in the Data Collector Advanced Configuration Snippet (Safety Valve) for ldap-login.conf field, then add the JAAS configuration properties to the same ldap-login.conf safety valve.

  If LDAP authentication is configured through the LDAP properties rather than the ldap-login.conf safety value, add the JAAS configuration properties to the Data Collector Advanced Configuration Snippet (Safety Valve) for generated-ldap-login-append.conf field.

  Add the following KafkaClient login section to the appropriate field as follows:

  KafkaClient {
       com.sun.security.auth.module.Krb5LoginModule required
       useKeyTab=true
       keyTab="_KEYTAB_PATH"
       principal="<principal name>/_HOST@<realm>";
  };

  For example:

  KafkaClient {
       com.sun.security.auth.module.Krb5LoginModule required
       useKeyTab=true
       keyTab="_KEYTAB_PATH"
       principal="sdc/_HOST@EXAMPLE.COM";
  };

  Cloudera Manager generates the appropriate keytab path and host name.

• Enter a Base64-encoded keytab in the Runtime Keytab property.

  Encode the keytab before entering it in the stage property. Be sure to remove unnecessary characters, such as newline characters, before encoding the keytab.

• Use a credential function to access a Base64-encoded keytab defined in a credential store.

  For more information, see Using a Credential Store.