Providing PLAIN Credentials

To connect to Kafka using SASL authentication with the PLAIN mechanism, provide the credentials in a Java Authentication and Authorization Service (JAAS) file.

Create a JAAS configuration file on the Data Collector machine. You can define a single JAAS file for Data Collector. As a result, every Kafka connection in every pipeline that uses SASL authentication with the PLAIN mechanism uses the same credentials.

Add the following KafkaClient login section to the file:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required 
    username="<username>" 
    password="<password>";
};

Then modify the Java configuration options used by the deployment to include the option that defines the path to the JAAS configuration file. In Control Hub, edit the deployment. In the Configure Engine section, click Advanced Configuration. Then, click Java Configuration. Add the following option to the Java Options property:

-Djava.security.auth.login.config=<JAAS config path>/kafka_client_jaas.conf

Add the configuration properties required for Kafka clients based on your installation and authentication type:

Tarball, RPM, or Cloudera Manager installation without LDAP authentication

If Data Collector does not use LDAP authentication, create a separate JAAS configuration file on the Data Collector machine. Add the following KafkaClient login section to the file:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required 
    username="<username>" 
    password="<password>";
};

Then modify the SDC_JAVA_OPTS environment variable to include the following option that defines the path to the JAAS configuration file:

-Djava.security.auth.login.config=<JAAS config path>/kafka_client_jaas.conf

Modify environment variables Modify environment variables using the method required by your installation type.

Tarball or RPM installation with LDAP authentication

If LDAP authentication is enabled in a tarball or RPM installation, add the properties to the JAAS configuration file used by Data Collector - the $SDC_CONF/ldap-login.conf file. Add the following KafkaClient login section to the end of the ldap-login.conf file:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required 
    username="<username>" 
    password="<password>";
};

Cloudera Manager installation with LDAP authentication

If LDAP authentication is enabled in a Cloudera Manager installation, enable the LDAP Config File Substitutions (ldap.login.file.allow.substitutions) property for the StreamSets service in Cloudera Manager.

If the Use Safety Valve to Edit LDAP Information (use.ldap.login.file) property is enabled and LDAP authentication is configured in the Data Collector Advanced Configuration Snippet (Safety Valve) for ldap-login.conf field, then add the JAAS configuration properties to the same ldap-login.conf safety valve.

If LDAP authentication is configured through the LDAP properties rather than the ldap-login.conf safety value, add the JAAS configuration properties to the Data Collector Advanced Configuration Snippet (Safety Valve) for generated-ldap-login-append.conf field.

Add the following KafkaClient login section to the appropriate field as follows:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required 
    username="<username>" 
    password="<password>";
};