MQTT

Available when using an authoring Data Collector version 5.0.0 or later.

To create an MQTT connection, the Basic stage library, streamsets-datacollector-basic-lib, must be installed on the selected authoring Data Collector.

For a description of the MQTT connection properties, see MQTT Connection Properties.

After you create an MQTT connection, you can use the connection in the following stages and locations:


Engine	Stages and Locations
Data Collector 5.0.0 or later	MQTT Subscriber origin MQTT Publisher destination Write to MQTT error record handling configured for a pipeline

MQTT Connection Properties

When creating an MQTT connection, configure the following properties on the MQTT tab:


MQTT Property	Description
Broker URL	MQTT Broker URL. Enter in the following format: `<tcp \| ssl>://<hostname>:<port>` Use ssl for secure connections to the broker. For example: `tcp://localhost:1883` For high availability MQTT clusters without a load balancer, specify a list of brokers from the cluster, separated by commas and without spaces. The stage connects to the first available broker, trying in the order listed. For example: `tcp://hostA:1883,tcp://hostB:1883,tcp://hostC:1883`
Client ID	MQTT Client ID. The ID must be unique across all clients connecting to the same broker. You can define an expression that evaluates to the client ID. For example, enter the following expression to use the unique pipeline ID as the client ID: `${pipeline:id()}` If a pipeline includes multiple MQTT stages and you want to use the unique pipeline ID as the client ID for both stages, prefix the client ID with a string like this: `sub-${pipeline:id()} and pub-${pipeline:id()}` Otherwise, all stages will use the same client ID. This can cause problems, such as messages disappearing.
Use Credentials	Enables entering credentials on the Credentials tab. Use when the MQTT broker requires a user name and password.

If you enabled credentials, specify a user name and password on the Credentials tab.

On the TLS tab, optionally configure the following SSL/TLS properties:


TLS Property	Description
Use TLS	Enables the use of TLS.
Use Remote Keystore	Enables loading the contents of the keystore from a remote credential store or from values entered in the stage properties. For more information, see Remote Keystore and Truststore.
Private Key	Private key used in the remote keystore. Enter a credential function that returns the key or enter the contents of the key. For more information about credential functions, see Credential Functions.
Certificate Chain	Each PEM certificate used in the remote keystore. Enter a credential function that returns the certificate or enter the contents of the certificate. Using simple or bulk edit mode, click the Add icon to add additional certificates. For more information about credential functions, see Credential Functions.
Keystore File	Path to the local keystore file. Enter an absolute path to the file or enter the following expression to define the file stored in the Data Collector resources directory: `${runtime:resourcesDirPath()}/keystore.jks` By default, no keystore is used.
Keystore Type	Type of keystore to use. Use one of the following types: Java Keystore File (JKS) PKCS #12 (p12 file) Default is Java Keystore File (JKS).
Keystore Password	Password to the keystore file. A password is optional, but recommended. Tip: To secure sensitive information, you can use credential stores or runtime resources.
Keystore Key Algorithm	Algorithm to manage the keystore. Default is SunX509.
Use Remote Truststore	Enables loading the contents of the truststore from a remote credential store or from values entered in the stage properties. For more information, see Remote Keystore and Truststore.
Trusted Certificates	Each PEM certificate used in the remote truststore. Enter a credential function that returns the certificate or enter the contents of the certificate. Using simple or bulk edit mode, click the Add icon to add additional certificates.
Truststore File	Path to the local truststore file. Enter an absolute path to the file or enter the following expression to define the file stored in the Data Collector resources directory: `${runtime:resourcesDirPath()}/truststore.jks` By default, no truststore is used.
Truststore Type	Type of truststore to use. Use one of the following types: Java Keystore File (JKS) PKCS #12 (p12 file) Default is Java Keystore File (JKS).
Truststore Password	Password to the truststore file. A password is optional, but recommended. Tip: To secure sensitive information, you can use credential stores or runtime resources.
Truststore Trust Algorithm	Algorithm to manage the truststore. Default is SunX509.
Use Default Protocols	Uses the default TLSv1.2 transport layer security (TLS) protocol. To use a different protocol, clear this option.
Transport Protocols	TLS protocols to use. To use a protocol other than the default TLSv1.2, click the Add icon and enter the protocol name. You can use simple or bulk edit mode to add protocols. Note: Older protocols are not as secure as TLSv1.2.
Cipher Suites	Uses a default cipher suite for the SSL/TLS handshake. To use a different cipher suite, clear this option.
Cipher Suites	Cipher suites to use. To use a cipher suite that is not a part of the default set, click the Add icon and enter the name of the cipher suite. You can use simple or bulk edit mode to add cipher suites. Enter the Java Secure Socket Extension (JSSE) name for the additional cipher suites that you want to use.