Installing for Tarball Using the Command Line

You can use the stagelibs command to install additional stage libraries for a core or common tarball installation.

The stagelibs command requires that curl version 7.18.1 or later and sha1sum utilities are installed on the machine. Verify that these utilities are installed before running the command.

Use the following commands to install additional tarball libraries:

To view the list of available libraries:

Run the following command from the $SDC_DIST directory:

bin/streamsets stagelibs -list

This provides a list of all available stage libraries and whether they are already installed. For more information about the stages installed with each stage library, see AvailableCommon Stage Libraries.

To install one or more stage libraries:

Run the following command from the $SDC_DIST directory:

bin/streamsets stagelibs -install=<libraryID>,<libraryID>,...

Use the full name of the libraries that you want to install, separating them with commas. Do not include spaces in the command.

For example, to install the Amazon S3 origin and destination, as well as the Cassandra destination, use the following command:

bin/streamsets stagelibs -install\
=streamsets-datacollector-aws-lib,streamsets-datacollector-cassandra_2-lib

When successful, the command line indicates that the stage libraries have been installed as follows:

Downloading: https://archives.streamsets.com/datacollector/<version>/tarball\
/streamsets-datacollector-aws-lib-<version>-SNAPSHOT.tgz
######################################################################## 100.0%
Downloading: https://archives.streamsets.com/datacollector/<version>/tarball\
streamsets-datacollector-jdbc-lib-<version>-SNAPSHOT.tgz
######################################################################## 100.0%
Downloading: https://archives.streamsets.com/datacollector/<version>/tarball\
streamsets-datacollector-rabbitmq-lib-<version>-SNAPSHOT.tgz
######################################################################## 100.0%

Stage library streamsets-datacollector-aws-lib installed
Stage library streamsets-datacollector-jdbc-lib installed
Stage library streamsets-datacollector-rabbitmq-lib installed

To generate the command required to perform the current installation (optional):

You can use the stagelibs command to generate the command to install the libraries that are installed on the current Data Collector. This allows you to easily replicate the installation elsewhere.

For example, say you installed three libraries above, and then installed another two. You can generate the command required to install all five libraries on additional machines.

To generate an installation script based on the current Data Collector installation, run the following command from the $SDC_DIST directory:

bin/streamsets stagelibs -installScript

The command returns an install command, such as the following:

=================================================================================
streamsets stagelibs -install=streamsets-datacollector-apache-kafka_0_8_1-lib,\
streamsets-datacollector-aws-lib,streamsets-datacollector-basic-lib,\
streamsets-datacollector-cdh_kafka_1_3-lib,streamsets-datacollector-jdbc-lib,\
streamsets-datacollector-jython_2_7-lib,streamsets-datacollector-rabbitmq-lib
=================================================================================

To uninstall libraries when necessary:

To uninstall a library, run the following command from the $SDC_DIST directory:

bin/streamsets stagelibs -uninstall=<libraryID>,<libraryID>,...

Use the full name of the libraries that you want to uninstall, separating them with commas. Do not include spaces in the command.