Storing Custom Libraries for Cloudera Manager
To store custom stage libraries for a Cloudera Manager installation, perform the following steps:
-
On every node that runs Data Collector, create a local directory for the custom stage libraries.
For example, you might create the custom stage library directory at:
/opt/sdc-user-libs
-
On every node that runs Data Collector, copy the custom stage libraries to the directory defined for the
USER_LIBRARIES_DIR environment variable. Use a unique subfolder for each custom
stage.
For example to store libraries for a custom stage named customstage1, you would copy them to the following directory:
/opt/sdc-user-libs/customstage1
- In Cloudera Manager, select the StreamSets service and then click Configuration.
-
On the Configuration page, in the Data
Collector Advanced Configuration Snippet (Safety Valve) for
sdc-env.sh field, add the USER_LIBRARIES_DIR environment
variable and point it to the custom stage library directory, as follows:
export USER_LIBRARIES_DIR="<custom stage library directory>"
For example:export USER_LIBRARIES_DIR="/opt/sdc-user-libs/"
-
When using the Java Security Manager, which is enabled by default, update the
Data Collector Advanced Configuration Snippet (Safety Valve) for
sdc-security.policy property to include the custom stage library
directory as follows:
// custom stage library directory grant codebase "file://<custom stage library directory>/-" { permission java.security.AllPermission; };
For example:// custom stage library directory grant codebase "file:///opt/sdc-user-libs/-" { permission java.security.AllPermission; };
- Restart Data Collector.