JDBC

Available when using an authoring Data Collector version 4.0.0 or later.

To create a JDBC connection, the JDBC stage library, streamsets-datacollector-jdbc-lib, must be installed on the selected authoring Data Collector.

For a description of the JDBC connection properties, see JDBC Connection Properties.

After you create a JDBC connection, you can use the connection in the following stages:
Engine Stages

Data Collector 4.0.0 or later
  • JDBC Multitable Consumer origin
  • JDBC Query Consumer origin
  • JDBC Lookup processor
  • JDBC Tee processor
  • JDBC Producer destination
  • JDBC Query executor

Transformer 4.0.0 or later
  • JDBC origin
  • JDBC Lookup processor
  • JDBC destination

JDBC Connection Properties

When creating a JDBC connection, configure the following properties on the JDBC tab:
JDBC Property Description
JDBC Connection String Connection string used to connect to the database. Use the connection string format required by the database vendor.

For example, use the following formats for these database vendors:

  • MySQL - jdbc:mysql://<host>:<port>/<database_name>
  • Oracle - jdbc:oracle:<driver_type>:@<host>:<port>:<service_name>
  • PostgreSQL - jdbc:postgresql://<host>:<port>/<database_name>
  • SQL Server - jdbc:sqlserver://<host>:<port>;databaseName=<database_name>
  • SQL Server 2019 Big Data Cluster accessed from a Transformer pipeline - jdbc:sqlserver://<ip>:<port>;databaseName=<database_name>

    Where <ip> is the IP address of the SQL Server master instance, and <port> is the port number of the SQL Server master instance. Use the SQL Server user name and password with this connection string. For more information, see SQL Server 2019 JDBC Connection Information in the Transformer documentation.

You can optionally include the user name and password in the connection string.
Use Credentials Enables entering credentials on the Credentials tab. Use when you do not include credentials in the JDBC connection string.
If you configured the connection to enter JDBC credentials separately from the JDBC connection string, then configure the following properties on the Credentials tab:
Credentials Property Description
Username User name for the JDBC connection.

The user account must have the correct permissions or privileges in the database. For example, the user account specified for an origin must be able to read data from the database. The user account specified for a destination must be able to write to the database.

Password Password for the JDBC user name.
Tip: To secure sensitive information, you can use credential stores or runtime resources.