Databricks Delta Lake
Available when using an authoring Data Collector version 3.19.0 or later.
- Data Collector 5.6.0 or later - Requires the Databricks stage library, streamsets-datacollector-sdc-databricks-lib.
- Data Collector
3.19.0 to 5.5.x - Requires
version 1.2.0 or later of the Databricks Enterprise stage library,
streamsets-datacollector-databricks-lib
.
For a description of the Databricks Delta Lake connection properties, see Databricks Delta Lake Connection Properties.
Engine | Stage |
---|---|
Data Collector 3.19.0 or later | Databricks Delta Lake destination |
Databricks Delta Lake Connection Properties
Databricks Delta Lake Property | Description |
---|---|
JDBC URL | JDBC URL used to connect to the Databricks
cluster. For connections that use Data Collector 5.6.0
and later, enter in the following format:
For connections that use earlier
versions of Data Collector, enter
in the following format:
Tip: In Databricks, you
can locate the JDBC URL for your cluster on the JDBC/ODBC tab in the
cluster configuration details. As a best practice, remove the
PWD parameter from the URL, and then enter the
personal access token value in the Token property below.
|
Token | Personal access token used to connect to the
Databricks cluster.
Tip: To secure sensitive
information, you can use credential stores or runtime
resources. |