SQL Server 2019 BDC Multitable Consumer (deprecated)

The SQL Server 2019 BDC Multitable Consumer origin reads data from Microsoft SQL Server 2019 Big Data Cluster (BDC) through a JDBC connection. For information about supported versions, see Supported Systems and Versions.

Use the origin to read multiple tables from one or more schemas in the same database. For example, you might use the origin to replicate a database.

When you configure the origin, you specify connection information and optional custom JDBC configuration properties to determine how the origin connects to SQL Server 2019 BDC. You specify a database and then define groups of tables to read from that database. The origin generates SQL queries based on the table configurations that you define, and then returns data as a map with column names and field values.

When you define the table configurations, you can optionally override the default key column and specify the initial offset to use. By default, the origin processes tables incrementally, using primary key columns or user-defined offset columns to track its progress. You can configure the origin to perform non-incremental processing to enable it to also process tables that do not have a key or offset column.

You can configure the origin to perform multithreaded partition processing, multithreaded table processing, or use the default - a mix of both. When configuring partitions, you can configure the offset size, number of active partitions, and offset conditions.

You define the strategy that the origin uses to create each batch of data and the number of batches to create from each result set. You can configure advanced properties, such as the initial order to read from tables, connection related properties, and transaction isolation. And you can specify what the origin does when encountering an unsupported data type.

When the pipeline stops, the SQL Server 2019 BDC Multitable Consumer origin notes where it stops reading. When the pipeline starts again, the origin continues processing from where it stopped by default. You can reset the origin to process all available data, using any initial offsets that you defined.

By default, the origin generates JDBC record header and field attributes that provide additional information about each record and field.

The origin can generate events for an event stream. For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.

Before you use the SQL Server 2019 BDC Multitable Consumer origin, you must install the SQL Server 2019 Big Data Cluster stage library as a custom stage library and complete other prerequisite tasks.