Teradata Consumer (deprecated)

Data Collector

The Teradata Consumer origin reads data from multiple Teradata Database tables in multiple schemas through a JDBC connection. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

Important: This stage is deprecated and may be removed in a future release.

The origin can process data from tables with simple numeric primary keys. The origin cannot process data from tables with compound or non-numeric primary keys.

With the Teradata Consumer origin, you can enable Teradata Fast Export to quickly retrieve large amounts of data. For example, you might use the origin to perform database replication.

Tip: Though you can also use the JDBC Multitable Consumer origin or the JDBC Query Consumer origin to read from Teradata Database tables, use the Teradata Consumer origin to enable Teradata Fast Export and retrieve large amounts of data quickly.

When you configure the origin, you specify connection information and custom JDBC configuration properties to determine how the origin connects to the database.

You define groups of database tables to read. The origin generates SQL queries based on the table configurations that you define, and then returns data as a map with column names and field values.

When you define the table configurations, you can optionally override the default key column and specify the initial offset to use. By default, the origin processes tables incrementally, using primary key columns or user-defined offset columns to track its progress. You can configure the origin to perform non-incremental processing to enable it to also process tables that do not have a key or offset column.

You can configure the origin to perform multithreaded partition processing, multithreaded table processing, or use the default - a mix of both. When configuring partitions, you can configure the offset size, number of active partitions, and offset conditions.

You define the strategy that the origin uses to create each batch of data and the number of batches to create from each result set. You can configure advanced properties, such as the initial order to read from tables, connection related properties, and transaction isolation. And you can specify what the origin does when encountering an unsupported data type: convert the data to string or stop the pipeline.

When the pipeline stops, the Teradata Consumer origin notes where it stops reading. When the pipeline starts again, the origin continues processing from where it stopped by default. You can reset the origin to process all available data, using any initial offsets that you defined.

By default, the origin generates JDBC record header and field attributes that provide additional information about each record and field.

You can configure advanced connection properties. To use a JDBC version older than 4.0, you specify the driver class name and define a health check query.

The origin can generate events for an event stream. For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.

Before you use the Teradata Consumer origin, you must complete a prerequisite task. The origin is available in the Teradata Enterprise stage library.the prerequisite tasks, including installing the Teradata stage library. The Teradata stage library is an Enterprise stage libraryEnterprise stage library. Releases of Enterprise stage libraries occur separately from Data Collector releases. For more information, see Enterprise Stage Libraries in the Data Collector documentation.