Google BigQuery

Supported pipeline types:

  • Data Collector
The Google BigQuery destination streams data into existing datasets and tables in Google BigQuery. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.
Tip: To process change data capture (CDC) data or to handle data drift, use the Google BigQuery (Enterprise) destination.

When you configure the destination, you define the existing BigQuery dataset and table to stream data into. The destination streams each record into a row in a BigQuery table. You can optionally define an expression to specify the insert ID to insert or update. The insert ID is a unique ID for each row. If you do not specify an insert ID, the destination inserts each record into a new row.

The destination maps fields from records to BigQuery columns based on matching names and compatible data types. You can configure the destination to ignore invalid columns when the destination cannot map a field to an existing BigQuery column. You can also configure the table cache size for the destination.

You define the project ID and credentials to use when connecting to Google BigQuery.

You can also use a connectionconnection to configure the destination.

For more information about streaming data into Google BigQuery, see the Google BigQuery documentation.