Google Big Query

The Google Big Query destination writes data to a Google BigQuery table. Use the destination in Dataproc cluster pipelines only.

The Google Big Query destination stages data in a Google Cloud Storage bucket before writing it to BigQuery. The destination can write data to a new or existing BigQuery table. You can configure the destination to create a table if the specified table does not exist. You can configure the destination to truncate the table before writing each batch.

The Google Big Query destination writes to a BigQuery table based on the specified write mode. The destination can insert data to the table or merge data with existing data in the table. When inserting data, the destination can add new nullable columns to the table schema and can relax required fields to allow null values. When merging data, the destination can insert, update, and delete records based on one or more key columns and the specified merge conditions.

When you configure the Google Big Query destination, you specify the dataset, table, and temporary storage bucket. You specify whether to create a new table or to truncate an existing table. You also select the write mode to use. When merging data, you specify the join key, merge conditions, and operations.

Before you configure the Google Big Query destination, complete the prerequisite task.