Google BigQuery

Supported pipeline types:

  • Data Collector
The Google BigQuery origin executes a query job and reads the result from Google BigQuery. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

The origin submits the query that you define, and then Google BigQuery runs the query as an interactive query. When the query is complete, the origin reads the query results to generate records. The origin runs the query once and then the pipeline stops when it finishes reading all query results. If you start the pipeline again, the origin submits the query again.

When you configure the origin, you define the query to run using valid BigQuery standard SQL or legacy SQL syntax. By default, BigQuery writes all query results to a temporary, cached results table. You can choose to disable retrieving cached results and force BigQuery to compute the query result.

You also define the project ID and credentials to use when connecting to Google BigQuery.

You can also use a connectionconnection to configure the origin.

The origin can generate events for an event stream. For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.