Google Cloud Storage

Supported pipeline types:

  • Data Collector
The Google Cloud Storage destination writes data to objects in Google Cloud Storage. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

The destination creates an object for each batch of data written to Google Cloud Storage.

With the Google Cloud Storage destination, you configure the bucket and common prefix to define where to write objects. You can use a partition prefix to specify the partition to write to. You can configure a prefix for the object name, and a time basis and data time zone for the stage. When using any data format except whole file, you can also configure a suffix for the object name and compress data with gzip before writing to Google Cloud Storage.

You also define the project ID and credentials to use when connecting to Google Cloud Storage.

You can also use a connectionconnection to configure the destination.

The destination can generate events for an event stream. For more information about the event framework, see Dataflow Triggers Overview.