HTTP Client

Supported pipeline types:

  • Data Collector

  • Data Collector Edge
The HTTP Client origin reads data from an HTTP resource URL. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

When you configure HTTP Client, you specify the resource URL, optional headers, and the method to use. For some methods, you can specify the request body and default content type.

You can configure the actions to take based on the response status and configure pagination properties to enable processing large volumes of data from paginated APIs. You can also enable the origin to read compressed and archived files.

The origin provides response header fields as record header attributes so you can use the information in the pipeline when needed.

The origin also provides several different authentication types to access data. You can enter credentials in the origin or you can secure the credentials in runtime resource files and reference the files in the origin. You can also configure the origin to use the OAuth 2 protocol to connect to an HTTP service.

You can optionally use an HTTP proxy and configure SSL/TLS properties. You can also configure the origin to log request and response information.
Tip: Data Collector provides several HTTP origins to address different needs. For a quick comparison chart to help you choose the right one, see Comparing HTTP Origins.

When the pipeline stops, HTTP Client notes where it stops reading. When the pipeline starts again, HTTP Client continues processing from where it stopped by default. You can reset the origin to process all requested files.