HTTP Server

Supported pipeline types:

  • Data Collector

  • Data Collector Edge
The HTTP Server origin listens on an HTTP endpoint and processes the contents of all authorized HTTP POST and PUT requests. Use the HTTP Server origin to read high volumes of HTTP POST and PUT requests using multiple threads. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

The HTTP Server origin can read requests containing messages with no compression or with the Gzip or Snappy compression format.

The HTTP Server origin can use multiple threads to enable parallel processing of data from multiple HTTP clients. The origin can require that requests specify an allowed application ID. Before you run a pipeline with the origin, perform additional steps to configure the HTTP clients.

When you configure the HTTP Server origin, you specify the maximum number of concurrent requests to determine how many threads to use. You define the listening port, allowed application IDs, and the maximum message size. You can also configure SSL/TLS properties, including default transport protocols and cipher suites.

Tip: Data Collector provides several HTTP origins to address different needs. For a quick comparison chart to help you choose the right one, see Comparing HTTP Origins. Also, if processing Flume events as HTTP requests, you will likely find the TCP Server origin more efficient.

The origin provides request header fields as record header attributes so you can use the information in the pipeline when needed.