REST Service

By default, the REST Service origin listens at an HTTP endpoint for requests. You can configure the origin to use Data Collector as an API gateway, which enables using the Data Collector URL as an API endpoint.

Use the REST Service origin to create a REST-based microservice. In a microservice pipeline, the REST Service origin works with one or more microservice destinations that specify the HTTP response code to pass back to the originating REST API client.

The REST Service origin generates a batch for each request that it receives. The REST Service origin can use multiple threads to enable parallel processing of requests from multiple clients. The origin can also read requests containing messages with no compression or with the Gzip or Snappy compression format.

When using the REST Service origin, you must configure the REST API clients to send data to the pipeline. The required client configuration depends on how you configure the origin.

When you configure the REST Service origin, you decide if you want to use a listening port or an API gateway to receive requests. To use a listening port, you specify the HTTP listening port. To use an API gateway, you specify the service name and whether to require gateway authentication.

When not using gateway authentication, you can enable Kerberos authentication using SPNEGO, or SSL/TLS with basic authentication.

For additional security, you can require each request to include a valid application ID by listing valid application IDs in the origin. And you can indicate if the application ID will be in the URL instead of the request header.

You can configure the maximum number of concurrent requests to determine how many threads to use. You also configure the format of generated responses.

When you start the pipeline, Data Collector displays the endpoint that the pipeline uses in monitor mode.