Provisioning Agent Communication

A Provisioning Agent is a containerized application that runs in a container orchestration framework, such as Kubernetes. The agent automatically provisions Data Collector containers in the Kubernetes cluster on which it runs.

Provisioning includes deploying, starting, stopping, and scaling the Data Collector containers to work with StreamSets Control Hub. Use provisioning to reduce the overhead of managing individual Data Collector installations.

You can provision both authoring and execution Data Collectors as long as you provision them in unique deployments. When you provision an authoring Data Collector, you must associate the deployment with a Kubernetes service to expose the Data Collector container outside the cluster.

After you create a Provisioning Agent and deploy the application to a container orchestration framework, the Provisioning Agent uses encrypted REST APIs to communicate with Control Hub. Provisioning Agents initiate outbound connections to Control Hub on the port number configured in the Control Hub system. The connection must use the same protocol, HTTP or HTTPS, as the Control Hub system.

After the Provisioning Agent deploys authoring or execution Data Collector containers, the Data Collector containers communicate with Control Hub the same way that any registered Data Collector communicates with Control Hub.

The following image shows how a Provisioning Agent communicates with Control Hub to provision authoring and execution Data Collectors: