Environments Overview

An environment defines where to deploy StreamSets engines. An environment represents the resources needed to run the engines.

After you create and enable an environment, you create a deployment that defines a group of identical engine instances to launch into that environment. A single environment can host multiple deployments.

You can create the following types of environments:
Self-managed
When using a self-managed environment, you take full control of procuring the resources needed to run engines and deploying engine instances to those resources. A self-managed environment can represent local on-premises machines or cloud computing machines.
Control Hub provides a default self-managed environment. In most cases, you won't need to create additional self-managed environments. You can simply use the default environment for all of your self-managed deployments.
Amazon Web Services (AWS)
When using an AWS environment, Control Hub connects to your AWS account, provisions the AWS resources needed to run engines, and deploys engine instances to those resources. You must create an Amazon virtual private cloud (VPC) and configure AWS credentials for Control Hub to use. You then create an AWS environment in Control Hub to connect to the VPC using these credentials.
Google Cloud Platform (GCP)
When using a GCP environment, Control Hub connects to your Google Cloud project, provisions the Google Cloud resources needed to run engines, and deploys engine instances to those resources. You must designate a project for the resources, create a Google virtual private cloud (VPC) network, and configure Google Cloud credentials for Control Hub to use. You then create a GCP environment in Control Hub to connect to the project using these credentials.

You can create multiple environments of the same type or of different types. For example, you might want to dedicate one AWS VPC to the designing and running of test pipelines, and another VPC to the running of production pipelines. You would create two AWS environments in Control Hub, one for the test VPC and another for the production VPC. Then you would create one deployment to provision resources and launch engine instances in the test AWS environment and another deployment to do the same in the production AWS environment.