Environments Overview

An environment defines where to deploy StreamSets engines. An environment represents the resources needed to run the engines.

Note: Environments are required for Data Collector and Transformer pipelines. They are not applicable for Transformer for Snowflake pipelines.

After you create and activate an environment, you create a deployment that defines a group of identical engine instances to launch into that environment. A single environment can host multiple deployments.

You can create the following types of environments:
Self-managed
In a self-managed environment, you take full control of procuring the resources needed to run engines and deploying engine instances to those resources. A self-managed environment can represent local on-premises machines or cloud computing machines.
Control Hub provides a default self-managed environment. In most cases, you won't need to create additional self-managed environments. You can simply use the default environment for all of your self-managed deployments.
Control Hub-managed
In a Control Hub-managed environment, Control Hub connects to the external system represented by the environment, automatically provisions the resources needed to run engines, and deploys engine instances to those resources.
After an administrator completes the required prerequisites, you can create the following types of Control Hub-managed environments:

You can create multiple environments of the same type or of different types. For example, you might want to dedicate one AWS VPC to the designing and running of test pipelines, and another VPC to the running of production pipelines. You would create two AWS environments in Control Hub, one for the test VPC and another for the production VPC. Then you would create one deployment to provision resources and launch engine instances in the test AWS environment and another deployment to do the same in the production AWS environment.

When you create an environment, you select the feature version to use for that environment and for all deployments created for the environment.