Monitoring Deployments

When you view deployment details in the Deployments view, you can monitor the following information about the deployment:

To view deployment details, click a deployment name in the Deployments view.

Deployment States

You can view the deployment state from the list of deployments in the Deployments view or from the deployment details.

The following table describes the possible states for a deployment:

Deployment State Description
Setup Incomplete Deployment is not completely configured.

A deployment encounters this state when you click Save & Exit or when you navigate away from the creation wizard before finishing the deployment configuration.

Activating Control Hub is in the process of activating the deployment.

For Control Hub-managed deployments, Control Hub is provisioning resources in the external system.

You cannot perform actions on activating deployments.

Active Deployment is active and ready to launch engine instances.
Note: An active Control Hub-managed deployment does not necessarily mean that StreamSets engines are actively running in the external system. After a deployment becomes active, it can take a few minutes for StreamSets engines to fully launch on provisioned resources.
Activation Error Control Hub is not able to start the deployment, or the already active deployment is critically impaired.

Self-managed deployments rarely encounter this state. Control Hub-managed deployments can encounter this state when prerequisites are incorrectly completed or when the deployment is incorrectly configured. For example, an Amazon EC2 deployment can encounter this state if the IAM policy used for AWS credentials does not grant permission to associate an instance profile with the provisioned EC2 instances.

Control Hub periodically attempts to transition deployments in an Activation Error state to an Active state.

Deactivating Control Hub is in the process of stopping the deployment.

For Control Hub-managed deployments, Control Hub is deleting the provisioned resources.

You cannot perform actions on deactivating deployments.

Deactivated Deployment is stopped and not ready to launch engine instances.

For Control Hub-managed deployments, all provisioned resources have been deleted.

Deactivation Error Control Hub is not able to stop the deployment.

Self-managed deployments rarely encounter this state. Control Hub-managed deployments can encounter this state when prerequisites are incorrectly completed or when the deployment is incorrectly configured. For example, an Amazon EC2 deployment can encounter this state when the IAM policy used for AWS credentials does not grant permission to delete AWS CloudFormation stacks.

Control Hub periodically attempts to transition deployments in a Deactivation Error state to a Deactivated state.

Event Logs

When you view the details of a deployment, you can view deployment event logs. The logs list all events related to the deployment, including errors that might occur when provisioning resources for one of the cloud service provider deployments or when running the engine installation script for a self-managed deployment.

For example, when you start an Amazon EC2 deployment, the event logs include details about creating the AWS CloudFormation stack, downloading the StreamSets engine installation files, and starting StreamSets engines on the provisioned instances.

Event logs display messages only about the current start of the deployment. Each time you restart the deployment, the previous event logs are removed.

To view deployment event logs, click a deployment name in the Deployments view and then locate the Deployment Event Logs section in the deployment details.

The following image shows sample event logs for an EC2 deployment:

Viewing Engines in a Deployment

When you view the details of a deployment, you can view the list of engine instances that belong to the deployment.

To view the list of engine instances, click a deployment name in the Deployments view and then locate the Engines in Deployment section in the deployment details.

The section lists each engine instance launched for the deployment with the following information:
  • Hostname
  • Number of pipelines running on the engine
  • Percentage of CPU and amount of memory being used by the engine
  • Last time that the engine reported its status to Control Hub

To view details about a specific engine instance, click the engine hostname. When you view engine details, you can monitor the performance of the engine, view the list of pipelines currently running on the engine, and access engine logs.

The following image shows two engine instances launched for an EC2 deployment: