Managing Engines
- Assign labels to engine instances.
- Restart engine instances.
- Balance jobs on Data Collector engine instances when the jobs are enabled for pipeline failover.
- Shut down engine instances. If you shut down an engine instance, you can later use the command line to run a command to start the instance.
- Delete engine instances.
You cannot shut down or delete engine instances belonging to Control Hub-managed deployments, such as GCE and Kubernetes deployments. Instead, engine instances are automatically shut down or deleted as needed when you edit the deployment. For example, if you edit a GCE deployment to decrease the number of engine instances from 3 to 2, Google Deployment Manager shuts down one of the engine instances and then deletes the engine and the VM instance where the engine runs.
Restarting Engines
Restart an engine instance to apply updates made to the engine configuration in the deployment.
For example, if you edit a deployment to add additional stage libraries, you must restart all engine instances managed by the deployment. During the restart, the stage libraries are installed on the engine instances.
When you edit an active deployment that updates the engine configuration, Control Hub prompts you to restart all engine instances. If you edit a deactivated deployment or if you choose not to restart the engine instances while editing an active deployment, you can restart the engine instances from the Engines view.
During the restart process, the engine instance shuts down and then automatically restarts.
Balancing Jobs on Engines
From the Engines view, you can balance all jobs enabled for failover and running on specific Data Collector engine instances.
When balancing the jobs, Control Hub redistributes the pipeline load evenly across all available Data Collectors that have the necessary labels and that have not exceeded any resource thresholds.
For more information about balancing jobs on Data Collector engines, see Balancing Jobs Enabled for Failover.
Shutting Down Engines for Self-Managed Deployments
You can shut down engine instances belonging to a self-managed deployment, and then manually start the engine instances at a later time.
You cannot shut down engine instances belonging to Control Hub-managed deployments.
- In the Control Hub Navigation panel, click .
- Click an engine type tab.
- Select one or more engine instances that you want to shut down.
-
In the toolbar above the engines list, click the More
icon (
) and then click Shut Down Engines.
- Click OK to confirm, and then click Close.
Starting Engines for Self-Managed Deployments
If you shut down an engine instance belonging to a self-managed deployment, you cannot use the Control Hub UI to manage the instance. You first must manually start the engine instance from the command line.
Starting from a Tarball Installation
You can manually start an engine instance that has been installed from a tarball.
Starting When Using a Proxy Server
To manually start an engine instance version 5.0.0 and later that has been installed from a tarball and has been configured to use a proxy server, you must define the proxy properties as environment variables when you start the engine. You can copy the required environment variables from the installation script generated for the self-managed deployment.
For more information about using a proxy server, see the Data Collector documentation or the Transformer documentation.
Starting from Docker
You can manually start an engine instance that has been installed from a Docker image.
Deleting Engines from a Self-Managed Deployment
You can delete an engine instance from a self-managed deployment.
You cannot delete engine instances from Control Hub-managed deployments. Instead, engine instances are automatically deleted as needed when you edit the deployment.