Control Hub Cloud User Guide

Home
Jobs

Getting Started
Release Notes
Tutorials
Pipeline Repository
Pipeline Design (Data Collector/Edge)
Pipeline Design (Transformer)
Connections
Jobs
- Jobs Overview
- Number of Pipeline Instances
  For Data Collector and Data Collector Edge jobs, you can manually scale out pipeline processing by increasing the number of pipeline instances that Control Hub runs for a job.
- Data Collector Pipeline Failover
  You can enable a Data Collector or Data Collector Edge job for pipeline failover. Enable pipeline failover to minimize downtime due to unexpected pipeline failures and to help you achieve high availability.
- Transformer Pipeline Failover
  You can enable a Transformer job for pipeline failover for some cluster types. Enable pipeline failover to prevent Spark applications from failing due to an unexpected Transformer shutdown.
- Runtime Parameters
  Runtime parameters are parameters that you define for a pipeline and call from within that same pipeline. When you define a runtime parameter, you enter the default value to use. When you create or edit a job that includes a pipeline with runtime parameters, you can specify another value to override the default. When the job starts, the value replaces the default value of the runtime parameter.
- Job Templates
  When you create a job for a pipeline that uses runtime parameters, you can enable the job to work as a job template. A job template lets you run multiple job instances with different runtime parameter values from a single job definition.
- Job Tags
  A job tag identifies similar jobs or job templates. Use job tags to easily search and filter jobs and job templates when viewing them in the Jobs view. For example, you might want to group jobs by the origin system or by the test or production environment.
- Managing Jobs
- Monitoring Jobs
  After you start a job, you can view statistics, error information, logs, and alerts about all remote pipeline instances run from the job. During pipeline test runs or job runs for Data Collector, you can also capture and review a snapshot of the data being processed.
- Troubleshooting
Scheduler
Topologies
Reports
Subscriptions
Data Collectors
Transformers
Edge Data Collectors
Provisioned Data Collectors
Labels
Export and Import
Organization Security
Glossary

Jobs

Jobs Overview
Number of Pipeline Instances
For Data Collector and Data Collector Edge jobs, you can manually scale out pipeline processing by increasing the number of pipeline instances that Control Hub runs for a job.
Data Collector Pipeline Failover
You can enable a Data Collector or Data Collector Edge job for pipeline failover. Enable pipeline failover to minimize downtime due to unexpected pipeline failures and to help you achieve high availability.
Transformer Pipeline Failover
You can enable a Transformer job for pipeline failover for some cluster types. Enable pipeline failover to prevent Spark applications from failing due to an unexpected Transformer shutdown.
Runtime Parameters
Runtime parameters are parameters that you define for a pipeline and call from within that same pipeline. When you define a runtime parameter, you enter the default value to use. When you create or edit a job that includes a pipeline with runtime parameters, you can specify another value to override the default. When the job starts, the value replaces the default value of the runtime parameter.
Job Templates
When you create a job for a pipeline that uses runtime parameters, you can enable the job to work as a job template. A job template lets you run multiple job instances with different runtime parameter values from a single job definition.
Job Tags
A job tag identifies similar jobs or job templates. Use job tags to easily search and filter jobs and job templates when viewing them in the Jobs view. For example, you might want to group jobs by the origin system or by the test or production environment.
Managing Jobs
Monitoring Jobs
After you start a job, you can view statistics, error information, logs, and alerts about all remote pipeline instances run from the job. During pipeline test runs or job runs for Data Collector, you can also capture and review a snapshot of the data being processed.
Troubleshooting

© 2023 StreamSets, Inc.