Monitoring Jobs

When you monitor a job, Control Hub displays the pipeline in the canvas and displays real-time statistics for the job in the monitor panel below the canvas.

Control Hub provides both a new job monitoring UI and a classic job monitoring UI. To switch the UI while monitoring a job, click the More icon () and then click Switch to Classic UI or Switch to New UI.

When you view the list of jobs in the Job Instances view or when you monitor a job, you can view the job status. You can also view the status of remote pipeline instances run from active jobs.

The job status is color-coded, providing an easy visual indicator of which jobs need your attention. A red status indicates that an error has occurred that you must resolve. A green status indicates that all is well.

The following table describes each job status:

Job Status	Description
	Job is inactive. A job that has never run has an inactive status. A job transitions from an active to an inactive status when you stop the job or when all remote pipeline instances run from the job have reached a finished state.
	Job is inactive after stopping automatically due to an error. For example, a red inactive status can occur when either the pipeline or Data Collector generates an error that causes Control Hub to stop the job.
	Job is inactive and has an error that you must acknowledge. This status can occur when at least one execution engine reported an error while attempting to stop the remote pipeline instance. For example, one Data Collector might have shut down and so could not properly stop the remote pipeline instance. You cannot perform actions on jobs with an inactive_error status until you acknowledge the error message. To acknowledge the error, view the job details or monitor the job and acknowledge reading the error message. For more information, see Acknowledging Job Errors.
	Control Hub is in the process of starting the job. You cannot perform actions on activating jobs.
	Job is active and remote pipeline instances are running on the execution engines assigned the same labels as the job.
	Job is active, but there are some issues you must look into. For example, a red active status can indicate one of the following issues: One of the assigned execution engines is not currently running. One of the assigned execution engines encountered an error while running the pipeline. All assigned execution engines have exceeded their resource thresholds.
	Control Hub is in the process of stopping a job as requested or as expected. Control Hub is communicating with the execution engines to stop all remote pipeline instances. You cannot perform actions on deactivating jobs.
	Control Hub is in the process of stopping a job automatically due to an error. Control Hub is communicating with the execution engines to stop all remote pipeline instances. You cannot perform actions on deactivating jobs.

When you start a Data Collector job, Control Hub sends an instance of the pipeline to a Data Collector engine. The engine remotely runs the pipeline instance, communicating with Control Hub at regular one minute intervals to report a heartbeat, pipeline status, and last-saved offset.

If a Data Collector engine fails to communicate with Control Hub before the maximum engine heartbeat interval expires, 5 minutes by default, then Control Hub considers the engine unresponsive.

Engines can become unresponsive for the following reasons:

Control Hub handles currently active jobs on unresponsive Data Collector engines differently, depending on the reason for the unresponsive engine and whether pipeline failover is enabled for the job.

By default when a job has an inactive error status, you cannot perform actions on the job until you acknowledge the error message. You can acknowledge job errors from the Job Instances view or when monitoring a job. You can also acknowledge job errors from a topology.

Acknowledging errors from the Job Instances view

To acknowledge job errors from the Job Instances view, click the row listing the job with the inactive error to display the job details. The details list the error message for the job. Review the message, taking action as needed, and then click Acknowledge Errors.

Acknowledging errors when monitoring a job

To acknowledge job errors when monitoring a job with an inactive error, click the Job Status tab in the monitoring panel. The Job Status tab lists the error message for the job. Review the message, taking action as needed, and then click the Acknowledge Errors icon () in the top toolbar, as displayed in the following image:

When a job is inactive, you can reset the metrics for the job by resetting the origin for the job. You might want to reset metrics when you are testing jobs and want to view the metrics from the current job run only.

For more information about resetting the origin, see Resetting the Origin for Jobs.

When you monitor an active Data Collector job, you can view the errors related to each pipeline stage. Stage-related errors include the error records that the stage produces and other errors encountered by the stage.

To view stage-related errors, select the stage in the canvas and then click the Errors tab in the monitor panel. The Errors tab displays the following tabs:

Error Records

Displays a sample of the 10 most recent error records with related error messages. You can expand and review the data in each error record. If the error was produced by an exception, you can click View Stack Trace to view the full stack trace.

Stage Errors

Displays a list of stage errors. Stage errors are operational errors, such as an origin being unable to create a record because of invalid source data.

The following image displays a sample Errors tab for a Field Type Converter processor that has encountered errors:

When monitoring an active Data Collector or Transformer job, you can view logs for the execution engine running the remote pipeline instance.

When you monitor a Transformer job for a cluster pipeline, the Monitoring panel provides URLs for the cluster or the Spark application that runs the pipeline.

Use the URL to access additional information about the cluster or Spark application. For example, the Spark web UI can include information such as completed jobs, memory usage, running executors, and the Spark driver log.

Cluster and Spark URLs display in the Runtime Statistics section of the Monitoring panel. For example, when you monitor a Databricks pipeline, the Databricks Job URL displays with the other runtime statistics, as follows:

A snapshot is a set of data captured as it moves through a running pipeline. You can capture and review snapshots during draft runs or job runs for Data Collector pipelines.

View a snapshot to verify how a Data Collector pipeline processes data. Like data preview, you can view how snapshot data moves through a pipeline stage by stage or across multiple stages. You can drill down to review the values of each record to determine if the stage or group of stages transforms data as expected.

Unlike data preview, you cannot edit data to perform testing when you review a snapshot. Instead, you can use the snapshot as source data for data preview. You might use a snapshot for data preview to test the pipeline with production data.

Snapshots captured for jobs are available for the duration of the job run. When you stop the job, all captured snapshots are deleted.

Snapshots captured for the previous draft run are still available after the draft run stops when using an authoring Data Collector 5.0.0 or later. For earlier Data Collector versions, all captured snapshots are deleted when you stop the draft run.

You can view the run history of a job and a summary of each run when you configure or monitor a job.

Control Hub displays timestamps using the browser time zone, which is determined by your local operating system.

Click the History tab in the job properties or monitor panel to view the run history. The following image shows a sample run history: