Resource Thresholds

Control Hub monitors the following resources used by each registered Data Collector Edge:
  • CPU load
  • Memory used (Data Collector Edge 3.13.x and later)
  • Number of running pipelines

For each registered Data Collector Edge, you can define a threshold for each of these resources. When starting, balancing, or synchronizing jobs on a Data Collector Edge, Control Hub verifies that the Data Collector Edge has not exceed any of its resource thresholds.

When multiple matching Edge Data Collectors have not exceeded their resource thresholds, Control Hub prioritizes Edge Data Collectors that are currently running the fewest number of pipelines. For example, you start a job and two matching Edge Data Collectors have the maximum CPU load set to 100%. Data Collector Edge A is currently using 70% of the CPU and running 1 pipeline. Data Collector Edge B is currently using 50% of the CPU and running 3 pipelines. Control Hub starts a pipeline instance for the job on Data Collector Edge A.

When all matching Edge Data Collectors have exceeded their resource thresholds, Control Hub randomly places jobs in a queue, giving the jobs a red active status. The job details display the following warning message:
JOBRUNNER_72 - Insufficient Data Collector Edge resources to run job. All matching Edge Data Collectors [<URLs>] have reached their maximum CPU usage limits.

When a matching Data Collector Edge no longer exceeds its resource thresholds, Control Hub randomly assigns a job from the queue to that Data Collector Edge, changing the job status to green active and running a pipeline instance on that Data Collector Edge.

Defining Resource Thresholds

You can define the resource thresholds for any registered SDC Edge.

  1. In the navigation panel, click Execute > Edge Data Collectors.
  2. Expand the details for the SDC Edge that you want to configure.
  3. Configure the following threshold values:
    Threshold Description
    Max CPU Load (%) Maximum percentage of CPU that can be used. Default is 100.
    Max Memory (%) Maximum percentage of memory that can be used. Default is 100.

    Monitored for SDC Edge 3.13.x and later.
    Max Running Pipeline Count Maximum number of pipelines that can be running. Default is 1,000,000.
  4. Click Save.