Always Migrate Job Offsets
By default when you stop and then restart a job that is disabled for failover, Control Hub sends the last-saved offset to the same Data Collector that originally ran the pipeline. You can configure Control Hub to always send job offsets to different Data Collectors with matching labels when you restart a job.
The always.migrate.offsets property in the
$DPM_CONF/jobrunner-app.properties file determines whether Control Hub
always migrates job offsets to different Data Collectors
when you stop and restart a job.
If set to true, Control Hub always sends the offset and pipeline instance to a different Data Collector when you restart the job. Set to true when all of your jobs include a pipeline origin that is not tied to a particular Data Collector machine. For example, if a pipeline reads from an external system such as a relational database or Elasticsearch, any Data Collector within the same network and with an identical configuration can continue processing from the last-saved offset recorded by another Data Collector.
- Failover is disabled - Control Hub sends the offset to the same Data Collector that originally ran the pipeline instance. In other words, Control Hub associates each pipeline instance with the same Data Collector.
- Failover is enabled - Control Hub sends the offset to another available Data Collector assigned all labels specified for the job.
Set to false when most of your jobs include a pipeline origin that is tied to a particular Data Collector machine. For example, if your jobs include a Directory or File Tail origin that reads from a local directory on the Data Collector machine.
Default is false.