Cron Scheduler

Supported pipeline types:
  • Data Collector

The Cron Scheduler origin generates records periodically based on a schedule. Use the origin to schedule tasks by triggering downstream stages in a pipeline.

The Cron Scheduler origin is an orchestration stage that you use in orchestration pipelines. Orchestration stages perform tasks, such as schedule and start pipelines and Control Hub jobs, that you can use to create an orchestrated workflow across the StreamSets platform. For example, an orchestration pipeline can use the Cron Scheduler origin to generate a record every Monday at 6 AM to trigger the Start Pipelines processor, which starts a pipeline that loads data from the previous week and generates a report.

When you configure the Cron Scheduler origin, you define the schedule for generating records. You specify the schedule with a cron expression and a time zone for that expression. At the scheduled time, the origin generates a record and passes it to the next stage in the orchestration pipeline.