Meet StreamSets Control Hub
Data Collector Communication
Transformer Communication
StreamSets Control Hub works with Transformer to design pipelines and to execute Transformer pipelines on Apache Spark, an open-source cluster-computing framework.
Provisioning Agent Communication
Control Hub Requirements
Control Hub User Interface
Jobs Overview
Number of Pipeline Instances
Data Collector Pipeline Failover
Transformer Pipeline Failover
You can enable a Transformer job for pipeline failover for some cluster types. Enable pipeline failover to prevent Spark applications from failing due to an unexpected Transformer shutdown.
Runtime Parameters
Job Templates
Job Tags
A job tag identifies similar jobs or job templates. Use job tags to easily search and filter jobs and job templates when viewing them in the Jobs view. For example, you might want to group jobs by the origin system or by the test or production environment.
Managing Jobs
Monitoring Jobs
Troubleshooting
Scheduler Overview
Scheduled Task Types
Time Zone
Each scheduled task has one time zone that applies to all the times configured for the task. By default, the time zone is UTC.
Cron Expression
Start and End Times
Missed Execution Handling
Creating Scheduled Tasks
Monitoring Scheduled Tasks
Managing Scheduled Tasks
Reports View
Report Definitions View
The Report Definitions view allows you to define a custom data delivery report that provides data processing metrics for a given job or topology. For example, you can use reports to view the number of records that were processed by a job or topology the previous day.
Creating Report Definitions
Generating Data Delivery Reports
To generate your data delivery report, do one of the following:
Viewing Data Delivery Reports
To view a data delivery report, click the name of the report in the Report Definitions view. The details of the report display. Then click Show Reports.
Troubleshooting
Subscriptions Overview
Enabling Events to Trigger Subscriptions
By default, events are disabled for organizations. An organization administrator can enable events to trigger subscriptions for the organization.
Events
A subscription is triggered by a Control Hub event. You can create a subscription for a single event type.
Actions
After an event triggers a subscription, the subscription performs an action - such as using a webhook to send an HTTP request to an external system or sending an email.
Parameters
Examples
Creating Subscriptions
Create a subscription to listen for Control Hub events and then complete an action when those events occur.
Troubleshooting
If a subscription doesn't complete the expected action, your organization administrator can view the subscription audit entries to help troubleshoot the issue.
Export and Import Overview
Pipeline, Fragment, and Topology Export and Import
Job Export and Import
Objects that Use Connections
When you export objects that use connections or objects with dependent objects that use connections, Control Hub exports the connection metadata only. The metadata includes the connection ID, name, and type. Control Hub does not export the connection credentials or other configured values.
Exporting Objects
Importing Objects
Organization Security Overview
Control Hub authenticates and authorizes user accounts to secure the integrity of your organization and data.
Authentication
Users and Groups
Roles
Control Hub provides several types of roles that allow you to customize the tasks that users can perform.
Permissions
Permissions determine the access level that users have on objects. When permission enforcement is enabled for your organization, you can share and grant permissions on objects.
Requirements for Common Tasks
Completing Control Hub tasks requires a combination of roles and permissions. The following sections list the requirements to complete common Control Hub tasks.
Organization Administration
As a user with the Organization Administrator role, you can use the Administration view to review and manage the organization configuration, to monitor active user sessions, and to view audit entries for your organization.
Rate Limiting
© 2023 StreamSets, Inc.