My Organization

Overview

An organization is a secure space provided to a set of Control Hub users.

All engines, pipelines, jobs, and other objects added by any user in the organization belong to that organization. You log in to Control Hub as a member of an organization and can only access data that belongs to that organization. The specific organization data that you can access depends on the roles and permissions assigned to your account.

When you are a member of multiple organizations, you select the organization to use when you log in. After logging in, you can switch organizations. You can also choose to leave an organization.

A user with the Organization Administrator role can review and configure organization properties. An organization administrator can also enable the organization to use SAML authentication.

Control Hub sets default system limits on the number of objects that can exist for the organization.

Multiple Organizations

A single user account can belong to multiple organizations.

For example, let's say that you sign up as a new user of the StreamSets platform. StreamSets creates a Control Hub organization using the company or organization name that you enter, and creates a user account using your email address. A colleague also signs up as a new user, and invites you to join her organization using your same email address. When you join your colleague's organization, StreamSets adds you as a member of that organization using your existing user account.

When you are a member of multiple organizations, you use a single set of credentials to log in to StreamSets. During the login, you select the organization to use for that session. For that session, you can only access data belonging to the selected organization.

To access data belonging to a different organization, you must switch organizations.

Switching Organizations

When you are a member of multiple organizations, you select the organization to use when you log in. After logging in, you can switch organizations.

  1. In the top right toolbar, click the My Account icon: .
  2. Click Switch Organization.
  3. Log in to your account again, selecting the other organization that you want to use.

Leaving an Organization

If you are not the organization administrator who created the organization, you can choose to leave the organization. Leaving the organization deletes your user account in that organization.

  1. In the top right toolbar, click the My Account icon (), and then click your user name.
  2. On the Account Settings tab, click Leave Organization.
  3. Click OK to confirm that you want to leave the organization.

Configuring Organization Properties

A user with the Organization Administrator role can review and configure organization properties.

  1. In the Navigation panel, click Manage > My Organization.
  2. Configure the following general organization properties:
    General Property Description
    Organization Name Name of the organization.
    Valid Domains List of trusted domains that can make authentication requests to Control Hub on behalf of your organization.
  3. Click Save to save changes made to general properties.
  4. Click Advanced to configure advanced organization properties.

    Some of the advanced properties can be modified only up to the maximum system limit. For example, the system limit for the maximum number of job runs is 100. You can configure a lower limit for the maximum number of job runs, such as 50, but cannot configure a limit higher than 100.

  5. Configure the following advanced properties:
    Advanced Property Description
    Enable events to trigger subscriptions Enables events so that Control Hub can trigger subscriptions for organizations. Disable events if you do not want users to use subscriptions.
    Enable WebSocket Tunneling for UI Communication Enables execution engines to use the WebSocket Secure (wss) protocol to establish a WebSocket tunnel with Control Hub over an encrypted SSL/TLS connection.
    Important: Before you disable this property, you must configure all engines to use the HTTPS protocol to directly communicate with the web browser.
    Enable Snowpark Preview Enables previewing data in Transformer for Snowflake pipelines, which allows Snowflake data to leave Snowflake servers for the preview.

    Disable this option to prevent all users in the organization from using data preview in Transformer for Snowflake pipelines.

    For more information, see the Transformer for Snowflake documentation.
    Enforce permissions during object access Enables permission enforcement to secure the integrity of organization data. Disable permission enforcement if you want all users in the organization to have full access to all objects.
    Default authoring engine timeout Number of milliseconds that Control Hub waits for a response from an authoring engine before considering the engine as not accessible.

    When the engine is not accessible, you cannot select that engine as the authoring engine when designing pipelines or creating connections.

    In most cases, the default value should be appropriate. Try increasing the value when the authoring engines are running, but the authoring engine selection pages indicate that engines are not accessible.

    Users can override this default value when configuring their browser settings within the My Account window.

    Default is 5000 milliseconds.
    Execution engine heartbeat interval Maximum number of seconds since the last reported Data Collector or Transformer heartbeat before Control Hub considers the engine unresponsive. In most cases, the default value of 300 seconds, or five minutes, is sufficient.

    System limit is 1,800.
    Maximum number of days before job status history is purged Maximum number of days to retain the run history for each job and draft run.

    System limit is 7.
    Maximum number of job runs Maximum number of job and draft runs to retain the run history for, including runs of job instances started from a job template.

    System limit is 100.
    Enable Multiple Job Start Synchronization Enables Control Hub to synchronize the start of multiple jobs. Use to ensure that the number of pipelines running on an engine does not exceed the configured resource threshold. Enabling the property can cause jobs to take longer to start.

    When disabled, the number of pipelines running on an engine can exceed the configured Max Running Pipeline Count only if you start multiple jobs at the exact same time using the scheduler or using the Control Hub REST API. For more information, see Resource Thresholds.
    Maximum number of scheduler runs Maximum number of scheduled task runs to retain the details for.

    System limit is 500.
    Maximum number of days before scheduler runs are purged Maximum number of days to retain run details for each scheduled task.

    System limit is 30.
    Inactivity period for session termination Maximum number of minutes that a user session can remain inactive before timing out. A user session is considered inactive when all browser tabs opened by the user and accessing Control Hub are not in focus or are closed.
  6. Click Save to save changes made to advanced properties.
  7. Click Configure SAML to enable SAML authentication for the organization.

    For more information about configuring SAML, see SAML Authentication Overview.

Organization Default System Limits

Control Hub sets default system limits on the number of objects that can exist in each organization. The limits protect the system from run-away scripts or unintended automation usage.

These limits are sufficient for most organizations. If you hit these system limits and have a paid account, contact the StreamSets Support team to discuss increasing the limits.

System limits cannot be increased for free trials.

The following table lists the default system limits within Control Hub:

Object System Limit
API credentials per user 10
Deployments 50
Engines 20
Environments 50
Groups

(including manually created groups and SCIM provisioned groups)

 1,000
Jobs 10,000
Active jobs running concurrently 1,000
Legacy Kubernetes deployments 50
Legacy Kubernetes Provisioning Agents 20
Pipelines

(including both and published pipelines)

 1,000
Pipeline versions or commits 3,000
Scheduled tasks 5,000
Subscriptions 100
Topologies

(including draft and published topologies)

 50
Topology versions or commits 500