My Organization

Overview

An organization is a secure space provided to a set of Control Hub users.

All engines, pipelines, jobs, and other objects added by any user in the organization belong to that organization. You log in to Control Hub as a member of an organization and can only access data that belongs to that organization. The specific organization data that you can access depends on the roles and permissions assigned to your account.

When you are a member of multiple organizations, you select the organization to use when you log in. After logging in, you can switch organizations. You can also choose to leave an organization.

A user with the Organization Administrator role can review and configure organization properties. An organization administrator can also enable the organization to use SAML authentication.

An organization has one primary organization administrator. When needed, the current primary organization administrator can change the primary administrator for the organization.

Control Hub sets default system limits on the number of objects that can exist for the organization.

Primary Organization Administrator

An organization can include multiple users assigned the Organization Administrator role, but only one primary organization administrator.

By default, the primary organization administrator is the initial user who signs up for an IBM StreamSets account and then invites other users to the organization.

The primary organization administrator differs from other users assigned the Organization Administrator role in the following ways:

  • Receives notification emails when IBM StreamSets encounters planned or unexpected outages.

  • Cannot be managed by any other organization administrator.

    For example, no other user can deactivate the primary organization administrator user account nor remove the Organization Administrator role from the primary organization administrator user account.

  • Can change the primary administrator for the organization.

For a description of the Organization Administrator role, see Role Descriptions.

Changing the Primary Organization Administrator

As the current primary organization administrator, you can change the primary administrator for the organization. You can select from all users assigned the Organization Administrator role.

  1. In the Navigation panel, click Manage > My Organization.
  2. Click the Edit icon () next to the primary organization administrator's email address.
  3. In the Change Primary Organization Administrator dialog box, select another user assigned the Organization Administrator role, and then click Save.
  4. Click Ok to confirm the change.

Multiple Organizations

A single user account can belong to multiple organizations.

For example, let's say that you sign up as a new user. IBM StreamSets creates a Control Hub organization using the company or organization name that you enter, and creates a user account using your email address. A colleague also signs up as a new user, and invites you to join her organization using your same email address. When you join your colleague's organization, IBM StreamSets adds you as a member of that organization using your existing user account.

When you are a member of multiple organizations, you use a single set of credentials to log in. During the login, you select the organization to use for that session. For that session, you can only access data belonging to the selected organization.

To access data belonging to a different organization, sign out of IBM StreamSets by clicking the My Account icon () in the top right toolbar, then clicking Sign Out. Log in to your account again, selecting the other organization that you want to use.

Leaving an Organization

If you are not the organization administrator who created the organization, you can choose to leave the organization. Leaving the organization deletes your user account in that organization.

  1. In the top right toolbar, click the My Account icon (), and then click your user name.
  2. On the Account Settings tab, click Leave Organization.
  3. Click OK to confirm that you want to leave the organization.

Configuring Organization Properties

A user with the Organization Administrator role can review and configure organization properties.

  1. In the Navigation panel, click Manage > My Organization.
  2. Configure the following general organization properties:
    General Property Description
    Organization Name Name of the organization.
    Valid Domains List of trusted domains that can make authentication requests to Control Hub on behalf of your organization.
  3. Click Save to save changes made to general properties.
  4. Click Advanced to configure advanced organization properties.

    Some of the advanced properties can be modified only up to the maximum system limit. For example, the system limit for the maximum number of job runs is 100. You can configure a lower limit for the maximum number of job runs, such as 50, but cannot configure a limit higher than 100.

  5. Configure the following advanced properties:
    Advanced Property Description
    Enable events to trigger subscriptions Enables events so that Control Hub can trigger subscriptions for organizations. Disable events if you do not want users to use subscriptions.
    Enable WebSocket Tunneling for UI Communication Enables execution engines to use the WebSocket Secure (wss) protocol to establish a WebSocket tunnel with Control Hub over an encrypted SSL/TLS connection.
    Important: Before you disable this property, you must configure all engines to use the HTTPS protocol to directly communicate with the web browser.
    Enable Snowpark Preview Enables previewing data in Transformer for Snowflake pipelines when using the Transformer for Snowflake engine hosted and managed by StreamSets. Enabling this option allows Snowflake data to leave Snowflake servers for the preview.

    Not applicable when your organization uses a deployed Transformer for Snowflake engine.

    Disable this option to prevent all users in the organization from using data preview in Transformer for Snowflake pipelines.

    For more information, see the Transformer for Snowflake documentation.
    Enforce permissions during object access Enables permission enforcement to secure the integrity of organization data. Disable permission enforcement if you want all users in the organization to have full access to all objects.
    Default authoring engine timeout Number of milliseconds that Control Hub waits for a response from an authoring engine before considering the engine as not accessible.

    When the engine is not accessible, you cannot select that engine as the authoring engine when designing pipelines or creating connections.

    In most cases, the default value should be appropriate. Try increasing the value when the authoring engines are running, but the authoring engine selection pages indicate that engines are not accessible.

    Users can override this default value when configuring their browser settings within the My Account window.

    Default is 5000 milliseconds.
    Execution engine heartbeat interval Maximum number of seconds since the last reported heartbeat before Control Hub considers an execution engine unresponsive. In most cases, the default value of 300 seconds, or five minutes, is sufficient.

    System limit is 1,800.
    Maximum number of days before job status history is purged Maximum number of days to retain the run history for each job and draft run.

    System limit is 7.
    Maximum number of job runs Maximum number of job and draft runs to retain the run history for, including runs of job instances started from a job template.

    System limit is 100.
    Enable Multiple Job Start Synchronization Enables Control Hub to synchronize the start of multiple jobs. Use to ensure that the number of pipelines running on an engine does not exceed the configured resource threshold. Enabling the property can cause jobs to take longer to start.

    When disabled, the number of pipelines running on an engine can exceed the configured Max Running Pipeline Count only if you start multiple jobs at the exact same time using the scheduler or using the Control Hub REST API. For more information, see Resource Thresholds.
    Default AWS Environment Feature Version Default feature version to use for new AWS environments.
    Default Azure Environment Feature Version Default feature version to use for new Azure environments.
    Default GCP Environment Feature Version Default feature version to use for new GCP environments.
    Default Kubernetes Environment Feature Version Default feature version to use for new Kubernetes environments.
    Default Self-Managed Environment Feature Version Default feature version to use for new self-managed environments.
    Show the Stage Library Mode UI field for CSP Deployments Displays the Stage Library Mode property for deployments so that you can use the user-provided stage library mode.
    Note: Use caution when enabling this property. In most situations, you can use the default managed stage library mode.
    Maximum number of scheduler runs Maximum number of scheduled task runs to retain the details for.

    System limit is 500.
    Maximum number of days before scheduler runs are purged Maximum number of days to retain run details for each scheduled task.

    System limit is 30.
    Inactivity period for session termination Maximum number of minutes that a user session can remain inactive before timing out. A user session is considered inactive when all browser tabs opened by the user and accessing Control Hub are not in focus or are closed.
  6. Click Save to save changes made to advanced properties.

    It can take a few minutes for the changes to take effect.

  7. Click Configure SAML to enable SAML authentication for the organization.

    For more information about configuring SAML, see SAML Authentication Overview.

Organization Default System Limits

Control Hub sets default system limits on the number of objects that can exist in each organization. The limits protect the system from run-away scripts or unintended automation usage.

These limits are sufficient for most organizations. If you hit these system limits and have a paid account, contact the StreamSets Support team to discuss increasing the limits.

System limits cannot be increased for free trials.

The following table lists the default system limits within Control Hub:

Object System Limit
API credentials per user 10
Deployments 50
Engines 20
Environments 50
Groups

(including manually created groups and SCIM provisioned groups)

 1,000
Jobs 10,000
Active jobs running concurrently 1,000
Legacy Kubernetes deployments 50
Legacy Kubernetes Provisioning Agents 20
Pipelines

(including both and published pipelines)

 1,000
Pipeline versions or commits 3,000
Scheduled tasks 5,000
Subscriptions 100
Topologies

(including draft and published topologies)

 50
Topology versions or commits 500