Completing Post Upgrade Tasks
In some situations, you must complete tasks within Control Hub after you upgrade.
Upgrade to MySQL 8.x
Starting with version 3.56.1, Control Hub supports MySQL version 8.x for the relational database that stores metadata written by Control Hub applications.
Although Control Hub continues to support MySQL 5.6 and 5.7, these earlier MySQL versions have reached end of life. After you upgrade Control Hub to version 3.56.x, you might consider upgrading to MySQL 8.x as a post-upgrade task.
To upgrade to MySQL 8.x, complete the following general steps:
-
Upgrade all Control Hub instances while still using MySQL 5.6 or 5.7.
The upgrade process renames several database columns that are required to support MySQL 8.x.
- Upgrade the MySQL 5.6 or 5.7 databases to version 8.x. For instructions, see the MySQL documentation.
-
Modify the following properties in the Control Hub application configuration files to use the connection URL, user name, and
password of the MySQL 8.x database:
db.openjpa.ConnectionURL
db.openjpa.ConnectionUserName
db.openjpa.ConnectionPassword
Modify the property values in all of the following files located in the Control Hub configuration directory, $DPM_CONF:
- connection-app.properties
- dpm.properties
- dynamic_preview-app.properties
- jobrunner-app.properties
- messaging-app.properties
- notification-app.properties
- pipelinestore-app.properties
- policy-app.properties
- provisioning-app.properties
- reporting-app.properties
- scheduler-app.properties
- sdp_classification-app.properties
- security-app.properties
- sla-app.properties
- timeseries-app.properties
- topology-app.properties
- Restart Control Hub for the changes to take effect.
Disable Logging for Non-Delivered Events
Control Hub version 3.51.0 includes a known issue where the logging of non-delivered events might cause out-of-memory errors to occur when the Messaging application retains a large number of requests that have not been retrieved.
To avoid these out-of-memory errors, StreamSets
recommends disabling this logging by setting the
non.delivered.events.checker.task.enabled
property to
false
in the $DPM_CONF/messaging-app.properties
configuration file.
This issue will be fixed in a future release.
Assign New Roles to Upgraded Users
- 3.0.0 introduced the Provisioning Operator role.
- 3.2.0 introduced the Reporting Operator and Scheduler Operator roles.
- 3.5.0 introduced the Classification Administrator and Policy Manager roles.
- 3.10.0 introduced the Control Hub Authentication role.
- 3.19.0 introduced the Connection Editor and Connection User roles.
Before upgraded users can complete the tasks that require these roles, you must assign the appropriate new roles to their user accounts or to the groups to which they belong. Users and groups are not assigned new roles during the upgrade process.
For a description of each role, see Role Descriptions.
- In the Navigation panel, click or .
- Click the name of a group or user.
- Select the required new roles as needed, and then click Save.
Update Provisioning Agents
Due to an important security fix, StreamSets recommends that you update all existing Provisioning Agents to use the StreamSets Control Agent Docker image version 3.17.0 or later.
If an existing Provisioning Agent uses latest
as the Control Agent
Docker image version and the imagePullPolicy
attribute in the
Provisioning Agent YAML specification is set to Always
, redeploy the
Provisioning Agent so that it is automatically updated to use version 3.17.0.
If an existing Provisioning Agent uses a specific Control Agent Docker image version,
update the Provisioning Agent YAML specification file to use the Control Agent Docker
image version 3.17.0
or to use latest
. For more
information, see Applying Changes to Provisioning Agents.
Update Deployments for Provisioned Data Collectors
Starting with the StreamSets Control Agent
Docker image version 3.10.0, the Control Agent requires that each YAML specification
file that defines a deployment use the Kubernetes API version
apps/v1
. Previously, the Control Agent Docker image required
that each YAML specification file use the API version
extensions/v1beta1
. Kubernetes has deprecated the
extensions/v1beta1
version.
If you upgrade a Provisioning Agent that uses a
Control Agent Docker image version earlier than 3.10.0, you must update all
deployment YAML specification files to use apps/v1
before you
redeploy the Provisioning Agent.
To upgrade a Provisioning Agent and then update deployments, complete the following steps:
-
Stop all deployments.
- In the Navigation panel, select .
- Select each deployment, and then click the Stop icon.
-
Upgrade the Provisioning Agent to use the Control Agent Docker image version
3.17.0 or later. If using
latest
as the Control Agent Docker image version, you can skip this step. -
Update all deployments.
Update the Data Collector Version Range
By default, StreamSets Control Hub can work with registered Data Collectors from version 2.1.0.0 to the current version of Control Hub. If you want Control Hub to work with later versions of Data Collector, you must update the Data Collector version range.
For example, if you upgrade Control Hub on-premises to version 3.3.0 and you upgrade registered Data Collectors to version 3.5.0, you must update the maximum Data Collector version that can work with Control Hub. As a best practice, configure the maximum Data Collector version to 3.5.10 to ensure that Data Collector upgrades to later minor versions, such as 3.5.1 or 3.5.2, will continue to work with Control Hub.
- Log in to Control Hub as the system administrator.
- In the Navigation panel, click .
- Click the Component Version Range icon: .
- Enter the minimum and maximum Data Collector versions that can work with Control Hub.
- If you enter a Snapshot version for the minimum version, enter the oldest
acceptable build date for that version using the following
format:
yyyy-MM-dd'T'HH:mm'Z'
For example:2016-08-22T06:30Z
Enter ANY to allow any build date from the specified Snapshot version.
Control Hub checks the build date only when the Data Collector version matches the specified minimum version.
Modify the System Sample Pipeline Organization
With version 3.0.0, you can configure another organization besides the default system
organization with an ID of admin
to manage system sample pipelines. If you
modified the organization that manages system sample pipelines, configure the new Control Hub version
to use the same organization to manage system sample pipelines.
pipeline.templates.organization
Restart Control Hub for the changed property to take effect.
For more information, see System Sample Pipelines.
Delete the Previous Applications
During the upgrade process, if you configured the upgraded Control Hub applications to use a different component ID from the previous version, delete the previous Control Hub applications from the Control Hub UI.
- In the Navigation panel, click .
- Select the previous Control Hub applications, and then click the Deactivate icon: .
- With the previous Control Hub applications still selected, click the Delete icon: .