Upgrade an Installation from the RPM Package
When you upgrade an installation from the RPM package, the new version uses the default Transformer configuration, data, log, and resource directories. If the previous version used the default directories, the new version has access to the files created in the previous version.
If the previous version used customized values for the directory environment variables, you must make the same customizations in the new version so that the new version can access the same files.
Use the same procedure to upgrade an installation from the RPM package when Spark runs locally on the Transformer machine or when Spark runs on a cluster.
Step 1. Shut Down the Previous Version
Stop all running pipelines and then shut down the previous version of Transformer.
-
Use one of the following methods to stop all running pipelines:
- If Transformer is
not registered to work with StreamSets Control Hub, stop
the pipelines using the Transformer UI.
From the Transformer Home page, select all running pipelines in the list and then click the Stop icon.
- If Transformer is
registered to work with StreamSets Control Hub, stop
all jobs running on Transformers using the
Control Hub UI.
From the Control Hub Jobs page, filter the jobs by engine and by engine label. Select all active jobs in the list and then click the Stop Jobs icon.
- If Transformer is
not registered to work with StreamSets Control Hub, stop
the pipelines using the Transformer UI.
-
Use one of the following methods to shut down Transformer:
- To use the command line for shutdown, use the required command for your
operating system.
For CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6, use:
service transformer stop
For CentOS 7, Oracle Linux 7, or Red Hat Enterprise Linux 7, use:
systemctl stop transformer
- To use the Transformer UI, click . When the confirmation dialog box appears, click Yes.
- To use the command line for shutdown, use the required command for your
operating system.
Step 2. Back Up the Previous Version
Before you install the new version, create a backup of the files in the data and resource directories in the previous version. You’ll also need to create a backup of the environment configuration file so that the file is not overwritten when you install the new version. That way, you can continue to run the previous version if needed.
- File that defines environment variables, based on the operating system:
- CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6 - the
$TRANSFORMER_DIST/libexec/transformerd-env.sh
file. - CentOS 7, Oracle Linux 7, or Red Hat Enterprise Linux 7 - the
/usr/lib/systemd/system/transformer.service
file.
- CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6 - the
- Data directory defined in the TRANSFORMER_DATA environment variable. Default is
/var/lib/transformer
. - If used, copy and rename the following directories as well:
- TRANSFORMER_EXTERNAL_RESOURCES - The Transformer directory for external resources.
- TRANSFORMER_RESOURCES - The Transformer directory for runtime resource files.
-
STREAMSETS_LIBRARIES_EXTRA_DIR - The Transformer directory for external libraries.
For example, if you are upgrading version 3.12.0 on CentOS 6, Oracle Linux 6, or Red Hat Enterprise Linux 6, back up the Transformer data
directory and name it as follows: /var/lib/transformer3120
. Create a
backup of the environment configuration file and name the backup file as follows:
transformerd-env-3120.sh
.
Step 3. Install the New Version
Install the new version of the RPM package on the same machine as the previous version.
Step 4. Update Environment Variables
Update the Transformer environment configuration file so that the new version of Transformer uses the same Java and Spark installation directories as the previous version.
Update the environment variables in the required file based on your installation type. For more information about the required file to edit, see Modifying Environment Variables.
Step 5. Update the Configuration Files
A new Transformer version can include new properties and configuration files required for Transformer to start or function properly.
When you install the new RPM package, the configuration files are written to the same
default directory as the previous version, /etc/transformer
. The
new versions of the configuration files are renamed with the following extension:
.rpmnew
. For example, the new version of the Transformer configuration file is renamed to
transformer.properties.rpmnew
.
.rpmnew
file extension. In this case, you do not
rename the configuration files, but must update the new files with any
customized values defined in the previous version.Step 6. Start the New Version of Transformer
Start the new version of Transformer, as described in Starting Transformer as a Service.
If the previous version of Transformer was registered with StreamSets Control Hub and you correctly updated the configuration files during the upgrade, then the new version of Transformer is automatically registered and enabled to work with Control Hub.