Pre Upgrade Tasks

Verify Installation Requirements

The minimum requirements for Data Collector can change with each version. Before you upgrade to a new Data Collector version, verify that the machine meets the latest minimum requirements as described in Installation Requirements.

Migrate to Java 8

Data Collector version 2.5.0.0 requires Java 8. If your previous Data Collector version ran on Java 7, you must migrate to Java 8 before upgrading to the latest Data Collector version.

All services that use Data Collector JAR files also must run on Java 8. This means that your Hadoop cluster must run on Java 8 if you are using cluster pipelines, the Spark Executor, or the MapReduce Executor.

To migrate to Java 8, complete the following steps before upgrading to the latest Data Collector version:

Shut down Data Collector.
Install Java 8 on the Data Collector machine.
If you customized Java configuration options in the SDC_JAVA7_OPTS environment variable and if those options are valid in Java 8, migrate those customizations to the SDC_JAVA8_OPTS environment variable.
Restart Data Collector and verify that it works as expected.
If any pipelines include the JavaScript Evaluator processor, open the pipelines and validate the scripts on Java 8.

Upgrade Cluster Streaming Pipelines

If you use cluster pipelines that run in cluster streaming mode and you are upgrading from a version earlier than 2.3.0.0, you must upgrade to Data Collector version 2.3.0.0 before upgrading to the latest version.

Prior to 2.3.0.0, Data Collector used the Spark checkpoint mechanism to recover cluster pipelines after a failure. Starting in version 2.3.0.0, Data Collector maintains the state of cluster pipelines without relying on Spark checkpoints.

Warning: If you upgrade from a version earlier than 2.3.0.0 directly to the latest version - without first upgrading to version 2.3.0.0 - cluster pipelines fail when starting.

Before you upgrade to the latest version, complete the following general tasks:

Upgrade to Data Collector version 2.3.0.0. For instructions, see Upgrade to version 2.3.0.0.
Start the upgraded Data Collector version 2.3.0.0 and run the cluster pipelines so that they process some data.

After verifying that the upgrade to Data Collector version 2.3.0.0 was successful, upgrade to the latest version.