Version History

A typical pipeline or pipeline fragment development cycle involves iterative changes. Control Hub maintains the version history for each pipeline and fragment.

When you first create a pipeline or fragment, it starts in a draft state so that you can configure the object. Draft objects are versioned as <version>-DRAFT, for example, v1-DRAFT or v2-DRAFT. When you finish editing a pipeline, publish the pipeline to indicate that it is final and is available to be added to a job. Publish a fragment to make it available for testing or use in pipelines.

Published pipelines and fragments are versioned by number, for example, v1 or v2. You cannot edit published objects. When viewing a published version, you can click edit to create a new version based on the version that displays in the canvas.

To view the version history, open a pipeline or fragment in the pipeline canvas, and then click the version number, as follows:

From the version history, you can manage the pipeline or fragment versions including viewing commit messages for each commit, comparing versions, creating tags for versions, and deleting versions.

Versions

The version history displays each pipeline or fragment commit along with the version number and commit message.

If you edit and then publish a previous version of an object, Control Hub updates the minor version number rather than the major version number.

For example, you have a pipeline with three versions. You get version 2 of the pipeline, edit the pipeline, and then publish the pipeline again. Control Hub versions the current pipeline as v2.1. So you now have four versions of the pipeline, with version 2.1 of the pipeline marked as the current version, as follows:

Comparing Versions

You can compare two versions of a pipeline or pipeline fragment. When you compare versions, Control Hub presents the versions side-by-side in the canvas, highlighting the differences between the versions. You can compare the pipeline or fragment configuration properties and all of the stages in the pipeline or fragment.

  1. Open a pipeline or pipeline fragment version.
  2. Click the More icon () in the pipeline canvas toolbar, and then click Compare Versions.

    The version currently open in the canvas and the previous version display side-by-side in the comparison window, highlighting any differences with a deleted, added, or updated icon, as follows:

    You can drill into the details of each stage, and compare the configuration of each stage between the versions.

  3. To compare with an older version, click the pipeline or fragment version number and select another version.
  4. Click the Open in Canvas icon () to open one of the versions in the pipeline canvas.

Deleting Versions

You can delete a specific pipeline or fragment version or all versions of a pipeline or fragment from the repository.

You can delete pipeline versions that are not included in jobs. To delete pipeline versions that are included in jobs, you must first delete the job or edit the job so that it no longer includes the pipeline version that you want to delete.

Similarly, you can only delete fragments that are not used in pipelines. To delete fragment versions that are included in pipelines, you must first remove the fragment versions from all pipelines or delete those pipelines.

  1. Open a pipeline or pipeline fragment.
  2. Click the pipeline or fragment version number to view the version history.
  3. Hover over the version that you want to delete, click the More icon () in the Actions column, and then click Delete Commit.
    Note: To delete all versions of a pipeline or fragment from the repository, click the More icon in the pipeline canvas toolbar, and then click Delete All Versions.

Deleting a Draft Version

While editing a draft version of a pipeline or pipeline fragment, you can delete that draft version to revert to the previous published version of the pipeline or fragment.

  1. While editing a draft version of a pipeline or fragment, click the More icon () in the pipeline canvas toolbar, and then click Delete Draft.
  2. Click OK to confirm that you want to delete the changes made to the draft.

    The pipeline canvas reverts your changes and displays the previous published version of the pipeline or fragment in the canvas.

Tags

The version history displays all tags defined for the pipeline or fragment.

Assign tags to pipeline or fragment versions to differentiate between the versions. You might want to add tags to versions to mark release points or to separate development and production environments.

For example, when you finish developing the first version of a fragment, you might add a Ready to Test tag to the version. Then, when the fragment passes muster, you can use a Ready for Use tag. These tags clearly inform the data engineer which fragments are ready to be tested in development pipelines or used in production pipelines.

Creating Tags

Create and assign tags to versions of pipelines and fragments to differentiate between the versions.

  1. Open a pipeline or pipeline fragment version.
  2. Click the pipeline or fragment version number to view the version history.
  3. Hover over the version that you want to tag, click the More icon () in the Actions column, and then click Create Tag.
  4. Enter a name and message for the tag.
    In the tag message, state the reason for adding the tag to this version.
  5. Click Create Tag.
    Control Hub displays the tag next to the version.

Removing Tags

If needed, you can remove tags from versions of pipelines and fragments.

  1. Open a pipeline or pipeline fragment version.
  2. Click the pipeline or fragment version number to view the version history.
  3. Click the X next to a tag name to remove the tag.