Version History

A typical pipeline or pipeline fragment development cycle involves iterative changes. Control Hub maintains the version history for each pipeline and fragment.

When you first create a pipeline or fragment, it starts in a draft state so that you can configure the object. Draft objects are versioned as <version>-DRAFT, for example, v1-DRAFT or v2-DRAFT. When you finish editing a pipeline, publish the pipeline to indicate that it is final and is available to be added to a job. Publish a fragment to make it available for testing or use in pipelines.

Published pipelines and fragments are versioned by number, for example, v1 or v2. You cannot edit published objects. When viewing a published version, you can click edit to create a new version based on the version that displays in the canvas.

To view the history of a pipeline or fragment from the Pipelines or Pipeline Fragments views, select a pipeline or fragment and then click the History icon: .

To view the history of a pipeline or fragment from the pipeline canvas, click the pipeline or fragment version number, as follows:

When you expand the version history, you can manage the pipeline or fragment versions including viewing commit messages for each commit, comparing versions, creating tags for versions, and deleting versions.

The following image shows the expanded version history of a pipeline:

Versions

The version history displays each pipeline or fragment commit along with the version number and commit message.

If you edit and then publish a previous version of an object, Control Hub updates the minor version number rather than the major version number.

For example, you have a pipeline with three versions. You get version 2 of the pipeline, edit the pipeline, and then publish the pipeline again. Control Hub versions the current pipeline as v2.1. So you now have four versions of the pipeline, with version 2.1 of the pipeline marked as the current version, as follows:

Comparing Versions

You can compare two versions of a pipeline or pipeline fragment. When you compare versions, Control Hub presents the versions side-by-side in the canvas, highlighting the differences between the versions. You can compare the pipeline or fragment configuration properties and all of the stages in the pipeline or fragment.

  1. Open a pipeline or pipeline fragment version.
  2. Click the pipeline or fragment version number, and then expand the version history.
  3. Hover over the version that you want to compare with the version currently open in the canvas, click the More icon (), and then click Compare with version.
    Tip: To compare with the previous version, you can simply click the Compare with Previous Version icon () in the toolbar.

    The two versions display side-by-side in the comparison window, highlighting any differences with a deleted, added, or updated icon, as follows:

    You can drill into the details of each stage, and compare the configuration of each stage between the versions.

Deleting Versions

You can delete a specific pipeline or fragment version or all versions of a pipeline or fragment from the repository.

You can delete pipeline versions that are not included in jobs. To delete pipeline versions that are included in jobs, you must first delete the job or edit the job so that it no longer includes the pipeline version that you want to delete.

Similarly, you can only delete fragments that are not used in pipelines. To delete fragment versions that are included in pipelines, you must first remove the fragment versions from all pipelines or delete those pipelines.

  1. Open a pipeline or pipeline fragment.
  2. Click the pipeline or fragment version number, and then expand the version history.
  3. Hover over the version that you want to delete, click the More icon (), and then click Delete commit.
    Note: To delete all versions of a pipeline or fragment from the repository, click the More icon () in the pipeline canvas toolbar, and then click Delete Pipeline or Delete Pipeline Fragment.

Tags

The version history displays all tags defined for the pipeline or fragment.

Assign tags to pipeline or fragment versions to differentiate between the versions. You might want to add tags to versions to mark release points or to separate development and production environments.

For example, when you finish developing the first version of a fragment, you might add a Ready to Test tag to the version. Then, when the fragment passes muster, you can use a Ready for Use tag. These tags clearly inform the data engineer which fragments are ready to be tested in development pipelines or used in production pipelines.

Creating Tags

Create and assign tags to versions of pipelines and fragments to differentiate between the versions.

  1. Open a pipeline or pipeline fragment version.
  2. Click the pipeline or fragment version number, and then expand the version history.
  3. Hover over the version that you want to tag, click the More icon (), and then click Tag commit.
  4. Enter a name and message for the tag.
    In the tag message, state the reason for adding the tag to this version.
  5. Click Create Tag.
    Control Hub displays the tag next to the version.

Removing Tags

If needed, you can remove tags from versions of pipelines and fragments.

  1. Open a pipeline or pipeline fragment version.
  2. Click the History icon () in the pipeline canvas toolbar.
  3. Click Pipeline Tags.
  4. Hover over the tag that you want to remove, and click the Remove Tag icon: .