Design in Data Collector or Transformer

In addition to designing pipelines in the Control Hub Pipeline Designer, you can also log into an authoring Data Collector or Transformer to design pipelines and then add them to the Control Hub pipeline repository. You cannot design pipeline fragments in Data Collector or Transformer.

You can add pipelines in the following ways:

Publish pipelines from registered Data Collectors or Transformers
You can publish pipelines from Data Collectors or Transformers that are registered with Control Hub.
Import pipelines from unregistered Data Collectors or Transformers
You can import pipelines from Data Collectors or Transformers that are not registered with Control Hub. Export the pipelines from the unregistered Data Collectors or Transformers using the Export for Control Hub option, and then import the pipelines into Control Hub.
When you export a pipeline for use in Control Hub, Data Collector or Transformer generates a JSON file named after the pipeline, as follows: <pipeline name>.json. The generated JSON file includes the definition of each stage library used in the pipeline.
Note: You can also export and import pipelines from registered Data Collectors and Transformers. However, when these are registered with Control Hub, it's simplest to publish the pipelines directly to Control Hub.

Publishing Pipelines

After you finish developing pipelines in a registered Data Collector or Transformer, you publish the pipelines to the Control Hub pipeline repository. You can publish pipelines that are valid.

  1. Log in to the registered Data Collector or Transformer.
  2. From the Home page, select pipelines in the list and then click the Publish Pipeline icon: . Or to publish a pipeline from the pipeline canvas, click the Control Hub Options icon (), and then click Publish Pipeline.
    The Publish Pipeline dialog box appears.
  3. Enter a commit message.
    As a best practice, state what changed in this pipeline version so that you can track the commit history of the pipeline.
    Note: If you are publishing multiple pipelines from the Home page, the same commit message is used for all of the pipelines.
  4. Click Publish Pipeline.
    When you log in to Control Hub, the Pipelines view lists the published pipelines.