Managing Connections

Managing connections includes filtering the list of displayed connections, testing connections, and editing connection values. You can also view the pipelines and fragments that use each connection.

When needed, you can delete a connection when it is not used in any pipelines or pipeline fragments.

Tip: Your organization administrator can view a history of the connections that have been created, updated, and deleted by viewing the connection audit entries.

Filtering Connections

In the Connections view, you can filter the list of displayed connections by connection type or tag. Or you can search for connections by name or description.

  1. In the Navigation panel, click Connections.
  2. If the Filter column does not display, click the Toggle Filter Column icon: .

    The following image displays the Filter column in the Connections view:

  3. To search for connections by name or description, enter text in the search field, and then press Return.
  4. Select a connection type or tag to additionally filter the list of connections.

    You can filter by connection types that you have access to. For example, if you do not have access to any Kudu connections, you cannot filter by the Kudu connection type.

  5. Select the Keep Filter Persistent checkbox to retain the filter when you return to the view.
    Tip: To share the applied filter, copy the URL and send it to another user in your organization.

Testing Connections

When you create or edit connections, you can test that the configured connection values are valid.

When you test a connection, you verify that the selected authoring Data Collector can connect to the external system using the configured values. Testing a connection does not guarantee that a running pipeline using the connection can also connect to the external system.

For example, let's say that you use credential functions in connection properties to retrieve sensitive values from a credential store. When you test the connection, you verify that the selected authoring Data Collector can retrieve the values from the credential store and connect to the external system. A successful test verifies that the credential store is correctly configured for the selected authoring Data Collector. If the same credential store is not correctly configured for the execution Data Collector, then when you start a job that includes a pipeline using this connection, the pipeline fails to connect to the external system.

When a connection type supports multiple stage libraries, such as a Kafka or Kudu connection, you select the stage library installed on the selected authoring Data Collector to use for testing. For example, let's say that both Apache Kafka and Cloudera CDH stage libraries are installed on the selected authoring Data Collector. To test a Kafka connection to brokers running in the Apache Kafka cluster, you select the Apache Kafka stage library.

  1. In the Navigation panel, click Connections.
  2. Select a connection and then click the Edit icon: .
  3. In the Define Connection section, verify that a valid authoring Data Collector is selected.
  4. In the Configure Connection section, verify that all required properties are configured.
  5. Click Test Connection.

    If the connection supports multiple stage libraries installed on the selected Data Collector, expand the list of libraries next to the Test Connection button and select the stage library to use for testing.

Viewing Connection Usage

You can view the pipelines and pipeline fragments that use a connection. You might want to view the list of pipelines and fragments before editing or deleting a connection.

  1. In the Navigation panel, click Connections.
  2. Expand the details of a connection.

    The Pipelines/Fragments section lists the most recently updated version of the pipelines and fragments that use the connection:

  3. Click a pipeline or fragment name to open the pipeline or fragment in the canvas.

Editing Connections

You can edit connections to change most connection properties. You cannot change the connection type.

When you edit a connection, all pipelines using that connection reflect the updated values in subsequent pipeline runs.
Tip: Before editing a connection, view the list of pipelines and pipeline fragments that use the connection. That way, you can ensure that you make valid changes for all objects that use the connection.

Edit connections from the Connections view. Hover over a connection, and click the Edit icon: .

In the Define Connection section of the dialog box, you can change the following information:
  • Name and description
  • Authoring Data Collector - You can select a different authoring Data Collector used to configure and test the connection. Ensure that the stage library required by the connection is installed on the selected authoring Data Collector.
  • Tags - You can assign and remove tags from the job to identify the connection in a different way.

In the Configure Connection section of the dialog box, you can change configuration properties for the connection type. For a description of each property, see Overview.

Deleting Connections

You can delete a connection when it is not used in any pipelines or pipeline fragments.
Tip: Before deleting a connection, view the list of pipelines and pipeline fragments that use the connection. Edit these pipelines and fragments to select another valid connection.
  1. In the Navigation panel, click Connections.
  2. Select connections in the list, and then click the Delete icon: .
  3. Click OK to confirm the deletion.