Searching for Objects

Overview

You can search for a specific object or set of objects belonging to your organization.

At this time, you can search for the following object types:

  • Fragments
  • Pipelines
  • Job templates
  • Job instances
  • Draft runs

StreamSets will implement search for additional object types in future releases.

Search replaces filtering and is enabled by default. If needed, you can disable search and revert to the original filtering functionality.

Each searchable object type includes a fixed set of properties that you create search conditions for. For example, you can search for pipelines by the Commit Message property, or search for jobs by the Failover property.

You can perform both basic and advanced searches. With basic search, you define search conditions by selecting the object properties, operators, and values that you want to search for.

With advanced search, you define search conditions by writing a search query using the StreamSets advanced query language (SAQL). Advanced search allows you to specify criteria that cannot be defined in basic search, such as defining complex conditions or changing the order of precedence for multiple conditions.

You can save both basic and advanced searches. You can star your most frequently used saved searches so that you can easily find them later.

Search Use Cases

You can search for objects to address the following common use cases:
Find similar pipelines to help with pipeline development
As a new pipeline developer, you want to explore pipelines created by other developers to solve a problem with your own pipeline.
You search for pipelines that contain the string Snowflake in the name and that have the label WestDataCenter.
Find jobs that have encountered errors and that are owned by multiple teams
As a manager, you want to view a list of running jobs that have encountered errors that are owned by your three development teams. Each development team assigns a unique engine label to the jobs that they run: tools, adaptors, or platform.
You search for jobs that are assigned the tools, adaptors, or platform engine label and that have an INACTIVE_ERROR status.
Find a specific pipeline version
As a pipeline developer, you want to quickly find version 5 of the SocialFeedsDataflow pipeline. It's an older version of the pipeline, and you don't want to spend time opening the most recent version of the pipeline in the pipeline canvas and then selecting the older version.
You search for a pipeline with the name SocialFeedsDataflow where the pipeline version is 5.
Find all pipelines developed by another user
A pipeline developer at your company has moved to another development team, and you need to take over all pipelines that she was working on. You search for pipelines that were committed or last modified by rita@mycompany.com.

Defining Search Conditions

To search for objects, you define the conditions for your search.

You can define search conditions in the following ways:

Basic search
Basic search provides a user interface where you simply select the object properties, operators, and values that you want to search for. For example, you can search for all pipelines with a name beginning with hd that were last modified after May 15, 2022.
To access basic search, select the appropriate view from the Navigation panel, such as Build > Pipelines, and then enter your search conditions in the search section above the list of objects, as follows:
For more information, see Basic Search.
Advanced search
Advanced search is a more powerful search that lets you write complex search queries using the StreamSets advanced query language (SAQL). With advanced search, you can specify criteria that cannot be defined in basic search, such as defining complex conditions or changing the order of precedence for multiple conditions.
To access advanced search, select the appropriate view from the Navigation panel, such as Build > Pipelines. In the search section above the list of objects, click the More icon () and then click Advanced Mode. Enter a search query using the required syntax, as follows:
For more information, see Advanced Search.

Changing the Search Mode

As you define search conditions, you can change the search mode.

When you change from basic to advanced mode, Control Hub automatically generates an advanced search query using the search conditions defined in basic mode.

When you change from advanced to basic mode, Control Hub cannot convert the advanced search query to basic mode because the query might include syntax not supported in basic mode. As a result, Control Hub deletes the advanced search query when you change to basic mode.

To change the search mode, click the More icon () in the search section above the list of objects, and then click Advanced Mode or Basic Mode.

Saved Searches

After you define search conditions that find the desired results, you can save the search for later use. When you save a search, you give the search a name. You can save both basic and advanced searches.

For example, you might define a search that finds all jobs that have encountered errors and that are running on engines belonging to the East Data Center deployment. You save the search with the name East Data Center - job errors, and then you can access and view the search results on a regular basis.

In addition to the searches that you define and save, Control Hub also includes several preset saved searches that solve common use cases.

You can star your most important saved searches so that you can easily find them later.

Note: Saving a search differs from persisting the last configured search.

Preset Searches

Control Hub includes several preset saved searches that solve common use cases.

For example, pipeline search includes the following preset searches:
  • Engine Type: Data Collector - Finds all Data Collector pipelines.
  • Engine Type: Transformer - Finds all Transformer pipelines.
  • Engine Type: Transformer for Snowflake - Finds all Transformer for Snowflake pipelines.
  • Pipeline Drafts - Finds all draft pipelines.
  • Pipeline Labels - Finds all pipelines with the labels that you select.

The preset searches are starred by default. When you manage saved searches, you can unstar the preset searches, but cannot edit or delete them.

Starred Searches

You can star your most important saved searches so that you can easily find them later. Control Hub also includes several preset searches that are starred by default. You can unstar these preset searches if you do not access them frequently.

To star and unstar searches, see Managing Saved Searches.

You can access starred searches from the following locations:
Starred Searches icon
In the search section above the list of objects, click the selector next to the Starred Searches icon, and then select the search that you want to access.
Saved Searches dialog box
In the search section above the list of objects, click the More icon () and then click Saved Searches. In the Saved Searches dialog box, filter by starred searches only, and then select the search that you want to access.

Saving a Search

After you've defined search conditions that find the desired results, you can save the search for later use.

You can save a maximum of 50 searches across all object types. Preset searches do not count towards this limit.

  1. Select the appropriate view from the Navigation panel, such as Build > Pipelines.
  2. In the search section above the list of objects, enter your search conditions in basic or advanced mode and then click Search.
  3. Click the More icon () and then click Save this Search.
  4. In the Save Search dialog box, enter a search name and then click Save.

Managing Saved Searches

You can view all saved searches for a specific object type. Access a saved search to view the results for that search.

The actions that you can complete for saved searches depend on the following search types:
  1. Select the appropriate view from the Navigation panel, such as Build > Pipelines.
  2. In the search section above the list of objects, click the More icon () and then click Saved Searches.
  3. Use one of the following methods to filter the list of saved searches:
    • Select Starred Only to display only starred searches.

    • Enter text in the search field to search for saved searches by name.

  4. Complete one of the following tasks:
    • Click the name of a saved search to view the results for that search.
    • In the Actions column for one of your saved searches, click the Edit icon () to edit the name of the saved search. When you finish editing the name, click the Done icon () to save the modified name.

      You cannot edit the names of preset searches.

    • In the Actions column for one of your saved searches, click the Delete icon () to delete the saved search. Click Ok to confirm the deletion, and then click Close.

      You cannot delete preset searches.

    • In the Actions column for any saved search, click the white Star icon () to star the saved search or the yellow Star () icon to unstar the saved search.

Persisting the Last Search

You can configure Control Hub to retain the last defined search.

If you do not persist the last search, then Control Hub displays basic search with no defined search conditions each time that you return to the view.

Note: Persisting the last configured search differs from saving a search.
  1. Select the appropriate view from the Navigation panel, such as Build > Pipelines.
  2. In the search section above the list of objects, click the More icon () and then enable the Persist Search property.

Reverting to Original Filtering

Search replaces filtering and is enabled by default. If needed, you can disable search and revert to the original filtering functionality.

  1. Select the appropriate view from the Navigation panel, such as Build > Pipelines.
  2. Click the More icon () above the list of objects, and then click Revert to Original Filtering.

    Control Hub disables search and reverts to filtering for all object types.

  3. To enable search again, click the More icon () above the list of objects, and then click Enable New Search.