Runtime Parameters

Runtime parameters are parameters that you define in a pipeline and then call from within the same pipeline. When the pipeline runs, the value replaces the name of the runtime parameter. Use runtime parameters to define values for stage and pipeline properties.

When you configure a job that runs the pipeline, you can override the parameter values defined in the pipeline. For example, you can define a targetTable parameter that points to a production table. Then during development, you can override it to point to a test table.

Or, you might define a set of parameters, one that defines the target table and one that defines the write mode, so you can use the same pipeline to write to different tables in different ways by defining different sets of parameters for each job.

You can use runtime parameters to represent the value for most properties. However, you cannot use them as part of a set of properties. You can tell when you are looking at a set of properties that won't allow runtime parameters when you see the Add Another and Bulk Edit Mode options. For example, the Column Type Converter processor uses sets of properties, so you cannot use runtime parameters in those properties:

To implement runtime parameters, perform the following steps:
  1. Define a runtime parameter.
  2. Call a runtime parameter.
  3. Optionally, override parameter values in job properties.

Step 1. Define a Runtime Parameter

Define runtime parameters in pipeline properties before you call them from the pipeline.

The parameter must evaluate to a valid option for the property type:
Text Box
Parameters called from a text box must be valid for that location.
For example, the targetTable parameter must be set to a valid table name.
Parameters called from a checkbox must be set to true or false.
Drop-down menus
Parameters called from a drop-down menu must evaluate to a valid key value. Each option in the menu has an associated key value.
To view a valid key value, select the desired option from the menu, then click the Use Parameter icon ().
For example, to find the key value for the append write mode, set the Write Mode property to Append to Table, as shown below. Then, click the Use Parameter icon. The second image shows APPEND as the key value for that option:

Defining a Runtime Parameter

Define runtime parameters in pipeline properties.
  1. In the pipeline properties, click the Parameters tab.
  2. Using simple or bulk edit mode, click the Add Another icon and define the name and the default value for each parameter.
    For example, the following pipeline has two runtime parameters defined. The first is the name of a target table, the second is the key value for the Append to Table write mode option:

Step 2. Call the Runtime Parameter

Use an expression to call a runtime parameter. To call a runtime parameter, use the following syntax in a stage or pipeline property:
${<parameter name>}

After you define a runtime parameter in pipeline properties, it becomes available through expression completion so you can avoid typographical errors.

To invoke expression completion, start typing the expression: ${ and the closing bracket appears. Then, start typing the name of the parameter that you want to use. Select the correct parameter when it appears.

In the following example, the parameter targetTable appears after typing ${t:

The error message resolves after the parameter is selected.

Calling from Text Boxes

To call a runtime parameter in a stage or pipeline property that displays as a text box, you simply specify the parameter name as described above.

For example, to use the target_database runtime parameter with the Snowflake Table destination, enter the following syntax for the Table property:
You can call a runtime parameter from within a StreamSets function by simply entering the parameter name. For example, the following expression trims leading and trailing spaces from the data in the address_column runtime parameter:
You can use a runtime parameter to represent a part of a property. For example, say you want to use a set of UDF in the Applied Functions processor that all have the same EU_ prefix. You can specify the prefix in the UDF Name property and use a UDFname runtime parameter for the rest of the name, as follows:

Calling from Checkboxes and Drop-Down Menus

To call a runtime parameter in a stage or pipeline property that displays as a checkbox or drop-down menu, you first must convert the property to a text box.

Click the Use Parameter icon () next to the checkbox or drop-down menu to convert the property to a text box, and then call the parameter using the required syntax.

For example, the following image shows the Snowflake Table destination with two runtime parameters. The Table property was already a text box. The Write Mode property is a drop-down menu converted to a text box to call the writeMode runtime parameter:

Step 3. Override the Parameter (Optional)

When needed, you can override a runtime parameter in the Control Hub job that runs the pipeline.

For more information about configuring parameters in jobs or job templates, see the Control Hub documentation.