Runtime Parameters
Runtime parameters are parameters that you define in a pipeline and then call from within the same pipeline. When the pipeline runs, the value replaces the name of the runtime parameter. Use runtime parameters to define values for stage and pipeline properties.
When you configure a job that runs the pipeline, you can override the parameter values
defined in the pipeline. For example, you can define a targetTable
parameter that points to a production table. Then during development, you can override
it to point to a test table.
Or, you might define a set of parameters, one that defines the target table and one that defines the write mode, so you can use the same pipeline to write to different tables in different ways by defining different sets of parameters for each job.
You can use runtime parameters to represent the value for most properties. However, you cannot use them as part of a set of properties. You can tell when you are looking at a set of properties that won't allow runtime parameters when you see the Add Another and Bulk Edit Mode options. For example, the Column Type Converter processor uses sets of properties, so you cannot use runtime parameters in those properties:
Step 1. Define a Runtime Parameter
Define runtime parameters in pipeline properties before you call them from the pipeline.
- Text Box
- Parameters called from a text box must be valid for that location.
- Checkboxes
- Parameters called from a checkbox must be set to true or false.
- Drop-down menus
- Parameters called from a drop-down menu must evaluate to a valid key value. Each option in the menu has an associated key value.
Defining a Runtime Parameter
- In the pipeline properties, click the Parameters tab.
-
Using simple or bulk edit
mode, click the Add Another icon and define
the name and the default value for each parameter.
For example, the following pipeline has two runtime parameters defined. The first is the name of a target table, the second is the key value for the Append to Table write mode option:
Step 2. Call the Runtime Parameter
${<parameter name>}
After you define a runtime parameter in pipeline properties, it becomes available through expression completion so you can avoid typographical errors.
To invoke expression completion, start typing the expression: ${ and the closing bracket appears. Then, start typing the name of the parameter that you want to use. Select the correct parameter when it appears.
In the following example, the parameter targetTable
appears after typing
${t
:
The error message resolves after the parameter is selected.
Calling from Text Boxes
To call a runtime parameter in a stage or pipeline property that displays as a text box, you simply specify the parameter name as described above.
target_database
runtime parameter with the
Snowflake Table destination, enter the following syntax for the Table
property:${target_table}
address_column
runtime
parameter: ${str:trim(address_column)}
EU_
prefix. You can specify the prefix in the UDF Name property and
use a UDFname
runtime parameter for the rest of the name, as
follows:EU_${UDFname}
Calling from Checkboxes and Drop-Down Menus
To call a runtime parameter in a stage or pipeline property that displays as a checkbox or drop-down menu, you first must convert the property to a text box.
Click the Use Parameter icon () next to the
checkbox or drop-down menu to convert the property to a text box, and then call the
parameter using the required
syntax.
For example, the following image shows the Snowflake Table destination with two runtime
parameters. The Table property was already a text box. The Write Mode property is a
drop-down menu converted to a text box to call the writeMode
runtime
parameter:
Step 3. Override the Parameter (Optional)
When needed, you can override a runtime parameter in the Control Hub job that runs the pipeline.
For more information about configuring parameters in jobs or job templates, see the Control Hub documentation.