Configuring SDC Edge

You can customize SDC Edge by editing the SDC Edge configuration file, <SDCEdge_home>/etc/edge.conf.

Use a text editor to edit the configuration file. To enable the changes, shut down and then restart SDC Edge.

The SDC Edge configuration file includes the following log property:
Execution Property Description
log-dir Directory where SDC Edge writes the log file.

Default is <SDCEdge_home>/log/. Be sure to uncomment the property if you change the value.

The SDC Edge configuration file includes the following execution property:
Execution Property Description
max-batch-size Maximum number of records included in a batch when the pipeline runs.

Default is 1000.

The SDC Edge configuration file includes the following process property:
Process Property Description
process-metrics-capture-interval Frequency in milliseconds that metrics about the SDC Edge process, such as memory usage, are refreshed.

Default is -1, which means that the metrics are refreshed when the HTTP REST API is called.

The SDC Edge configuration file includes the following HTTP properties:
HTTP Property Description
enabled Specifies whether a web server runs inside the SDC Edge process. When enabled, you can use the Data Collector UI or REST API to manage edge pipelines deployed to SDC Edge. Managing edge pipelines includes previewing, validating, starting, stopping, and monitoring pipelines as well as resetting the origin for pipelines.

Set to true in a development environment to simplify edge pipeline development.

Set to false in a production environment to secure the edge pipelines deployed to SDC Edge. When false, you manage edge pipelines using Control Hub or by starting pipelines when you start SDC Edge.

Default is true.

bind-address Address that SDC Edge binds to. The address is localhost and the defined port number.

Default port number is 18633.

base-http-url SDC Edge URL used to communicate with Control Hub.
Default is http://<hostname>:<port> where:
  • <hostname> is determined by SDC Edge using the following command: hostname -f.
  • <port> is the port number defined in the bind-address property.

You might want to change the default value to more easily identify multiple Edge Data Collectors registered with Control Hub. Be sure to uncomment the property if you change the value.

The SDC Edge configuration file includes the following Control Hub properties that define how SDC Edge works with Control Hub:
Control Hub Property Description
enabled Specifies whether the SDC Edge is enabled to work with Control Hub.

Default is false.

base-url URL to access Control Hub.

Set to https://cloud.streamsets.com.

Set to the Control Hub URL provided by your system administrator. For example, https://<hostname>:18631.

app-auth-token Authentication token for this SDC Edge instance when enabled to work with Control Hub.
job-labels Labels to assign to this SDC Edge. Use labels to group Edge Data Collectors registered with Control Hub. To assign multiple labels, enter a comma-separated list of labels.

Default is "all", which you can use to run a job on all registered instances of SDC Edge.

events-recipient Name of the internal Control Hub messaging queue to which SDC Edge sends events.

Do not change this value.

ping-frequency Frequency in milliseconds that SDC Edge notifies Control Hub that it is running.

Default is 6,000.

status-events-interval Frequency in milliseconds that SDC Edge informs Control Hub of the following information:
  • Status of all local and published pipelines that are running on this SDC Edge.
  • Performance information for this SDC Edge - including CPU load and memory usage.

Default is 60,000.