Administer SDC Edge

Configuring SDC Edge

You can customize SDC Edge by editing the SDC Edge configuration file, <SDCEdge_home>/etc/edge.conf.

Use a text editor to edit the configuration file. To enable the changes, shut down and then restart SDC Edge.

The SDC Edge configuration file includes the following log property:

Execution Property	Description
log-dir	Directory where SDC Edge writes the log file. Default is `<SDCEdge_home>/log/`. Be sure to uncomment the property if you change the value.

The SDC Edge configuration file includes the following execution property:

Execution Property	Description
max-batch-size	Maximum number of records included in a batch when the pipeline runs. Default is 1000.

The SDC Edge configuration file includes the following process property:

Process Property	Description
process-metrics-capture-interval	Frequency in milliseconds that metrics about the SDC Edge process, such as memory usage, are refreshed. Default is -1, which means that the metrics are refreshed when the HTTP REST API is called.

The SDC Edge configuration file includes the following HTTP properties:

HTTP Property Description

enabled

HTTP Property	Description
enabled	Specifies whether a web server runs inside the SDC Edge process. When enabled, you can use the Data Collector UI or REST API to manage edge pipelines deployed to SDC Edge. Managing edge pipelines includes previewing, validating, starting, stopping, and monitoring pipelines as well as resetting the origin for pipelines. Set to true in a development environment to simplify edge pipeline development. Set to false in a production environment to secure the edge pipelines deployed to SDC Edge. When false, you manage edge pipelines using Control Hub or by starting pipelines when you start SDC Edge. Default is true.
bind-address	Address that SDC Edge binds to. The address is localhost and the defined port number. Default port number is 18633.
base-http-url	SDC Edge URL used to communicate with Control Hub. Default is `http://<hostname>:<port>` where: `<hostname>` is determined by SDC Edge using the following command: `hostname -f`. `<port>` is the port number defined in the `bind-address` property. You might want to change the default value to more easily identify multiple Edge Data Collectors registered with Control Hub. Be sure to uncomment the property if you change the value.

Specifies whether a web server runs inside the SDC Edge process. When enabled, you can use the Data Collector UI or REST API to manage edge pipelines deployed to SDC Edge. Managing edge pipelines includes previewing, validating, starting, stopping, and monitoring pipelines as well as resetting the origin for pipelines.

Set to true in a development environment to simplify edge pipeline development.

Set to false in a production environment to secure the edge pipelines deployed to SDC Edge. When false, you manage edge pipelines using Control Hub or by starting pipelines when you start SDC Edge.

Default is true.

bind-address

Address that SDC Edge binds to. The address is localhost and the defined port number.

Default port number is 18633.

base-http-url

SDC Edge URL used to communicate with Control Hub.

Default is http://<hostname>:<port> where:

<hostname> is determined by SDC Edge using the following command: hostname -f.
<port> is the port number defined in the bind-address property.

You might want to change the default value to more easily identify multiple Edge Data Collectors registered with Control Hub. Be sure to uncomment the property if you change the value.

The SDC Edge configuration file includes the following Control Hub properties that define how SDC Edge works with Control Hub:

Control Hub Property	Description
enabled	Specifies whether the SDC Edge is enabled to work with Control Hub. Default is false.
base-url	URL to access Control Hub. Set to `https://cloud.streamsets.com`.
app-auth-token	Authentication token for this SDC Edge instance when enabled to work with Control Hub.
job-labels	Labels to assign to this SDC Edge. Use labels to group Edge Data Collectors registered with Control Hub. To assign multiple labels, enter a comma-separated list of labels. Default is "all", which you can use to run a job on all registered instances of SDC Edge.
events-recipient	Name of the internal Control Hub messaging queue to which SDC Edge sends events. Do not change this value.
ping-frequency	Frequency in milliseconds that SDC Edge notifies Control Hub that it is running. Default is 6,000.
status-events-interval	Frequency in milliseconds that SDC Edge informs Control Hub of the following information: Status of all local and published pipelines that are running on this SDC Edge. Performance information for this SDC Edge - including CPU load and memory usage. Default is 60,000.

Starting SDC Edge

You can start SDC Edge manually or as a service:

Manual start

To manually start SDC Edge, run the following command from the SDC Edge home directory:

bin/edge

Note: In a test or development environment, you can start SDC Edge so that it skips verifying trusted certificates. To do so, run the following command from the SDC Edge home directory to manually start SDC Edge:

bin/edge
                -insecureSkipVerify

. StreamSets highly recommends that you configure SDC Edge to verify trusted certificates in a production environment.

To modify the log level when you manually start SDC Edge, see Logs.

Service start

Before you can start SDC Edge as a service, you must register SDC Edge as a service on the edge device.

To start SDC Edge as a service, run the following command as an administrator from the SDC Edge home directory:

bin/edge -service start

For example, you might run the following command on Linux:

sudo opt/local/streamsets-datacollector-edge/bin/edge -service start

Shutting Down SDC Edge

You can shut down and then manually launch SDC Edge to apply changes to the SDC Edge configuration file.

Shut down SDC Edge based on whether SDC Edge is started manually or as a service:

Manual start

To shut down SDC Edge, type Ctrl+C in the command prompt.

Service start

To shut down SDC Edge started as a service, run the following command as an administrator from the SDC Edge home directory:

bin/edge -service stop

For example, you might run the following command on Linux:

sudo opt/local/streamsets-datacollector-edge/bin/edge -service stop

Restarting SDC Edge

When SDC Edge runs as a service, you can restart SDC Edge to apply changes to the SDC Edge configuration file. During the restart process, SDC Edge shuts down and then automatically restarts.

To restart SDC Edge, run the following command as an administrator from the SDC Edge home directory:

bin/edge -service restart

For example, you might run the following command on Linux:

sudo opt/local/streamsets-datacollector-edge/bin/edge -service restart

Logs

By default, SDC Edge writes log messages at the info severity level to the <SDCEdge_home>/log/edge.log file. To view the logs, simply open the edge.log file in a text editor.

To change the default log file directory, modify the SDC Edge configuration file.

You can modify the log level or send the log messages to the console when you manually start SDC Edge. You cannot modify this log information when SDC Edge runs as a service.

To modify logs, use the appropriate command:

To set the log level to debug for troubleshooting, run the following command from the SDC Edge home directory:
```
bin/edge -debug
```
To send the log messages to the console instead of to the edge.log file, run the following command from the SDC Edge home directory:
```
bin/edge -logToConsole
```