Firewall Configuration

When you deploy StreamSets engines to on-premise or cloud computing machines that reside behind a firewall or in a system that limits access to specific IP addresses, allow the required inbound and outbound traffic to each machine.

Inbound Connections

Control Hub does not directly send requests to machines running StreamSets engines. However, you must configure your firewall to allow the following inbound connections to the machines, depending on the engine type and configuration:

Engine Type Port Protocol Usage
Transformer Transformer port - 19630 by default TCP The Apache Spark cluster must be able to access Transformer at this port number to send the status, metrics, and offsets for running pipelines.
Data Collector and Transformer HTTPS port defined in the engine advanced configuration properties of the deployment TCP When using direct engine REST APIs for browser to engine communication, web browsers must be able to reach engines on the configured HTTPS port number.

In most cases, you can use the default WebSocket tunneling communication method and do not need to allow an inbound connection to the HTTPS port number. For more information, see Engine Communication.

In addition, if you want to use SSH to connect to machines running StreamSets engines, configure your firewall to allow the following inbound connection to the machines. Control Hub does not require SSH access to the machines. However, you might want to enable access for troubleshooting purposes.

Engine Type Port Protocol Usage
Data Collector and Transformer 22 TCP Optionally connect to the machine using SSH.

Outbound Connections

StreamSets engines make outbound connections to the following systems. Ensure that your firewall allows outbound connectivity to these systems.

Important: The StreamSets systems are available in several geographic regions operated by different infrastructure providers. Some of the IP addresses listed below are not within the control of StreamSets and might change. As a best practice, allow the provided DNS names so that your firewall is always up to date.
System DNS and IP Address Port Protocol Usage
StreamSets authentication service cloud.login.streamsets.com

Allow all of the following:

  • 3.133.68.10
  • 3.134.148.19
  • 3.137.15.194
 443 TCP, TLS 1.2 or later User authentication with StreamSets.
StreamSets identity provider Allow all of the following:
  • Current - identitytoolkit.googleapis.com
  • Future - idp.login.streamsets.com
 443 TCP, TLS 1.2 or later Identity management for username/password and social logins.

SAML logins use the StreamSets authentication service.
StreamSets Control Hub Allow all of the following:
  • na01.hub.streamsets.com - 34.145.124.26
  • na02.hub.streamsets.com - 35.237.72.69
  • eu01.hub.streamsets.com - 35.246.207.204
  • eu02.hub.streamsets.com - 35.195.241.100
  • ap01.hub.streamsets.com - 34.87.199.210
 443 TCP, TLS 1.2 or later Engine communication with Control Hub.
StreamSets server that hosts engine installation and stage library files archives.streamsets.com 443 TCP, TLS 1.2 Engine and stage library file downloads.
StreamSets telemetry server Allow all of the following:
  • telemetry.streamsets.com
  • prod-customer-support-bundles.s3.amazonaws.com
 443 HTTPS Telemetry data collection.
External origin and destination systems Depends on the system Depends on the system Depends on the system External system connections so that pipeline stages can process your data.