Firewall Configuration

Firewall Configuration Overview

If you access the StreamSets platform from machines that reside behind a firewall or in a system that limits access to specific DNS names and IP addresses, you must allow the required inbound and outbound traffic to each machine.

The requirements differ, based on whether the machines are used to launch a web browser to access the Control Hub UI or are used to run StreamSets engines.

Browser

When you use a web browser to access the Control Hub UI from a machine that resides behind a firewall or in a system that limits access to specific IP addresses, ensure that your firewall allows outbound connectivity to the following systems.

Important: The StreamSets systems are available in several geographic regions operated by different infrastructure providers. Some of the IP addresses listed below are not within the control of StreamSets and might change. As a best practice, allow the provided DNS names so that your firewall is always up to date.
System DNS and IP Address Port Protocol Usage
StreamSets authentication service cloud.login.streamsets.com

Allow all of the following:

  • 3.133.68.10
  • 3.134.148.19
  • 3.137.15.194
 443 TCP, TLS 1.2 or later User authentication with StreamSets.
StreamSets identity provider identitytoolkit.googleapis.com 443 TCP, TLS 1.2 or later Identity management for username/password and social logins.

SAML logins use the StreamSets authentication service.
StreamSets Control Hub Allow all of the following:
  • na01.hub.streamsets.com - 34.145.124.26
  • na02.hub.streamsets.com - 35.237.72.69
  • na03.hub.streamsets.com - 35.225.184.207
  • eu01.hub.streamsets.com - 35.246.207.204
  • eu02.hub.streamsets.com - 35.195.241.100
  • ap01.hub.streamsets.com - 34.87.199.210
  • me36.hub.streamsets.com - 20.174.57.95
 443 TCP, TLS 1.2 or later Web browser access to the Control Hub UI.
Deployed StreamSets engines Location where engines are running HTTPS port defined in the engine advanced configuration properties of the deployment TCP, TLS 1.2 When using direct engine REST APIs for browser to engine communication, web browsers must be able to directly reach engines.

In most cases, you can use the default WebSocket tunneling communication method and do not need to allow outbound connections from browser machines to the engine machines. For more information, see Engine Communication.

StreamSets Engines

When you deploy StreamSets engines to on-premise or cloud computing machines that reside behind a firewall or in a system that limits access to specific IP addresses, allow the required inbound and outbound traffic to each machine.

Note: These requirements are for deployed engines only. When your organization uses the Transformer for Snowflake engine hosted by StreamSets, you must configure a different Snowflake network policy requirement, as described in the Transformer for Snowflake documentation.

Inbound Connections

Control Hub does not directly send requests to machines running StreamSets engines. However, you must configure your firewall to allow the following inbound connections to the machines, depending on the engine type and configuration:

Engine Type Port Protocol Usage
Transformer Transformer port - 19630 by default TCP The Apache Spark cluster must be able to access Transformer at this port number to send the status, metrics, and offsets for running pipelines.
All HTTPS port defined in the engine advanced configuration properties of the deployment TCP When using direct engine REST APIs for browser to engine communication, web browsers must be able to reach engines on the configured HTTPS port number.

In most cases, you can use the default WebSocket tunneling communication method and do not need to allow an inbound connection to the HTTPS port number. For more information, see Engine Communication.

In addition, if you want to use SSH to connect to machines running StreamSets engines, configure your firewall to allow the following inbound connection to the machines. Control Hub does not require SSH access to the machines. However, you might want to enable access for troubleshooting purposes.

Engine Type Port Protocol Usage
All 22 TCP Optionally connect to the machine using SSH.

Outbound Connections

StreamSets engines make outbound connections to the following systems. Ensure that your firewall allows outbound connectivity to these systems.

Important: The StreamSets systems are available in several geographic regions operated by different infrastructure providers. Some of the IP addresses listed below are not within the control of StreamSets and might change. As a best practice, allow the provided DNS names so that your firewall is always up to date.
System DNS and IP Address Port Protocol Usage
StreamSets Control Hub Allow all of the following:
  • na01.hub.streamsets.com - 34.145.124.26
  • na02.hub.streamsets.com - 35.237.72.69
  • na03.hub.streamsets.com - 35.225.184.207
  • eu01.hub.streamsets.com - 35.246.207.204
  • eu02.hub.streamsets.com - 35.195.241.100
  • ap01.hub.streamsets.com - 34.87.199.210
  • me36.hub.streamsets.com - 20.174.57.95
 443 TCP, TLS 1.2 or later Engine communication with Control Hub.
StreamSets server that hosts engine installation and stage library files archives.streamsets.com 443 TCP, TLS 1.2 Engine and stage library file downloads.
StreamSets telemetry server Allow all of the following:
  • telemetry.streamsets.com
  • prod-customer-support-bundles.s3.amazonaws.com
 443 HTTPS Telemetry data collection.
External origin and destination systems Depends on the system Depends on the system Depends on the system External system connections so that pipeline stages can process your data.
In addition, if you deploy a StreamSets engine as a Docker image, configure your firewall to allow the following outbound connections to Docker Hub. Outbound connections to Docker Hub are not required when you deploy a StreamSets engine from a tarball file.
System DNS Port Protocol Usage
Docker Hub Allow all of the following:
  • hub.docker.com
  • registry-1.docker.io
 443 TCP, TLS 1.2 or later Pull StreamSets engine images from Docker Hub.