Web Client
Available when using an authoring Data Collector version 5.11.0 or later.
- Data Collector 5.11.0 or later - Requires the OkHttp Web Client stage library, streamsets-datacollector-webclient-impl-okhttp-lib.
For a description of the Web Client connection properties, see Web Client Connection Properties.
Engine | Stages |
---|---|
Data Collector 5.11.0 or later |
|
Web Client Connection Properties
When creating a Web Client connection, configure the following properties on the Connection tab:
Connection Property | Description |
---|---|
Base URL | Fixed base URL or request endpoint of the requested resource. For
example, http://localhost:8000 or
http://localhost:8000/basic/authenticate . |
Authentication Scheme |
Determines the authentication type used to connect to the server:
|
Keystore Management | Determines the authentication
and encryption used to connect to the HTTP server.
|
Keystore Location |
Where to load the keystore from:
Required for manual keystore management. |
Keystore File |
Path to the local keystore file. Enter an absolute path to the file or enter the following expression to define the file stored in the Data Collector resources directory:
Available for manual keystore management. |
Keystore Type | Type of keystore to use. Use one of the following types:
Default is Java Keystore File (JKS). |
Keystore Password |
Password to the keystore file. A password is optional, but recommended. |
Private Key | Private key used in the remote keystore. Enter a credential function that returns the key or enter the contents of the key. |
Certificate Chain | Each PEM certificate used in the remote keystore. Enter a
credential function that returns the certificate or enter the
contents of the certificate. Using simple or bulk edit mode, click the Add icon to add additional certificates. |
Keystore Key Algorithm |
Algorithm to manage the keystore. Default is SunX509. |
Truststore File |
Path to the local truststore file. Enter an absolute path to the file or enter the following expression to define the file stored in the Data Collector resources directory:
By default, no truststore is used. |
Truststore Type |
Type of truststore to use. Use
one of the following types:
Default is Java Keystore File (JKS). |
Truststore Password |
Password to the truststore file. A password is optional, but recommended. |
Truststore Key Algorithm |
Algorithm to manage the truststore. Default is SunX509. Required for manual keystore management. |
Default Protocol Versions |
Use only modern default secure protocol versions. Available for manual keystore management. |
Default Cipher Suites |
Use only modern default cipher suites. Available for manual keystore management. |
Use Proxy Server | Enables using a proxy server to connect to the system. |
Proxy Server | Proxy server endpoint. |
Proxy User | User name for proxy credentials. |
Proxy Password | Password for proxy credentials. |
Test Endpoint | Test endpoint of the URL of the requested resource. For example,
/basic/authenticate .The test endpoint is used only to verify the connection URL and credentials, and is not used in stages that use the connection. |
Test Method | HTTP method to apply to the requested resource. The test method is used only to verify the connection URL and credentials, and is not used in stages that use the connection. |
Connection Property | Description |
---|---|
User |
Authentication username. |
Password |
Authentication password. |
Bearer Property | Description |
---|---|
Token | Authentication token to access the requested resource. |
OAuth 1 Property | Description |
---|---|
Consumer Key |
Name of the OAuth consumer key. |
Consumer Secret |
OAuth consumer secret. |
Access Token |
OAuth 1.0 access token. |
Token Secret |
OAuth 1.0 token secret. |
OAuth 2 Property | Description |
---|---|
Grant Type |
Grant type required by the HTTP service. |
Token Endpoint |
URL to request the access token. |
Client ID |
Client ID that the HTTP service uses to identify the HTTP client. Enter for the client credentials grant that uses a client ID and secret for authentication. Or, for the resource owner password credentials grant that requires a client ID and secret. Required for the Client Credentials grant. |
Client Secret |
Client secret that the HTTP service uses to authenticate the HTTP client. Enter for the client credentials grant that uses a client ID and secret for authentication. Or, for the resource owner password credentials grant that requires a client ID and secret. Required for the Client Credentials grant. |
Signing Algorithm | Algorithm used to sign the access token. Required for the Access Token grant. |
Signing Key | Private key that the selected signing algorithm uses to sign the
access token. Required for the Access Token grant using a signing algorithm. |
Token Headers | Headers to include in the access token Available for the Access Token grant. |
Token Claims | Claims to include in the access token. Specify in JSON format. Enter
each claim required to obtain an access token. You can include the
expression language in the token claims. For example, to request an access token to read from Google service accounts, enter the following claims with the appropriate values:
Required for the Access Tokens grant. |
Owner User | Resource owner user name. Required for the Owner Credentials grant. |
Owner Password | Resource owner password. Required for the Owner Credentials grant. |
Owner Client ID | Resource owner client ID. Available for the Owner Credentials grant. |
Owner Client Secret | Resource owner client secret. Required for the Owner Credentials grant. |
Additional Parameters | Optional parameters to send to the token endpoint when requesting
an access token. For example, you can define the OAuth 2
scope request parameter.Click the Add icon to add additional key-value pairs. |
Request Property | Description |
---|---|
Request Headers |
Common headers to include in the request. Using simple or bulk edit mode, click Add to add additional common headers. For information about simple and bulk edit mode, see Simple and Bulk Edit Mode in the Data Collector documentation. |