Commands¶
The stf script installed as part of this project makes available a number of commands. For full
details, the usage page is the definitive resource:
$ stf -h
usage: stf [-h] [--version] [-v] [--docker-image image] [--release {3.x,4.x}]
[--docker-image-dont-pull] [--docker-network network]
[--sdc-resources-directory dir]
[--testframework-config-directory dir] [--env-var var]
[--streamsets-sdk-directory dir] [--testframework-directory dir]
{test,benchmark,shell,build,stub,start,info} ...
positional arguments:
{benchmark,build,info,shell,start,stub,test}
Test Framework subcommands
benchmark Run performance metrics
build Build STF Docker images
info Sub-system info
shell Open a shell within the STF container
start Start a sub-system
stub Generate a Test Stub
test Run STF tests
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbose Be noisier (default: False)
--docker-image image Docker image to use for the STF container (default:
None)
--release {3.x,4.x} Release version to use the appropriate STF image
(default: None)
--docker-image-dont-pull
Don't pull STF Docker image (default: False)
--docker-network network
Docker network to which to attach the STF container
(default: cluster)
--sdc-resources-directory dir
A directory containing resources to mount into the SDC
container (default: None)
--testframework-config-directory dir
A directory containing STF configuration files to
mount into the STF container (default:
/Users/username/.streamsets/testframework)
--env-var var One or more environment variables to set. Retrieves
host environment variable(s) as given by this argument
unless the value is passed with "=", in which case it
is used as environment variable value (default: None)
advanced arguments:
Stuff for StreamSets devs, mostly
--streamsets-sdk-directory dir
Path to the an `sdk` folder containing source code of
the StreamSets SDK for Python (default: None)
--testframework-directory dir
A testframework directory to mount into the STF
container (for use when making STF changes that don't
require a rebuild of the image) (default: None)
Run 'stf <command> -h' for more information on a specific subcommand.
stf test¶
Tests are executed by cd-ing into a directory containing the test source and then running
the stf test command. Any arguments after test are passed into the Test Framework
container as arguments to an invocation of the pytest command.
stf shell¶
To get an interactive shell within the Test Framework environment, run stf shell. This
command is particularly useful during the test development process, where a user may want to
explore the streamsets.testframework package from an interpreter.
stf build¶
StreamSets publishes Docker images of StreamSets Data Collector and Data Collector stage libraries
to Docker Hub. There are a number of images, however, which we are not allowed to distribute
because of the terms of the underlying binaries’ EULA. stf build is the command to use
to create these images locally.
stf start¶
The stf start command leverages the Docker orchestration capabilities of the StreamSets Test
Framework to start StreamSets Data Collector. As an example, to start version 5.1.0 of
StreamSets Data Collector with the AWS and JDBC stage libraries preinstalled and bind it to an SCH, you could run
$ stf start sdc --version 5.1.0 --stage-lib aws jdbc --aster-server-url $ASTER_SERVER_URL --sch-credential-id $CRED_ID --sch-token $CRED_TOKEN
2022-08-09 04:17:24 PM [INFO] [streamsets.testframework.cli] Pulling Docker image streamsets/testframework-4.x:latest ...
2022-08-09 04:19:35 PM [INFO] [streamsets.testframework.sdc] Starting StreamSets Data Collector 5.1.0 container ...
2022-08-09 04:19:51 PM [INFO] [streamsets.testframework.sdc] SDC 5.1.0 (git:8225991372db2da5c962d0e0d5f0b83436995e4f) is now running. SDC container (nwsrxuhn.cluster:18630) can be followed along on ASTER_SERVER_URL is the page used to log into your SCH instance, and CRED_ID and CRED_TOKEN are the API
Credentials obtained in the Manage > API Credentials section of the SCH.
