Commands

The stf script installed as part of this project makes available a number of commands. For full details, the usage page is the definitive resource:

$ stf -h
usage: stf [-h] [--version] [-v] [--docker-image image] [--release {3.x,4.x}]
           [--docker-image-dont-pull] [--docker-network network]
           [--sdc-resources-directory dir]
           [--testframework-config-directory dir] [--env-var var]
           [--streamsets-sdk-directory dir] [--testframework-directory dir]
           {test,benchmark,shell,build,stub,start,info} ...

positional arguments:
  {benchmark,build,info,shell,start,stub,test}
                        Test Framework subcommands
    benchmark           Run performance metrics
    build               Build STF Docker images
    info                Sub-system info
    shell               Open a shell within the STF container
    start               Start a sub-system
    stub                Generate a Test Stub
    test                Run STF tests

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v, --verbose         Be noisier (default: False)
  --docker-image image  Docker image to use for the STF container (default:
                        None)
  --release {3.x,4.x}   Release version to use the appropriate STF image
                        (default: None)
  --docker-image-dont-pull
                        Don't pull STF Docker image (default: False)
  --docker-network network
                        Docker network to which to attach the STF container
                        (default: cluster)
  --sdc-resources-directory dir
                        A directory containing resources to mount into the SDC
                        container (default: None)
  --testframework-config-directory dir
                        A directory containing STF configuration files to
                        mount into the STF container (default:
                        /Users/username/.streamsets/testframework)
  --env-var var         One or more environment variables to set. Retrieves
                        host environment variable(s) as given by this argument
                        unless the value is passed with "=", in which case it
                        is used as environment variable value (default: None)

advanced arguments:
  Stuff for StreamSets devs, mostly

  --streamsets-sdk-directory dir
                        Path to the an `sdk` folder containing source code of
                        the StreamSets SDK for Python (default: None)
  --testframework-directory dir
                        A testframework directory to mount into the STF
                        container (for use when making STF changes that don't
                        require a rebuild of the image) (default: None)

Run 'stf <command> -h' for more information on a specific subcommand.

stf test

Tests are executed by cd-ing into a directory containing the test source and then running the stf test command. Any arguments after test are passed into the Test Framework container as arguments to an invocation of the pytest command.

stf shell

To get an interactive shell within the Test Framework environment, run stf shell. This command is particularly useful during the test development process, where a user may want to explore the streamsets.testframework package from an interpreter.

stf build

StreamSets publishes Docker images of StreamSets Data Collector and Data Collector stage libraries to Docker Hub. There are a number of images, however, which we are not allowed to distribute because of the terms of the underlying binaries’ EULA. stf build is the command to use to create these images locally.

stf start

The stf start command leverages the Docker orchestration capabilities of the StreamSets Test Framework to start StreamSets Data Collector. As an example, to start version 5.1.0 of StreamSets Data Collector with the AWS and JDBC stage libraries preinstalled and bind it to an SCH, you could run


$ stf start sdc --version 5.1.0 --stage-lib aws jdbc --aster-server-url $ASTER_SERVER_URL --sch-credential-id $CRED_ID --sch-token $CRED_TOKEN
2022-08-09 04:17:24 PM [INFO] [streamsets.testframework.cli] Pulling Docker image streamsets/testframework-4.x:latest ...
2022-08-09 04:19:35 PM [INFO] [streamsets.testframework.sdc] Starting StreamSets Data Collector 5.1.0 container ...
2022-08-09 04:19:51 PM [INFO] [streamsets.testframework.sdc] SDC 5.1.0 (git:8225991372db2da5c962d0e0d5f0b83436995e4f) is now running. SDC container (nwsrxuhn.cluster:18630) can be followed along on 
2022-08-09 04:19:51 PM [INFO] [streamsets.testframework.sdc] SDC info file available at /Users/username/.streamsets/testframework/datacollector_system_info.json
2022-08-09 04:19:51 PM [INFO] [streamsets.testframework.sdc] SDC ID: 

ASTER_SERVER_URL is the page used to log into your SCH instance, and CRED_ID and CRED_TOKEN are the API Credentials obtained in the Manage > API Credentials section of the SCH.