Installation¶
Prerequisites¶
Prior to installing the StreamSets Test Framework, a number of prerequisites must be satisfied:
Python¶
The StreamSets Test Framework has been tested against Python 3.6 and this is the version we
recommend be used when invoking stf commands. If your machine has a different version of Python 3+
installed, feel free to try it out, but if things seem broken, we recommend using pyenv to
quickly and non-destructively switch to a different version of Python.
Docker¶
Any recent version of Docker should work with the StreamSets Test Framework.
Note
If using Docker for Mac, the StreamSets Test Framework requires that you go into your Docker
Preferences and deselect the “Securely store Docker logins in macOS keychain” if you want to
pull any Docker images stored on private repositories. This is needed in order to allow credentials
entered via docker login to be passed into the STF container at runtime.
Installation from PyPI¶
To install the most recent stable release of the project, use your Python 3 installation’s instance of pip:
$ pip3 install streamsets-testframework
To do a simple check to verify that the installation was successful, simply run
$ stf --version
stf 4.0.0
Docker images¶
The stf command launches most of its sub-commands in a Docker container started from the
streamsets/testframework-4.x:latest image on Docker Hub.
Many STF tests of StreamSets Data Collector support the automatic starting and stopping of SDC containers from Docker images. These tests will also handle orchestration of stage libraries, external libraries and environment libraries, as required by the tests, which are also packaged as Docker images. As part of our automated build process, StreamSets creates and pushes to Docker Hub images of StreamSets Data Collector and its stage libraries for stable releases and nightly builds. Because of license restrictions, though, there are a number of Docker images required to run certain STF tests that we do not distribute. These include images containing the JDBC drivers as part of external libraries and images containing libraries to interact with an environment.
To build external libraries, simply run the stf build extras command:
$ stf build extras
2018-09-19 07:33:41 AM [INFO] [streamsets.testframework.sdc] 2018-09-21 10:23:19 AM [INFO] [streamsets.testframework.cli.build] This will build images for version 3.4.0 ...
...
Similarly, to build environment libraries, run stf build environments:
$ stf build environments
2018-09-19 07:33:41 AM [INFO] [streamsets.testframework.sdc] 2018-09-21 10:23:19 AM [INFO] [streamsets.testframework.cli.build] This will build images for version 3.4.0 ...
...
