Directory Path

The File destination writes files to a directory in HDFS or a local file system.

To specify the directory, enter the path to the directory. The format of the directory path depends on the file system that you want to write to:

HDFS
To write files to HDFS, use the following format for the directory path:
hdfs://<authority>/<path>
For example, to write to the /user/hadoop/files directory on HDFS, enter the following path:
hdfs://nameservice/user/hadoop/files
Local file system
To write files to a local file system, use the following format for the directory path:
file:///<directory>
For example, to write to the /Users/transformer/output directory on the local file system, enter the following path:
file:///Users/transformer/output

The destination creates the directory if it doesn't exist. The user that Transformer uses to launch the Spark application must have write access to the root of the specified directory path. For a cluster pipeline, the user that launches the Spark application depends on the cluster type configured for the pipeline. For a local pipeline, the user that launches the Spark application is the user that runs the Transformer process.