Init Script

When you configure a cloud service provider deployment, you can define an initialization script that Control Hub runs on each newly provisioned instance in your cloud account. The script runs in parallel to the provisioning process.

Note: You cannot define an init script for a self-managed or Kubernetes deployment.

Use the script to set up provisioned instances with additional software as required by your organization. For example, you might use an init script to install required certificates or packages on each provisioned instance.

You define the init script in the Configure Autoscaling Group step of the deployment wizard.

By default, Control Hub automatically sets up each provisioned instance with the software required to run a StreamSets engine. For details, see the following topics:

Script Requirements

The init script must meet the following requirements:
  • The script must be a valid shell script with a maximum size of 8 KB.
  • The script must start with the following line:
  • The script should contain one or more meaningful exit codes.

    The exit code is written to the deployment event logs. You can view the exit code to determine whether the script succeeded. As a best practice, include both success and failure exit codes in the script. Ensure that the failure exit codes indicate the reason for the failure.

    If you do not include an exit code in the script, then the exit code of the last successful script command displays in the deployment event logs.

  • When you configure a deployment, you can directly enter the script or you can upload a saved shell script file and then edit the contents of the script. Uploaded files must use an .sh extension.

Sample Init Script

The following sample init script for an Amazon EC2 deployment installs a package and then attaches an AWS volume to the provisioned EC2 instance. If the volume is successfully attached, the script mounts the volume to a folder, downloads a file from an AWS S3 bucket, and returns a success exit code. If the volume is not successfully attached, the script returns the exit code of the failed attach command.
# Install git
yum -qy install git
# Get the id of this instance
instanceid=$(curl -s
# Attach an existing volume
aws ec2 attach-volume --volume-id vol-01ab2cd345ef6g78 --instance-id ${instanceid} --device /dev/xvdk --region us-west-2
# Check the return code to verify the volume was attached successfully
if [ $? == 0 ];
     # Mount the successfully attached volume
     mkdir /mnt/data
     # it takes some time before the device is available
     sleep 5
     mount /dev/xvdk /mnt/data
     mkdir /mnt/data/test
     # Download a file from S3
     aws s3 cp s3://example-data-bucket/lorem.txt /mnt/data/test
     ls -alR /mnt/data
     # Return exit code 0 to indicate successful execution
     exit 0
     # Return the exit code of the failed attach command
     exit $?

Script Execution

When the deployment provisions a new instance in your cloud account, the init script runs in parallel to the provisioning process. Control Hub does not monitor the success or failure of the script. If the script fails, the deployment continues provisioning new instances.

You can monitor the script execution in the following ways:
View deployment event logs
The deployment event logs include messages about the start and finish of the init script. The finished message includes the script exit code, indicating whether the script succeeded or failed.
For more details about viewing deployment event logs, see Event Logs.
View the init script output
If the script returns a failure exit code, you can view the init script output. Use SSH to connect to the provisioned instance and then locate the script output in the /var/log/cloud-init-output.log file.
Important: To connect to the provisioned instance, you must allow inbound traffic to each instance and configure SSH access for the deployment.