Deployment Sample

Define only a deployment in the YAML specification file when creating a deployment for one or more execution Data Collectors that can be manually scaled.

The following sample YAML specification file defines only a deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: datacollector-deployment
  namespace: <agentNamespace>
spec:
  replicas: 1
  selector:
    matchLabels:
      app: <deploymentLabel>
  template:
    metadata:
      labels:
        app : <deploymentLabel>
        kerberosEnabled: true
        krbPrincipal: <KerberosUser>
    spec:
      containers:
      - name : datacollector
        image: <privateImage>
        ports:
        - containerPort: 18360
        volumeMounts:
         - name: krb5conf
           mountPath: /etc/krb5.conf
           subPath: krb5.conf
           readOnly: true
        env:
        - name: HOST
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: PORT0
          value: "18630"
      imagePullSecrets:
      - name: <imagePullSecrets>
      volumes:
      - name: krb5conf
        secret:
          secretName: krb5conf

If not enabling Kerberos authentication, you'd remove the following Kerberos attributes from the sample file:

...
      kerberosEnabled: true
      krbPrincipal: <KerberosUser>
...
      volumeMounts:
         - name: krb5conf
           mountPath: /etc/krb5.conf
           subPath: krb5.conf
           readOnly: true
...
      volumes:
     - name: krb5conf
       secret:
         secretName: krb5conf

Replace the following variables in the sample file with the appropriate attribute values:


Variable	Description
agentNamespace	Namespace used for the Provisioning Agent that manages this deployment.
deploymentLabel	Label for this deployment. Must be unique for all deployments managed by the Provisioning Agent.
KerberosUser	User for the Kerberos principal when enabling Kerberos authentication. This attribute is optional. If you remove this attribute, the Provisioning Agent uses `sdc` as the Kerberos user. The Provisioning Agent creates a unique Kerberos principal for each deployed Data Collector container using the following format: `<KerberosUser>/<host>@<realm>`. The agent determines the host and realm to use, creates the Kerberos principal, and generates the keytab for that principal. For example, if you define the `KerberosUser` attribute as `marketing` and the Provisioning Agent deploys two Data Collector containers, the agent creates the following Kerberos principals: `marketing/10.60.1.25@EXAMPLE.COM marketing/10.60.1.26@EXAMPLE.COM`
privateImage	Path to your private Data Collector Docker image stored in your private repository. Or, if using the public StreamSets Data Collector Docker image, modify the attribute as follows: `image: streamsets/datacollector:<version>` Where `<version>` is the Data Collector version. For example: `image: streamsets/datacollector:4.1.0`
imagePullSecrets	Pull secrets required for the private image stored in your private repository. If using the public StreamSets Data Collector Docker image, remove these lines.