Attributes for AWS Fargate with EKS

When provisioning Data Collectors to AWS Fargate with Amazon Elastic Kubernetes Service (EKS), add the following additional attributes to the deployment YAML specification file:

Required attribute

Add the following required environment variable to avoid having to configure the maximum open file limit on the virtual machines provisioned by AWS Fargate:

- name: SDC_FILE_LIMIT
  value: 0

Optional attribute

Add the following optional resources attribute to define the size of the virtual machines that AWS Fargate provisions. Set the values of the cpu and memory attributes as needed:

resources:
  limits:
    cpu: 500m
    memory: 2G
  requests:
    cpu: 200m
    memory: 2G

For example, to define a deployment only on AWS Fargate with EKS using the public StreamSets Data Collector Docker image, use the following sample YAML specification file. The additional attributes used by AWS Fargate are in bold:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: datacollector-deployment
  namespace: <agentNamespace>
spec:
  replicas: 1
  selector:
    matchLabels:
      app: <deploymentLabel>
  template:
    metadata:
      labels:
        app : <deploymentLabel>
    spec:
      containers:
      - name : datacollector
        image: <privateImage>
        ports:
        - containerPort: 18360
        env:
        - name: HOST
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: PORT0
          value: "18630"
        - name: SDC_FILE_LIMIT
          value: 0
        resources:
           limits:
             cpu: 500m
             memory: 2G
           requests:
             cpu: 200m
             memory: 2G