Resetting the Kinesis Consumer Origin

You can reset the Kinesis Consumer origin to process all available data instead of continuing from the last-saved offset. For general information about resetting the origin, see Resetting the OriginResetting the Origin in the Data Collector documentation.

When you reset the origin for a Kinesis Consumer pipeline, Data Collector resets the origin for every Kinesis Consumer pipeline that uses the same application name.

Note the following requirement and guidelines before resetting the origin:

Requirement: Delete table permission on DynamoDB

Because Kinesis Streams stores offsets in DynamoDB, the IAM user or instance profile associated with the pipeline must have delete table permissions for tables matching the application name used in your pipelines, as follows:

If you use access keys to authenticate with Amazon Web Services, the IAM user associated with the access keys must have delete table permission.
If Data Collector uses instance profiles to authenticate, the instance profile for the EC2 instance must have delete permission.

Guidelines for resetting the origin

To reset the origin for a Kinesis Consumer pipeline, perform the following steps:

Stop the pipeline that you want to reset.
If other pipelines use the same application name, stop all of the pipelines.
Reset the origin: In the pipeline canvas or from the Home page, click the More icon , then click Reset Origin.
When working with multiple pipelines that use the same application name, you can use any of the pipelines. You only need to reset one of them.
Start the pipeline with the reset origin.
If you stopped any related pipelines, restart those pipelines.

For example, say you have three pipelines that read from Kinesis Streams using the Sales application name: NorthSales, CentralSales, and SouthSales. To reset the origin for all pipelines, stop all three pipelines, reset the origin for one of the pipelines, say SouthSales. Then restart the SouthSales pipeline, and then you can restart the other pipelines.