Offset Field and Initial Offset

MongoDB uses the offset field to track the data to read. By default, the MongoDB origin uses the _id field as the offset field.

You can use a nested offset field, such as o._id. Or, you can use any Object ID, date, or string field as the offset field. The results for using any field besides the default _id field are not guaranteed.

When you use a date or Object ID field, specify a timestamp to use as the initial offset. Object ID fields include an embedded timestamp that the origin uses to determine where in the collection to begin reading. When you define the initial offset for a date or Object ID field, use the following format:

YYYY-MM-DD HH:mm:ss

When you use a string field, specify the initial string to use as the initial offset.

Note: If you change the offset field type for the origin after the pipeline runs and then stops, you must reset the origin before you can run the pipeline again.