Generated Records

The MySQL Binary Log origin generates one record for each transaction recorded in the binary logs. The record includes fields and record header attributes that contain the operation type, change data capture information, and the changed data.

Note: If a transaction recorded in the binary logs is rolled back in the MySQL database, MySQL retains the original transaction in the log and then also records the rolled back transaction. As a result, the MySQL Binary Log origin can process the changes recorded in the original transaction and in the rolled back transaction.

Each generated record includes the following information:

CRUD operation type

The CRUD operation type is stored in the Type record field and also in the sdc.operation.type record header attribute.

The Type field contains one of the following string values: INSERT, DELETE, or UPDATE.

The sdc.operation.type record header attribute contains one of the following numeric values:

1 for insert data
2 for delete data
3 for update data

You can use either the Type record field or the sdc.operation.type record header attribute to write records to the destination system. CRUD-enabled destinations such as JDBC Producer and Kudu use the sdc.operation.type header attribute. For an overview of Data Collector CDC and a list of CRUD-enabled destinations, see Processing Changed Data.

Change data capture information

The change data capture information is stored in the following record fields:

BinLogFilename - when GTID is disabled
BinLogPosition- when GTID is disabled
SeqNo - when GTID is enabled
GTID - when GTID is enabled
Table
ServerId
Database
Timestamp
Offset

New data

Fields that contain new data to be inserted or updated appears in a Data map field.

Changed data

Fields that contain old data to be updated, or fields in a delete record appear in an OldData map field.

For example, the origin generates the following record when it reads from binary log files for a MySQL server with GTID enabled: