DDL Schema Format

To use DDL to specify a custom schema, use a comma-separated list of field names and data types. Here's an example of the basic structure:

<first field name> <data type>, <second field name> <data type>, <third field name> <data type>

To define a List field, use the Array data type and specify the data types of the subfields as follows:

<list field name> Array <subfields data type>

To define a Map field, use Struct data type, then specify the names and types of the subfields as follows:

<map field name> Struct < <first subfield name>:<data type>, <second subfield name>:<data type> >

Tip: You can use back ticks ( ` ) to escape field names that can be mistaken for reserved words, such as `count`.

Example

The following DDL custom schema includes, in order, a String, Boolean, Map, and List field:

TransactionID String, Verified Boolean, User Struct <ID:Integer, Name:String>, Items Array <String>

When processing the following JSON data:

{"Verified":true,"User":{"ID":23005,"Name":"Marnee Gehosephat"},"Items":["T-35089", "M-00352", "Q-11044"],"TransactionID":"G-23525-3350"}

The origin generates the following record:

Notice the User Map field with the Integer and String subfields and the Items List field with String subfields. Also note that the order of the fields now match the order in the custom schema.