JavaScript Scripting

record

An object that contains fields and values to process. Create new record objects with the sdc.createRecord(<String record ID>) method. The methods available to the object depend on the origin configuration. You can configure the record type to be native objects or Data Collector records.

For more information see Accessing Record Details.

batch

An object that collects records to process together. Create new batch objects with the sdc.createBatch() method. The object includes the following methods:

• add(<record>) - Appends a record to the batch.
• add(<record[]>) - Appends a list of records to the batch.
• addError(<record>,<String message>) - Appends an error record to the batch. The appended error record contains the associated error message.
• addEvent(<event record>) - Appends an event to the batch. Verify that the stage enables event generation before implementing event methods.
• size() - Returns the number of records in the batch.
• process(<String entity>, <String offset>) - Processes the batch and commits the offset for the named entity.
• getSourceResponseRecords() - After processing a batch, retrieves any response records returned by downstream stages.

log

An object that writes messages to the log4j log. Use sdc.log to access the object configured for the stage. The object includes methods that correspond to the level in the log file:

• info(<message template>, <arguments>...)
• warn(<message template>, <arguments>...)
• error(<message template>, <arguments>...)
• trace(<message template>, <arguments>...)

The message template can contain positional variables, indicated by curly brackets: { }. When writing messages, the method replaces each variable with the argument in the corresponding position - that is, the method replaces the first { } occurrence with the first argument, and so on.

sdc

A wrapper object that accesses the constants, methods, and objects available to the user script.

The sdc object contains the following constants:

• lastOffsets - Dictionary that contains the last-saved offset for each entity. Use at the start of the script to read the last value associated with a successfully processed batch.
  Note: The constant does not update while the pipeline runs.
• batchSize - Number of records to create in a single batch. Configured on the Performance tab with the Batch Size property.
• numThreads - Number of threads to run simultaneously. Configured on the Performance tab with the Number of Threads property.
• userParams - Dictionary that contains the script parameters and values configured on the Advanced tab with the Parameters in Script property.

The sdc object contains the following methods:

• createBatch() - Returns a new batch.
• createRecord(<String record ID>) - Returns a new record with the passed ID. Pass a string that uniquely identifies the record and includes enough information to track the record source.
• isStopped() - Returns a Boolean value that indicates whether the pipeline has been stopped.
• isPreview() - Returns a Boolean value that indicates whether the pipeline is in preview mode.
• getFieldNull(<record>, <String field path>) - Returns one of the following:
  • The value of the field at the specified path if the value is not null
  • The null object defined for the field type, such as NULL_INTEGER or NULL_STRING, if the value is null
  • The unassigned null object NULL if there is no field at the specified path
• createMap(<Boolean list-map>) - Returns a map for use as a field in a record. Pass true to create a list-map field, or false to create a map field.
• createEvent(<String type>, <Integer version>) - Returns a new event record with the specified event type and version. Verify that the stage enables event generation before implementing event methods.

• Create a String field named new and set its value to new-value.
• Update the existing field named old to set the value of the attr attribute to attr-value.

Data type of null values

You can associate null values with a data type. For example, if the script assigns a null value to an Integer field, the field is returned to the pipeline as an integer with a null value.

Use constants in the JavaScript code to create a new field of a specific data type with a null value. For example, you can create a new String field with a null value by assigning the NULL_STRING type constant to the field as follows:

records[i].value.new_field = sdc.NULL_STRING

Date fields

Use the String data type to create a new field to store a date with a specific format. For example, the following sample code creates a new String field that stores the current date using the format YYYY-MM-dd:

 // Define a date object to record the current date
var date = new Date();
var curBatch = sdc.createBatch();

for(var i = 0; i < records.length; i++) {
 try {
   // Create a string field to store the current date with the specified format
   records[i].value.date = date.getFullYear()+ "-" + date.getMonth() + "-" + date.getDate();
   // Add record to the current batch
   curBatch.add(record);
 } catch (e) {
   // Send record to error
   curBatch.addError(records[i], e.toString());
 }
}

// Process the current batch
curBatch.process(entityName, offset.toString());

Values retain their original type

Values retain their original type regardless of whether a script modifies the value.

1. On the General tab, select the Produce Events property.

   This enables the event output stream for use.

2. Include both of the following methods in the script:
   • sdc.createEvent(<String type>, <Integer version>) - Creates an event record with the specified event type and version number. You can create a new event type or use an existing event type. Existing event types are documented in other event-generating stages.

     The event record contains no record fields. Generate record fields as needed.

   • batch.toEvent(<record>) - Use to append an event record to a batch and pass events to the event output stream.

• record.<header name> - Use to return the value of a header attribute.
• record.attributes - Use to return a map of custom record header attributes, or to create or update a specific record header attribute.

var <class name> = Java.type('<package>.<class name>');

var DigestSHA3 = Java.type('org.bouncycastle.jcajce.provider.digest.SHA3.DigestSHA3');