Scripted Data Generator Component

The Scripted Data Generator Component creates data using a provided Groovy script. The script must declare a class which defines the two required methods: futureGenerationTime  and generate. Optional helper methods may also be included. The provided com.datashyft.pipeline.scripting.ScriptedDataGenerator  interface defines these methods for convenience.

• generate  — Called when new data should be generated. It returns a list of DataOut  objects, each containing the generated data and the output channel name. Any output channel used by the script must be defined on the component in the blueprint. Data targeting undefined channels is discarded.
• futureGenerationTime  — Returns a duration (in milliseconds) after which generate  should next be called. If no more data should be generated, return Long.MAX_VALUE .

Data Governance

Data items generated by this component are automatically registered with the Data Governance System.

Auditing

Scripts can log audit events using an AuditLogger  object. Extend the com.datashyft.pipeline.scripting.ScriptedAuditor  abstract class to access one via getAuditLogger() . Use sendAuditLog(category, message)  to record messages:

this.auditLogger.sendAuditLog(LogCategory.INFO, "Generated a data item")

Categories include DEBUG , INFO , WARN , and ERROR .

Example Script

Here is an example script.  It generates a new String Data object every second and outputs it onto the Output Channel named “output”.  This script will run repeatedly until the deployment containing it is terminated.

import com.datashyft.pipeline.dataobjects.StringData;
import com.datashyft.pipeline.util.AbstractGroovyComponent.AuditLogger
import com.topiatechnology.mdci.strategies.DataOut;
import java.util.Collections;
import java.util.List;
import com.datashyft.pipeline.scripting.ScriptedGenerator;

// @OutputChannel("output", "StringData")
public class TimedGenerator implements ScriptedGenerator {

  private long _count = 0;

  public List<DataOut> generate() {
    StringData stringData =
    new StringData(null, "string", "Data Item " + _count++) ;

    return Collections.singletonList(
      new DataOut("output", stringData));
  }

  public long futureGenerationTime() {
    return 1000;
  }
}

Input Channels

<None>

Output Channels

channelN — Custom output channels must be defined for data output. The component outputs PipelineData objects generated by the script. See the Custom Input and Output Channels page for details on how to specify the custom output channels.

error – Outputs ErrorOutput  objects if something goes wrong in the script.

Parameters

script – The Groovy script to use to generate data.