Remote Data Exchange Component
The Remote Data Exchange component allows deployments operating on different runtimes to pass data back and forth between them. Data passed into a Remote Data Exchange component in the deployed blueprint on one runtime is returned from the corresponding Remote Data Exchange component in the deployed blueprint on the other runtime. The data flow is bidirectional allowing data to flow in both directions.
Remote Data Exchange components on each runtime must be provided with a shared authentication token that is used to verify the remote component before sending data across the connection.
If the Data Governance is enabled on the deployments, then the deployments that are communicating using the Remote Data Exchange component must be using the same DGS Context. If they aren't, the components will fail to connect, and errors will be reported in the deployment audit logs. To set the DGS Context, add a blueprint setting called dgs.context to each of the blueprints and set them all to the same unique value. Alternatively, for programmatically deployed blueprints, you can specify the DGS Context setting as a setting override during deployment of the blueprints.
If multiple independent channels of data need to be passed through this communication channel, it is recommended that a Multiplexer and Demultiplexer be placed in front of the Remote Data Exchange components so that the streams can be cleanly aggregated and divided on either side.
Data Governance
If Data Governance is enabled, data items received by the component from the partner pipeline will be marked as received. If the data item's type is not recognized, the SerializedObjectData item will be recorded as having been derived from the original data item.
Furthermore, the deployments that are communicating using the Remote Data Exchange component must be using the same DGS Context. To set the DGS Context, add a blueprint setting called dgs.context to each of the blueprints and set them all to the same unique value. Alternatively, for programmatically deployed blueprints, you can specify the DGS Context setting as a setting override during deployment of the blueprints.
Input Channels
data — Accepts a PipelineData object and transfers it to the partner component.
Output Channels
data — Returns any PipelineData object that it receives from its partner component. If the received item's data type is not recognized, returns a SerializedObjectData object containing the serialized form of the original item. If the component receives a SerializedObjectData object from the partner system and it recognizes its data type, it will reconstitute the original data item and return it.
Parameters
resource — The Remote Data Exchange Resource that specifies the information for connecting to the remote runtime.
connectionID — The unique ID shared by the two components in the data exchange that allows them to locate each other and exchange data.
authenticationToken — A shared authentication token that is used by the components to authenticate each other. Both components must have the same authentication token. You should create unique authentication tokens for each pair of Remote Data Exchange components that you set up.