DGS Filter Component

A DGS Filter takes data as input, filters that data according to its filter rules, and outputs the result, if any. The criteria by which data should be filtered is retrieved from the DGS Storage. The format of this criteria is described in the Data Governance System Introduction article. A DGS Filter will have an ID so that it is able to retrieve the filters it should apply from the DGS Storage. The filter component will load its criteria from the DGS Storage on startup. 

A filter is configured with a default action that is applied if none of the filter rules specify a specific action.  This default action is either DISCARD, which will throw out data items that aren’t specifically allowed, or ALLOW, which pass through all data items that are not explicitly discarded. A filter will reload its criteria on some interval automatically if configured to do so. Please see the Data Governance System Documentation for more information.

Input Channels

data — Accepts a PipelineData data item and applies the filters to it.

reload— Reloads the filter criteria from the DGS Storage.

Output Channels

data — Returns the input PipelineData data item, if allowed by the filters or default action, modified according to any applicable filters.

Parameters

filterID — (String) The ID used to retrieve filter criteria from storage.

defaultAction — (String, default DISCARD) The default action to take when the filter criteria do not specify an action to take on the data item, or when the filter criteria haven’t been loaded from storage. Valid values are ALLOW and DISCARD.

refreshInterval — (int, default 0) How frequently in seconds to reload the filter criteria from storage. A value of 0 or less disables auto-refresh.