Blueprint Editor

Overview

A Blueprint is a collection of Systems and Components put together by a customer in such a way as to fulfill their functional requirements.  The collection is linked together by Connections to its various components via their input and output channels.  Components have parameters that can be set to configure their behavior.  To allow quick modification of shared configuration values, blueprints can define Settings that can be referenced in the configuration of multiple components.  More details on the general structure and purpose of blueprints can be found on the Blueprint Overview page.

Systems

A System is a location where components can be run.  Systems correspond with deployed DataShyft Customer Runtimes.  By placing a component onto a system, you are informing DataShyft of where the component should be run.  A blueprint must have at least one system in order for it to be deployed but can encompass as many systems as are necessary to accomplish the blueprints goals.

Adding Systems to a Blueprint

To add a system to a blueprint, drag it from the System palette on the left side of the editor onto the editing canvas.  The system can be resized using the handle in its bottom right corner.  When you add components to a system, the system will automatically resize to fit its content.

Removing Systems from a Blueprint

To Remove a System, click the X button in the top right corner of the system.  You will be prompted to confirm the removal of the system.  

Note: Removing a system will also remove all the components it contains.

Swapping Systems in a Blueprint

To switch from one system to another in your blueprint, click the Swap Systems button on the right side of the System title bar.  You be presented with a dialog that allows you to pick the system you wish to replace this system with.  Once you select the new system and click Swap Systems, the system will be swapped.  All existing components will be moved to the newly selected system, and the editor will update the name of the system to the selected one.

Components

A Component is a piece of software that performs a specific task within a Blueprint.  For example, DataShyft provides components for accessing databases and FTP servers, transforming data from one format to another, and routing data to various components in the blueprint.  Each of these components has a specific set of configuration parameters that control how it operates.

Adding Components to a Blueprint

Components are added to ta blueprint by dragging them from the Available Components palette on the left side of the blueprint editor into a system on the canvas.  Components can only be placed inside systems, so you must add a system to the canvas before attempting to add a components.  Dropping a component on the empty canvas will not do anything.

Removing Components from a Blueprint

Components can be removed from a Blueprint by clicking the delete button in the upper left corner of the component.  The button will appear when you hover the mouse over the component.  You will be prompted to confirm the removal.  Removing a component will also remove all connections to that component.

Configuring Components on a Blueprint

Most components have configuration options that control their behavior, many of which are required and must be set before the blueprint can be deployed.  You can access these component configuration options by clicking on the components title.  This will open the Component Details view on the right side of the editor.  The details panel lists all the information and configuration options for the component.

Component ID

The component ID is shown at the top of the details panel.  This ID must be unique within the blueprint and is used to identify the component.  You can change a component's ID by clicking on the name, editing it, and hitting enter to commit the change.

Type

The type field shows the specific type of component this is.  Hovering over the type name will provide a description of the component and its purpose.

Input Channels

A list of all the input channels defined on this component along with their data types.  This list corresponds to the input channel list shown on the component element on the editor canvas.

Some components allow you to create custom input channels on them (e.g., Multiplexer).  Such components will display a plus button next to the table.  Clicking this button will open a dialog where you can specify the name and data type for this channel.  Once the Add Channel button is clicked to commit the change, the new input channel is added to the list.  Customer input channels will be identifiable by the delete button displayed next to them.  Clicking this delete button will prompt you to confirm the channel removal.

The Scripting components have an alternate way of specifying custom input channels inside the script itself.  See Customizing Input and Output Channels on Scripting Components for more details on specifying custom channels on scripting components.

Output Channels

A list of all the output channels defined on this component along with their data types.  This list corresponds to the output channel list shown on the component element on the editor canvas.

Some components allow you to create custom output channels on them (e.g., Demultiplexer).  Such components will display a plus button next to the table.  Clicking this button will open a dialog where you can specify the name and data type for this channel.  Once the Add Channel button is clicked to commit the change, the new output channel is added to the list.  Customer output channels will be identifiable by the delete button displayed next to them.  Clicking this delete button will prompt you to confirm the channel removal.

The Scripting components have an alternate way of specifying custom output channels inside the script itself.  See Customizing Input and Output Channels on Scripting Components for more details on specifying custom channels on scripting components.

Parameters

The Parameters section lists out all of the component's parameters.  Optional parameters typically have default values that will be shown as placeholder text.  Full details on the parameters for a specific component can be found on the component's help page.

Hovering on a parameter will show the parameter description, data type, and any default value it may have.

If a component parameter has an issue, it will display a yellow warning icon next to the parameter name.  Hovering over the icon will display a pop up describing the issue with the parameter.  Issues can include the parameter not being set, use of a resource that doesn't work on the target system, or use of an unrecognized blueprint setting.

Connections

For a blueprint to perform useful work, the components need to be connected to one another to allow data and control signals to flow between them.  Connections within a Blueprint specify the component and output channel the data will come from, and the component and input channel the data will be delivered to.  Multiple connections can specify the same sources or destinations if data needs to flow to multiple components within a blueprint.  

If a connection specifies a source and destination that are on different systems, DataShyft will automatically setup a secure data communication channel to handle the transfer of data between the systems.

Adding Connections to a Blueprint

Connections between the output and input channels of components can be added by dragging from the output channel to the input channel.  When you start the drag, the input channels on the components will highlight to let you know if a connection would be valid or not.  Connections that highlight in green are known good connection points.  This means that data types of the output channel and input channel are compatible.  Connections that highlight in yellow are allowable, but may or may not work depending on the specific connection.  This occurs when the input channel's type is a superclass of the output channel's type (e.g. output is FileData and input is PipelineData).  Input channels that are not compatible will not highlight at all and you cannot make connections to them (e.g. output is FileData and input is MappedData).

Removing Connections from a Blueprint

To remove a connection from the blueprint, hover over the connection.  This will display the connection removal button.  Clicking the button will prompt you for confirmation before removing the connection.

Blueprint Settings

Blueprint settings are accessed via the Blueprint Settings button on the Toolbar.  Clicking it will open up the Blueprint Settings dialog.  In this dialog, you can add, update, or remove blueprint settings as well as configure the Data Governance settings for the blueprint.

Managing Blueprint Settings

The Blueprint Settings table lists all of the blueprint settings defined on the blueprint.  The Settings table colors each row to indicate what is changing about it.  New rows have a green background behind the name and value.  Edited rows have a yellow background behind the edit fields.  Deleted settings have a red background and show a strike throw styling.

Adding a Blueprint Setting

The Add Row button will add a row to the table so you can add a new setting.  Once the row is added to the table, enter the setting name and value into the respective fields.  The settings will be saved when you click the Update button and dismiss the dialog.

Editing a Blueprint Setting

Existing Blueprint Settings can be edited by changing their name or value.  The changes will be saved when you click the Update button and dismiss the dialog.  An edited setting will show a revert button.  Clicking this button will revert the setting row back to its original name and value.

Deleting a Blueprint Setting

Existing Blueprint Settings can be delete by clicking the delete icon next to the row.  If you delete an existing setting, the row will show a red background and strikethrough of the name and value.  If you delete a newly added setting, the setting is removed from the table immediately.  The changes will be saved when you click the Update button and dismiss the dialog.

Configuring Data Governance

Data Governance configuration for the Blueprint is configured in the Blueprint Settings dialog.  The Data Governance dropdown at the top of the dialog allows you to select the Data Governance resource that is to be used with this blueprint, or disable data governance if it is not required.

Blueprint Tools

The Blueprint Editor toolbar provides access to most of the Blueprint-level functions of the editor.

Save Blueprint

The Save button will immediately save the current blueprint to the server.  This button is only enabled if there are unsaved changes on the blueprint.  Blueprint will auto-save periodically to help prevent data loss.

Duplicate Blueprint

The Duplicate Blueprint button allows you to make a copy of this blueprint.  A dialog will be shown asking you for the name of the copied blueprint.  Once copied, the editor will reload with the new blueprint to allow editing.

Export Blueprint

The Export Blueprint button will download a JSON version of the blueprint that can be imported into this tenant, or another tenant, to recreate this blueprint.

Delete Blueprint

The Delete Blueprint button allows you to delete this blueprint from the system.  You will be asked to confirm the deletion of the blueprint since it is not reversible.

Blueprint Name

The Blueprint name can be edited by clicking on the name, making desired changes, and either hitting enter or clicking outside the text box.

Lock/Unlock Blueprint

The Lock/Unlock Blueprint button allows you to lock a blueprint.  A locked blueprint is only editable by you.  If a blueprint is locked by another user, you will be unable to unlock it.

Note: Administrators are able to unlock any blueprint at any time.

Publish Blueprint

The Publish Blueprint button will publish the blueprint making it uneditable by anyone.  On published blueprints, this button will become an Unpublished Blueprint button, which will switch the blueprint back to Draft status and allow editing.

Archive Blueprint

The Archive Blueprint button will archive the blueprint.  Archived blueprints cannot be edited and are excluded from the default Blueprint list filter.  It provides a way to mark blueprints as no longer being relevant without outright deleting them.

Deploy Blueprint

The Deploy Blueprint button will attempt to deploy the blueprint.  You will be prompted for a name for the new deployment, with the name defaulting to the blueprint name.  As part of the deployment process, the blueprint will be validated to ensure that it is in a deployable state.  If the blueprint cannot be deployed, an error dialog will be present informing you of all the issues detected in the blueprint.  Once the deployment is successfully launched, the UI will navigate to the Deployment List and open the deployment details for the newly launched deployment.