DataShyft Runtime Security

The DataShyft Customer Runtime maintains the security of its deployments, communications, and customer data.  This section covers how the Runtimes provide this security and details what traffic is allowed into and out of the runtimes.

Runtime Security

The DataShyft Customer Runtime is provided by Topia Technology as an OCI-compliant container image.  This image can be deployed into any OCI-compatiable host, such as Docker, AWS, or Azure.  Once a customer has deployed a Runtime instance within their data center or Virtual Private Cloud, Topia Technology has no access to it; the container is under the control of the customer.  

Management Interface Security

The DataShyft Customer Runtime’s Management Interface is available on port 8752, a custom HTTPS port. Accessing the Management Interface requires users to authenticate.  Unauthenticated users are prevented from proceeding past the login screen. 

Communication Security

The DataShyft Customer Runtime has two communication pathways, command and control communication with the DataShyft Platform, and Pipeline data transfer to and from other runtimes.  These two pathways handle different classes of data and secure that data in appropriate ways.

Command and Control

Command and Control messages are sent back and forth between the DataShyft Platform Tenant and the Runtimes.  These messages include heartbeats, pings, deployments, audit logs, and status.  These messages do not include any customer data.  All command-and-control information sent to the platform is stored in the DataShyft Platform Tenant's database and is used only by that Tenant and its users.

Command and Control messages sent to the Runtime on port 8752, a custom HTTPS port shared with the management interface.  Information on installing custom certificates on this port are available elsewhere in this guide.  Command and Control messages sent by the Runtime to the DataShyft Platform Tenant are on Port 443, the standard HTTPS port.

Heartbeats

Blueprint Heartbeats are sent by the Runtime to the Platform Tenant on a periodic basis.  These messages serve two purposes.  They let the Platform Tenant know that the Runtime is still active and operational and let the Platform know when the configuration of the runtime has changed.  These messages are digitally signed to verify that they come from the Runtime.  Heartbeat messages are sent approximately every 10 seconds, and on shutdown to provide the platform with immediate notice of the system going offline.

Pings

The DataShyft Platform Tenant sends periodic ping messages to the Runtime to verify communication and collect Runtime configuration information.  The Runtime validates the Ping request and responds by sending its current configuration information back to the platform.  This information includes a list of the IDs of all the local credentials setup on the runtime, a list of all the libraries installed on the runtime, and the port used for runtime-to-runtime communication.  The request is digitally signed by the Platform Tenant to verify its authenticity.  The request and response are sent over an HTTPS connection to ensure security.

Deployments

When a Blueprint is deployed, the DataShyft Tenant Platform makes requests to the runtime and transfers over the configuration and code needed to execute the data orchestration and integration process defined in the blueprint.  These requests are made over an HTTPS connection to ensure security.  In addition to the encryption provided by the HTTPS connection, the DataShyft Platform generates a secret key using a Diffie-Hellman Elliptic Curve (DHEC) key agreement process based on the runtime’s registered public key and the Platform Tenant’s private key.  This secret key is used to encrypt the individual fields in the transfer messages.  The transfer messages include cryptographic hashes of the code and configuration that are sent as part of the transfer.  This allows the runtime to validate that the data has not been corrupted during transit.

Audit Logs

The Runtime sends deployment audit logs to the Platform Tenant as needed.  The audit logs contain messages generated by the deployment that communicate the information, warnings, and errors.  The audit logs are sent to the Platform over an HTTPS connection to ensure security.  The logs are stored in the Platform Tenant's database where they are only available to the tenant.

Status

The Runtime sends status and metrics to the Platform Tenant periodically.  The status and metrics messages contain information about the status of all the components that make up a deployment as well as metrics for each component (e.g. messages processed, bytes transferred, execution time, etc).  These messages are sent to the Platform over an HTTPS connection to ensure security.  The status and metrics information are stored in the Platform Tenant's database where they are only available to the tenant.

Customer Data

Customer Data is processed by Deployments running on the DataShyft Customer Runtime.  Customer Data is transferred between Runtimes when a deployment’s blueprint directs data from one system to another.  Customer data is only moved as defined in the Blueprint.  Customer data only leaves a Runtime as defined by the Blueprint, such as when it is stored in a database, written to an FTP Server, etc.  

Customer Data is never sent to the DataShyft Platform.

In multi-system deployments, Secure Bridge components are used to securely transfer data between the runtimes for a deployment.  The Secure Bridges establish secure, encrypted connections between them themselves.  This is a multi-part process that ensures that connections are made between the correct components and that all data is protected against disclosure while in transit. Secure Bridges establish connections on port 32100.

Once the connection has been verified, the bridges negotiate ephemeral keys that are used to encrypt all data passing through them.  The Bridges use Diffie-Hellman Elliptic Curve (DHEC) key agreement to securely derive a shared encryption key.  This shared key is used to encrypt all traffic sent between the bridges. Periodically, the Secure Bridge components will trigger a renegotiation process that reruns the DHEC key agreement process to generate a new key.  Once the new key has been derived, all future traffic is encrypted using the new key. 

Network Security Recommendations

The DataShyft Customer Runtime should be setup with a Firewall or Security Group that limits access to it.  The Firewall or Security Group should limit the access as follows: