Configuration Overview | Teradata VantageCore with Dell ECS Object Storage - Configuration Overview

Dell ECS Object Storage acts as a repository for semi-structured data that can be accessed using either Vantage, or via other third-party tools. Vantage can query and analyze data within the object store that is generated from other data sources, provided that the objects are of a supported format.

Dell ECS Object Storage is accessed using a supported Vantage platform through the customer’s network infrastructure, as illustrated in the following high-level diagram showing the data flow.

ECS network connectivity overview (data)

As shown in the diagram, both the Vantage platform (left) and the Dell ECS Object Storage solution (right) are connected to the customer network. The Analytics Database engine within the Vantage platform can operate on data on the object store via NOS queries over the customer network (top).

As a resource attached to the customer network, Dell ECS Object Storage is accessible to Vantage as well as other third-party data processing and analytic applications that the customer may have in their data center. Vantage can operate on CSV, JSON, or Parquet formatted objects generated from other data sources and applications using NOS.

The Dell ECS Object Storage solution comprises the following major infrastructure components.

Component	Description
Object Storage Nodes	The Dell ECS Object Storage nodes provide the storage compute domain where the Object Store software runs, serving up an S3-compatible object store to clients. Each node contains compute and storage media (HDDs or SSDs), leveraged by the Dell ECS Object Storage SDS. The Object Storage nodes work together to form fault-tolerant, highly durable, and a highly available cluster (called Virtual Data Center) underpinned by robust 12+4 Erasure Coding (EC). Dell ECS Object Storage supports a single namespace per cluster (VDC), which can be comprised of a single minimum configuration rack or comprised of hundreds of nodes to reach a scale of exabytes. Exabytes is a Dell ECS Object Storage theoretical limit. Refer to ECS EX500 Cluster Configurations or ECS EX5000D Cluster Configurations for the range of clusters offered with this Teradata validated architecture.
ECS Connection Manager	The ECS Connection Managers function as load balancers to ensure an even distribution of IO across the Dell ECS Object Storage nodes, and to redirect front-end I/O as appropriate when an object storage node is offline.
Object Storage Networking	As a distributed, scalable storage solution, Dell ECS Object Storage requires both front-end and back-end networking to facilitate data/traffic between object storage nodes and to clients, as well as for management. The front-end network provides connectivity between ECS and the customer network, and is used to enable: Data traffic from ECS to client (such as NOS). Internode traffic, including messages sent between nodes to process I/O requests. (Erasure Coded write operations, for example, generate inter-node traffic). In a single-site, multi-rack deployment, internode traffic traverses from the front-end switches up to the customer network and then to the other rack front-end switches to process requests. Replication traffic. If replication is configured, the associated I/O to replicate objects occurs out the front-end network switches, over the customer network, back to the front-end switches on the remote cluster (Virtual Data Center). Management traffic. Management of the Dell ECS Object Storage (and ECS Connection Managers) occurs over the front-end (physical) network, using Web UI, CLI, or RESTful APIs. Alert/event reporting (using SNMP traps) and communication with customer-provided services (such as DNS and NTP) are examples of other types of management traffic that occur over the front-end network. Segregation of traffic into separate VLANs is fully supported and a best-practice recommended by Dell. Customers should minimally leverage separate VLANs to logically isolate ECS management traffic from other traffic types. Customers are directed to follow the processes outlined in Dell ECS: Networking Best Practices (H15718.10), available on the Dell ECS InfoHub for details The back-end network provides connectivity between ECS nodes for certain service operations, such as install, re-install, and upgrades.

Component

Description

Object Storage Nodes

The Dell ECS Object Storage nodes provide the storage compute domain where the Object Store software runs, serving up an S3-compatible object store to clients. Each node contains compute and storage media (HDDs or SSDs), leveraged by the Dell ECS Object Storage SDS. The Object Storage nodes work together to form fault-tolerant, highly durable, and a highly available cluster (called Virtual Data Center) underpinned by robust 12+4 Erasure Coding (EC).

Dell ECS Object Storage supports a single namespace per cluster (VDC), which can be comprised of a single minimum configuration rack or comprised of hundreds of nodes to reach a scale of exabytes.

Exabytes is a Dell ECS Object Storage theoretical limit. Refer to ECS EX500 Cluster Configurations or ECS EX5000D Cluster Configurations for the range of clusters offered with this Teradata validated architecture.

ECS Connection Manager

The ECS Connection Managers function as load balancers to ensure an even distribution of IO across the Dell ECS Object Storage nodes, and to redirect front-end I/O as appropriate when an object storage node is offline.

Object Storage Networking

As a distributed, scalable storage solution, Dell ECS Object Storage requires both front-end and back-end networking to facilitate data/traffic between object storage nodes and to clients, as well as for management.

The front-end network provides connectivity between ECS and the customer network, and is used to enable:

Data traffic from ECS to client (such as NOS).
Internode traffic, including messages sent between nodes to process I/O requests. (Erasure Coded write operations, for example, generate inter-node traffic). In a single-site, multi-rack deployment, internode traffic traverses from the front-end switches up to the customer network and then to the other rack front-end switches to process requests.
Replication traffic. If replication is configured, the associated I/O to replicate objects occurs out the front-end network switches, over the customer network, back to the front-end switches on the remote cluster (Virtual Data Center).
Management traffic. Management of the Dell ECS Object Storage (and ECS Connection Managers) occurs over the front-end (physical) network, using Web UI, CLI, or RESTful APIs. Alert/event reporting (using SNMP traps) and communication with customer-provided services (such as DNS and NTP) are examples of other types of management traffic that occur over the front-end network.

Segregation of traffic into separate VLANs is fully supported and a best-practice recommended by Dell. Customers should minimally leverage separate VLANs to logically isolate ECS management traffic from other traffic types. Customers are directed to follow the processes outlined in Dell ECS: Networking Best Practices (H15718.10), available on the Dell ECS InfoHub for details

The back-end network provides connectivity between ECS nodes for certain service operations, such as install, re-install, and upgrades.

Configuration Overview | Teradata VantageCore with Dell ECS Object Storage - Configuration Overview - Teradata VantageCore

Teradata® VantageCore with Dell ECS Object Storage - Architectural Reference Specification