Inter-Cabinet Server Management Network Topology to 1,458 Nodes (324-Port Switches) - Appliance for Hadoop

Appliance for Hadoop 5 Platform Product and Site Preparation Guide

prodname
Appliance for Hadoop
created_date
February 2018
category
Product Guide
featnum
B035-6030-075K
Server Management Network for System with Maximum 1,458 Nodes, 2 InfiniBand Switch Cabinets

  • This configuration is built on the cabinet and collective topology defined in Collectives.
  • This configuration illustrates topology for one to eighty-one processing/storage cabinets.
  • Each processing/storage cabinet shown above is in a single collective. One collective includes the InfiniBand switch cabinet.
  • The InfiniBand switch cabinet contains one primary server management Ethernet switch and one secondary server management Ethernet switch.
    Although a cabinet number is not shown for the InfiniBand cabinet, the cabinet number must be unique in the system.
  • The System VMS in the base cabinet (Cabinet 1) hosts the SWS VM, Viewpoint VM, and CMIC VM. Each expansion cabinet contains a Cabinet VMS that hosts the CMIC VM.
    In systems with an Enterprise SWS and an Enterprise Viewpoint Server, the System VMS hosts only a CMIC VM.
  • This configuration has a pair of InfiniBand cabinets (Network-0 Cabinet and Network-1 Cabinet). Each InfiniBand cabinet contains one primary server management Ethernet switch or one secondary server management Ethernet switch.
    Although cabinet numbers are not shown for the InfiniBand cabinets, the cabinet numbers for these cabinets must be unique in the system.
  • Separate paths for the primary and secondary server management networks ensure network redundancy.
  • The primary and secondary networks are configured as follows:
    • The primary Ethernet switch in the InfiniBand Network-0 cabinet acts as the gathering switch for the primary switches in all odd-numbered cabinets in the system. The primary switches in those odd-numbered cabinets act as gathering switches for the primary switches in the even-numbered cabinets.
    • The secondary Ethernet switch in the InfiniBand Network-1 cabinet acts as the gathering switch for the secondary switches in all even-numbered cabinets in the system. The secondary switches in those even-numbered cabinets act as gathering switches for the secondary switches in the odd-numbered cabinets.
      The exception is Cabinet 81, which is the only odd-numbered cabinet connected to the secondary switch in the Network-1 cabinet.
  • If the system is included in a UDA environment, UDA ports will be accommodated by open ports on the primary and secondary server management switches for UDA connections to the other system or systems in the UDA environment. These UDA connections are not shown in the illustration above. For more information, see the Teradata Unified Data Architecture InfiniBand and Server Management Configuration Guide, B035-6001.
  • For port-to-port cabling, see the appropriate Hardware Installation Guide.