IntelliFlex systems can accommodate optional second-tier switches. The second-tier switch interconnects the first-tier switches and provides the connection point for the uplinks that connect to the UDA fabric. A second-tier switch also allows for future expansion to additional nodes and cabinets.
When second-tier switches are used, each clique is considered a group, and the cabinets in the group connect to the same first-tier switches. The number of uplinks to each second-tier switch depends on the number of second-tier switches in use, which depends on the number of nodes and cabinets.
Second-tier switches are installed by default in Cabinet 2.
System Network Topology for a Two-Group IntelliFlex System
System Network Topology for a Nine-Group IntelliFlex System
This configuration allows for connection to the UDA, and for future expansion beyond 9 groups, or up to 117 nodes.
- The first cabinet contains IntelliFlex processing nodes.
- Additional cabinets in a single group may also contain processing nodes.
- The first cabinet contains a System VMS for Server Management, and may contain additional TMSs.
- A cabinet VMS for Server Management is added for every 50 servers.
- 36-port InfiniBand switches in cabinets 1 and 3 provide node-to-switch connections for the redundant InfiniBand networks (Network-0 and Network-1). Each node contains a dual-port InfiniBand adapter that connects to the InfiniBand switches.
- Four ports on each of the first-tier InfiniBand switches in cabinets 1 and 3 are used to provide the uplink connections to the second-tier switch.
- When connected to a UDA network, the second-tier switches can be used for UDA connections.