ThunderX2 Pod 2
Overview
The ThunderX2 Pod 1 consists of 3 Arm®v8 based Gigabyte R281-T91 servers and 1 Ampere HR330A jumphost server with the following specifications per server:
Gigabyte R281-T91 | Ampere HR330A | |
---|---|---|
CPU | 2x 28 core 2.0 GHz Marvell TX2 Processors | 1x 32 core 3.3 GHz Ampere ARMv8 64-bit Processor |
RAM | 4x 32GB RDIMM-Tx2 | 8x 16GB DDR4-DIMM |
Storage | 1x 480GB SSD | 1x 480GB SSD |
Networking | 2x 10Gbe SFP+ (not connected) | 2x 10Gbe SFP+ (connected) 1x 1Gbe BASE-T (not connected) 1x IPMI / Lights-out Management |
Usage & Users
This information can now be found on the Akraino dashboard. Check the Shared Community Lab page for more information.
Server Access
For the meantime when you request access to the Akraino Pods at UNH-IOL we will ask that you send us your public ssh key so we can add it to the root users of those machines.
If you wish to have your own user we can get that setup, but it is much quicker and easier if you just stick with the root user
IPMI Access
When booking a pod, the Akraino dashboard will provide IPMI credentials via email.
You can access an IPMI interface like the example below:
thunder2-gigabyte1-ipmi.akr.iol.unh.edu
Then enter the username and password provided from the file in /opt
and now you can manage the system on your and will even be able to add your own public keys in the event of a reinstall of the machine.
If you have any issues with any of the IPMI interfaces please email akraino-lab@iol.unh.edu and we will assist you in anyway we can.
Networking
The networking within the pod is currently provided by a 10Gbe BASE-T switch, which only supports static configurations (i.e. no SDN control). This switch will be replaced with an Edgecore 7816-64X that is currently on order, along with 40Gbe NICs to be installed on each server.
Network | IP Network | VLAN ID | Description |
---|---|---|---|
IPMI / Management | 10.11.17.0/24 GW: 10.11.17.1 | 3017 | Connections to all IPMI interfaces on each server and the switch management interface |
Public | 10.11.18.0/24 GW: 10.11.18.1 | 3018 | Public network (able to reach the Internet), available to each server. |
Servers and Switches
The 3 Gigabyte R281-T91 have 3 sets of dual port interfaces:
Mellanox Technologies MT27700 on PCI-e expansion slot - two ports on PCI addresses 0000:91:00.0 and 0000:91:00.0 (enp145s0f0 and enp14s0f1)
QLogic Corp. FastLinQ QL41000 Series integrated on the main board - two ports on PCI addresses 0000:0c:00.0 and 0000:0c:00.1 (enp12s0f0 and enp12s0f1 or eno1 and eno2)
Intel 10G X550T on PCI-e expansion slot - two ports on PCI address 0000:05:00.0 and 0000:05:00.1 (enp5s0f0 and enp5s0f1); not connected at the moment
In the table below, the switch ports for the 3 gigabyte servers are in the following order:
Mellanox MT27700 port 1
Mellanox MT27700 port 0
QLogic FaslinQ QL41000 port 0
QLogic FaslinQ QL41000 port 1
The ampere-jumphost2 has the following network interfaces:
Mellanox MT27710 on PCI-e expansion slot - two ports at PCI addresses 0000:01:00.0 and 0000:01:00.1 (enp1s0f0 and enp1s0f1)
Intel Gigabit I210 on PCI-e expansion slowt - one port at PCI address 0002:01:00.0 (enP2p1s0); not connected at the moment
In the table below, the switch ports are in the following order:
MT27710 port 1
MT27710 port 0
Server Name | IPMI Address | IPMI VLAN ID | Public Network Address | Public Network VLAN ID | Switch Port(s) | Serial Connection | OS Installed |
---|---|---|---|---|---|---|---|
thunder2-gigabyte1 | 10.11.17.11 | 3017 | 10.11.18.11 | 3018 | gigabyte-edgecore2: left 40Bbe Port 1, right 40Gbe Port 2, left 10Gbe Port 7 Breakout 1, right 10Gbe Port 7 Breakout 2 | Async 1/6 | CentOS 7.6 |
thunder2-gigabyte2 | 10.11.17.12 | 3017 | 10.11.18.12 | 3018 | gigabyte-edgecore2: left 40Bbe Port 3, right 40Gbe Port 4, left 10Gbe Port 7 Breakout 3, right 10Gbe Port 7 Breakout 4 | Async 1/7 | Ubuntu 18.04 |
thunder2-gigabyte3 | 10.11.17.13 | 3017 | 10.11.18.13 | 3018 | gigabyte-edgecore2: left 40Bbe Port 5, right 40Gbe Port 6, left 10Gbe Port 9 Breakout 1, right 10Gbe Port 9 Breakout 2 | Async 1/8 | Ubuntu 18.04 |
thunder2-jumphost | 10.11.17.14 | 3017 | 10.11.18.14 | 3018 | gigabyte-edgecore2: left 10 Gbe Port 9 breakout 3, right 10 Gbe Port 9 breakout 4 | N/A | Ubuntu 18.04 |
Switch Name | Mgmt Address | VLAN ID | Serial Connection | OS Installed |
---|---|---|---|---|
thunder2-egdecore | 10.11.17.5 | 3017 | Async 1/2 | SONiC |