A few years ago nVidia added the Tesla M10 GPU to their already mind-bending lineup of NVIDIA GRID accelerators. This dual-slot PCI Express board is explicitly designed for data centers requiring graphics acceleration for high-density VDI environments. The Tesla M10 can support 64 concurrent users while raising the bar when it comes to shared vGPU allocation.
Although Nvidia has done a great job of creating an intuitive installation wizard and step-by-step documentation around the initial install process, most power users see the guides as a horrible game of Where's Waldo. This quick and dirty reference blog post can help those who are familiar with their VDI environment and want more GPU horsepower. In this post, we will cover license server installation, VM, and ESXi host configuration. You’ll find links to all the supporting documentation and hopefully a walkthrough video next week. To start, make sure to install Java runtime environment 32-bit (even if you are installing it in a 64-bit server).
Nvidia M10 Installation and Setup
License Server Install
- Nvidia Grid license server can co-exist with other license server roles like Citrix.
Reference article for License Server setup: http://docs.nvidia.com/grid/latest/grid-license-server-user-guide/index.html
- You must install Java runtime environment 32-bit even if it's a 64-bit server
- During the install you are prompted with a checkbox if you want to remotely manage the Grid server, if so, select open port 8080.
- The steps to install the license server are very intuitive and step-by-step in the above document, so there is no real need to reproduce in this blog. The main take-away here is that the license is based upon the Mac address of the server that you are using. If you decide to change license servers or add an HA license server configuration, then you would need to re-download a new license file from the Nvidia customer portal.
Nvidia license portal
- External URL (Browse from and computer on the LAN) Http://Server FQDN:8080/licserver
- Internal URL (Directly on the License server) URL http://localhost:8080/licserver
Note: Licensed GPU enabled VMs; licensed by MAC address of the VM
Reference article: http://images.nvidia.com/content/pdf/grid/guides/quickstartguide.pdf
- First steps are you need to install a new Shared PCI device on the VM
- Set the GPU Profile to Grid_m 10-1b (1GB GPU Profile)
- This is only a recommendation there are higher and lower memory settings depending on your particular use case. In this instance, I chose 1 GB GPU profile
- Click the button Reserve all memory
Warning: The VM will not power on until its memory reservations equal its memory size.
Next you need to install the NVIDIA Graphics Driver onto the Guest OS. If you try to install the driver prior to adding the GPU hardware devices you will get an error that no hardware was found.
- Install GRID vGPU driver.
- Select Custom (Advanced).
- Select Clean installation.
- Restart VM and connect to VM.
- Once you install the Nvidia driver Device Manager should show the new display driver w any warnings.
- Next you have to assign a Nvidia license to your VM.
- Right-click on the Desktop and select Nvidia Control Panel
- Select the Licensing Menu/Manage Licenses
- Enter the Primary License Server:
- Port Number: 7070
Note: You will no longer be able to use the vSphere Console after installing the Nvidia driver to access the VM. You will need to use RDP or VNC to access the VM prior to pulling it into XenDesktop or Horizon.
ESXI Host Configuration for Nvida Grid
- Copy the .vib file for vSphere to the ESXi host and put the host in Maintenance Mode:
$ esxcli system maintenanceMode set –-enable true
- Install the Nvidia vib file by running the following command:
$ esxcli software vib install –v /tmp/NVIDIA**.vib )
Note: NVIDIA** should be the full NVIDIA vib file name.
- Exit out of the maintenance mode using the following command:
$ esxcli system maintenanceMode set –-enable false
- The ESXi host is rebooted.
- Verify the NVIDIA GPU card after the reboot using the following command: $ nvidia-smi
[root@Host:~] esxcli software vib install -v /tmp/NVIDIA-VMware_ESXi_6.5_Host_Driver_384.99-1OEM.618.104.22.16898673.vib
Message: Operation finished successfully.
Reboot Required: false
VIBs Installed: NVIDIA_bootbank_NVIDIA-VMware_ESXi_6.5_Host_Driver_384.99-1OEM.622.214.171.12498673