# Performance acceleration with GPU instances GPU instances GPU-based instances provide access to NVIDIA GPUs with thousands of compute cores. You can use these instances to accelerate scientific, engineering, and rendering applications by leveraging the CUDA or Open Computing Language (OpenCL) parallel computing frameworks. You can also use them for graphics applications, including game streaming, 3-D application streaming, and other graphics workloads. Before you can activate or optimize a GPU-based instance, you must install the appropriate drivers, as follows: + To install NVIDIA drivers on an instance with an attached NVIDIA GPU, such as a P3 or G4dn instance, see [NVIDIA drivers](install-nvidia-driver.md). + To install AMD drivers on an instance with an attached AMD GPU, such as a G4ad instance, see [AMD drivers](install-amd-driver.md). **Topics** + [Activate NVIDIA GRID Virtual Applications](activate_grid.md) + [Optimize GPU settings](optimize_gpu.md) + [Set up dual 4K displays on G4ad](activate_g4ad_4k.md) + [ # Get started with GPU accelerated instances ](gpu-instances-started.md) # Activate NVIDIA GRID Virtual Applications on your Amazon EC2 GPU-based instances Activate NVIDIA GRID Virtual Applications To activate the GRID Virtual Applications on GPU-based instances that have NVIDIA GPUs (NVIDIA GRID Virtual Workstation is enabled by default), you must define the product type for the driver. The process that you use depends on the operating system of your instance. ## Linux instances **To activate GRID Virtual Applications on your Linux instances** 1. Create the `/etc/nvidia/gridd.conf` file from the provided template file. ``` [ec2-user ~]$ sudo cp /etc/nvidia/gridd.conf.template /etc/nvidia/gridd.conf ``` 1. Open the `/etc/nvidia/gridd.conf` file in your favorite text editor. 1. Find the `FeatureType` line, and set it equal to `0`. Then add a line with `IgnoreSP=TRUE`. ``` FeatureType=0 IgnoreSP=TRUE ``` 1. Save the file and exit. 1. Reboot the instance to pick up the new configuration. ``` [ec2-user ~]$ sudo reboot ``` ## Windows instances **To activate GRID Virtual Applications on your Windows instances** 1. Run **regedit.exe** to open the registry editor. 1. Navigate to `HKEY_LOCAL_MACHINE\SOFTWARE\NVIDIA Corporation\Global\GridLicensing`. 1. Open the context (right-click) menu on the right pane and choose **New**, **DWORD**. 1. For **Name**, enter **FeatureType** and type `Enter`. 1. Open the context (right-click) menu on **FeatureType** and choose **Modify**. 1. For **Value data**, enter `0` for NVIDIA GRID Virtual Applications and choose **OK**. 1. Open the context (right-click) menu on the right pane and choose **New**, **DWORD**. 1. For **Name**, enter **IgnoreSP** and type `Enter`. 1. Open the context (right-click) menu on **IgnoreSP** and choose **Modify**. 1. For **Value data**, type `1` and choose **OK**. 1. Close the registry editor. # Optimize GPU settings on Amazon EC2 instances Optimize GPU settings There are several GPU setting optimizations that you can perform to achieve the best performance on NVIDIA GPU instances. With some of these instance types, the NVIDIA driver uses an autoboost feature, which varies the GPU clock speeds. By disabling autoboost and setting the GPU clock speeds to their maximum frequency, you can consistently achieve the maximum performance with your GPU instances. ## Optimize GPU settings on Linux 1. Configure the GPU settings to be persistent. This command can take several minutes to run. ``` [ec2-user ~]$ sudo nvidia-persistenced ``` 1. [G3, and P2 instances only] Disable the autoboost feature for all GPUs on the instance. ``` [ec2-user ~]$ sudo nvidia-smi --auto-boost-default=0 ``` 1. Set all GPU clock speeds to their maximum frequency. Use the memory and graphics clock speeds specified in the following commands. Some versions of the NVIDIA driver do not support setting the application clock speed, and display the error `"Setting applications clocks is not supported for GPU..."`, which you can ignore. + G3 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 2505,1177 ``` + G4dn instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 5001,1590 ``` + G5 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 6250,1710 ``` + G6, G6f, Gr6, and Gr6f instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 6251,2040 ``` + G6e instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 9001,2520 ``` + G7e instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 12481,2430 ``` + P2 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 2505,875 ``` + P3 and P3dn instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 877,1530 ``` + P4d instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 1215,1410 ``` + P4de instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 1593,1410 ``` + P5 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 2619,1980 ``` + P5e and P5en instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 3201,1980 ``` + P6-B200 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 3996,1965 ``` + P6-B300 instances: ``` [ec2-user ~]$ sudo nvidia-smi -ac 3996,2032 ``` ## Optimize GPU settings on Windows 1. Open a PowerShell window and navigate to the NVIDIA installation folder. ``` PS C:\> cd "C:\Windows\System32\DriverStore\FileRepository\nvgridsw_aws.inf_*\" ``` 1. [G3, and P2 instances only] Disable the autoboost feature for all GPUs on the instance. ``` PS C:\> .\nvidia-smi --auto-boost-default=0 ``` 1. Set all GPU clock speeds to their maximum frequency. Use the memory and graphics clock speeds specified in the following commands. Some versions of the NVIDIA driver do not support setting the application clock speed, and display the error `"Setting applications clocks is not supported for GPU..."`, which you can ignore. + G3 instances: ``` PS C:\> .\nvidia-smi -ac "2505,1177" ``` + G4dn instances: ``` PS C:\> .\nvidia-smi -ac "5001,1590" ``` + G5 instances: ``` PS C:\> .\nvidia-smi -ac "6250,1710" ``` + G6, G6f, Gr6, and Gr6f instances: ``` PS C:\> .\nvidia-smi -ac "6251,2040" ``` + G6e instances: ``` PS C:\> .\nvidia-smi -ac "9001,2520" ``` + P2 instances: ``` PS C:\> .\nvidia-smi -ac "2505,875" ``` + P3 and P3dn instances: ``` PS C:\> .\nvidia-smi -ac "877,1530" ``` # Set up Dual 4K displays on G4ad Linux instances Set up dual 4K displays on G4ad After you launch a G4ad instance, you can set up dual 4K displays. **To install the AMD drivers and configure dual screens** 1. Connect to your Linux instance to get the PCI Bus address of the GPU you want to target for dual 4K (2x4k): ``` lspci -vv | grep -i amd ``` You will get output similar to the following: ``` 00:1e.0 Display controller: Advanced Micro Devices, Inc. [*AMD*/ATI] Device 7362 (rev c3) Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device 0a34 ``` 1. Note the PCI bus address is 00:1e.0 in the above output. Create a file named `/etc/modprobe.d/amdgpu.conf` and add: ``` options amdgpu virtual_display=0000:00:1e.0,2 ``` 1. To install the AMD drivers on Linux, see [AMD drivers for your EC2 instance](install-amd-driver.md). If you already have the AMD GPU driver installed, you will need to rebuild the amdgpu kernel modules through dkms. 1. Use the below xorg.conf file to define the dual (2x4K) screen topology and save the file in `/etc/X11/xorg.conf:` ``` ~$ cat /etc/X11/xorg.conf Section "ServerLayout" Identifier "Layout0" Screen 0 "Screen0" Screen 1 "Screen1" InputDevice "Keyboard0" "CoreKeyboard" InputDevice "Mouse0" "CorePointer" Option "Xinerama" "1" EndSection Section "Files" ModulePath "/opt/amdgpu/lib64/xorg/modules/drivers" ModulePath "/opt/amdgpu/lib/xorg/modules" ModulePath "/opt/amdgpu-pro/lib/xorg/modules/extensions" ModulePath "/opt/amdgpu-pro/lib64/xorg/modules/extensions" ModulePath "/usr/lib64/xorg/modules" ModulePath "/usr/lib/xorg/modules" EndSection Section "InputDevice" # generated from default Identifier "Mouse0" Driver "mouse" Option "Protocol" "auto" Option "Device" "/dev/psaux" Option "Emulate3Buttons" "no" Option "ZAxisMapping" "4 5" EndSection Section "InputDevice" # generated from default Identifier "Keyboard0" Driver "kbd" EndSection Section "Monitor" Identifier "Virtual" VendorName "Unknown" ModelName "Unknown" Option "Primary" "true" EndSection Section "Monitor" Identifier "Virtual-1" VendorName "Unknown" ModelName "Unknown" Option "RightOf" "Virtual" EndSection Section "Device" Identifier "Device0" Driver "amdgpu" VendorName "AMD" BoardName "Radeon MxGPU V520" BusID "PCI:0:30:0" EndSection Section "Device" Identifier "Device1" Driver "amdgpu" VendorName "AMD" BoardName "Radeon MxGPU V520" BusID "PCI:0:30:0" EndSection Section "Extensions" Option "DPMS" "Disable" EndSection Section "Screen" Identifier "Screen0" Device "Device0" Monitor "Virtual" DefaultDepth 24 Option "AllowEmptyInitialConfiguration" "True" SubSection "Display" Virtual 3840 2160 Depth 32 EndSubSection EndSection Section "Screen" Identifier "Screen1" Device "Device1" Monitor "Virtual" DefaultDepth 24 Option "AllowEmptyInitialConfiguration" "True" SubSection "Display" Virtual 3840 2160 Depth 32 EndSubSection EndSection ``` 1. Set up DCV by following the instructions in setting up an [interactive desktop](#amd-interactive-desktop). 1. After the DCV set up is complete, reboot. 1. Confirm that the driver is functional: ``` dmesg | grep amdgpu ``` The response should look like the following: ``` Initialized amdgpu ``` 1. You should see in the output for `DISPLAY=:0 xrandr -q` that you have 2 virtual displays connected: ``` ~$ DISPLAY=:0 xrandr -q Screen 0: minimum 320 x 200, current 3840 x 1080, maximum 16384 x 16384 Virtual connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 0mm x 0mm 4096x3112 60.00 3656x2664 59.99 4096x2160 60.00 3840x2160 60.00 1920x1200 59.95 1920x1080 60.00 1600x1200 59.95 1680x1050 60.00 1400x1050 60.00 1280x1024 59.95 1440x900 59.99 1280x960 59.99 1280x854 59.95 1280x800 59.96 1280x720 59.97 1152x768 59.95 1024x768 60.00 59.95 800x600 60.32 59.96 56.25 848x480 60.00 59.94 720x480 59.94 640x480 59.94 59.94 Virtual-1 connected 1920x1080+1920+0 (normal left inverted right x axis y axis) 0mm x 0mm 4096x3112 60.00 3656x2664 59.99 4096x2160 60.00 3840x2160 60.00 1920x1200 59.95 1920x1080 60.00 1600x1200 59.95 1680x1050 60.00 1400x1050 60.00 1280x1024 59.95 1440x900 59.99 1280x960 59.99 1280x854 59.95 1280x800 59.96 1280x720 59.97 1152x768 59.95 1024x768 60.00 59.95 800x600 60.32 59.96 56.25 848x480 60.00 59.94 720x480 59.94 640x480 59.94 59.94 ``` 1. When you connect into DCV, change the resolution to 2x4K, confirming the dual monitor support is registered by DCV. ![\[DCV resolution changes\]](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/images/dm-dcv-example.png) ## Set up an interactive desktop for Linux After you confirm that your Linux instance has the AMD GPU driver installed and amdgpu is in use, you can install an interactive desktop manager. We recommend the MATE desktop environment for the best compatibility and performance. **Prerequisite** Open a text editor and save the following as a file named `xorg.conf`. You'll need this file on your instance. ``` Section "ServerLayout" Identifier "Layout0" Screen 0 "Screen0" InputDevice "Keyboard0" "CoreKeyboard" InputDevice "Mouse0" "CorePointer" EndSection Section "Files" ModulePath "/opt/amdgpu/lib64/xorg/modules/drivers" ModulePath "/opt/amdgpu/lib/xorg/modules" ModulePath "/opt/amdgpu-pro/lib/xorg/modules/extensions" ModulePath "/opt/amdgpu-pro/lib64/xorg/modules/extensions" ModulePath "/usr/lib64/xorg/modules" ModulePath "/usr/lib/xorg/modules" EndSection Section "InputDevice" # generated from default Identifier "Mouse0" Driver "mouse" Option "Protocol" "auto" Option "Device" "/dev/psaux" Option "Emulate3Buttons" "no" Option "ZAxisMapping" "4 5" EndSection Section "InputDevice" # generated from default Identifier "Keyboard0" Driver "kbd" EndSection Section "Monitor" Identifier "Monitor0" VendorName "Unknown" ModelName "Unknown" EndSection Section "Device" Identifier "Device0" Driver "amdgpu" VendorName "AMD" BoardName "Radeon MxGPU V520" BusID "PCI:0:30:0" EndSection Section "Extensions" Option "DPMS" "Disable" EndSection Section "Screen" Identifier "Screen0" Device "Device0" Monitor "Monitor0" DefaultDepth 24 Option "AllowEmptyInitialConfiguration" "True" SubSection "Display" Virtual 3840 2160 Depth 32 EndSubSection EndSection ``` **To set up an interactive desktop on Amazon Linux 2** 1. Install the EPEL repository. ``` [ec2-user ~]$ sudo amazon-linux-extras install epel -y ``` 1. Install the MATE desktop. ``` [ec2-user ~]$ sudo amazon-linux-extras install mate-desktop1.x -y [ec2-user ~]$ sudo yum groupinstall "MATE Desktop" -y [ec2-user ~]$ sudo systemctl disable firewalld ``` 1. Copy the `xorg.conf` file to `/etc/X11/xorg.conf`. 1. Reboot the instance. ``` [ec2-user ~]$ sudo reboot ``` 1. (Optional) [Install the Amazon DCV server](https://docs.aws.amazon.com/dcv/latest/adminguide/setting-up-installing.html) to use Amazon DCV as a high-performance display protocol, and then [connect to a Amazon DCV session](https://docs.aws.amazon.com/dcv/latest/userguide/using-connecting.html) using your preferred client. **To set up an interactive desktop on Ubuntu** 1. Install the MATE desktop. ``` $ sudo apt install xorg-dev ubuntu-mate-desktop -y $ sudo apt purge ifupdown -y ``` 1. Copy the `xorg.conf` file to `/etc/X11/xorg.conf`. 1. Reboot the instance. ``` $ sudo reboot ``` 1. Install the AMF encoder for the appropriate version of Ubuntu. ``` $ sudo apt install ./amdgpu-pro-20.20-*/amf-amdgpu-pro_20.20-*_amd64.deb ``` 1. (Optional) [Install the Amazon DCV server](https://docs.aws.amazon.com/dcv/latest/adminguide/setting-up-installing.html) to use Amazon DCV as a high-performance display protocol, and then [connect to a Amazon DCV session](https://docs.aws.amazon.com/dcv/latest/userguide/using-connecting.html) using your preferred client. 1. After the DCV installation give the DCV User video permissions: ``` $ sudo usermod -aG video dcv ``` **To set up an interactive desktop on CentOS** 1. Install the EPEL repository. ``` $ sudo yum update -y $ sudo yum install epel-release -y ``` 1. Install the MATE desktop. ``` $ sudo yum groupinstall "MATE Desktop" -y $ sudo systemctl disable firewalld ``` 1. Copy the `xorg.conf` file to `/etc/X11/xorg.conf`. 1. Reboot the instance. ``` $ sudo reboot ``` 1. (Optional) [Install the Amazon DCV server](https://docs.aws.amazon.com/dcv/latest/adminguide/setting-up-installing.html) to use Amazon DCV as a high-performance display protocol, and then [connect to a Amazon DCV session](https://docs.aws.amazon.com/dcv/latest/userguide/using-connecting.html) using your preferred client. # Get started with GPU accelerated instances The latest generation of GPU accelerated instance types, such as those shown in the following list deliver the highest performance capabilities for deep learning and high performance computing (HPC) applications. Select the instance type link to learn more about its capabilities. + [P6 family](https://aws.amazon.com/ec2/instance-types/p6/) + [P6 family](https://aws.amazon.com/ec2/instance-types/p6/) + [P5 family](https://aws.amazon.com/ec2/instance-types/p5/) For a complete list of instance type specifications for accelerated instance types, see [Accelerated computing](https://docs.aws.amazon.com/ec2/latest/instancetypes/ac.html) in the *Amazon EC2 Instance Types* reference. **Software configuration** The easiest way to get started with the latest generation GPU accelerated instance types is to launch an instance from an AWS Deep Learning AMI that's preconfigured with all of the required software. For the latest AWS Deep Learning AMIs for use with GPU accelerated instance types, see [P6 Supported DLAMI](https://docs.aws.amazon.com/dlami/latest/devguide/p6-support-dlami.html) in the *AWS Deep Learning AMIs Developer Guide*. If you need to build a custom AMI to launch instances that host deep learning or HPC applications, we recommend that you install the following minimum software versions on top of your base image. | Instance type | NVIDIA driver | CUDA | NVIDIA GDRCopy | EFA installer | NCCL | EFA K8s ¹ | | --- | --- | --- | --- | --- | --- | --- | | G7e | 575 | 12.9 | 2.5 | 1.45.0 | 2.28.3 | 0.5.10 | | P5 | 530 | 12.1 | 2.3 | 1.24.1 | 2.18.3 | 0.4.4 | | P5.4xlarge | 530 | 12.1 | 2.3 | 1.43.1 ² | 2.18.3 | 0.4.4 | | P5e | 550 | 12.1 | 2.3 | 1.24.1 | 2.18.3 | 0.5.5 | | P5en | 550 | 12.1 | 2.3 | 1.24.1 | 2.18.3 | 0.5.6 | | P6-B200 | 570 | 12.8 | 2.5 | 1.41.0 | 2.26.2-1 | 0.5.10 | | P6e-GB200 | 570 | 12.8 | 2.5 | 1.41.0 | 2.26.2-1 | 0.5.10 | | P6-B300 | 580 | 13.0 | 2.5 | 1.44.0 | 2.28.3 | 0.5.10 | ** ¹** The **EFA K8s** column contains the minimum recommended version for `aws-efa-k8s-device-plugin`. ** ²** There is compatibility issue that affects `P5.4xlarge` instances when GPU-to-GPU communication uses Elastic Fabric Adapter (EFA) and the NVIDIA Collective Communications Library (NCCL). To mitigate the issue, set the environment variable `FI_HMEM_DISABLE_P2P` to `1`, and ensure that you install EFA version 1.43.1 or newer. **Note** If you use version 1.41.0 of the EFA installer, the `aws-ofi-nccl plugin` comes with it. For earlier versions of the EFA installer, use `aws-ofi-nccl plugin` version `1.7.2-aws` or later. We also recommend that you configure the instance to not use deeper C-states. For more information, see [High performance and low latency by limiting deeper C-states](https://docs.aws.amazon.com/linux/al2/ug/processor_state_control.html#c-states) in the *Amazon Linux 2 User Guide*. The latest AWS Deep Learning Base GPU AMIs are preconfigured to not use deeper C-states. For networking and Elastic Fabric Adapter (EFA) configuration see [Maximize network bandwidth on Amazon EC2 instances with multiple network cards](efa-acc-inst-types.md).