# Troubleshooting your gateway
Following, you can find information about best practices and troubleshooting issues related to gateways, host platforms, virtual tapes, high availability, data recovery, and security. The on-premises gateway troubleshooting information covers gateways deployed on supported virtualization platforms. The troubleshooting information for high availability issues covers gateways running on VMware vSphere High Availability (HA) platform.
**Topics**
+ [Troubleshooting: gateway offline issues](troubleshooting-gateway-offline.md) - Learn how to diagnose problems that can cause your gateway to appear offline in the Storage Gateway console.
+ [Troubleshooting: internal error during gateway activation](troubleshooting-gateway-activation.md) - Learn what to do if you receive an internal error message when attempting to activate your Storage Gateway.
+ [Troubleshooting on-premises gateway issues](troubleshooting-on-premises-gateway-issues.md) - Learn about typical issues that you might encounter working with your on-premises gateways, and how to allow Support to connect to your gateway to assist with troubleshooting.
+ [Troubleshooting Microsoft Hyper-V setup](troubleshooting-hyperv-setup.md) - Learn about typical issues that you might encounter when deploying Storage Gateway on the Microsoft Hyper-V platform.
+ [Troubleshooting Amazon EC2 gateway issues](troubleshooting-EC2-gateway-issues.md) - Find information about typical issues that you might encounter when working with gateways deployed on Amazon EC2.
+ [Troubleshooting hardware appliance issues](troubleshooting-hardware-appliance-issues.md) - Learn how to resolve issues that you might encounter with the Storage Gateway Hardware Appliance.
+ [Troubleshooting virtual tape issues](Main_TapesIssues-vtl.md) - Learn about actions you can take if you experience unexpected issues with your virtual tapes.
+ [Troubleshooting high availability issues](troubleshooting-ha-issues.md) - Learn what to do if you experience issues with gateways that are deployed in a VMware HA environment.
# Troubleshooting: gateway offline issues
Use the following troubleshooting information to determine what to do if the AWS Storage Gateway console shows that your gateway is offline.
Your gateway might be showing as offline for one or more of the following reasons:
+ The gateway can't reach the Storage Gateway service endpoints.
+ The gateway shut down unexpectedly.
+ A cache disk associated with the gateway has been disconnected or modified, or has failed.
To bring your gateway back online, identify and resolve the issue that caused your gateway to go offline.
## Check the associated firewall or proxy
If you configured your gateway to use a proxy, or you placed your gateway behind a firewall, then review the access rules of the proxy or firewall. The proxy or firewall must allow traffic to and from the network ports and service endpoints required by Storage Gateway. For more information, see [Network and firewall requirements](https://docs.aws.amazon.com/storagegateway/latest/tgw/Requirements.html#networks).
## Check for an ongoing SSL or deep-packet inspection of your gateway's traffic
If an SSL or deep-packet inspection is currently being performed on the network traffic between your gateway and AWS, then your gateway might not be able to communicate with the required service endpoints. To bring your gateway back online, you must disable the inspection.
## Check for a power outage or hardware failure on the hypervisor host
A power outage or hardware failure on the hypervisor host of your gateway can cause your gateway to shut down unexpectedly and become unreachable. After you restore the power and network connectivity, your gateway will become reachable again.
After your gateway is back online, be sure to take steps to recover your data. For more information, see [Best practices for recovering your data](https://docs.aws.amazon.com/storagegateway/latest/tgw/recover-data-from-gateway.html).
## Check for issues with an associated cache disk
Your gateway can go offline if at least one of the cache disks associated with your gateway was removed, changed, or resized, or if it is corrupted.
**If a working cache disk was removed from the hypervisor host:**
1. Shut down the gateway.
1. Re-add the disk.
**Note**
Make sure you add the disk to the same disk node.
1. Restart the gateway.
**If a cache disk is corrupted, was replaced, or was resized:**
1. Shut down the gateway.
1. Reset the cache disk.
1. Reconfigure the disk for cache storage.
1. Restart the gateway.
For more information on troubleshooting a corrupted cache disk for a tape gateway, see [You need to recover a virtual tape from a malfunctioning cache disk](https://docs.aws.amazon.com/storagegateway/latest/tgw/Main_TapesIssues-vtl.html#creating-recovery-tape-vtl).
# Troubleshooting: internal error during gateway activation
Storage Gateway activation requests traverse two network paths. Incoming activation requests sent by a client connect to the gateway's virtual machine (VM) or Amazon Elastic Compute Cloud (Amazon EC2) instance over port 80. If the gateway successfully receives the activation request, then the gateway communicates with the Storage Gateway endpoints to receive an activation key. If the gateway can't reach the Storage Gateway endpoints, then the gateway responds to the client with an internal error message.
Use the following troubleshooting information to determine what to do if you receive an internal error message when attempting to activate your AWS Storage Gateway.
**Note**
Make sure you deploy new gateways using the latest virtual machine image file or Amazon Machine Image (AMI) version. You will receive an internal error if you attempt to activate a gateway that uses an outdated AMI.
Make sure that you select the correct gateway type that you intend to deploy before you download the AMI. The .ova files and AMIs for each gateway type are different, and they are not interchangeable.
## Resolve errors when activating your gateway using a public endpoint
To resolve activation errors when activating your gateway using a public endpoint, perform the following checks and configurations.
### Check the required ports
For gateways deployed on-premises, check that the ports are open on your local firewall. For gateways deployed on an Amazon EC2 instance, check that the ports are open on the instance's security group. To confirm that the ports are open, run a telnet command on the public endpoint from a server. This server must be in the same subnet as the gateway. For example, the following telnet commands test the connection to port 443:
```
telnet d4kdq0yaxexbo.cloudfront.net 443
telnet storagegateway.region.amazonaws.com 443
telnet dp-1.storagegateway.region.amazonaws.com 443
telnet proxy-app.storagegateway.region.amazonaws.com 443
telnet client-cp.storagegateway.region.amazonaws.com 443
telnet anon-cp.storagegateway.region.amazonaws.com 443
```
To confirm that the gateway itself can reach the endpoint, access the gateway's local VM console (for gateways deployed on-premises). Or, you can SSH to the gateway's instance (for gateways deployed on Amazon EC2). Then, run a network connectivity test. Confirm that the test returns `[PASSED]`. For more information, see [Testing Your Gateway Connection to the Internet](https://docs.aws.amazon.com/storagegateway/latest/tgw/manage-on-premises-common.html#MaintenanceTestGatewayConnectivity-common).
**Note**
The default login user name for the gateway console is `admin`, and the default password is `password`.
### Make sure firewall security does not modify packets sent from the gateway to the public endpoints
SSL inspections, deep packet inspections, or other forms of firewall security can interfere with packets sent from the gateway. The SSL handshake fails if the SSL certificate is modified from what the activation endpoint expects. To confirm that there's no SSL inspection in progress, run an OpenSSL command on the main activation endpoint ( `anon-cp.storagegateway.region.amazonaws.com`) on port 443. You must run this command from a machine that's in the same subnet as the gateway:
```
$ openssl s_client -connect anon-cp.storagegateway.region.amazonaws.com:443 -servername anon-cp.storagegateway.region.amazonaws.com
```
**Note**
Replace *region* with your AWS Region.
If there's no SSL inspection in progress, then the command returns a response similar to the following:
```
$ openssl s_client -connect anon-cp.storagegateway.us-east-2.amazonaws.com:443 -servername anon-cp.storagegateway.us-east-2.amazonaws.com
CONNECTED(00000003)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = anon-cp.storagegateway.us-east-2.amazonaws.com
verify return:1
---
Certificate chain
0 s:/CN=anon-cp.storagegateway.us-east-2.amazonaws.com
i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
1 s:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
i:/C=US/O=Amazon/CN=Amazon Root CA 1
2 s:/C=US/O=Amazon/CN=Amazon Root CA 1
i:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
3 s:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
i:/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2 Certification Authority
---
```
If there is an ongoing SSL inspection, then the response shows an altered certificate chain, similar to the following:
```
$ openssl s_client -connect anon-cp.storagegateway.ap-southeast-1.amazonaws.com:443 -servername anon-cp.storagegateway.ap-southeast-1.amazonaws.com
CONNECTED(00000003)
depth=0 DC = com, DC = amazonaws, OU = AWS, CN = anon-cp.storagegateway.ap-southeast-1.amazonaws.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 DC = com, DC = amazonaws, OU = AWS, CN = anon-cp.storagegateway.ap-southeast-1.amazonaws.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:/DC=com/DC=amazonaws/OU=AWS/CN=anon-cp.storagegateway.ap-southeast-1.amazonaws.com
i:/C=IN/O=Company/CN=Admin/ST=KA/L=New town/OU=SGW/emailAddress=admin@company.com
---
```
The activation endpoint accepts SSL handshakes only if it recognizes the SSL certificate. This means that the gateway's outbound traffic to the endpoints must be exempt from inspections performed by firewalls in your network. These inspections might be an SSL inspection or a deep packet inspection.
### Check gateway time synchronization
Excessive time skews can cause SSL handshake errors. For on-premises gateways, you can use the gateway's local VM console to check your gateway's time synchronization. The time skew should be no larger than 60 seconds. For more information, see [Synchronizing Your Gateway VM Time](https://docs.aws.amazon.com/storagegateway/latest/tgw/MaintenanceTimeSync-hyperv.html).
The **System Time Management** option isn't available on gateways that are hosted on Amazon EC2 instances. To make sure Amazon EC2 gateways can properly synchronize time, confirm that the Amazon EC2 instance can connect to the following NTP server pool list over ports UDP and TCP 123:
+ 0.amazon.pool.ntp.org
+ 1.amazon.pool.ntp.org
+ 2.amazon.pool.ntp.org
+ 3.amazon.pool.ntp.org
## Resolve errors when activating your gateway using an Amazon VPC endpoint
To resolve activation errors when activating your gateway using an Amazon Virtual Private Cloud (Amazon VPC) endpoint, perform the following checks and configurations.
### Check the required ports
Make sure the required ports within your local firewall (for gateways deployed on-premises) or security group (for gateways deployed in Amazon EC2) are open. The ports required for connecting a gateway to a Storage Gateway VPC endpoint differ from those required when connecting a gateway to public endpoints. The following ports are required for connecting to a Storage Gateway VPC endpoint:
+ TCP 443
+ TCP 1026
+ TCP 1027
+ TCP 1028
+ TCP 1031
+ TCP 2222
For more information, see [Creating a VPC endpoint for Storage Gateway](https://docs.aws.amazon.com/storagegateway/latest/tgw/gateway-private-link.html#create-vpc-endpoint).
Additionally, check the security group that's attached to your Storage Gateway VPC endpoint. The default security group attached to the endpoint might not allow the required ports. Create a new security group that allows traffic from your gateway's IP address range over the required ports. Then, attach that security group to the VPC endpoint.
**Note**
Use the [Amazon VPC console](https://console.aws.amazon.com//vpc/) to verify the security group that's attached to the VPC endpoint. View your Storage Gateway VPC endpoint from the console, and then choose the **Security Groups** tab.
To confirm that the required ports are open, you can run telnet commands on the Storage Gateway VPC Endpoint. You must run these commands from a server that's in the same subnet as the gateway. You can run the tests on the first DNS name that doesn't specify an Availability Zone. For example, the following telnet commands test the required port connections using the DNS name vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:
```
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 443
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 1026
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 1027
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 1028
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 1031
telnet vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com 2222
```
### Make sure firewall security does not modify packets sent from the gateway to your Storage Gateway Amazon VPC endpoint
SSL inspections, deep packet inspections, or other forms of firewall security can interfere with packets sent from the gateway. The SSL handshake fails if the SSL certificate is modified from what the activation endpoint expects. To confirm that there's no SSL inspection in progress, run an OpenSSL command on your Storage Gateway VPC endpoint. You must run this command from a machine that's in the same subnet as the gateway. Run the command for each required port:
```
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:443 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1026 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1027 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1028 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1031 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
$ openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:2222 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
```
If there's no SSL inspection in progress, then the command returns a response similar to the following:
```
openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1027 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
CONNECTED(00000005)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = anon-cp.storagegateway.us-east-1.amazonaws.com
verify return:1
---
Certificate chain
0 s:CN = anon-cp.storagegateway.us-east-1.amazonaws.com
i:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
1 s:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
i:C = US, O = Amazon, CN = Amazon Root CA 1
2 s:C = US, O = Amazon, CN = Amazon Root CA 1
i:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
3 s:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
i:C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
---
```
If there is an ongoing SSL inspection, then the response shows an altered certificate chain, similar to the following:
```
openssl s_client -connect vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com:1027 -servername vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
CONNECTED(00000005)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 DC = com, DC = amazonaws, OU = AWS, CN = anon-cp.storagegateway.us-east-1.amazonaws.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:/DC=com/DC=amazonaws/OU=AWS/CN=anon-cp.storagegateway.us-east-1.amazonaws.com
i:/C=IN/O=Company/CN=Admin/ST=KA/L=New town/OU=SGW/emailAddress=admin@company.com
---
```
The activation endpoint accepts SSL handshakes only if it recognizes the SSL certificate. This means that the gateway's outbound traffic to your VPC endpoint over required ports is exempt from inspections performed by your network firewalls. These inspections might be SSL inspections or deep packet inspections.
### Check gateway time synchronization
Excessive time skews can cause SSL handshake errors. For on-premises gateways, you can use the gateway's local VM console to check your gateway's time synchronization. The time skew should be no larger than 60 seconds. For more information, see [Synchronizing Your Gateway VM Time](https://docs.aws.amazon.com/storagegateway/latest/tgw/MaintenanceTimeSync-hyperv.html).
The **System Time Management** option isn't available on gateways that are hosted on Amazon EC2 instances. To make sure Amazon EC2 gateways can properly synchronize time, confirm that the Amazon EC2 instance can connect to the following NTP server pool list over ports UDP and TCP 123:
+ 0.amazon.pool.ntp.org
+ 1.amazon.pool.ntp.org
+ 2.amazon.pool.ntp.org
+ 3.amazon.pool.ntp.org
### Check for an HTTP proxy and confirm associated security group settings
Before activation, check if you have an HTTP proxy on Amazon EC2 configured on the on-premises gateway VM as a Squid proxy on port 3128. In this case, confirm the following:
+ The security group attached to the HTTP proxy on Amazon EC2 must have an inbound rule. This inbound rule must allow Squid proxy traffic on port 3128 from the gateway VM's IP address.
+ The security group attached to the Amazon EC2 VPC endpoint must have inbound rules. These inbound rules must allow traffic on ports 1026-1028, 1031, 2222, and 443 from the IP address of the HTTP proxy on Amazon EC2.
## Resolve errors when activating your gateway using a public endpoint and there is a Storage Gateway VPC endpoint in the same VPC
To resolve errors when activating your gateway using a public endpoint when there is a Amazon Virtual Private Cloud (Amazon VPC) enpoint in the same VPC, perform the following checks and configurations.
### Confirm that the **Enable Private DNS Name** setting isn't enabled on your Storage Gateway VPC endpoint
If **Enable Private DNS Name** is enabled, you can't activate any gateways from that VPC to the public endpoint.
**To disable the private DNS name option:**
1. Open the [Amazon VPC console](https://console.aws.amazon.com//vpc/).
1. In the navigation pane, choose **Endpoints**.
1. Choose your Storage Gateway VPC endpoint.
1. Choose **Actions**.
1. Choose **Manage Private DNS Names**.
1. For **Enable Private DNS Name**, clear **Enable for this Endpoint**.
1. Choose **Modify Private DNS Names** to save the setting.
# Troubleshooting on-premises gateway issues
You can find information following about typical issues that you might encounter working with your on-premises gateways, and how to activate Support to help troubleshoot your gateway.
The following table lists typical issues that you might encounter working with your on-premises gateways.
| Issue | Action to Take |
| --- | --- |
| You cannot find the IP address of your gateway. | Use the hypervisor client to connect to your host to find the gateway IP address. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-on-premises-gateway-issues.html) If you are still having trouble finding the gateway IP address: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-on-premises-gateway-issues.html) |
| You're having network or firewall problems. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-on-premises-gateway-issues.html) |
| Your gateway's activation fails when you click the **Proceed to Activation** button in the Storage Gateway Management Console. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-on-premises-gateway-issues.html) |
| You need to remove a disk allocated as upload buffer space. For example, you might want to reduce the amount of upload buffer space for a gateway, or you might need to replace a disk used as an upload buffer that has failed. | For instructions about removing a disk allocated as upload buffer space, see [Removing Disks from Your Gateway](add-remove-disks.md). |
| You need to improve bandwidth between your gateway and AWS. | You can improve the bandwidth from your gateway to AWS by setting up your internet connection to AWS on a network adapter (NIC) separate from that connecting your applications and the gateway VM. Taking this approach is useful if you have a high-bandwidth connection to AWS and you want to avoid bandwidth contention, especially during a snapshot restore. For high-throughput workload needs, you can use [Direct Connect](https://aws.amazon.com/directconnect/) to establish a dedicated network connection between your on-premises gateway and AWS. To measure the bandwidth of the connection from your gateway to AWS, use the `CloudBytesDownloaded` and `CloudBytesUploaded` metrics of the gateway. For more on this subject, see [Measuring Performance Between Your Tape Gateway and AWS](PerfGatewayAWS-vtl-common.md). Improving your internet connectivity helps to ensure that your upload buffer does not fill up. |
| Throughput to or from your gateway drops to zero. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-on-premises-gateway-issues.html) You can view the throughput to and from your gateway from the Amazon CloudWatch console. For more information about measuring throughput to and from your gateway and AWS, see [Measuring Performance Between Your Tape Gateway and AWS](PerfGatewayAWS-vtl-common.md). |
| You are having trouble importing (deploying) Storage Gateway on Microsoft Hyper-V. | See [Troubleshooting Microsoft Hyper-V setup](troubleshooting-hyperv-setup.md), which discusses some of the common issues of deploying a gateway on Microsoft Hyper-V. |
| You receive a message that says: "The data that has been written to the volume in your gateway isn't securely stored at AWS". | You receive this message if your gateway VM was created from a clone or snapshot of another gateway VM. If this isn’t the case, contact Support. |
## Allowing Support to help troubleshoot your gateway hosted on-premises
Storage Gateway provides a local console you can use to perform several maintenance tasks, including activating Support to access your gateway to assist you with troubleshooting gateway issues. By default, Support access to your gateway is deactivated. You provide this access through the host's local console. To give Support access to your gateway, you first log in to the local console for the host, navigate to the Storage Gateway's console, and then connect to the support server.
**To allow Support access to your gateway**
1. Log in to your host's local console.
+ VMware ESXi – for more information, see [Accessing the Gateway Local Console with VMware ESXi](accessing-local-console.md#MaintenanceConsoleWindowVMware-common).
+ Microsoft Hyper-V – for more information, see [Access the Gateway Local Console with Microsoft Hyper-V](accessing-local-console.md#MaintenanceConsoleWindowHyperV-common).
1. At the prompt, enter the corresponding numeral to select **Gateway Console**.
1. Enter **h** to open the list of available commands.
1.
Do one of the following:
+ If your gateway is using a public endpoint, in the **AVAILABLE COMMANDS** window, enter **open-support-channel** to connect to customer support for Storage Gateway. Allow TCP port 22 so you can open a support channel to AWS. When you connect to customer support, Storage Gateway assigns you a support number. Make a note of your support number.
+ If your gateway is using a VPC endpoint, in the **AVAILABLE COMMANDS** window, enter **open-support-channel**. If your gateway is not activated, provide the VPC endpoint or IP address to connect to customer support for Storage Gateway. Allow TCP port 22 so you can open a support channel to AWS. When you connect to customer support, Storage Gateway assigns you a support number. Make a note of your support number.
**Note**
The channel number is not a Transmission Control Protocol/User Datagram Protocol (TCP/UDP) port number. Instead, the gateway makes a Secure Shell (SSH) (TCP 22) connection to Storage Gateway servers and provides the support channel for the connection.
1. After the support channel is established, provide your support service number to Support so Support can provide troubleshooting assistance.
1. When the support session is completed, enter **q** to end it. Don't close the session until Amazon Web Services Support notifies you that the support session is complete.
1. Enter **exit** to log out of the gateway console.
1. Follow the prompts to exit the local console.
# Troubleshooting Microsoft Hyper-V setup
The following table lists typical issues that you might encounter when deploying Storage Gateway on the Microsoft Hyper-V platform.
| Issue | Action to Take |
| --- | --- |
| You try to import a gateway and receive the following error message: "A server error occurred while attempting to import the virtual machine. Import failed. Unable to find virtual machine import files under location [...]. You can import a virtual machine only if you used Hyper-V to create and export it." | This error can occur for the following reasons: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/storagegateway/latest/tgw/troubleshooting-hyperv-setup.html) |
| You try to import a gateway and receive the following error message: "A server error occurred while attempting to import the virtual machine. Import failed. Import task failed to copy file from [...]: The file exists. (0x80070050)" | If you have already deployed a gateway and you try to reuse the default folders that store the virtual hard disk files and virtual machine configuration files, then this error will occur. To fix this problem, specify new locations under **Server** in the panel on the left side of the **Hyper-V Settings** dialog box. |
| You try to import a gateway and receive the following error message: "A server error occurred while attempting to import the virtual machine. Import failed. Import failed because the virtual machine must have a new identifier. Select a new identifier and try the import again." | When you import the gateway make sure you select **Copy the virtual machine** and check the **Duplicate all files** box in the **Import Virtual Machine** dialog box to create a new unique ID for the VM. |
| You try to start a gateway VM and receive the following error message: "An error occurred while attempting to start the selected virtual machine(s). The child partition processor setting is incompatible with parent partition. 'AWS-Storage-Gateway' could not initialize. (Virtual machine ID [...])" | This error is likely caused by a CPU discrepancy between the required CPUs for the gateway and the available CPUs on the host. Ensure that the VM CPU count is supported by the underlying hypervisor. For more information about the requirements for Storage Gateway, see [Requirements for setting up Tape Gateway](Requirements.md). |
| You try to start a gateway VM and receive the following error message: "An error occurred while attempting to start the selected virtual machine(s). 'AWS-Storage-Gateway' could not initialize. (Virtual machine ID [...]) Failed to create partition: Insufficient system resources exist to complete the requested service. (0x800705AA)" | This error is likely caused by a RAM discrepancy between the required RAM for the gateway and the available RAM on the host. For more information about the requirements for Storage Gateway, see [Requirements for setting up Tape Gateway](Requirements.md). |
| Your snapshots and gateway software updates are occurring at slightly different times than expected. | The gateway VM's clock might be offset from the actual time, known as clock drift. Check and correct the VM's time using local gateway console's time synchronization option. For more information, see [Synchronize VM time with Hyper-V or Linux KVM host time](MaintenanceTimeSync-hyperv.md). |
| You need to put the unzipped Microsoft Hyper-V Storage Gateway files on the host file system. | Access the host as you do a typical Microsoft Windows server. For example, if the hypervisor host is name `hyperv-server`, then you can use the following UNC path `\\hyperv-server\c$`, which assumes that the name `hyperv-server` can be resolved or is defined in your local hosts file. |
| You are prompted for credentials when connecting to hypervisor. | Add your user credentials as a local administrator for the hypervisor host by using the Sconfig.cmd tool. |
| You may notice poor network performance if you turn on virtual machine queue (VMQ) for a Hyper-V host that's using a Broadcom network adapter. | For information about a workaround, see the Microsoft documentation, see [Poor network performance on virtual machines on a Windows Server 2012 Hyper-V host if VMQ is turned on](https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/poor-network-performance-hyper-v-host-vm). |
# Troubleshooting Amazon EC2 gateway issues
In the following sections, you can find typical issues that you might encounter working with your gateway deployed on Amazon EC2. For more information about the difference between an on-premises gateway and a gateway deployed in Amazon EC2, see [Deploy a customized Amazon EC2 instance for Tape Gateway](ec2-gateway-common.md).
**Topics**
+ [Your gateway activation hasn't occurred after a few moments](#activation-issues)
+ [You can't find your EC2 gateway instance in the instance list](#find-instance)
+ [You created an Amazon EBS volume but can't attach it to your EC2 gateway instance](#ebs-volume-issue)
+ [You get a message that you have no disks available when you try to add storage volumes](#no-disk)
+ [You want to remove a disk allocated as upload buffer space to reduce upload buffer space](#uploadbuffer-issue)
+ [Throughput to or from your EC2 gateway drops to zero](#gateway-throughput-issue)
+ [You want Support to help troubleshoot your EC2 gateway](#EC2-EnableAWSSupportAccess)
+ [You want to connect to your gateway instance using the Amazon EC2 serial console](#ec2-serial-console)
## Your gateway activation hasn't occurred after a few moments
Check the following in the Amazon EC2 console:
+ Port 80 is activated in the security group that you associated with the instance. For more information about adding a security group rule, see [Adding a security group rule](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html#adding-security-group-rule) in the *Amazon EC2 User Guide*.
+ The gateway instance is marked as running. In the Amazon EC2 console, the **State** value for the instance should be RUNNING.
+ Make sure that your Amazon EC2 instance type meets the minimum requirements, as described in [Storage requirements](Requirements.md#requirements-storage).
After correcting the problem, try activating the gateway again. To do this, open the Storage Gateway console, choose **Deploy a new Gateway on Amazon EC2**, and re-enter the IP address of the instance.
## You can't find your EC2 gateway instance in the instance list
If you didn't give your instance a resource tag and you have many instances running, it can be hard to tell which instance you launched. In this case, you can take the following actions to find the gateway instance:
+ Check the name of the Amazon Machine Image (AMI) on the **Description** tab of the instance. An instance based on the Storage Gateway AMI should start with the text **aws-storage-gateway-ami**.
+ If you have several instances based on the Storage Gateway AMI, check the instance launch time to find the correct instance.
## You created an Amazon EBS volume but can't attach it to your EC2 gateway instance
Check that the Amazon EBS volume in question is in the same Availability Zone as the gateway instance. If there is a discrepancy in Availability Zones, create a new Amazon EBS volume in the same Availability Zone as your instance.
## You get a message that you have no disks available when you try to add storage volumes
For a newly activated gateway, no volume storage is defined. Before you can define volume storage, you must allocate local disks to the gateway to use as an upload buffer and cache storage. For a gateway deployed to Amazon EC2, the local disks are Amazon EBS volumes attached to the instance. This error message likely occurs because no Amazon EBS volumes are defined for the instance.
Check block devices defined for the instance that is running the gateway. If there are only two block devices (the default devices that come with the AMI), then you should add storage. For more information on doing so, see [Deploy a customized Amazon EC2 instance for Tape Gateway](ec2-gateway-common.md). After attaching two or more Amazon EBS volumes, try creating volume storage on the gateway.
## You want to remove a disk allocated as upload buffer space to reduce upload buffer space
Follow the steps in [Determining the size of upload buffer to allocate](decide-local-disks-and-sizes.md#CachedLocalDiskUploadBufferSizing-common).
## Throughput to or from your EC2 gateway drops to zero
Verify that the gateway instance is running. If the instance is starting due to a reboot, for example, wait for the instance to restart.
Also, verify that the gateway IP has not changed. If the instance was stopped and then restarted, the IP address of the instance might have changed. In this case, you need to activate a new gateway.
You can view the throughput to and from your gateway from the Amazon CloudWatch console. For more information about measuring throughput to and from your gateway and AWS, see [Measuring Performance Between Your Tape Gateway and AWS](PerfGatewayAWS-vtl-common.md).
## You want Support to help troubleshoot your EC2 gateway
Storage Gateway provides a local console you can use to perform several maintenance tasks, including activating Support to access your gateway to assist you with troubleshooting gateway issues. By default, Support access to your gateway is deactivated. You provide this access through the Amazon EC2 local console. You log in to the Amazon EC2 local console through a Secure Shell (SSH). To successfully log in through SSH, your instance's security group must have a rule that opens TCP port 22.
**Note**
If you add a new rule to an existing security group, the new rule applies to all instances that use that security group. For more information about security groups and how to add a security group rule, see [Amazon EC2 security groups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html) in the *Amazon EC2 User Guide*.
To let Support connect to your gateway, you first log in to the local console for the Amazon EC2 instance, navigate to the Storage Gateway's console, and then provide the access.
**To activate Support access to a gateway deployed on an Amazon EC2 instance**
1. Log in to the local console for your Amazon EC2 instance. For instructions, go to [Connect to your instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html) in the *Amazon EC2 User Guide*.
You can use the following command to log in to the EC2 instance's local console.
```
ssh –i PRIVATE-KEY admin@INSTANCE-PUBLIC-DNS-NAME
```
**Note**
The *PRIVATE-KEY* is the `.pem` file containing the private certificate of the EC2 key pair that you used to launch the Amazon EC2 instance. For more information, see [Retrieving the public key for your key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#retriving-the-public-key) in the *Amazon EC2 User Guide*.
The *INSTANCE-PUBLIC-DNS-NAME* is the public Domain Name System (DNS) name of your Amazon EC2 instance that your gateway is running on. You obtain this public DNS name by selecting the Amazon EC2 instance in the EC2 console and clicking the **Description** tab.
1. At the prompt, enter **6 - Command Prompt** to open the Support Channel console.
1. Enter **h** to open the **AVAILABLE COMMANDS** window.
1. Do one of the following:
+ If your gateway is using a public endpoint, in the **AVAILABLE COMMANDS** window, enter **open-support-channel** to connect to customer support for Storage Gateway. Allow TCP port 22 so you can open a support channel to AWS. When you connect to customer support, Storage Gateway assigns you a support number. Make a note of your support number.
+ If your gateway is using a VPC endpoint, in the **AVAILABLE COMMANDS** window, enter **open-support-channel**. If your gateway is not activated, provide the VPC endpoint or IP address to connect to customer support for Storage Gateway. Allow TCP port 22 so you can open a support channel to AWS. When you connect to customer support, Storage Gateway assigns you a support number. Make a note of your support number.
**Note**
The channel number is not a Transmission Control Protocol/User Datagram Protocol (TCP/UDP) port number. Instead, the gateway makes a Secure Shell (SSH) (TCP 22) connection to Storage Gateway servers and provides the support channel for the connection.
1. After the support channel is established, provide your support service number to Support so Support can provide troubleshooting assistance.
1. When the support session is completed, enter **q** to end it. Don't close the session until Support notifies you that the support session is complete.
1. Enter **exit** to exit the Storage Gateway console.
1. Follow the console menus to log out of the Storage Gateway instance.
## You want to connect to your gateway instance using the Amazon EC2 serial console
You can use the Amazon EC2 serial console to troubleshoot boot, network configuration, and other issues. For instructions and troubleshooting tips, see [Amazon EC2 Serial Console](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-serial-console.html) in the *Amazon Elastic Compute Cloud User Guide*.
# Troubleshooting hardware appliance issues
The following topics discuss issues that you might encounter with the Storage Gateway Hardware Appliance, and suggestions on troubleshooting these.
## You can't determine the service IP address
When attempting to connect to your service, make sure that you are using the service's IP address and not the host IP address. Configure the service IP address in the service console, and the host IP address in the hardware console. You see the hardware console when you start the hardware appliance. To go to the service console from the hardware console, choose **Open Service Console**.
## How do you perform a factory reset?
If you need to perform a factory reset on your appliance, contact the Storage Gateway Hardware Appliance team for support, as described in the Support section following.
## How do you perform a remote restart?
If you need to perform a remote restart of your appliance, you can do so using the Dell iDRAC management interface. For more information, see [iDRAC9 Virtual Power Cycle: Remotely power cycle Dell EMC PowerEdge Servers](https://infohub.delltechnologies.com/en-us/p/idrac9-virtual-power-cycle-remotely-power-cycle-dell-emc-poweredge-servers/) on the Dell Technologies InfoHub website.
## Where do you obtain Dell iDRAC support?
The Dell PowerEdge server comes with the Dell iDRAC management interface. We recommend the following:
+ If you use the iDRAC management interface, you should change the default password. For more information about the iDRAC credentials, see [Dell PowerEdge - What is the default sign-in credentials for iDRAC?](https://www.dell.com/support/article/en-us/sln306783/dell-poweredge-what-is-the-default-username-and-password-for-idrac?lang=en).
+ Make sure that the firmware is up-to-date to prevent security breaches.
+ Moving the iDRAC network interface to a normal (`em`) port can cause performance issues or prevent the normal functioning of the appliance.
## You can't find the hardware appliance serial number
You can find the serial number for your Storage Gateway Hardware Appliance using the Storage Gateway console.
**To find the hardware appliance serial number:**
1. Open the Storage Gateway console at [https://console.aws.amazon.com/storagegateway/home](https://console.aws.amazon.com/storagegateway/).
1. Choose **Hardware** from the navigation menu on the left side of the page.
1. Select your hardware appliance from the list.
1. Locate the **Serial Number** field on the **Details** tab for your appliance.
## Where to obtain hardware appliance support
To contact AWS about technical support for your hardware appliance, see [Support](https://aws.amazon.com/contact-us).
The Support team might ask you to activate the support channel to troubleshoot your gateway issues remotely. You don't need this port to be open for the normal operation of your gateway, but it is required for troubleshooting. You can activate the support channel from the hardware console as shown in the procedure following.
**To open a support channel for AWS**
1. Open the hardware console.
1. Choose **Open Support Channel** at the bottom of the main page of the hardware console, and then press `Enter`.
The assigned port number should appear within 30 seconds if there are no network connectivity or firewall issues. For example:
**Status: Open on port 19599**
1. Note the port number and provide it to Support.
# Troubleshooting virtual tape issues
You can find information following about actions to take if you experience unexpected issues with your virtual tapes.
**Topics**
+ [Recovering a Virtual Tape From An Unrecoverable Gateway](#recovery-tapes)
+ [Troubleshooting Irrecoverable Tapes](#IrrecoverableTapes)
+ [High Availability Health Notifications](#troubleshooting-ha-notifications)
## Recovering a Virtual Tape From An Unrecoverable Gateway
Although it is rare, your Tape Gateway might encounter an unrecoverable failure. Such a failure can occur in your hypervisor host, the gateway itself, or the cache disks. If a failure occurs, you can recover your tapes by following the troubleshooting instructions in this section.
**Topics**
+ [You Need to Recover a Virtual Tape from a Malfunctioning Tape Gateway](#creating-recovery-tape-vtl)
+ [You Need to Recover a Virtual Tape from a Malfunctioning Cache Disk](#recover-from-failed-disk)
### You Need to Recover a Virtual Tape from a Malfunctioning Tape Gateway
If your Tape Gateway or the hypervisor host encounters an unrecoverable failure, you can recover any data that has already been uploaded to AWS to another Tape Gateway.
Note that the data written to a tape might not be completely uploaded until that tape has been successfully archived into VTS. The data on tapes recovered to another gateway in this manner may be incomplete or empty. We recommend performing an inventory on all recovered tapes to ensure they contain the expected content.
**To recover a tape to another Tape Gateway**
1. Identify an existing functioning Tape Gateway to serve as your recovery target gateway. If you don't have a Tape Gateway to recover your tapes to, create a new Tape Gateway. For information about how to create a gateway, see [Creating a Gateway](https://docs.aws.amazon.com/storagegateway/latest/tgw/create-gateway-vtl.html).
1. Open the Storage Gateway console at [https://console.aws.amazon.com/storagegateway/home](https://console.aws.amazon.com/storagegateway/).
1. In the navigation pane, choose **Gateways**, and then choose the Tape Gateway you want to recover tapes from.
1. Choose the **Details** tab. A tape recovery message is displayed in the tab.
1. Choose **Create recovery tapes** to deactivate the gateway.
1. In the dialog box that appears, choose **Disable gateway**.
This process permanently halts normal function of your Tape Gateway and exposes any available recovery points. For instructions, see [Deactivating your Tape Gateway](https://docs.aws.amazon.com/storagegateway/latest/tgw/disabling-gateway-vtl.html).
1. From the tapes that the deactivated gateway displays, choose the virtual tape and the recovery point you want to recover. A virtual tape can have multiple recovery points.
1. To begin recovering any tapes you need to the target Tape Gateway, choose **Create recovery tape**.
1. In the **Create recovery tape** dialog box, verify the barcode of the virtual tape you want to recover.
1. For **Gateway**, choose the Tape Gateway you want to recover the virtual tape to.
1. Choose **Create recovery tape**.
1. Delete the failed Tape Gateway so you don't get charged. For instructions, see [Deleting your gateway and removing associated resources](deleting-gateway-common.md).
Storage Gateway moves the tape from the failed Tape Gateway to the Tape Gateway you specified. The Tape Gateway marks the tape status as RECOVERED.
### You Need to Recover a Virtual Tape from a Malfunctioning Cache Disk
If your cache disk encounters an error, the gateway prevents read and write operations on virtual tapes in the gateway. For example, an error can occur when a disk is corrupted or removed from the gateway. The Storage Gateway console displays a message about the error.
In the error message, Storage Gateway prompts you to take one of two actions that can recover your tapes:
+ **Shut Down and Re-Add Disks **– Take this approach if the disk has intact data and has been removed. For example, if the error occurred because a disk was removed from your host by accident but the disk and the data is intact, you can re-add the disk. To do this, see the procedure later in this topic.
+ **Reset Cache Disk** – Take this approach if the cache disk is corrupted or not accessible. If the disk error causes the cache disk to be inaccessible, unusable, or corrupted, you can reset the disk. If you reset the cache disk, tapes that have clean data (that is, tapes for which data in the cache disk and Amazon S3 are synchronized) will continue to be available for you to use. However, tapes that have data that is not synchronized with Amazon S3 are automatically recovered. The status of these tapes is set to RECOVERED, but the tapes will be read-only. For information about how to remove a disk from your host, see [Determining the size of upload buffer to allocate](decide-local-disks-and-sizes.md#CachedLocalDiskUploadBufferSizing-common).
**Important**
If the cache disk you are resetting contains data that has not been uploaded to Amazon S3 yet, that data can be lost. After you reset cache disks, no configured cache disks will be left in the gateway, so you must configure at least one new cache disk for your gateway to function properly.
To reset the cache disk, see the procedure later in this topic.
**To shut down and re-add a disk**
1. Shut down the gateway. For information about how to shut down a gateway, see [Shutting Down Your Gateway VM](MaintenanceShutDown-common.md).
1. Add the disk back to your host, and make sure the disk node number of the disk has not changed. For information about how to add a disk, see [Determining the size of upload buffer to allocate](decide-local-disks-and-sizes.md#CachedLocalDiskUploadBufferSizing-common).
1. Restart the gateway. For information about how to restart a gateway, see [Shutting Down Your Gateway VM](MaintenanceShutDown-common.md).
After the gateway restarts, you can verify the status of the cache disks. The status of a disk can be one of the following:
+ **present** – The disk is available to use.
+ **missing** – The disk is no longer connected to the gateway.
+ **mismatch** – The disk node is occupied by a disk that has incorrect metadata, or the disk content is corrupted.
**To reset and reconfigure a cache disk**
1. In the **A disk error has occurred** error message illustrated preceding, choose **Reset Cache Disk**.
1. On the **Configure gateway** page, configure the disk for cache storage. For information about how to do so, see [Configure your Tape Gateway](https://docs.aws.amazon.com/storagegateway/latest/tgw/create-gateway-vtl.html#configure-gateway-tape).
1. After you have configured cache storage, shut down and restart the gateway as described in the previous procedure.
The gateway should recover after the restart. You can then verify the status of the cache disk.
**To verify the status of a cache disk**
1. Open the Storage Gateway console at [https://console.aws.amazon.com/storagegateway/home](https://console.aws.amazon.com/storagegateway/).
1. In the navigation pane, choose **Gateways**, and then choose your gateway.
1. For **Actions**, choose **Configure Local Storage** to display the **Configure Local Storage** dialog box. This dialog box shows all local disks in the gateway.
The cache disk node status is displayed next to the disk.
**Note**
If you don't complete the recovery process, the gateway displays a banner that prompts you to configure local storage.
## Troubleshooting Irrecoverable Tapes
If your virtual tape fails unexpectedly, Storage Gateway sets the status of the failed virtual tape to IRRECOVERABLE. The action you take depends on the circumstances. You can find information following on some issues you might find, and how to troubleshoot them.
### You Need to Recover Data From an IRRECOVERABLE Tape
If you have a virtual tape with the status IRRECOVERABLE, and you need to work with it, try one of the following:
+ Activate a new Tape Gateway if you don't have one activated. For more information, see [Creating a Gateway](https://docs.aws.amazon.com/storagegateway/latest/tgw/create-gateway-vtl.html).
+ Deactivate the Tape Gateway that contains the irrecoverable tape, and recover the tape from a recovery point to the new Tape Gateway. For more information, see [You Need to Recover a Virtual Tape from a Malfunctioning Tape Gateway](#creating-recovery-tape-vtl).
**Note**
You have to reconfigure your iSCSI initiator and backup application to use the new Tape Gateway. For more information, see [Connecting your VTL devices](GettingStartedAccessTapesVTL.md).
### You Don't Need an IRRECOVERABLE Tape That Isn't Archived
If you have a virtual tape with the status IRRECOVERABLE, you don't need it, and the tape has never been archived, you should delete the tape. For more information, see [Deleting virtual tapes from your Tape Gateway](deleting-tapes-vtl.md).
### A Cache Disk in Your Gateway Encounters a Failure
If one or more cache disks in your gateway encounters a failure, the gateway prevents read and write operations to your virtual tapes and volumes. To resume normal functionality, reconfigure your gateway as described following:
+ If the cache disk is inaccessible or unusable, delete the disk from your gateway configuration.
+ If the cache disk is still accessible and useable, reconnect it to your gateway.
**Note**
If you delete a cache disk, tapes or volumes that have clean data (that is, for which data in the cache disk and Amazon S3 are synchronized) will continue to be available when the gateway resumes normal functionality. For example, if your gateway has three cache disks and you delete two, tapes or volumes that are clean will have AVAILABLE status. Other tapes and volumes will have IRRECOVERABLE status.
If you use ephemeral disks as cache disks for your gateway or mount your cache disks on an ephemeral drive, your cache disks will be lost when you shut down the gateway. Shutting down the gateway when your cache disk and Amazon S3 are not synchronized can result in data loss. As a result, we don't recommend using ephemeral drives or disks.
## High Availability Health Notifications
When running your gateway on the VMware vSphere High Availability (HA) platform, you may receive health notifications. For more information about health notifications, see [Troubleshooting high availability issues](troubleshooting-ha-issues.md).
# Troubleshooting high availability issues
You can find information following about actions to take if you experience availability issues.
**Topics**
+ [Health notifications](#ha-health-notifications)
+ [Metrics](#ha-health-notification-metrics)
## Health notifications
When you run your gateway on VMware vSphere HA, all gateways produce the following health notifications to your configured Amazon CloudWatch log group. These notifications go into a log stream called `AvailabilityMonitor`.
**Topics**
+ [Notification: Reboot](#troubleshoot-reboot-notification)
+ [Notification: HardReboot](#troubleshoot-hardreboot-notification)
+ [Notification: HealthCheckFailure](#troubleshoot-healthcheckfailure-notification)
+ [Notification: AvailabilityMonitorTest](#troubleshoot-availabilitymonitortest-notification)
### Notification: Reboot
You can get a reboot notification when the gateway VM is restarted. You can restart a gateway VM by using the VM Hypervisor Management console or the Storage Gateway console. You can also restart by using the gateway software during the gateway's maintenance cycle.
**Action to Take**
If the time of the reboot is within 10 minutes of the gateway's configured [maintenance start time](MaintenanceManagingUpdate-common.md), this is probably a normal occurrence and not a sign of any problem. If the reboot occurred significantly outside the maintenance window, check whether the gateway was restarted manually.
### Notification: HardReboot
You can get a `HardReboot` notification when the gateway VM is restarted unexpectedly. Such a restart can be due to loss of power, a hardware failure, or another event. For VMware gateways, a reset by vSphere High Availability Application Monitoring can launch this event.
**Action to Take**
When your gateway runs in such an environment, check for the presence of the `HealthCheckFailure` notification and consult the VMware events log for the VM.
### Notification: HealthCheckFailure
For a gateway on VMware vSphere HA, you can get a `HealthCheckFailure` notification when a health check fails and a VM restart is requested. This event also occurs during a test to monitor availability, indicated by an `AvailabilityMonitorTest` notification. In this case, the `HealthCheckFailure` notification is expected.
**Note**
This notification is for VMware gateways only.
**Action to Take**
If this event repeatedly occurs without an `AvailabilityMonitorTest` notification, check your VM infrastructure for issues (storage, memory, and so on). If you need additional assistance, contact Support.
### Notification: AvailabilityMonitorTest
For a gateway on VMware vSphere HA, you can get an `AvailabilityMonitorTest` notification when you [run a test](vmware-ha.md#vmware-ha-test-failover) of the [Availability and application monitoring](https://docs.aws.amazon.com/storagegateway/latest/APIReference/API_StartAvailabilityMonitorTest.html) system in VMware.
## Metrics
The `AvailabilityNotifications` metric is available on all gateways. This metric is a count of the number of availability-related health notifications generated by the gateway. Use the `Sum` statistic to observe whether the gateway is experiencing any availability-related events. Consult with your configured CloudWatch log group for details about the events.