# Domain management
Domain management

This chapter describes how to manage Amazon OpenSearch Service domains through the Centralized Logging with OpenSearch console. An Amazon OpenSearch Service domain is synonymous with an Amazon OpenSearch Service cluster.

In this chapter, you will learn:
+  [Import and remove an Amazon OpenSearch Service Domain](domain-operations.md) 
+  [Create an access proxy](access-proxy-1.md) 
+  [Create recommended alarms](domain-alarms.md) 

You can read the [Getting Started](getting-started.md) chapter first and walk through the basic steps for using the Centralized Logging with OpenSearch solution.

# Domain Operations


Once logged into the Centralized Logging with OpenSearch console, you can import an Amazon OpenSearch Service domain.

 **Prerequisites** 

1. Centralized Logging with OpenSearch supports Amazon OpenSearch Service, and engine version OpenSearch 1.3 or later.

1. Centralized Logging with OpenSearch supports OpenSearch clusters within VPC. If you don’t have an Amazon OpenSearch Service domain yet, you can create an Amazon OpenSearch Service domain within VPC. See [Launching your Amazon OpenSearch Service domains within a VPC](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/vpc.html).

1. Centralized Logging with OpenSearch supports OpenSearch clusters with [fine-grained access control](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html) only. In the security configuration, the Access policy should look like the following image:

    **Sample access policy.**   
![\[image27\]](http://docs.aws.amazon.com/solutions/latest/centralized-logging-with-opensearch/images/image27.png)

## Import an Amazon OpenSearch Service Domain


1. Sign in to the Centralized Logging with OpenSearch console.

1. In the left navigation panel, under **Domains**, choose **Import OpenSearch Domain**.

1. On the **Select domain** page, choose a domain from the dropdown list. The dropdown list will display only domains in the same Region as the solution.

1. Choose **Next**.

1. On the **Configure network** page, under **Network creation**, choose **Manual** and choose **Next**; or choose **Automatic**, and go to step 9.

1. Under **VPC**, choose a VPC from the list. By default, the solution creates a standalone VPC, and you can choose the one named `LogHubVpc/DefaultVPC`. You can also choose the same VPC as your Amazon OpenSearch Service domains.

1. Under **Log Processing Subnet Group**, select at least 2 subnets from the dropdown list. By default, the solution creates two private subnets. You can choose subnets named `LogHubVpc/DefaultVPC/privateSubnet1` and `LogHubVpc/DefaultVPC/privateSubnet2`.

1. Under **Log Processing Security Group**, select one from the dropdown list. By default, the solution creates one Security Group named `ProcessSecurityGroup`.

1. On the **Create tags** page, add tags if needed.

1. Choose **Import**.

## Set up VPC Peering


By default, the solution creates a standalone VPC. You must create VPC Peering to allow the log processing layer to have access to your Amazon OpenSearch Service domains.

**Note**  
Automatic mode will create VPC peering and configure route table automatically. You do not need to set up VPC peering again.

 **VPC peering connecting the solution and an OpenSearch VPC.** 

![\[setup vpc peering\]](http://docs.aws.amazon.com/solutions/latest/centralized-logging-with-opensearch/images/setup-vpc-peering.png)


Follow this section to create VPC peering, update your security group, and update route tables.

### Create VPC Peering Connection


1. Sign in to the Centralized Logging with OpenSearch console.

1. In the left navigation panel, under **Domains**, select **OpenSearch Domains**.

1. Find the domain that you imported and select the domain name.

1. Choose the **Network** tab.

1. Copy the VPC ID in both sections **OpenSearch domain network** and **Log processing network**. You will create a Peering Connection between these two VPCs.

1. Navigate to [VPC Console Peering Connections](https://console.aws.amazon.com/vpc/home#PeeringConnections).

1. Select the Create peering connection button.

1. On the **Create peering connection** page, enter a name.

1. For the Select a local VPC to peer with, VPC ID (Requester), select the VPC ID of the Log processing network.

1. For the Select another VPC to peer with, VPC ID (Accepter), select the VPC ID of the OpenSearch domain network.

1. Choose **Create peering connection**, and navigate to the peering connection detail page.

1. Choose the **Actions** button and choose **Accept request**.

### Update Route Tables


1. Go to the Centralized Logging with OpenSearch console.

1. In the **OpenSearch domain network** section, choose the subnet under **Availability Zone and Subnets** to open the subnet console in a new tab.

1. Select the subnet, and choose the **Route table** tab.

1. Select the associated route table of the subnet to open the route table configuration page.

1. Select the **Routes** tab, and choose **Edit routes**.

1. Add a route 10.255.0.0/16 (the CIDR of Centralized Logging with OpenSearch, if you created the solution with existing VPC, change this value) pointing to the Peering Connection you created.

1. Go back to the Centralized Logging with OpenSearch console.

1. Choose the VPC ID under the **OpenSearch domain network** section.

1. Select the VPC ID on the VPC Console and find its **IPv4 CIDR**.

1. On the Centralized Logging with OpenSearch console, in the **Log processing network** section, choose the subnets under **Availability Zone and Subnets** to open the subnets in new tabs.

1. Repeat step 3, 4, 5, 6 to add an opposite route. Namely, configure the IPv4 CIDR of the OpenSearch VPC to point to the Peering Connection. You must repeat the steps for each subnet of the Log processing network.

### Update Security Group of OpenSearch Domain


1. On the Centralized Logging with OpenSearch console, under the **OpenSearch domain network** section, select the Security Group ID in **Security Groups** to open the Security Group in a new tab.

1. On the console, select **Edit inbound rules**.

1. Add the rule `ALLOW TCP/443 from 10.255.0.0/16` (the CIDR of Centralized Logging with OpenSearch, if you created Centralized Logging with OpenSearch with existing VPC, change this value).

1. Choose **Save rules**.

**Note**  
If you prefer to use Transit Gateway rather than VPC peering for connectivity between OpenSearch domain VPC and the solution VPC, select manual network create option during domain import. After creation, you’ll need to configure your route tables to direct traffic through the Transit Gateway instead of VPC peering connection.

## Remove an Amazon OpenSearch Service domain


If needed, you can remove the Amazon OpenSearch Service domains.

**Important**  
Removing the domain from Centralized Logging with OpenSearch will NOT delete the Amazon OpenSearch Service domain in your AWS account. It will NOT impact any existing log analytics pipelines.

1. Sign in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch Domains**.

1. Select the domain from the table.

1. Choose **Remove**.

1. In the confirmation dialog box, choose **Remove**.

# Access proxy


By default, an Amazon OpenSearch Service domain within VPC cannot be accessed from the internet. Centralized Logging with OpenSearch creates a highly available [NGINX cluster](https://aws.amazon.com/premiumsupport/knowledge-center/opensearch-outside-vpc-nginx/) that allows you to access the OpenSearch Dashboards from the internet. Alternatively, you can choose to access the Amazon OpenSearch Service domains using [SSH Tunnel](https://aws.amazon.com/premiumsupport/knowledge-center/opensearch-outside-vpc-ssh/). Refer to the [Access proxy architecture](access-proxy.md) for more implementation details.

This section covers the following:

1.  [Create a proxy](#create-a-proxy) 

1.  [Create an associated DNS record](#create-an-associated-dns-record) 

1.  [Access Amazon OpenSearch Service via proxy](#access-amazon-opensearch-service-via-proxy) 

1.  [Delete a proxy](#delete-a-proxy) 

## Create a proxy


You can create the NGINX-based proxy using the Centralized Logging with OpenSearch console or by deploying a standalone CloudFormation stack.

 **Prerequisites** 
+ Make sure an Amazon OpenSearch Service **domain** within VPC is available.
+ The domain associated **SSL certificate** is created or uploaded in [AWS Certificate Manager (ACM)](https://aws.amazon.com/certificate-manager).
+ Make sure you have the EC2 private key (.pem) file.

### （Option 1）Using the Centralized Logging with OpenSearch console


1. Log in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch domains**.

1. Select the domain from the table.

1. Under **General configuration**, choose **Enable** at the **Access Proxy** label.
**Note**  
Once the access proxy is enabled, a link to the access proxy will be available.

1. On the **Create access proxy** page, choose the **Proxy Instance Type** and **Proxy Instance Number**.

1. Under **Public access proxy**, select at least 2 subnets for **Public Subnets**. You can choose 2 public subnets named `CLVPC/DefaultVPC/publicSubnetX`, which are created by Centralized Logging with OpenSearch by default.

1. Choose a Security Group of the Application Load Balancer in **Public Security Group**. You can choose a security group named `ProxySecurityGroup`, which is created by Centralized Logging with OpenSearch default.

1. Choose the NGINX Instance Key Name.

1. Enter the **Domain Name**.

1. Choose **Load Balancer SSL Certificate** associated with the domain name.

1. Choose **Create**.

### (Option 2) Using the CloudFormation stack


This automated AWS CloudFormation template deploys the *Centralized Logging with OpenSearch - NGINX access proxy* solution in the AWS Cloud.

1. Log in to the AWS Management Console and select the button to launch the AWS CloudFormation template. [https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Fsolutions-reference%2Fcentralized-logging-with-opensearch%2Flatest%2FNginxForOpenSearch.template](https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Fsolutions-reference%2Fcentralized-logging-with-opensearch%2Flatest%2FNginxForOpenSearch.template) 

You can also [download the template](https://s3.amazonaws.com/solutions-reference/centralized-logging-with-opensearch/latest/NginxForOpenSearch.template) as a starting point for your own implementation. . To launch the stack in a different AWS Region, use the Region selector in the console navigation bar. . On the **Create stack** page, verify that the correct template URL shows in the **Amazon S3 URL** text box and choose **Next**. . On the **Specify stack details** page, assign a name to your stack. . Under **Parameters**, review the parameters for the template and modify them as necessary. This solution uses the following parameters.

\$1


| Parameter | Default | Description | 
| --- | --- | --- | 
|  VPCId  |   *<Requires input>*   |  The VPC to deploy the NGINX proxy resources, for example, `vpc-bef13dc7`.  | 
|  PublicSubnetIds  |   *<Requires input>*   |  The public subnets where Elastic Load Balancing is deployed. You must select at least two public subnets, for example, `subnet-12345abc`, `subnet-54321cba`.  | 
|  ELBSecurityGroupId  |   *<Requires input>*   |  The security group being associated with the Elastic Load Balancing, for example, `sg-123456`.  | 
|  ELBDomain  |   *<Requires input>*   |  The custom domain name of the Elastic Load Balancing, for example, `dashboard.example.com`.  | 
|  ELBDomainCertificateArn  |   *<Requires input>*   |  The SSL certificate ARN associated with the ELBDomain. The certificate must be created from ACM.  | 
|  PrivateSubnetIds  |   *<Requires input>*   |  The private subnets where NGINX instances are deployed. You must select at least two private subnets, for example, `subnet-12345abc`, `subnet-54321cba`.  | 
|  NginxSecurityGroupId  |   *<Requires input>*   |  The security group associated with the NGINX instances. The security group must allow access from Elastic Load Balancing security group.  | 
|  KeyName  |   *<Requires input>*   |  The PEM key name of the NGINX instances.  | 
|  EngineType  |   `OpenSearch`   |  The engine type of the OpenSearch. Select OpenSearch.  | 
|  Endpoint  |   *<Requires input>*   |  The OpenSearch endpoint, for example, `vpc-your_opensearch_domain_name-xcvgw6uu2o6zafsiefxubwuohe.us-east-1.es.amazonaws.com`.  | 
|  CognitoEndpoint  |   *Optional input*   |  The Amazon Cognito User Pool endpoint URL of the OpenSearch domain, for example, mydomain.auth.us-east-1.amazoncognito.com. Leave empty if your OpenSearch domain is not authenticated through Amazon Cognito User Pool.  | 

1. Choose **Next**.

1. On the **Configure stack options** page, choose **Next**.

1. On the **Review and create** page, review and confirm the settings. Check the box acknowledging that the template creates IAM resources.

1. Choose **Submit** to deploy the stack.

You can view the status of the stack in the AWS CloudFormation console in the **Status** column. You should receive a **CREATE\$1COMPLETE** status in approximately 15 minutes.

### Recommended Proxy Configuration


The following table provides a list of recommended proxy configuration examples for different number of concurrent users. You can create a proxy according to your own use cases.


| Number of Concurrent Users | Proxy Instance Type | Number of Proxy Instances | 
| --- | --- | --- | 
|  4  |  t3.nano  |  1  | 
|  6  |  t3.micro  |  1  | 
|  8  |  t3.nano  |  2  | 
|  10  |  t3.small  |  1  | 
|  12  |  t3.micro  |  2  | 
|  20  |  t3.small  |  2  | 
|  25  |  t3.large  |  1  | 
|  50\$1  |  t3.large  |  2  | 

## Create an associated DNS record


After provisioning the proxy infrastructure, you must create an associated DNS record in your DNS resolver. The following introduces how to find the Application Load Balancer domain, and then create a CNAME record pointing to this domain.

1. Log in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch domains**.

1. Select the domain from the table.

1. Choose the **Access Proxy** tab. You can see the **Load Balancer Domain**, which is the Application Load Balancer domain.

1. Go to the DNS resolver, create a CNAME record pointing to this domain. If your domain is managed by [Amazon Route 53](https://aws.amazon.com/route53), refer to [Creating records by using the Amazon Route 53 console](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-creating.html).

## Access Amazon OpenSearch Service via proxy


After the DNS record takes effect, you can access the Amazon OpenSearch Service built-in dashboard from anywhere via proxy. You can enter the domain of the proxy in your browser, or choose the **Link** button under **Access Proxy** in the **General Configuration** section.

 **Example General configuration screen.** 

![\[image30\]](http://docs.aws.amazon.com/solutions/latest/centralized-logging-with-opensearch/images/image30.png)


## Delete a Proxy


1. Log in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch domains**.

1. Select the domain from the table.

1. Choose the **Access Proxy** tab.

1. Choose the **Delete**.

1. On the confirmation prompt, choose **Delete**.

# Domain alarms


Amazon OpenSearch Service provides a set of [recommended CloudWatch alarms](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/cloudwatch-alarms.html) to monitor the health of Amazon OpenSearch Service domains. Centralized Logging with OpenSearch helps you to create the alarms automatically, and send a notification to your email (or SMS) via Amazon SNS.

## Create alarms


### (Option 1) Using the Centralized Logging with OpenSearch console


1. Log in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch domains**.

1. Select the domain from the table.

1. Under **General configuration**, choose **Enable** at the **Alarms label**.

1. Enter the Email.

1. Choose the alarms that you want to create and adjust the settings if necessary.

1. Choose **Create**.

### (Option 2) Using the CloudFormation stack


This automated AWS CloudFormation template deploys the *Centralized Logging with OpenSearch - Alarms* solution in the AWS Cloud.

1. Log in to the AWS Management Console and select the button to launch the AWS CloudFormation template. [https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Fsolutions-reference%2Fcentralized-logging-with-opensearch%2Flatest%2FAlarmForOpenSearch.template](https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Fsolutions-reference%2Fcentralized-logging-with-opensearch%2Flatest%2FAlarmForOpenSearch.template) 

You can also [download the template](https://s3.amazonaws.com/solutions-reference/centralized-logging-with-opensearch/latest/AlarmForOpenSearch.template) as a starting point for your own implementation. . To launch the stack in a different AWS Region, use the Region selector in the console navigation bar. . On the **Create stack** page, verify that the correct template URL shows in the **Amazon S3 URL** text box and choose **Next**. . On the **Specify stack details** page, assign a name to your stack. . Under **Parameters**, review the parameters for the template and modify them as necessary. This solution uses the following parameters.

\$1


| Parameter | Default | Description | 
| --- | --- | --- | 
|  Endpoint  |   *<Requires input>*   |  The endpoint of the OpenSearch domain, for example, `vpc-your_opensearch_domain_name-xcvgw6uu2o6zafsiefxubwuohe.us-east-1.es.amazonaws.com`.  | 
|  DomainName  |   *<Requires input>*   |  The name of the OpenSearch domain.  | 
|  Email  |   *<Requires input>*   |  The notification email address. Alarms will be sent to this email address via Amazon SNS.  | 
|  ClusterStatusRed  |   `Yes`   |  Whether to enable alarm when at least one primary shard and its replicas are not allocated to a node.  | 
|  ClusterStatusYellow  |   `Yes`   |  Whether to enable alarm when at least one replica shard is not allocated to a node.  | 
|  FreeStorageSpace  |   `10`   |  Whether to enable alarm when a node in your cluster is down to the free storage space you entered in GiB. We recommend setting it to 25% of the storage space for each node. 0 means that the alarm is disabled.  | 
|  ClusterIndexWritesBlocked  |   `1`   |  Index writes blocked error occurs for >= x times in 5 minutes, 1 consecutive time. Input 0 to disable this alarm.  | 
|  UnreachableNodeNumber  |   `3`   |  Nodes minimum is < x for 1 day, 1 consecutive time. 0 means that the alarm is disabled.  | 
|  AutomatedSnapshotFailure  |   `Yes`   |  Whether to enable alarm when an automated snapshot failed. AutomatedSnapshotFailure maximum is >= 1 for 1 minute, 1 consecutive time.  | 
|  CPUUtilization  |   `Yes`   |  Whether to enable alarm when sustained high usage of CPU occurred. CPUUtilization or WarmCPUUtilization maximum is >= 80% for 15 minutes, 3 consecutive times.  | 
|  JVMMemoryPressure  |   `Yes`   |  Whether to enable alarm when JVM RAM usage peak occurred. JVMMemoryPressure or WarmJVMMemoryPressure maximum is >= 80% for 5 minutes, 3 consecutive times.  | 
|  MasterCPUUtilization  |   `Yes`   |  Whether to enable alarm when sustained high usage of CPU occurred in master nodes. MasterCPUUtilization maximum is >= 50% for 15 minutes, 3 consecutive times.  | 
|  MasterJVMMemoryPressure  |   `Yes`   |  Whether to enable alarm when JVM RAM usage peak occurred in master nodes. MasterJVMMemoryPressure maximum is >= 80% for 15 minutes, 1 consecutive time.  | 
|  KMSKeyError  |   `Yes`   |  Whether to enable alarm when the AWS KMS encryption key is disabled. KMSKeyError is >= 1 for 1 minute, 1 consecutive time.  | 
|  KMSKeyInaccessible  |   `Yes`   |  Whether to enable alarm when the AWS KMS encryption key has been deleted or has revoked its grants to OpenSearch Service. KMSKeyInaccessible is >= 1 for 1 minute, 1 consecutive time.  | 

1. Choose **Next**.

1. On the **Configure stack options** page, choose **Next**.

1. On the **Review and create** page, review and confirm the settings. Check the box acknowledging that the template creates AWS Identity and Access Management (IAM) resources.

1. Choose **Submit** to deploy the stack.

You can view the status of the stack in the AWS CloudFormation console in the **Status** column. You should receive a **CREATE\$1COMPLETE** status in approximately 5 minutes.

Once you have created the alarms, a confirmation email will be sent to your email address. You must choose the **Confirm** link in the email.

Go to the CloudWatch Alarms page by choosing the **General configuration > Alarms > CloudWatch Alarms** link on the Centralized Logging with OpenSearch console, and the link location is shown as follows:

 **General configuration screen.** 

![\[image31\]](http://docs.aws.amazon.com/solutions/latest/centralized-logging-with-opensearch/images/image31.png)


Make sure that all the alarms are in **OK** status because you might have missed the notification if the alarms have changed its status before subscription.

**Note**  
The alarm will not send an Amazon SNS notification to your email address if triggered before subscription. We recommend you check the alarms status after enabling the OpenSearch alarms. If you see any alarm, which is in **In Alarm** status, you should fix that issue first.

## Delete alarms


1. Log in to the Centralized Logging with OpenSearch console.

1. In the navigation pane, under **Domains**, choose **OpenSearch domains**.

1. Select the domain from the table.

1. Choose the **Alarms** tab.

1. Choose the **Delete**.

1. On the confirmation prompt, choose **Delete**.