# Monitoring Amazon Q Business and Q Apps Monitoring Monitoring is an important part of maintaining the reliability, availability, and performance of Amazon Q Business and your other AWS solutions. AWS provides the following monitoring tools to monitor Amazon Q Business, report when something is wrong, and take automatic actions when appropriate: + *AWS CloudTrail* captures API calls and related events made by or on behalf of your AWS account and delivers the log files to an Amazon S3 bucket that you specify. You can identify which users and accounts called AWS, the source IP address from which the calls were made, and when the calls occurred. For more information, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). + *Amazon CloudWatch* monitors your AWS resources and the applications you run on AWS in real time. You can collect and track metrics, create customized dashboards, and set alarms that notify you or take actions when a specified metric reaches a threshold that you specify. For example, you can have CloudWatch track CPU usage or other metrics of your Amazon EC2 instances and automatically launch new instances when needed. For more information, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/). + You can use [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html) to monitor and analyze user conversations and response feedback in Amazon Q Business. CloudWatch Logs can deliver logs to multiple locations, such as Amazon CloudWatch, Amazon S3, or Amazon Data Firehose (standard rates apply). We recommend that you set up conversation and feedback logging with Amazon CloudWatch within five minutes of creating your Amazon Q Business Application environment. For more information, see [Monitoring Amazon Q Business user conversations with Amazon CloudWatch Logs](monitoring-cloudwatch-logs.md). **Topics** + [ # Logging Amazon Q Business API calls using AWS CloudTrail ](logging-using-cloudtrail.md) + [ # Logging Amazon Q Apps API calls using AWS CloudTrail ](logging-qapps-using-cloudtrail.md) + [ # Monitoring Amazon Q Business and Amazon Q Apps with Amazon CloudWatch ](monitoring-cloudwatch.md) + [ # Monitoring Amazon Q Business user conversations with Amazon CloudWatch Logs ](monitoring-cloudwatch-logs.md) + [ # Viewing Amazon Q Business and Q App metrics in analytics dashboards ](analytics-dashboard.md) # Logging Amazon Q Business API calls using AWS CloudTrail Amazon Q Business CloudTrail logs Amazon Q Business is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon Q Business. CloudTrail captures all API calls for Amazon Q Business as events. The calls captured include calls from the Amazon Q console and code calls to the Amazon Q Business API operations. A *trail* enables CloudTrail to deliver log files to an Amazon S3 bucket. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon Q Business. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon Q Business, the IP address from which the request was made, who made the request, when it was made, and additional details. For more information about CloudTrail, including how to configure and activate it, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html). ## Amazon Q Business information in CloudTrail CloudTrail is activated on your AWS account when you create the account. When activity occurs in Amazon Q Business, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your AWS account. For more information, see [Viewing events with CloudTrail Event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html) in the *AWS CloudTrail User Guide*. For an ongoing record of events in your AWS account, including events for Amazon Q, create a trail. A *trail* enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following topics: + [Creating a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html) + [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html) + [Receiving CloudTrail log files from multiple Regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) ## Control plane events in CloudTrail CloudTrail supports logging the following Amazon Q Business actions documented in the [Amazon Q Business API Reference](https://docs.aws.amazon.com/amazonq/latest/api-reference/Welcome.html): + [CreateApplication](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreateApplication.html) + [DeleteApplication](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteApplication.html) + [GetApplication](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetApplication.html) + [ListApplications](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListApplications.html) + [UpdateApplication](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateApplication.html) + [DeleteChatControlsConfiguration](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteChatControlsConfiguration.html) + [GetChatControlsConfiguration](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetChatControlsConfiguration.html) + [UpdateChatControlsConfiguration](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateChatControlsConfiguration.html) + [CreateDataSource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreateApplication.html) + [DeleteDataSource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteDataSource.html) + [GetDataSource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListDataSources.html) + [ListDataSources](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListDataSources.html) + [UpdateDataSource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateDataSource.html) + [CreateWebExperience](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreateWebExperience.html) + [DeleteWebExperience](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteWebExperience.html) + [ListWebExperiences](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListTagsForResource.html) + [UpdateWebExperience](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateWebExperience.html) + [CreateIndex](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreateIndex.html) + [DeleteIndex](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteIndex.html) + [GetIndex](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetIndex.html) + [ListIndices](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListIndices.html) + [UpdateIndex](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateIndex.html) + [CreatePlugin](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreatePlugin.html) + [DeletePlugin](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteIndex.html) + [GetPlugin](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetPlugin.html) + [ListPlugins](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetPlugin.html) + [UpdatePlugin](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateIndex.html) + [CreateRetriever](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_CreatePlugin.html) + [DeleteRetriever](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_DeleteRetriever.html) + [GetRetriever](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetRetriever.html) + [ListRetrievers](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListRetrievers.html) + [UpdateRetriever](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_UpdateRetriever.html) + [ListTagsForResource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_ListTagsForResource.html) + [TagResource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_TagResource.html) + [UntagResource](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_TagResource.html) Every event or log entry contains information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root or AWS Identity and Access Management (IAM) user credentials. + Whether the request was made with temporary security credentials for a role or federated user. + Whether the request was made by another AWS service. For more information, see [CloudTrail userIdentity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html) in the *AWS CloudTrail User Guide*. ## Data plane events in CloudTrail [Data events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html#logging-data-events) provide information about the resource operations performed on or in a resource (for example, reading or writing to an Amazon S3 object). These are also known as *data plane operations*. By default, CloudTrail doesn't log data events. The following table shows the Amazon Q Business API operations logged to CloudTrail as *data events*. The **Data event type (console)** column shows the appropriate selection in the CloudTrail console. The **Amazon Q Business resource types** column shows the `resources.type` value that you would specify to log data events for the resource. | Data event type (console) | Amazon Q Business resource types | Supported data events | | --- | --- | --- | | Amazon Q Business application | AWS::QBusiness::Application | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/logging-using-cloudtrail.html) | | Amazon Q Business data resource | AWS::QBusiness::DataSource | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/logging-using-cloudtrail.html) | | Amazon Q Business index | AWS::QBusiness::Index | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/logging-using-cloudtrail.html) | You can log these API operations by configuring advanced event selectors to record data events for the Amazon Q Business resource types: `AWS::QBusiness::Application`, `AWS::QBusiness::DataSource`, and `AWS::QBusiness::Index`. To configure advanced event selectors, you can use either the CloudTrail console or the AWS CLI: + From the CloudTrail console, choose the **Data event type** for which you want to log data events. Additionally, you can filter on the `eventName` and `resources.ARN` fields by choosing a custom log selector template. For more information, see [Logging data events with the AWS Management Console](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html#logging-data-events-console) in the *AWS CloudTrail User Guide*. + From the AWS CLI, specify the `resources.type` value for which you want to log data events and set the `eventCategory` equal to `Data`. For more information, see [ Logging data events with the AWS CLI](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html#creating-data-event-selectors-with-the-AWS-CLI) in the *AWS CloudTrail User Guide*. The following example shows how to configure a trail to log all Amazon Q Business data events for all Amazon Q Business resource types. ``` aws cloudtrail put-event-selectors --trail-name trailName \ --advanced-event-selectors \ '[ { "Name": "Log all data events on an Amazon Q Business application", "FieldSelectors": [ { "Field": "eventCategory", "Equals": ["Data"] }, { "Field": "resources.type", "Equals": ["AWS::QBusiness::Application"] } ] }, { "Name": "Log all data events on an Amazon Q Business data source", "FieldSelectors": [ { "Field": "eventCategory", "Equals": ["Data"] }, { "Field": "resources.type", "Equals": ["AWS::QBusiness::DataSource"] } ] }, { "Name": "Log all data events on an Amazon Q Business index", "FieldSelectors": [ { "Field": "eventCategory", "Equals": ["Data"] }, { "Field": "resources.type", "Equals": ["AWS::QBusiness::Index"] } ] } ]' ``` You can additionally filter on the `eventName` and `resources.ARN` fields. For more information about configuring these fields, see [AdvancedFieldSelector](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_AdvancedFieldSelector.html) in the *AWS CloudTrail API Reference*. Additional charges apply for data events. For more information about CloudTrail pricing, see [AWS CloudTrail Pricing](https://aws.amazon.com/cloudtrail/pricing/). ## Amazon Q Business management events in CloudTrail [Management events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-events-with-cloudtrail.html#logging-management-events) provide information about management operations that are performed on resources in your AWS account. These management events are also known as *control plane operations*. CloudTrail logs management event API operations by default. Amazon Q Business logs the remainder of Amazon Q Business API operations as management events. For a list of the Amazon Q Business API operations that Amazon Q logs to CloudTrail, see the [Amazon Q Business API Reference](https://docs.aws.amazon.com/amazonq/latest/api-reference/Welcome.html). ## Understanding Amazon Q Business log file entries A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order. The following example shows a CloudTrail log entry that demonstrates the `CreateApplication` action. ``` { "eventVersion": "1.08", "userIdentity": { "type": "AssumedRole", "principalId": "principal ID", "arn": "ARN", "accountId": "account ID", "accessKeyId": "access key ID", "sessionContext": { "sessionIssuer": { "type": "Role", "principalId": "principal ID", "arn": "ARN", "accountId": "account ID", "userName": "user name" }, "webIdFederationData": {}, "attributes": { "creationDate": "yyyy-mm-ddThh:mm:ssZ", "mfaAuthenticated": "false" } } }, "eventTime": "yyyy-mm-ddThh:mm:ssZ", "eventSource": "qbusiness.amazonaws.com", "eventName": "CreateApplication", "awsRegion": "region", "sourceIPAddress": "region", "userAgent": "user agent", "requestParameters": { "name": "name", "roleArn": "description", "clientToken": "client token" }, "responseElements": { "applicationId": "application ID" }, "requestID": "request ID", "eventID": "event ID", "readOnly": false, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "account ID", "eventCategory": "Management", "tlsDetails": { "tlsVersion": "TLS version", "cipherSuite": "cipher suite", "clientProvidedHostHeader": "qbusiness.us-west-2.api.aws" } } ``` # Logging Amazon Q Apps API calls using AWS CloudTrail Amazon Q Apps CloudTrail logs Amazon Q Apps is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon Q Apps. CloudTrail captures all API calls for Amazon Q Apps as events. The calls captured include calls from theAmazon Q Apps web experience, console and code calls to theAmazon Q Apps API operations. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon Q Apps. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon Q Apps, the IP address from which the request was made, who made the request, when it was made, and additional details. For more information about CloudTrail, including how to configure and activate it, see the [https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html). ## Amazon Q Apps information in CloudTrail CloudTrail is activated on your AWS account when you create the account. When activity occurs in Amazon Q Apps, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history** in the CloudTrail console. You can view, search, and download recent events in your AWS account. For more information, see [Viewing events with CloudTrail Event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html) in the *AWS CloudTrail User Guide*. For an ongoing record of events in your AWS account, including events for Amazon Q Apps, create a trail. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following topics: + [Creating a trail for your AWS account](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html) + [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html) + [Receiving CloudTrail log files from multiple regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) ## Management events [Management events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-events-with-cloudtrail.html#logging-management-events) provide information about management operations that are performed on resources in your AWS account. These management events are also known as control plane operations. CloudTrail logs management event API operations by default. CloudTrail supports logging the followingAmazon Q Apps actions: + `CreateLibraryItem` + `UpdateLibraryItem` + `DeleteLibraryItem` + `GetLibraryItem` + `ListLibraryItems` + `TagResource` + `UntagResource` + `ListTagsForResource` Every event or log entry contains information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root or AWS Identity and Access Management (IAM) user credentials. + Whether the request was made with temporary security credentials for a role or federated user. + Whether the request was made by another AWS service. For more information, see [CloudTrail userIdentity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html) in the *AWS CloudTrail User Guide*. ## Data events [Data events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html) provide visibility into the resource operations performed on or within a resource. These are also known as data plane operations. Data events are often high-volume activities. By default, CloudTrail doesn't log data events. The following table shows theAmazon Q Apps API operations logged to CloudTrail as data events. The **Data event type (console)** column shows the appropriate selection in the CloudTrail console. The **Amazon Q Apps resource types column** shows the `resources.type` value that you would specify to log data events for the resource. | Data event type (console) | Amazon Q Apps resource types | Supported data events | | --- | --- | --- | | Amazon Q Apps | AWS::QApps::QApp | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/logging-qapps-using-cloudtrail.html) | | Amazon Q Business application | AWS::QBusiness::Application | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/logging-qapps-using-cloudtrail.html) | You can log these API operations by configuring advanced event selectors to record data events for theAmazon Q Apps resource types: `AWS::QApps::QApp` and `AWS::QBusiness::Application`. To configure advanced event selectors, you can use either the CloudTrail console or the AWS CLI: + From the CloudTrail console, choose the **Data event type** for which you want to log data events. Additionally, you can filter on the `eventName` and `resources.ARN` fields by choosing a custom log selector template. For more information, see [Logging data events with the AWS Management Console](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html#logging-data-events-console) in the *AWS CloudTrail User Guide*. + From the AWS CLI, specify the `resources.type` value for which you want to log data events and set the `eventCategory` equal to `Data`. For more information, see [Logging data events with the AWS CLI](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html#creating-data-event-selectors-with-the-AWS-CLI) in the *AWS CloudTrail User Guide*. The following example shows how to configure a trail to log allAmazon Q Apps data events for allAmazon Q Apps resource types. ``` aws cloudtrail put-event-selectors --trail-name trailName \ --advanced-event-selectors \ '[ { "Name": "Log all data events on anAmazon Q Apps", "FieldSelectors": [ { "Field": "eventCategory", "Equals": ["Data"] }, { "Field": "resources.type", "Equals": ["AWS::QApps::QApp"] } ] } ]' ``` You can additionally filter on the `eventName` and `resources.ARN` fields. For more information about configuring these fields, see [AdvancedFieldSelector](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_AdvancedFieldSelector.html) in the *AWS CloudTrail API Reference*. **Note** Additional charges apply for data events. For more information about CloudTrail pricing, see [AWS CloudTrail Pricing](https://aws.amazon.com/cloudtrail/pricing/). ## UnderstandingAmazon Q Apps log file entries A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the API calls, so they don't appear in any specific order. The following example shows a CloudTrail log entry that demonstrates the `GetLibraryItem` action. ``` { "eventVersion": "1.09", "userIdentity": { "type": "AssumedRole", "principalId": "principal ID", "arn": "ARN", "accountId": "account ID", "accessKeyId": "access key ID", "sessionContext": { "sessionIssuer": { "type": "Role", "principalId": "principal ID", "arn": "ARN", "accountId": "account ID", "userName": "user name" }, "attributes": { "creationDate": "yyyy-mm-ddThh:mm:ssZ", "mfaAuthenticated": "false" } }, "onBehalfOf": { "userId": "user ID", "identityStoreArn": "ARN" } }, "eventTime": "yyyy-mm-ddThh:mm:ssZ", "eventSource": "qapps.amazonaws.com", "eventName": "GetLibraryItem", "awsRegion": "region", "sourceIPAddress": "source IP address", "userAgent": "user agent", "requestParameters": { "input": "query input", "idc-application-arn": "ARN", "application-id": "Q application ID" }, "requestID": "request ID", "eventID": "event ID", "readOnly": true, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "account ID", "eventCategory": "Management" } ``` # Monitoring Amazon Q Business and Amazon Q Apps with Amazon CloudWatch CloudWatch metrics You can monitor Amazon Q Business and Amazon Q Apps with Amazon CloudWatch, which collects raw data and processes it into readable, near real-time metrics. These statistics are kept for 15 months, so that you can access historical information and gain a better perspective on how your web application or service is performing. You can also set alarms that watch for certain thresholds, and send notifications or take actions when those thresholds are met. For more information, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/). **Topics** + [ ## Using CloudWatch Metrics ](#using-metrics) + [ ## Viewing metrics ](#how-to-access) + [ # Creating a CloudWatch alarm ](alarms.md) + [ # Amazon Q Business chat metrics ](qbusiness-metrics-chat.md) + [ # Amazon Q Business API operation metrics ](qbusiness-metrics-api.md) + [ # Amazon Q Business index metrics ](qbusiness-metrics-index.md) + [ # Amazon Q Apps metrics ](qapps-metrics.md) ## Using CloudWatch Metrics To use metrics, you must specify the following information: + The metric namespace. A *namespace* is a CloudWatch container Amazon Q uses to publish its metrics into. If you are using the CloudWatch [ListMetrics](https://docs.aws.amazon.com//AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) API or the [list-metrics](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudwatch/list-metrics.html) command to view the metrics for Amazon Q Business or Amazon Q Apps, specify `AWS/QBusiness` or `AWS/QApps` for the namespace. + The metric dimension. A *dimension* is a name-value pair that helps you to uniquely identify a metric. Common dimension names include: + `ApplicationId` - Identifies the specific Amazon Q Business application + `IndexId` - Identifies the specific index (for index-related metrics) + `DataSourceId` - Identifies the specific data source (for data source metrics) + `PluginId` - Identifies the specific plugin (for plugin-related metrics) + `API name` - Identifies the specific API operation (for performance metrics like Chat) + `MethodType` - Identifies the specific method being called (for API operation metrics) + `UsefulnessReason` - Identifies feedback categories (for feedback metrics) + The metric name. For example, `DocumentsIndexed`. You can get monitoring data for Amazon Q Business or Q Apps with the AWS Management Console, the AWS CLI, or the CloudWatch API. You can also use the CloudWatch API through one of the AWS SDKs or the CloudWatch API tools. The console displays a series of graphs based on the raw data from the CloudWatch API. Depending on your needs, you might prefer to use either the graphs displayed in the console or retrieved from the API. ### The following table shows some common uses for the metrics. These are suggestions to get you started, not a comprehensive list. | How do I? | Relevant metrics | | --- | --- | | How do I track how many documents were indexed successfully? | Use the `DocumentsIndexed` metrics. | | How do I monitor end user experience for Amazon Q Business? | Use the `ThumbsUpCount` and `ThumbsDownCount` metrics. | | How do I track how many users used Q Apps | Use the `ActiveUsers` metric. | | How do I track API operation success rates? | Use the `success` and `failure` metrics with appropriate `MethodType` dimensions. | | How do I monitor individual API operation performance? | Use the `latency` metric with specific `MethodType` dimensions. | | How do I monitor chat response performance and latency? | Use the `TimeToFirstToken` and `Latency` metrics with API name dimension set to "Chat". | You must have the appropriate CloudWatch permissions to monitor Amazon Q Business with CloudWatch. For more information, see [Identity and access management for Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/auth-and-access-control-cw.html) in the *Amazon CloudWatch User Guide*. ## Viewing metrics The following steps show how to access Amazon Q Business or Amazon Q Apps metrics using the CloudWatch console. **To view metrics (console)** 1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch). 1. Choose **Metrics**, choose the **All Metrics** tab, and then choose **AWS/QBusiness** or **AWS/QApps**. **Note** Some Q Apps metrics might be emitted using a different namespace. 1. Choose the metric dimension. 1. Choose the metric that you want from the list, and choose a time period for the graph. # Creating a CloudWatch alarm You can create a CloudWatch alarm that sends an Amazon Simple Notification Service (Amazon SNS) message when the alarm changes state. An alarm watches a single metric over a time period that you specify. It performs one or more actions based on the value of the metric relative to a given threshold over a number of time periods. The action is a notification sent to an Amazon SNS topic or an Auto Scaling policy. Alarms invoke actions for sustained state changes only. CloudWatch alarms don't invoke actions simply because they are in a particular state. The state must have changed and have been maintained for a specified number of time periods. To create an alarm based on an Amazon Q Business or Q Apps metric, see [Create a CloudWatch Alarm Based on a CloudWatch Metric](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ConsoleAlarms.html). **To set an alarm (console)** 1. Sign in to the AWS Management Console and open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). 1. In the navigation pane, choose **Alarms**, and choose **Create Alarm**. This opens the **Create Alarm Wizard**. 1. Choose **Select metric**. 1. In the **All metrics** tab, choose an AWS/QBusiness or AWS/QApps metric for your application, index, and data source. Also set the time as set number of hours, days, weeks, or custom. 1. Choose your statistic. For example, **Average**. Also choose your alarm trigger time period as a set number of minutes, hours, per day, or custom. 1. Choose your threshold to trigger the alarm, whether to use a static value or a band and the condition to meet for the threshold. 1. Choose the alarm state for the trigger, whether the metric must fall outside your set threshold, or another state. Select who/which email to send the alarm notification to. 1. Choose **Next**. Add a name and optional description for your alarm. Choose **Next**. 1. Choose **Create Alarm**. **Note** You can use the `Sum` statistic to aggregate all metrics with the unit `Count`. For more information on CloudWatch statistics and how to use them, see [CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html) in the *Amazon CloudWatch User Guide*. # Amazon Q Business chat metrics The following table shows the [Chat and conversation management](conversation-api.md) metrics that Amazon Q Business sends to CloudWatch in real time. | Metric name | Unit | Description | | --- | --- | --- | | `ActionErrorCount` | Count | The number of errors because of actions. Valid dimensions: `ApplicationId`, `PluginId` | | `ActionInvocationCount` | Count | The number of actions invoked. Valid dimensions: `ApplicationId`, `PluginId` | | `BlockedChatMessages` | Count | The number of chat messages that were blocked by Amazon Q Business due to an admin guardrail configuration. For example, a `BlockedTopic` or `Blocked Phrase`. . Valid dimensions: `ApplicationId` | | `ChatMessages` | Count | The number of chat messages This metric is emitted every time a chat message is processed. Valid dimensions: `ApplicationId` | | `ChatMessagesWithAttachment` | Count | The number of chat messages with file uploads. Valid dimensions: `ApplicationId` | | `ChatMessagesWithNoAnswer` | Count | The number of chat messages that resulted in no answer. Valid dimensions: `ApplicationId` | | `HallucinatedChatMessages` | Count | The number of system-generated chat messages with hallucination. You can create a hallucination rate metric by combining this metric with the `ChatMessages` metric. Valid dimensions: `ApplicationId` | | `TimeToFirstToken` | Milliseconds | The time taken to generate the first token in a chat response. This metric measures the initial response latency for chat interactions. Valid dimensions: `API name`, `ApplicationId` | | `Latency` | Milliseconds | The total time taken to complete a chat API request from start to finish. This metric measures the end-to-end response time for chat interactions. Valid dimensions: `API name`, `ApplicationId` | | `DailyActiveUsers` | Count | The number of active users from the previous day. This metric is calculated using the *Maximum* statistic. For more information, see [CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html). Valid dimensions: `ApplicationId` | | `MonthlyActiveUsers` | Count | The total number of unique month-to-date active users. This metric will be calculated from the 00:00 UTC on the first day of the month till 00:00 UTC from the current day. This metric is calculated using the *Maximum* statistic. For more information, see [CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html). Valid dimensions: `ApplicationId` | | `WeeklyActiveUsers` | Count | The total number of unique week-to-date active users. This metric will be calculated from 00:00 UTC on Sunday till 00:00 UTC from the current day. This metric is calculated using the *Maximum* statistic. For more information, see [CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html). Valid dimensions: `ApplicationId` | | `NewConversations` | Count | The number of new conversations started. Valid dimensions: `ApplicationId` | | `ThumbsDownCount` | Count | The feedback count for thumbs down. Valid dimensions: `ApplicationId`, `UsefulnessReason` | | `ThumbsUpCount` | Count | The feedback count for thumbs up. Valid dimensions: `ApplicationId`, `UsefulnessReason` | # Amazon Q Business API operation metrics The following table shows the API operation metrics that Amazon Q Business sends to CloudWatch. | Metric name | Unit | Description | | --- | --- | --- | | `success` | Count | The number of successful API operation calls. This metric is emitted for each successful API operation execution. Valid dimensions: `MethodType`, `ApplicationId` | | `failure` | Count | The number of failed API operation calls. This metric is emitted for each failed API operation execution. Valid dimensions: `MethodType`, `ApplicationId` | | `latency` | Milliseconds | The time taken to complete an API operation call. This metric measures the response time for individual API operations. Valid dimensions: `MethodType`, `ApplicationId` | The `MethodType` dimension can include values such as: + `ListPlugins` + (Additional method types may be available depending on API usage) # Amazon Q Business index metrics The following table shows the [Index](concepts-terms.md#index) metrics that Amazon Q Business sends to CloudWatch in real time. | Metric name | Unit | Description | | --- | --- | --- | | `DocumentCount` | Count | The number of documents. This metric is published every 15 minutes. Valid dimensions: `ApplicationId`, `IndexId` | | `DocumentsIndexed` | Count | The number of documents that were indexed. Valid dimensions: `ApplicationId`, `IndexId`, `DataSourceId` | | `DocumentsFailedToIndex` | Count | The number of documents that failed to index. Valid dimensions: `ApplicationId`, `IndexId`, `DataSourceId` | | `DocumentsFailedToIndexDueToCDE` | Count | The number of documents that failed to index because of custom document enrichment. Valid dimensions: `ApplicationId`, `IndexId`, `DataSourceId` | | `ExtractedTextSize` | MB | Size of the extracted text Valid dimensions: `ApplicationId`, `IndexId` | | MonthlyDataSyncDuration | Count | Duration of data synchronization operations over a monthly period. Valid dimensions: `ApplicationId`, `IndexId`, `DataSourceId` | # Amazon Q Apps metrics The following table shows the metrics that Amazon Q Apps sends to CloudWatch in real time. | Metric name | Unit | Description | Namespace | | --- | --- | --- | --- | | `ActiveQAppsUsers` | Count | The number of Active Q App users. A user is a person who creates, edits or runs the apps. The metric is emitted once every day on UTC midnight. Valid dimensions: `ApplicationId` | AWS/QApps | | `ActiveQAppsCreators` | Count | The number of Active Q App creators. A creator is a user who creates a Q App in any way, including through magic wand, magic builder, create blank app or duplicate an app. The metric is emitted once every day on UTC midnight. Valid dimensions: `ApplicationId` | AWS/QApps | | `QAppCreated` | Count | The number of Q Apps created. This metric is emitted every time a Q App is created Valid dimensions: `ApplicationId` | AWS/QApps | | `ActiveQApps` | Count | The number of active Q Apps. The active Q App is defined as the app being created, updated or run. Valid dimensions: `ApplicationId` | AWS/QApps | | `QAppPublished` | Count | The number of Q Apps that are shared to the library. This metric is emitted every time a Q App is shared to the library. Valid dimensions: `ApplicationId` | AWS/QApps | | `QAppExecuted` | Count | The number of Q Apps run. This metric is emitted every time a Q App is run. Valid dimensions: `ApplicationId` | AWS/QApps | | `ResourceCount (QAppCountPerApplication)` | Count | The number of total Q Apps in the application environment. This metric is emitted every 5 minutes with Resource dimension populated as `QAppCountPerApplication`. Valid dimensions: `Resouce, ResourceId, Service, Type`. | Usage | | `ResourceCount (QAppCountPerUser)` | Count | The number of total users set up for the application environment. This metric is emitted every 5 minutes with Resource dimension populated as `QAppCountPerUser`. Valid dimensions: `Resource, ResourceId, Service, Type`. | Usage | # Monitoring Amazon Q Business user conversations with Amazon CloudWatch Logs Amazon Q Business CloudWatch Logs You can use [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) to deliver user conversations and response feedback in Amazon Q Business for you to analyze. These logs can be delivered to multiple destinations, such as Amazon CloudWatch, Amazon S3, or Amazon Data Firehose (standard rates apply). We recommend that you set up conversation and feedback logging within five minutes of creating your Amazon Q Business application environment. The following are examples of tasks you can complete with logs from conversations and response feedback in Amazon Q Business: + Identify common user queries and pain points by reviewing the chat message content. + Identify number of system-generated messages that have hallucination. + Monitor the quality of responses by looking at metrics like `isMessageWithNoAnswer`. + Understand user sentiment and satisfaction by analyzing the feedback data, including comments and usefulness ratings. + Generate custom dashboards and reports to track key metrics and trends over time. Amazon Q Business supports the `EVENT_LOGS` log type that tracks the specifics of conversations in an application. You can use `EVENT_LOGS` to monitor Amazon Q Business in all AWS regions where Amazon Q Business is offered. For more information about the AWS Regions and endpoints currently supported by Amazon Q Business, see [Supported Regions](quotas-regions.md#regions). Logs from conversations might include sensitive or personally identifiable data passed in the chats. You can filter out this information from your logs with the Amazon Q Business console. Or you can mask this data on your logs using CloudWatch Logs masking policies. For more information, see [Help protect sensitive log data with masking](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/mask-sensitive-log-data.html). **Topics** + [ # Amazon Q Business chat message and feedback log examples ](cw-log-examples.md) + [ # Permissions for monitoring Amazon Q Business with Amazon CloudWatch Logs ](cw-logs-permissions.md) + [ # Enabling Amazon Q Business user conversation logging ](cw-logs-enable-logging.md) + [ # Amazon Q Business conversation log query examples ](cw-logs-common-queries.md) # Amazon Q Business chat message and feedback log examples Log examples The following are examples of Amazon Q Business chat message and feedback logs in CloudWatch Logs. **Chat message** ``` { "application_id": "", "event_timestamp": "2024-09-05T19:43:58Z", "log_type": "Message", "account_id": "", "conversation_id": "", "system_message_id": "", "user_message_id": "", "user_message": string, "system_message": string, "output_metrics_is_message_blocked": boolean, "output_metrics_is_message_with_no_answer": boolean, "user_email": "" } ``` **Amazon Q Business generated chat message (hallucination detected)** ``` { "hallucinated_message": "string", "application_id": "", "event_timestamp": "2024-09-05T19:43:58Z", "log_type": "Message", "account_id": "", "conversation_id": "", "system_message_id": "", "user_message_id": "", "user_message": string, "system_message": string, "output_metrics_is_message_blocked": boolean, "output_metrics_is_message_with_no_answer": boolean, "user_email": "" } ``` **Amazon Q Business generated chat message (no hallucination detected)** ``` { "hallucinated_message": "NO HALLUCINATION DETECTED", "application_id": "", "event_timestamp": "2024-09-05T19:43:58Z", "log_type": "Message", "account_id": "", "conversation_id": "", "system_message_id": "", "user_message_id": "", "user_message": string, "system_message": string, "output_metrics_is_message_blocked": boolean, "output_metrics_is_message_with_no_answer": boolean, "user_email": "" } ``` For system generated messages that have hallucination, you'll see one of the following log descriptions: + Hallucination mitigation disabled – `hallucinated_message: DISABLED` + Hallucination mitigation enabled but not triggered – `hallucinated_message: NOT TRIGGERED` + Hallucination mitigation enabled and triggered, but no hallucinations detected – `hallucinated_message: NO HALLUCINATION DETECTED` + Hallucination mitigation feature enabled and triggered, and hallucinations detected – `hallucinated_message: string` **Feedback** ``` { "application_id": "", "event_timestamp": "2024-09-05T13:13:27Z", "log_type": "Feedback", "account_id": "", "conversation_id": "", "system_message_id": "", "user_message_id": "", "user_message": string, "system_message": string, "usefulness_reason": "NOT_FACTUALLY_CORRECT" | "HARMFUL_OR_UNSAFE" | "INCORRECT_OR_MISSING_SOURCES" | "NOT_HELPFUL" | "FACTUALLY_CORRECT" | "COMPLETE" | "RELEVANT_SOURCES" | "HELPFUL" | "NOT_BASED_ON_DOCUMENTS" | "NOT_COMPLETE" | "NOT_CONCISE" | "OTHER", "usefulness": "NOT_USEFUL" | "USEFUL", "comment": string, "user_email": "" } ``` # Permissions for monitoring Amazon Q Business with Amazon CloudWatch Logs Permissions To set up Amazon CloudWatch Logs for Amazon Q Business, use the following IAM policy to grant the necessary permissions. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "CloudWatchLogsDeliveryPermissions", "Effect": "Allow", "Action": "logs:CreateDelivery", "Resource": [ "arn:aws:logs:us-east-1:111122223333:delivery-source:*", "arn:aws:logs:us-east-1:111122223333:delivery:*", "arn:aws:logs:us-east-1:111122223333:delivery-destination:*" ] }, { "Sid": "QBusinessLogDeliveryPermissions", "Effect": "Allow", "Action": "qbusiness:AllowVendedLogDeliveryForResource", "Resource": [ "arn:aws:qbusiness:us-east-1:111122223333:application/application-id" ] } ] } ``` ------ For example IAM policies with all the required permissions for your specific logging destination, see [Enable logging from AWS services](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) in the *Amazon CloudWatch Logs User Guide*. # Enabling Amazon Q Business user conversation logging Enabling logging You can enable Amazon Q Business user conversation logging within the Amazon Q Business console or using the Amazon CloudWatch Logs API operations. When you enable logging, you specify a delivery destination for the logs. If you choose Amazon S3, the prefix of logs delivered to the Amazon S3 bucket is `AWSLogs/account-id/AmazonQBusinessLogs/your-region/application-id/year/month/day/hour/.` The files are compressed and named with `Feedback-20240905T19Z_501fec0f.log.gz ` or `VendedAnalyticsChat-20240905T19Z_d26ccf9e.log.gz` formats. **Important** Logs might include sensitive or personally identifiable data passed in the chats. You can filter out this information from your logs with the Amazon Q Business console. Or you can mask this data on your logs using CloudWatch Logs masking policies. For more information, see [Help protect sensitive log data with masking](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/mask-sensitive-log-data.html). **Topics** + [ ## Enabling user conversation logging with the Amazon Q Business console ](#cws-logs-enable-logging-console) + [ ## Enabling user conversation logging with the Amazon CloudWatch Logs API operations ](#cws-logs-enable-logging-api) ## Enabling user conversation logging with the Amazon Q Business console To enable user conversation logging with the Amazon Q Business console, use the admin controls for your environment to configure log delivery, optionally filter out sensitive information, and then enable logging to start streaming conversation and feedback data. **To enable logging** 1. Open the Amazon Q Business console at [Amazon Q Business](https://console.aws.amazon.com/amazonq/business/) and sign in to your account. 1. In **Applications**, choose the name of your application environment. 1. In the navigation pane, choose **Enhancements** and choose **Admin Controls and Guardrails**. 1. In **Log delivery**, choose **Add** and choose one of the following options. + **Amazon CloudWatch Logs** – Enter the **Destination log group ** where the logs will be stored. To filter out sensitive or personally identifiable information, choose **Additional settings - optional** and specify the fields to be logged, the output format, and field delimiter. For more information about log groups, see [Working with log groups and log streams](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html) in the *Amazon CloudWatch Logs* user guide. + **Amazon S3** – To add delivery to Amazon S3, choose the **Log type** and specify a **Destination S3 bucket**. To filter out sensitive or personally identifiable information, in **Additional settings - optional** specify the fields to be logged, whether to use hive compatible S3 paths, the output format, and the field delimiter. + **Amazon Data Firehose** – To add delivery to Amazon Data Firehose, choose the **Log type** and specify a **Destination delivery stream**. To filter out sensitive or personally identifiable information, in **Additional settings - optional** specify the fields to be logged, the output format, and the field delimiter. For information about creating a delivery stream, see [Create a Firehose delivery stream](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CreateFirehoseStream.html). **Note** If you want the users' email recorded in your logs, it must be added explicitly as a field in **Additional settings**. 1. Choose **Enable logging** start streaming conversation and feedback data to your logging destination. ## Enabling user conversation logging with the Amazon CloudWatch Logs API operations To enable user conversation logging with the Amazon CloudWatch Logs API operations, you call the PutDeliverySource, PutDeliveryDesintation, and CreateDelivery API operations. For information about quotas for these API operations, see [Service quotas](https://docs.aws.amazon.com//general/latest/gr/cwl_region.html#limits_cloudwatch_events). **Note** To enable conversation logging, you need the Amazon Resource Name (ARN) of your environment. To get this ARN, you can use the Amazon Q Business console or the [GetApplication](https://docs.aws.amazon.com/amazonq/latest/api-reference/API_GetApplication.html) API operation. An ARN follows this format: `arn:aws:qbusiness:region:account-id:application/application-id`. **To enable user conversation logging** 1. Create a delivery source with the [PutDeliverySource](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliverySource.html) Amazon CloudWatch Logs API operation. Give the delivery source a name and for `resourceArn`, specify the ARN of your application. For `logType`, specify `EVENT_LOGS`. ``` { "logType": "EVENT_LOGS", "name": "my-q-business-application-delivery-source", "resourceArn": "arn:aws:qbusiness:your-region:your-account-id:application/application-id" } ``` 1. Configure the log delivery destination with the [PutDeliveryDestination](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliveryDestination.html) Amazon CloudWatch Logs API operation. You can choose either Amazon CloudWatch Logs, Amazon S3, or Amazon Data Firehose as the destination for storing logs. You must specify the Amazon Resource Name of one of the destination options for where your logs will be stored. The `outputFormat` of the logs can be one of the following: json, plain, w3c, raw, or parquet. The following shows how to specify an Amazon S3 bucket as a log delivery destination with an `outputFormat` of `json`. ``` { "deliveryDestinationConfiguration": { "destinationResourceArn": "arn:aws:s3:::bucket-name" }, "name": "s3-delivery-destination", "outputFormat": "json", "tags": { "key": "value" } } ``` 1. Enable monitoring with the [CreateDelivery](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateDelivery.html) Amazon CloudWatch Logs API operation. This API operation links the delivery source to the destination you created in the previous steps. ``` { "deliveryDestinationArn": "string", "deliverySourceName": "string", "tags": { "string": "string" } } ``` **Note** If you want the users' email recorded in your logs, it must be added explicitly as a field along with the [other fields](https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/cw-log-examples.html) that you want in the [https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateDelivery.html#CWL-CreateDelivery-request-recordFields](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateDelivery.html#CWL-CreateDelivery-request-recordFields) parameter as part of calling the `CreateDelivery` operation . # Amazon Q Business conversation log query examples Log query examples You can use [CloudWatch Logs insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html) to interact with conversation and feedback logs from Amazon Q Business. The following are examples of common queries. + Query for all the feedback type logs. ``` filter log_type = "Feedback" ``` + Query for all the chat type logs. ``` filter log_type = "VendedAnalyticsChat" ``` + Query for chat logs related to particular conversation. ``` filter conversation_id = ``` + Query for chat logs where customer message with certain pattern. ``` filter customer_message like /pattern/ ``` + Query for chat logs where system response message with certain pattern. ``` filter system_message like /pattern/ ``` + Query for chat logs where system not able to provide answer. ``` filter system_message like /Sorry, I could not find relevant information to complete your request./ ``` + Query for chat logs where system not able to provide answer. ``` filter output_metrics_is_message_with_no_answer = 1 ``` + Query for chat logs where customer message was blocked. ``` filter output_metrics_is_message_blocked = 1 ``` + Query for all the feedback logs where system answer was marked useful. ``` filter usefulness = "USEFUL" ``` + Query for all the feedback logs where system answer was marked not useful. ``` filter usefulness = "NOT_USEFUL" ``` + Query for all the feedback logs where system answer was marked not useful with reason “Other”. ``` filter usefulness = "NOT_USEFUL" and usefulness_reason = "OTHER" ``` + Query all feedback logs where system message was hallucinated ``` filter hallucinated_message != "NOT TRIGGERED" and hallucinated_message != NO HALLUCINATION DETECTED ``` # Viewing Amazon Q Business and Q App metrics in analytics dashboards Analytics dashboards The analytics dashboards in Amazon Q Business offer a comprehensive view of key usage metrics and trends about the Amazon Q application environment and Amazon Q Apps. This centralized hub presents the CloudWatch metrics as interactive charts and visualizations for you to review. There are two dashboards: + Amazon Q Business Analytics dashboard – Use this dashboard to learn how users are interacting with a specific Amazon Q Business application environment. For example, you can identify trends in usage, conversation, feedback, and queries. + Amazon Q App Analytics dashboard – Use this dashboard to learn how your users are interacting with Q Apps. For example, you can identify the average daily users creating, updating, or running Q Apps over a specific time period. As an Amazon Q admin, you can view the dashboards with the Amazon Q Business console. You can view the metrics in these dashboards over different pre-selected time intervals. They are available at no additional charge in all Regions where the Amazon Q Business service is offered. **Topics** + [ # Viewing the analytics dashboards ](analytics-dashboard-view.md) + [ # Amazon Q Business Analytics dashboard metrics ](analytics-dashboard-metrics.md) + [ # Amazon Q Apps Analytics dashboard metrics ](q-apps-analytics-dashboard-metrics.md) # Viewing the analytics dashboards Viewing the analytics dashboards To view the Amazon Q Business Analytics dashboard or Amazon Q Apps Analytics dashboard with the Amazon Q Business console, you choose your application environment and navigate to the Analytics dashboard page. **To view the analytics dashboards** 1. Sign in to the Amazon Q Business console at [https://console.aws.amazon.com/amazonq/business/](https://console.aws.amazon.com/amazonq/business). 1. In the left navigation pane, choose **Applications**. 1. From the **Applications** section in the center pane, choose the *Name* of application environment that you want to see the Analytics for. 1. In the left navigation pane, choose **Insights**.Choose between **Amazon Q Business** or **Amazon Q Apps** to view the relevant dashboard for your application environment. For information about the metrics that appear, see [Amazon Q Business Analytics dashboard metrics](analytics-dashboard-metrics.md) or[Amazon Q Apps Analytics dashboard metrics](q-apps-analytics-dashboard-metrics.md). 1. Use the date picker to specify the time interval for the graphs. You can choose from the following preset intervals. + Last week – This interval starts from the previous Sunday at 00:00 and goes until the following Sunday at 00:00 in the UTC time zone. + This week – This interval starts this Sunday at 00:00 and goes until 00:00 today in the UTC time zone. + Last month – This interval begins on the 1st day of the last month at 00:00 and ends on the first day of this month at 00:00 in the UTC time zone. + This month – This interval begins on the 1st day of this month at 00:00 and goes until 00:00 today in the UTC time zone. # Amazon Q Business Analytics dashboard metrics The Amazon Q Business dashboard provides a comprehensive view of key metrics to help admins understand the performance and usage of every Amazon Q application environment from the Amazon Q Business console. The following analytics are available in the dashboard for the specified time period. + **Average daily active users** This tile gives the average number of unique daily users for the application environment that participated in at least one chat session for the chosen period. **Note** This metric is calculated using the *Maximum* statistic. For more information, see [CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html). + **Total users** All the users of this application environment. + **Weekly active users** This tile gives the total number of unique users for the application environment who participated in at least one chat session for the week in the chosen period. + **Monthly active users** This tile gives the total number of unique users for the application environment who participated in at least one chat session for the month in the chosen period. + **Total conversations** This tile gives the total number of chat conversations that have occurred in this application environment. Conversations are different from individual chat queries and represent a collection of related queries and responses over a relevant session or subject. + **Total queries** This tile gives the total number of chat message queries received by this application environment. + **Average queries per user** This tile gives the average of daily chat queries for divided by the daily active unique users for that application environment for the chosen period. + **Average queries per conversation** This tile gives the average of total chat queries to an application environment divided by the total individual conversations in the application environment. Conversations refer to new chat threads, which are listed on the left navigation pane of the web experience for the application environment. + **Total plugins** This tile shows the total number of plug-ins currently used by this application environment. + **Most frequently used plugin** This tile shows the plug-in most frequently used by this application environment. + The **Overview** trend This line chart displays two trend lines - one showing the daily active users for each day for the application environment, and another showing the total daily individual chat queries received by the application environment. + The **Customer feedback** trend This line chart displays three trend lines for the application environment. + The number of thumbs up feedback. + The number of thumbs down feedback. + The number of chat message responses that did not receive any feedback thumbs up or thumbs down). + The **Thumbs down reasons** chart This pie chart represents the different feedback reasons that users choose when providing a thumbs down feedback, along with the count or percentage of times each reason was selected. You can also filter by feedback reasons to further see the top queries that resulted in the feedback or choose **view details to resolve issues** to see all the queries for further evaluation. + The **Thumbs-down feedback queries** This page shows the list of queries that resulted in an unsuccessful response for further evaluation. You can choose any query to view details and see **Recommendations** on how to possibly resolve the issue. **Note** A maximum of *5000* query responses will be loaded initially for evaluation. You can always search for specific queries. The empty columns in some rows indicate that you need to update the **Vended log deliveries** configuration in **Admin-controls-guardrails page**. + The **Unsuccessful query responses** chart This pie chart represents the breakdown between unsuccessful query responses because the answer was not found or the answer was blocked due to your guardrails settings. You can also filter by response type further to see the top queries that resulted in the response or choose **view details to resolve** issues to see all the queries for further evaluation. + The **Unsuccessful response queries** This page shows the list of queries that resulted in an unsuccessful response for further evaluation. You can choose any query to view details and see **Recommendations** on how to possibly resolve the issue. **Note** A maximum of *5000* query responses will be loaded initially for evaluation. You can always search for specific queries. The empty columns in some rows indicate that you need to update the **Vended log deliveries** configuration in **Admin-controls-guardrails page**. + The **Queries** trend This chart displays three trend lines. + The number of chat queries that included a file upload. + The number of instances where the chat queries was blocked by the guardrails set. + The number of instances where the application environment was unable to find a relevant answer to the user's query. + The **Plugins** chart This bar chart shows the number of times users have selected each of the available plugins per day. For example, *ServiceNow*, *Salesforce*, *JIRA*, *Zendesk*, *Custom*, etc. when sending a query to the application environment. + The **Conversations** chart This bar chart shows the total number of conversations per day for the specified date range. Conversations are different from individual chat queries, and represent a collection of related queries and responses over a relevant session. + The **Average queries per conversation** trend This bar chart calculates the average number of queries per conversation by dividing the total number of daily queries by the total number of conversations per day for the application environment. Conversations are listed on the left navigation pane of the web experience, and each conversation represents a collection of related chat messages. The chart indicates the average length of conversations, as measured by the total number of chat messages from the first to the last message in a given conversation thread. # Amazon Q Apps Analytics dashboard metrics The Amazon Q Apps dashboard provides a comprehensive view of key metrics to help admins understand the performance and usage of Q Apps. If the metrics are not populated in the dashboard, it is likely due to the absence of a Q Apps service role or the role is not configured properly in the Amazon Q Business application environment. Usually a banner will be displayed on the Amazon Q Business console to prompt creating and registering the Q Apps service role. Depending on how you set up the Amazon Q Business application environment prior to the introduction of analytics dashboard feature, the banner might not always appear. To resolve the issue, go to the **Admin controls and guardrails** page for the Amazon Q Business application environment, choose **Edit Global Control** and re-save the configuration without changes. The metrics should start populating in 10 minutes. The following analytics are available in the Amazon Q Apps Analytics dashboard for the specified time period. + **Active users** The average unique daily users creating, updating, or running Q Apps. + **Active creators** The average unique daily users creating or updating Q Apps. + **Total Q Apps** The average of the total Q Apps per day. + **Active Q Apps** The average number of daily Q Apps ran or updated. + **Q App Participants trend** A trend of the daily active users and the daily active creators. + **Q App trend** A trend of the total daily Q Apps created and the daily active Q Apps. + **Total Q Apps Runs trend** A trend of the total daily Q App runs. + **Published Q App trend** A trend of the total daily published Q Apps.