# Fleet indexing
You can use fleet indexing to index, search, and aggregate your devices' data from the following sources: [AWS IoT registry](thing-registry.md), [AWS IoT Device Shadow](iot-device-shadows.md), [AWS IoT connectivity](life-cycle-events.md), [AWS IoT Device Management Software Package Catalog](software-package-catalog.md), and [AWS IoT Device Defender](https://docs.aws.amazon.com/iot-device-defender/latest/devguide/what-is-device-defender.html) violations. You can query a group of devices, and aggregate statistics on device records that are based on different combinations of device attributes, including state, connectivity, and device violations. With fleet indexing, you can organize, investigate, and troubleshoot your fleet of devices.
Fleet indexing provides the following capabilities.
## Managing index updates
You can set up a fleet index to index updates for your thing groups, thing registries, device shadows, device connectivity, and device violations. When you activate fleet indexing, AWS IoT creates an index for your things or thing groups. `AWS_Things` is the index created for all of your things. `AWS_ThingGroups` is the index that contains all of your thing groups. After fleet indexing is active, you can run queries on your index. For example, you can find all devices that are handheld and have more than 70 percent battery life. AWS IoT updates the index continually with your latest data. For more information, see [Managing fleet indexing](managing-fleet-index.md).
## Querying connectivity status for a specific device
This API provides low-latency, high-throughput access to the most recent device-specific connectivity information. For more information, see [Device connectivity status.](https://docs.aws.amazon.com/iot/latest/developerguide/device-connectivity-status.html)
## Searching across data sources
You can create a query string based on [a query language](query-syntax.md) and use it to search across data sources. You also need to configure data sources in the fleet indexing setting so that the indexing configuration contains the data sources you want to search from. The query string describes the things that you want to find. You can create queries by using AWS managed fields, custom fields, and any attributes from your indexed data sources. For more information about data sources that support fleet indexing, see [Managing thing indexing](managing-index.md).
## Querying for aggregate data
You can search your devices for aggregate data and return statistics, percentile, cardinality, or a list of things with search queries about particular fields. You can run aggregations on AWS managed fields or any attributes you configure as custom fields within fleet indexing settings. For more information about aggregation query, see [Querying for aggregate data](index-aggregate.md).
## Monitoring aggregate data and creating alarms by using fleet metrics
You can use fleet metrics to send aggregate data to CloudWatch automatically, analyze trends, and create alarms to monitor the aggregate state of your fleet based on pre-defined thresholds. For more information about fleet metrics, see [Fleet metrics](iot-fleet-metrics.md).
# Managing fleet indexing
Fleet indexing manages two types of indexes for you: thing indexing and thing group indexing.
## Thing indexing
The index created for all of your things is called `AWS_Things`. Thing indexing supports the following data sources: [AWS IoT registry](thing-registry.md) data, [AWS IoT Device Shadow ](iot-device-shadows.md)data, [AWS IoT connectivity](life-cycle-events.md) data, and [AWS IoT Device Defender](https://docs.aws.amazon.com/iot-device-defender/latest/devguide/what-is-device-defender.html) violations data. By adding these data sources to your fleet indexing configuration, you can search for things, query for aggregate data, and create dynamic thing groups and fleet metrics based on your search queries.
**Registry**-AWS IoT provides a registry that helps you manage things. You can add the registry data to your fleet indexing configuration to search for devices based on the thing names, descriptions, and other registry attributes. For more information about the registry, see [How to manage things with the registry](thing-registry.md).
**Shadow**-The [AWS IoT Device Shadow service](iot-device-shadows.md) provides shadows that help you store your device state data. Thing indexing supports both classic unnamed shadows and named shadows. To index named shadows, activate your named shadow settings and specify your shadow names in thing indexing configuration. By default, you can add up to 10 shadow names per AWS account. To see how to increase the number of shadow names limit, see [AWS IoT Device Management Quotas](https://docs.aws.amazon.com/general/latest/gr/iot_device_management.html#fleet-indexing-limits) in the *AWS General Reference*.
To add named shadows for indexing:
+ If you use the [AWS IoT console](https://console.aws.amazon.com/iot/home), turn on **Thing indexing**, choose **Add named shadows**, and add your shadow names through **Named shadow selection**.
+ If you use the AWS Command Line Interface (AWS CLI), set `namedShadowIndexingMode` to be `ON`, and specify shadow names in [https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html](https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html). To see example CLI commands, see [Manage thing indexing](managing-index.md#enable-index).
**Important**
July 20, 2022 is the General Availability (GA) release of the AWS IoT Device Management fleet indexing integration with AWS IoT Core named shadows and AWS IoT Device Defender detect violations. With this GA release, you can index specific named shadows by specifying shadow names. If you added your named shadows for indexing during this feature's public preview period from November 30, 2021 to July 19, 2022, we encourage you to reconfigure your fleet indexing settings and choose specific shadow names to reduce indexing cost and optimize performance.
For more information about shadows, see [AWS IoT Device Shadow service](iot-device-shadows.md).
**Connectivity**-Device connectivity data helps you identify the connection status of your devices. This connectivity data is driven by [lifecycle events](life-cycle-events.md). When a client connects or disconnects, AWS IoT publishes lifecycle events with messages to MQTT topics. A connect or disconnect message can be a list of JSON elements that provide details of the connection status. For more information about device connectivity, see [Lifecycle events](life-cycle-events.md).
**Device Defender violations**-AWS IoT Device Defender violations data helps identify anomalous device behaviors against the normal behaviors that you define in a Security Profile. A Security Profile contains a set of expected device behaviors. Each behavior uses a metric that specifies the normal behavior of your devices. For more information about Device Defender violations, see [AWS IoT Device Defender detect](https://docs.aws.amazon.com/iot-device-defender/latest/devguide/device-defender-detect.html).
For more information, see [Managing thing indexing](managing-index.md).
## Thing group indexing
`AWS_ThingGroups` is the index that contains all of your thing groups. You can use this index to search for groups based on group name, description, attributes, and all parent group names.
For more information, see [Managing thing group indexing](thinggroup-index.md).
## Managed fields
Managed fields contain data associated with things, thing groups, device shadows, device connectivity, and Device Defender violations. AWS IoT defines the data type in managed fields. You specify the values of each managed field when you create an AWS IoT thing. For example, thing names, thing groups, and thing descriptions are all managed fields. Fleet indexing indexes managed fields based on the indexing mode that you specify. Managed fields can't be changed or appear in `customFields`. For more information, see [Custom fields](#custom-field).
The following lists managed fields for thing indexing:
+ Managed fields for the registry
```
"managedFields" : [
{name:thingId, type:String},
{name:thingName, type:String},
{name:registry.version, type:Number},
{name:registry.thingTypeName, type:String},
{name:registry.thingGroupNames, type:String},
]
```
+ Managed fields for classic unnamed shadows
```
"managedFields" : [
{name:shadow.version, type:Number},
{name:shadow.hasDelta, type:Boolean}
]
```
+ Managed fields for named shadows
```
"managedFields" : [
{name:shadow.name.shadowName.version, type:Number},
{name:shadow.name.shadowName.hasDelta, type:Boolean}
]
```
+ Managed fields for thing connectivity
```
"managedFields" : [
{name:connectivity.timestamp, type:Number},
{name:connectivity.version, type:Number},
{name:connectivity.connected, type:Boolean},
{name:connectivity.disconnectReason, type:String}
]
```
+ Managed fields for Device Defender
```
"managedFields" : [
{name:deviceDefender.violationCount, type:Number},
{name:deviceDefender.securityprofile.behaviorname.metricName, type:String},
{name:deviceDefender.securityprofile.behaviorname.lastViolationTime, type:Number},
{name:deviceDefender.securityprofile.behaviorname.lastViolationValue, type:String},
{name:deviceDefender.securityprofile.behaviorname.inViolation, type:Boolean}
]
```
+ Managed fields for thing groups
```
"managedFields" : [
{name:description, type:String},
{name:parentGroupNames, type:String},
{name:thingGroupId, type:String},
{name:thingGroupName, type:String},
{name:version, type:Number},
]
```
The following table lists managed fields that are not searchable.
| Data source | Managed field that is unsearchable |
| --- | --- |
| Registry | registry.version |
| Unnamed shadows | shadow.version |
| Named shadows | shadow.name.\$1.version |
| Device Defender | deviceDefender.version |
| Thing groups | version |
## Custom fields
You can aggregate thing attributes, Device Shadow data, and Device Defender violations data by creating custom fields to index them. The `customFields` attribute is a list of field name and data type pairs. You can perform aggregation queries based on data type. The indexing mode that you choose affects fields can be specified in `customFields`. For example, if you specify the `REGISTRY` indexing mode, you can't specify a custom field from a thing shadow. You can use the [update-indexing-configuration](https://docs.aws.amazon.com/cli/latest/reference/iot/update-indexing-configuration.html) CLI command to create or update the custom fields (see an example command in [Updating indexing configuration examples](managing-index.md#update-index-examples)).
+ **Custom field names**
Custom field names for thing and thing group attributes begin with `attributes.`, followed by the attribute name. If unnamed shadow indexing is on, things can have custom field names that begin with `shadow.desired` or `shadow.reported`, followed by the unnamed shadow data value name. If named shadow indexing is on, things can have custom field names that begin with `shadow.name.*.desired.` or `shadow.name.*.reported.`, followed by the named shadow data value. If Device Defender violations indexing is on, things can have custom field names that begin with `deviceDefender.`, followed by the Device Defender violations data value.
The attribute or data value name that follows the prefix can have only alphanumeric, - (hyphen), and \$1 (underscore) characters. It can't have any spaces.
If there' a type inconsistency between a custom field in your configuration and the value being indexed, fleet indexing ignores the inconsistent value for aggregation queries. CloudWatch Logs are helpful when troubleshooting aggregation query problems. For more information, see [Troubleshooting aggregation queries for the fleet indexing service](fleet-indexing-troubleshooting.md#aggregation-troubleshooting).
+ **Custom field types**
Custom field types have the following supported values: `Number`, `String`, and `Boolean`.
# Manage thing indexing
The index created for all of your things is `AWS_Things`. You can control what to index from the following data sources: [AWS IoT registry](thing-registry.md) data, [AWS IoT Device Shadow ](iot-device-shadows.md)data, [AWS IoT connectivity](life-cycle-events.md) data, and [AWS IoT Device Defender](https://docs.aws.amazon.com/iot-device-defender/latest/devguide/what-is-device-defender.html) violations data.
**Topics**
+ [
## Enabling thing indexing
](#enable-index)
+ [
## Describing a thing index
](#describe-index)
+ [
## Querying a thing index
](#search-index)
+ [
## Restrictions and limitations
](#index-limitations)
+ [
## Authorization
](#query-auth)
## Enabling thing indexing
You use the [update-indexing-configuration](https://docs.aws.amazon.com/cli/latest/reference/iot/update-indexing-configuration.html) CLI command or the [UpdateIndexingConfiguration](https://docs.aws.amazon.com/iot/latest/apireference/API_UpdateIndexingConfiguration.html) API operation to create the `AWS_Things` index and control its configuration. By using the `--thing-indexing-configuration` (`thingIndexingConfiguration`) parameter, you control what kind of data (for example, registry, shadow, device connectivity data, and Device Defender violations data) is indexed.
The `--thing-indexing-configuration` parameter takes a string with the following structure:
```
{
"thingIndexingMode": "OFF"|"REGISTRY"|"REGISTRY_AND_SHADOW",
"thingConnectivityIndexingMode": "OFF"|"STATUS",
"deviceDefenderIndexingMode": "OFF"|"VIOLATIONS",
"namedShadowIndexingMode": "OFF"|"ON",
"managedFields": [
{
"name": "string",
"type": "Number"|"String"|"Boolean"
},
...
],
"customFields": [
{
"name": "string",
"type": "Number"|"String"|"Boolean"
},
...
],
"filter": {
"namedShadowNames": [ "string" ],
"geoLocations": [
{
"name": "String",
"order": "LonLat|LatLon"
}
]
}
}
```
### Thing indexing modes
You can specify different thing indexing modes in your indexing configuration, depending on what data sources you want to index and search devices from:
+ `thingIndexingMode`: Controls if registry or shadow is indexed. When `thingIndexingMode` is set to be `OFF`, thing indexing is disabled.
+ `thingConnectivityIndexingMode`: Specifies if thing connectivity data is indexed.
+ `deviceDefenderIndexingMode`: Specifies if Device Defender violations data is indexed.
+ `namedShadowIndexingMode`: Specifies if named shadow data is indexed. To select named shadows to add to your fleet indexing configuration, set `namedShadowIndexingMode` to be `ON` and specify your named shadow names in [https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html](https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html).
The table below shows the valid values for each indexing mode and the data source that's indexed for each value.
****
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/managing-index.html)
### Managed fields and custom fields
**Managed fields**
Managed fields contain data associated with things, thing groups, device shadows, device connectivity, and Device Defender violations. AWS IoT defines the data type in managed fields. You specify the values of each managed field when you create an AWS IoT thing. For example, thing names, thing groups, and thing descriptions are all managed fields. Fleet indexing indexes managed fields based on the indexing mode that you specify. Managed fields can't be changed or appear in `customFields`.
**Custom fields**
You can aggregate attributes, Device Shadow data, and Device Defender violations data by creating custom fields to index them. The `customFields` attribute is a list of field name and data type pairs. You can perform aggregation queries based on data type. The indexing mode that you choose affects fields can be specified in `customFields`. For example, if you specify the `REGISTRY` indexing mode, you can't specify a custom field from a thing shadow. You can use the [update-indexing-configuration](https://docs.aws.amazon.com/cli/latest/reference/iot/update-indexing-configuration.html) CLI command to create or update the custom fields (see an example command in [Updating indexing configuration examples](#update-index-examples)). For more information, see [Custom fields](managing-fleet-index.md#custom-field).
### Indexing filter
Indexing filter provides additional selections for named shadows and geolocation data.
**`namedShadowNames`**
To add named shadows to your fleet indexing configuration, set `namedShadowIndexingMode` to be `ON` and specify your named shadow names in `namedShadowNames` filter.
**Example**
```
"filter": {
"namedShadowNames": [ "namedShadow1", "namedShadow2" ]
}
```
`geoLocations`
To add geolocation data to your fleet indexing configuration:
+ If your geolocation data is stored in a classic (unnamed) shadow, set `thingIndexingMode` to be REGISTRY\$1AND\$1SHADOW, and specify your geolocation data in `geoLocations` filter.
The example filter below specifies a geoLocation object in a classic (unnamed) shadow:
```
"filter": {
"geoLocations": [
{
"name": "shadow.reported.location",
"order": "LonLat"
}
]
}
```
+ If your geolocation data is stored in a named shadow, set `namedShadowIndexingMode` to be ON, add the shadow name in `namedShadowNames` filter, and specify your geolocation data in `geoLocations` filter.
The example filter below specifies a geoLocation object in a named shadow (`nameShadow1`):
```
"filter": {
"namedShadowNames": [ "namedShadow1" ],
"geoLocations": [
{
"name": "shadow.name.namedShadow1.reported.location",
"order": "LonLat"
}
]
}
```
For more information, see [ IndexingFilter](https://docs.aws.amazon.com/iot/latest/apireference/API_IndexingFilter.html) from *AWS IoT* *API Reference*.
### Updating indexing configuration examples
To update your indexing configuration, use the AWS IoT **update-indexing-configuration** CLI command . The following examples show how to use **update-indexing-configuration**.
Short syntax:
```
aws iot update-indexing-configuration --thing-indexing-configuration \
'thingIndexingMode=REGISTRY_AND_SHADOW, deviceDefenderIndexingMode=VIOLATIONS,
namedShadowIndexingMode=ON,filter={namedShadowNames=[thing1shadow]}, thingConnectivityIndexingMode=STATUS,
customFields=[{name=attributes.version,type=Number},
{name=shadow.name.thing1shadow.desired.DefaultDesired, type=String}, {name=shadow.desired.power, type=Boolean},
{name=deviceDefender.securityProfile1.NUMBER_VALUE_BEHAVIOR.lastViolationValue.number, type=Number}]'
```
JSON syntax:
```
aws iot update-indexing-configuration --cli-input-json \ '{
"thingIndexingConfiguration": { "thingIndexingMode": "REGISTRY_AND_SHADOW",
"thingConnectivityIndexingMode": "STATUS",
"deviceDefenderIndexingMode": "VIOLATIONS",
"namedShadowIndexingMode": "ON",
"filter": { "namedShadowNames": ["thing1shadow"]},
"customFields": [ { "name": "shadow.desired.power", "type": "Boolean" },
{"name": "attributes.version", "type": "Number"},
{"name": "shadow.name.thing1shadow.desired.DefaultDesired", "type": "String"},
{"name": "deviceDefender.securityProfile1.NUMBER_VALUE_BEHAVIOR.lastViolationValue.number", "type": Number} ] } }'
```
This command doesn't produce any output.
To check the thing index status, run the `describe-index` CLI command:
```
aws iot describe-index --index-name "AWS_Things"
```
The output of the `describe-index` command looks like the following:
```
{
"indexName": "AWS_Things",
"indexStatus": "ACTIVE",
"schema": "MULTI_INDEXING_MODE"
}
```
**Note**
It can take a moment for fleet indexing to update the fleet index. We recommend waiting until the `indexStatus` shows ACTIVE before using it. You can have different values in the schema field depending on what data sources you've configured. For more information, see [Describing a thing index](#describe-index).
To get your thing indexing configuration details, run the `get-indexing-configuration` CLI command:
```
aws iot get-indexing-configuration
```
The output of the `get-indexing-configuration` command looks like the following:
```
{
"thingIndexingConfiguration": {
"thingIndexingMode": "REGISTRY_AND_SHADOW",
"thingConnectivityIndexingMode": "STATUS",
"deviceDefenderIndexingMode": "VIOLATIONS",
"namedShadowIndexingMode": "ON",
"managedFields": [
{
"name": "connectivity.disconnectReason",
"type": "String"
},
{
"name": "registry.version",
"type": "Number"
},
{
"name": "thingName",
"type": "String"
},
{
"name": "deviceDefender.violationCount",
"type": "Number"
},
{
"name": "shadow.hasDelta",
"type": "Boolean"
},
{
"name": "shadow.name.*.version",
"type": "Number"
},
{
"name": "shadow.version",
"type": "Number"
},
{
"name": "connectivity.version",
"type": "Number"
},
{
"name": "connectivity.timestamp",
"type": "Number"
},
{
"name": "shadow.name.*.hasDelta",
"type": "Boolean"
},
{
"name": "registry.thingTypeName",
"type": "String"
},
{
"name": "thingId",
"type": "String"
},
{
"name": "connectivity.connected",
"type": "Boolean"
},
{
"name": "registry.thingGroupNames",
"type": "String"
}
],
"customFields": [
{
"name": "shadow.name.thing1shadow.desired.DefaultDesired",
"type": "String"
},
{
"name": "deviceDefender.securityProfile1.NUMBER_VALUE_BEHAVIOR.lastViolationValue.number",
"type": "Number"
},
{
"name": "shadow.desired.power",
"type": "Boolean"
},
{
"name": "attributes.version",
"type": "Number"
}
],
"filter": {
"namedShadowNames": [
"thing1shadow"
]
}
},
"thingGroupIndexingConfiguration": {
"thingGroupIndexingMode": "OFF"
}
}
```
To update the custom fields, you can run the `update-indexing-configuration` command. An example is as follows:
```
aws iot update-indexing-configuration --thing-indexing-configuration
'thingIndexingMode=REGISTRY_AND_SHADOW,customFields=[{name=attributes.version,type=Number},{name=attributes.color,type=String},{name=shadow.desired.power,type=Boolean},{name=shadow.desired.intensity,type=Number}]'
```
This command added `shadow.desired.intensity` to the indexing configuration.
**Note**
Updating the custom field indexing configuration overwrites all existing custom fields. Make sure to specify all custom fields when calling **update-indexing-configuration**.
After the index is rebuilt, you can use an aggregation query on the newly added fields, search registry data, shadow data, and thing connectivity status data.
When changing the indexing mode, make sure all of your custom fields are valid by using the new indexing mode. For example, if you start off using `REGISTRY_AND_SHADOW` mode with a custom field called `shadow.desired.temperature`, you must delete the `shadow.desired.temperature` custom field before changing the indexing mode to `REGISTRY`. If your indexing configuration contains custom fields that aren't indexed by the indexing mode, the update fails.
## Describing a thing index
The following command shows you how to use the **describe-index** CLI command to retrieve the current status of the thing index.
```
aws iot describe-index --index-name "AWS_Things"
```
The response of the command can look like the following:
```
{
"indexName": "AWS_Things",
"indexStatus": "BUILDING",
"schema": "REGISTRY_AND_SHADOW_AND_CONNECTIVITY_STATUS"
}
```
The first time that you fleet indexing, AWS IoT builds your index. When `indexStatus` is in the `BUILDING` state, you can't query the index. The `schema` for the things index indicates which type of data (`REGISTRY_AND_SHADOW_AND_CONNECTIVITY_STATUS`) is indexed.
Changing the configuration of your index causes the index to be rebuilt. During this process, the `indexStatus` is `REBUILDING`. You can run queries on data in the things index while it's being rebuilt. For example, if you change the index configuration from `REGISTRY` to `REGISTRY_AND_SHADOW` while the index is being rebuilt, you can query registry data, including the latest updates. However, you can't query the shadow data until the rebuild is complete. The amount of time it takes to build or rebuild the index depends on the amount of data.
You can see different values in the schema field depending on the data sources that you've configured. The following table shows the different schema values and the corresponding descriptions:
| Schema | Description |
| --- | --- |
| OFF | No data sources are configured or indexed. |
| REGISTRY | Registry data is indexed. |
| REGISTRY\$1AND\$1SHADOW | Registry data and unnamed (classic) shadow data are indexed. |
| REGISTRY\$1AND\$1CONNECTIVITY | Registry data and connectivity data are indexed. |
| REGISTRY\$1AND\$1SHADOW\$1AND\$1CONNECTIVITY\$1STATUS | Registry data, unnamed (classic) shadow data, and connectivity data are indexed. |
| MULTI\$1INDEXING\$1MODE | Named shadow or Device Defender violations data is indexed, in addition to registry, unnamed (classic) shadow or connectivity data. |
## Querying a thing index
Use the **search-index** CLI command to query data in the index.
```
aws iot search-index --index-name "AWS_Things" --query-string
"thingName:mything*"
```
```
{
"things":[{
"thingName":"mything1",
"thingGroupNames":[
"mygroup1"
],
"thingId":"a4b9f759-b0f2-4857-8a4b-967745ed9f4e",
"attributes":{
"attribute1":"abc"
},
"connectivity": {
"connected":false,
"timestamp":1556649874716,
"disconnectReason": "CONNECTION_LOST"
}
},
{
"thingName":"mything2",
"thingTypeName":"MyThingType",
"thingGroupNames":[
"mygroup1",
"mygroup2"
],
"thingId":"01014ef9-e97e-44c6-985a-d0b06924f2af",
"attributes":{
"model":"1.2",
"country":"usa"
},
"shadow":{
"desired":{
"location":"new york",
"myvalues":[3, 4, 5]
},
"reported":{
"location":"new york",
"myvalues":[1, 2, 3],
"stats":{
"battery":78
}
},
"metadata":{
"desired":{
"location":{
"timestamp":123456789
},
"myvalues":{
"timestamp":123456789
}
},
"reported":{
"location":{
"timestamp":34535454
},
"myvalues":{
"timestamp":34535454
},
"stats":{
"battery":{
"timestamp":34535454
}
}
}
},
"version":10,
"timestamp":34535454
},
"connectivity": {
"connected":true,
"timestamp":1556649855046
}
}],
"nextToken":"AQFCuvk7zZ3D9pOYMbFCeHbdZ+h=G"
}
```
In the JSON response, `"connectivity"` (as enabled by the `thingConnectivityIndexingMode=STATUS` setting) provides a Boolean value, a timestamp, and a disconnectReason that indicates whether the device is connected to AWS IoT Core. The device `"mything1"` disconnected (`false`) at POSIX time `1556649874716` due to `CONNECTION_LOST`. For more information about disconnect reasons, see [Lifecycle events](life-cycle-events.md).
```
"connectivity": {
"connected":false,
"timestamp":1556649874716,
"disconnectReason": "CONNECTION_LOST"
}
```
The device `"mything2"` connected (`true`) at POSIX time `1556649855046`:
```
"connectivity": {
"connected":true,
"timestamp":1556649855046
}
```
Timestamps are given in milliseconds since epoch, so `1556649855046` represents 6:44:15.046 PM on Tuesday, April 30, 2019 (UTC).
**Important**
If a device has been disconnected for approximately an hour, the `"timestamp"` value and the `"disconnectReason"` value of the connectivity status might be missing.
## Restrictions and limitations
These are the restrictions and limitations for `AWS_Things`.
**Shadow fields with complex types**
A shadow field is indexed only if the value of the field is a simple type, such as a JSON object that doesn't contain an array, or an array that consists entirely of simple types. Simple type means a string, number, or one of the literals `true` or `false`. For example, given the following shadow state, the value of field `"palette"` isn't indexed because it's an array that contains items of complex types. The value of field `"colors"` is indexed because each value in the array is a string.
```
{
"state": {
"reported": {
"switched": "ON",
"colors": [ "RED", "GREEN", "BLUE" ],
"palette": [
{
"name": "RED",
"intensity": 124
},
{
"name": "GREEN",
"intensity": 68
},
{
"name": "BLUE",
"intensity": 201
}
]
}
}
}
```
**Nested shadow field names**
The names of nested shadow fields are stored as a period (.) delimited string. For example, given a shadow document:
```
{
"state": {
"desired": {
"one": {
"two": {
"three": "v2"
}
}
}
}
}
```
The name of field `three` is stored as `desired.one.two.three`. If you also have a shadow document, it's stored like this:
```
{
"state": {
"desired": {
"one.two.three": "v2"
}
}
}
```
Both match a query for `shadow.desired.one.two.three:v2`. As a best practice, don't use periods in shadow field names.
**Shadow metadata**
A field in a shadow's metadata section is indexed, but only if the corresponding field in the shadow's `"state"` section is indexed. (In the previous example, the `"palette"` field in the shadow's metadata section isn't indexed either.)
**Unregistered devices**
Fleet indexing indexes the connectivity status for a device whose connection `clientId` is the same as the `thingName` of a registered thing in [Registry](https://docs.aws.amazon.com//iot/latest/developerguide/thing-registry.html).
**Unregistered shadows**
If you use [UpdateThingShadow](https://docs.aws.amazon.com/iot/latest/apireference/API_iotdata_UpdateThingShadow.html) to create a shadow using a thing name that hasn't been registered in your AWS IoT account, fields in this shadow aren't indexed. This applies to both classic unnamed shadow and named shadow.
**Numeric values**
If any registry or shadow data is recognized by the service as a numeric value, it's indexed as such. You can form queries involving ranges and comparison operators on numeric values (for example, `"attribute.foo<5"` or `"shadow.reported.foo:[75 TO 80]"`). To be recognized as numeric, the value of the data must be a valid, literal type JSON number. The value can be an integer in the range -2^53...2^53-1, a double-precision floating point with optional exponential notation, or part of an array that contains only these values.
**Null values**
Null values aren't indexed.
**Maximum values**
The maximum number of custom fields for aggregation queries is 5.
The maximum number of requested percentiles for aggregation queries is 100.
## Authorization
You can specify the things index as an Amazon Resource Name (ARN) in an AWS IoT policy action, as follows.
****
| Action | Resource |
| --- | --- |
| `iot:SearchIndex` | An index ARN (for example, `arn:aws:iot:your-aws-regionyour-aws-account:index/AWS_Things`). |
| `iot:DescribeIndex` | An index ARN (for example, `arn:aws:iot:your-aws-region:index/AWS_Things`). |
**Note**
If you have permissions to query the fleet index, you can access the data of things across the entire fleet.
# Manage thing group indexing
`AWS_ThingGroups` is the index that contains all of your thing groups. You can use this index to search for groups based on group name, description, attributes, and all parent group names.
## Enabling thing group indexing
You can use the `thing-group-indexing-configuration` setting in the [UpdateIndexingConfiguration](https://docs.aws.amazon.com/iot/latest/apireference/API_UpdateIndexingConfiguration.html) API to create the `AWS_ThingGroups` index and control its configuration. You can use the [GetIndexingConfiguration](https://docs.aws.amazon.com/iot/latest/apireference/API_GetIndexingConfiguration.html) API to retrieve the current indexing configuration.
To update the thing group indexing configurations, run the **update-indexing-configuration** CLI command:
```
aws iot update-indexing-configuration --thing-group-indexing-configuration thingGroupIndexingMode=ON
```
You can also update configurations for both thing and thing group indexing in a single command, as follows:
```
aws iot update-indexing-configuration --thing-indexing-configuration thingIndexingMode=REGISTRY --thing-group-indexing-configuration thingGroupIndexingMode=ON
```
The following are valid values for `thingGroupIndexingMode`.
OFF
No indexing/delete index.
ON
Create or configure the `AWS_ThingGroups` index.
To retrieve the current thing and thing group indexing configurations, run the **get-indexing-configuration** CLI command:
```
aws iot get-indexing-configuration
```
The response of the command looks like the following:
```
{
"thingGroupIndexingConfiguration": {
"thingGroupIndexingMode": "ON"
}
}
```
## Describing group indexes
To retrieve the current status of the `AWS_ThingGroups` index, use the **describe-index** CLI command:
```
aws iot describe-index --index-name "AWS_ThingGroups"
```
The response of the command looks like the following:
```
{
"indexStatus": "ACTIVE",
"indexName": "AWS_ThingGroups",
"schema": "THING_GROUPS"
}
```
AWS IoT builds your index the first time that you indexing. You can't query the index if the `indexStatus` is `BUILDING`.
## Querying a thing group index
To query data in the index, use the **search-index** CLI command:
```
aws iot search-index --index-name "AWS_ThingGroups" --query-string "thingGroupName:mythinggroup*"
```
## Authorization
You can specify the thing groups index as a resource ARN in an AWS IoT policy action, as follows.
****
| Action | Resource |
| --- | --- |
| `iot:SearchIndex` | An index ARN (for example, `arn:aws:iot:your-aws-region:index/AWS_ThingGroups`). |
| `iot:DescribeIndex` | An index ARN (for example, `arn:aws:iot:your-aws-region:index/AWS_ThingGroups`). |
# Device connectivity status queries
AWS IoT Fleet Indexing supports individual device connectivity querying, allowing you to efficiently retrieve connectivity status and related metadata for specific devices. This feature complements existing fleet-wide indexing and querying capabilities.
## How it works
Device connectivity query support can be used for optimized single-device connectivity status retrieval. This API provides low-latency, high-throughput access to the most recent device-specific connectivity information. Once you enable connectivity indexing you will have access to this query API which will be charged as standard queries. For more information, see [AWS IoT Device Management pricing](https://aws.amazon.com/iot-device-management/pricing/#:~:text=Search%20queries%20(per%2010%2C000%20queries))
## Features
With device connectivity query support, you can:
1. Query the current connectivity state (connected or disconnected) for a given device using its `thingName`.
1. Retrieve additional connectivity metadata, including:
1. Disconnect reason
1. Timestamps for the most recent connect or disconnect event.
**Note**
Fleet indexing indexes the connectivity status for a device whose connection `clientId` is the same as the `thingName` of a registered thing in [Registry.](https://docs.aws.amazon.com/iot/latest/developerguide/thing-registry.html)
## Benefits
1. **Low latency:** Reflects the most recent device connectivity state and offers low latency to reflect connection state changes from IoT Core. IoT Core determines a device as disconnected either as soon as it receives a disconnect request from the device or in case of a device disconnecting without sending a disconnect request. IoT core will wait 1.5x of the configured keep-alive time before the client is determined to be disconnected. The Connectivity status API will reflect these changes typically under one second after IoT Core determines a device’s connected state change.
1. **High throughput:** Supports 350 Transactions Per Second (TPS) by default, and can be adjustable to higher upon request.
1. **Data retention: **Stores event data indefinitely when Fleet Indexing (FI) ConnectivityIndexing mode is enabled and the thing is not deleted. If you disable Connectivity Indexing, the records will not be retained.
**Note**
If connectivity status indexing was enabled before the launch of this API, Fleet Indexing begins tracking connectivity status changes after the API launch and reflects the updated status based on those changes.
## Prerequisites
To use device connectivity query support:
1. [Set up an AWS account](https://docs.aws.amazon.com/iot/latest/developerguide/setting-up.html)
1. Onboard and register devices to AWS IoT Core in your preferred region
1. [Enable Fleet Indexing](https://docs.aws.amazon.com/iot/latest/developerguide/managing-index.html) with Connectivity indexing
**Note**
No additional setup is required if you already have connectivity indexing enabled
For detailed setup instructions, refer to the [AWS IoT Developer Guide](https://docs.aws.amazon.com/iot/latest/developerguide/setting-up.html)
## Examples
```
aws iot get-thing-connectivity-data --thing-name myThingName
```
```
{
"connected": true,
"disconnectReason": "NONE",
"thingName": "myThingName",
"timestamp": "2024-12-19T10:00:00.000000-08:00"
}
```
+ `thingName`: The name of the device as indicated by the request. This also matches the clientId used to connect to AWS IoT Core.
+ `disconnectReason`: Reason for disconnect. Will be NONE for a connected device.
+ `connected`: The boolean value true indicating this device is currently connected.
+ `timestamp`: The timestamp representing the device’s most recent disconnect in milliseconds.
```
aws iot get-thing-connectivity-data --thing-name myThingName
```
```
{
"connected": false,
"disconnectReason": "CLIENT_INITIATED_DISCONNECT",
"thingName": "myThingName",
"timestamp": "2024-12-19T10:30:00.000000-08:00"
}
```
+ `thingName`: The name of the device as indicated by the request. This also matches the clientId used to connect to AWS IoT Core.
+ `disconnectReason`: Reason for disconnect is CLIENT\$1INITIATED\$1DISCONNECT indicating the client indicated to AWS IoT Core that it would disconnect.
+ `connected`: The boolean value false indicating this device is currently disconnected.
+ `timestamp`: The timestamp representing the device’s most recent disconnect in milliseconds.
```
aws iot get-thing-connectivity-data --thing-name neverConnectedThing
```
```
{
"connected": false,
"disconnectReason": "UNKNOWN",
"thingName": "neverConnectedThing"
}
```
+ `thingName`: The name of the device as indicated by the request. This also matches the clientId used to connect to AWS IoT Core.
+ `disconnectReason`: Reason for disconnect. Will be “UNKNOWN” for a device which has never been connected or for which Fleet Indexing does not have the last disconnect reason stored.
+ `connected`: The boolean value false indicating this device is currently disconnected.
+ `timestamp`: The timestamp is not returned for a device which has never been connected or for which Fleet Indexing does not have the last timestamp stored.
# Querying for aggregate data
AWS IoT provides four APIs (`GetStatistics`, `GetCardinality`, `GetPercentiles`, and `GetBucketsAggregation`) that allow you to search your device fleet for aggregate data.
**Note**
For issues with missing or unexpected values for the aggregation APIs, read [Fleet indexing troubleshooting guide](fleet-indexing-troubleshooting.md).
## GetStatistics
The [GetStatistics](https://docs.aws.amazon.com/iot/latest/apireference/API_GetStatistics.html) API and the **get-statistics** CLI command return the count, average, sum, minimum, maximum, sum of squares, variance, and standard deviation for the specified aggregated field.
The **get-statistics** CLI command takes the following parameters:
`index-name`
The name of the index to search. The default value is `AWS_Things`.
`query-string`
The query used to search the index. You can specify `"*"` to get the count of all indexed things in your AWS account.
`aggregationField`
(Optional)The field to aggregate. This field must be a managed or custom field defined when you call **update-indexing-configuration**. If you don't specify an aggregation field, `registry.version` is used as the aggregation field.
`query-version`
The version of the query to use. The default value is `2017-09-30`.
The type of aggregation field can affect the statistics returned.
### GetStatistics with string values
If you aggregate on a string field, calling `GetStatistics` returns a count of devices that have attributes that match the query. For example:
```
aws iot get-statistics --aggregation-field 'attributes.stringAttribute'
--query-string '*'
```
This command returns the number of devices that contain an attribute named `stringAttribute`:
```
{
"statistics": {
"count": 3
}
}
```
### GetStatistics with Boolean values
When you call `GetStatistics` with a Boolean aggregation field:
+ AVERAGE is the percentage of devices that match the query.
+ MINIMUM is 0 or 1 according to the following rules:
+ If all the values for the aggregation field are `false`, MINIMUM is 0.
+ If all the values for the aggregation field are `true`, MINIMUM is 1.
+ If the values for the aggregation field are a mixture of `false` and `true`, MINIMUM is 0.
+ MAXIMUM is 0 or 1 according to the following rules:
+ If all the values for the aggregation field are `false`, MAXIMUM is 0.
+ If all the values for the aggregation field are `true`, MAXIMUM is 1.
+ If the values for the aggregation field are a mixture of `false` and `true`, MAXIMUM is 1.
+ SUM is the sum of the integer equivalent of the Boolean values.
+ COUNT is the count of things that match the query string criteria and contain a valid aggregation field value.
### GetStatistics with numerical values
When you call `GetStatistics` and specify an aggregation field of type `Number`, `GetStatistics` returns the following values:
count
The count of things that match the query string criteria and contain a valid aggregation field value.
average
The average of the numerical values that match the query.
sum
The sum of the numerical values that match the query.
minimum
The smallest of the numerical values that match the query.
maximum
The largest of the numerical values that match the query.
sumOfSquares
The sum of the squares of the numerical values that match the query.
variance
The variance of the numerical values that match the query. The variance of a set of values is the average of the squares of the differences of each value from the average value of the set.
stdDeviation
The standard deviation of the numerical values that match the query. The standard deviation of a set of values is a measure of how spread out the values are.
The following example shows how to call **get-statistics** with a numerical custom field.
```
aws iot get-statistics --aggregation-field 'attributes.numericAttribute2'
--query-string '*'
```
```
{
"statistics": {
"count": 3,
"average": 33.333333333333336,
"sum": 100.0,
"minimum": -125.0,
"maximum": 150.0,
"sumOfSquares": 43750.0,
"variance": 13472.22222222222,
"stdDeviation": 116.06990230986766
}
}
```
For numerical aggregation fields, if the field values exceed the maximum double value, the statistics values are empty.
## GetCardinality
The [GetCardinality](https://docs.aws.amazon.com/iot/latest/apireference/API_GetCardinality.html) API and the **get-cardinality** CLI command return the approximate count of unique values that match the query. For example, you might want to find the number of devices with battery levels at less than 50 percent:
```
aws iot get-cardinality --index-name AWS_Things --query-string "batterylevel
> 50" --aggregation-field "shadow.reported.batterylevel"
```
This command returns the number of things with battery levels at more than 50 percent:
```
{
"cardinality": 100
}
```
`cardinality` is always returned by **get-cardinality** even if there are no matching fields. For example:
```
aws iot get-cardinality --query-string "thingName:Non-existent*"
--aggregation-field "attributes.customField_STR"
```
```
{
"cardinality": 0
}
```
The **get-cardinality** CLI command takes the following parameters:
`index-name`
The name of the index to search. The default value is `AWS_Things`.
`query-string`
The query used to search the index. You can specify `"*"` to get the count of all indexed things in your AWS account.
`aggregationField`
The field to aggregate.
`query-version`
The version of the query to use. The default value is `2017-09-30`.
## GetPercentiles
The [GetPercentiles](https://docs.aws.amazon.com/iot/latest/apireference/API_GetPercentiles.html) API and the **get-percentiles** CLI command groups the aggregated values that match the query into percentile groupings. The default percentile groupings are: 1,5,25,50,75,95,99, although you can specify your own when you call `GetPercentiles`. This function returns a value for each percentile group specified (or the default percentile groupings). The percentile group "1" contains the aggregated field value that occurs in approximately one percent of the values that match the query. The percentile group "5" contains the aggregated field value that occurs in approximately five percent of the values that match the query, and so on. The result is an approximation, the more values that match the query, the more accurate the percentile values.
The following example shows how to call the **get-percentiles** CLI command.
```
aws iot get-percentiles --query-string "thingName:*" --aggregation-field
"attributes.customField_NUM" --percents 10 20 30 40 50 60 70 80 90 99
```
```
{
"percentiles": [
{
"value": 3.0,
"percent": 80.0
},
{
"value": 2.5999999999999996,
"percent": 70.0
},
{
"value": 3.0,
"percent": 90.0
},
{
"value": 2.0,
"percent": 50.0
},
{
"value": 2.0,
"percent": 60.0
},
{
"value": 1.0,
"percent": 10.0
},
{
"value": 2.0,
"percent": 40.0
},
{
"value": 1.0,
"percent": 20.0
},
{
"value": 1.4,
"percent": 30.0
},
{
"value": 3.0,
"percent": 99.0
}
]
}
```
The following command shows the output returned from **get-percentiles** when there are no matching documents.
```
aws iot get-percentiles --query-string "thingName:Non-existent*"
--aggregation-field "attributes.customField_NUM"
```
```
{
"percentiles": []
}
```
The **get-percentile** CLI command takes the following parameters:
`index-name`
The name of the index to search. The default value is `AWS_Things`.
`query-string`
The query used to search the index. You can specify `"*"` to get the count of all indexed things in your AWS account.
`aggregationField`
The field to aggregate, which must be of `Number` type.
`query-version`
The version of the query to use. The default value is `2017-09-30`.
`percents`
(Optional)You can use this parameter to specify custom percentile groupings.
## GetBucketsAggregation
The [GetBucketsAggregation](https://docs.aws.amazon.com/iot/latest/apireference/API_GetBucketsAggregation.html) API and the **get-buckets-aggregation** CLI command return a list of buckets and the total number of things that fit the query string criteria.
The following example shows how to call the get-buckets-aggregation CLI command.
```
aws iot get-buckets-aggregation --query-string '*' --index-name AWS_Things --aggregation-field 'shadow.reported.batterylevelpercent' --buckets-aggregation-type 'termsAggregation={maxBuckets=5}'
```
This command returns the following:
```
{
"totalCount": 20,
"buckets": [
{
"keyValue": "100",
"count": 12
},
{
"keyValue": "90",
"count": 5
},
{
"keyValue": "75",
"count": 3
}
]
}
```
The get-buckets-aggregation CLI command takes the following parameters:
`index-name`
The name of the index to search. The default value is `AWS_Things`.
`query-string`
The query used to search the index. You can specify `"*"` to get the count of all indexed things in your AWS account.
`aggregation-field`
The field to aggregate.
`buckets-aggregation-type`
The basic control of the response shape and the bucket aggregation type to perform.
## Authorization
You can specify the thing groups index as a resource ARN in an AWS IoT policy action, as follows.
| Action | Resource |
| --- | --- |
| `iot:GetStatistics` | An index ARN (for example, `arn:aws:iot:your-aws-region:index/AWS_Things` or `arn:aws:iot:your-aws-region:index/AWS_ThingGroups`). |
# Query syntax
In fleet indexing, you use a query syntax to specify queries.
## Supported features
The query syntax supports the following features:
+ Terms and phrases
+ Searching fields
+ Prefix search
+ Range search
+ Boolean operators `AND`, `OR`, `NOT`, and `–`. The hyphen is used to exclude something from search results (for example, `thingName:(tv* AND -plasma)`).
+ Grouping
+ Field grouping
+ Escaping special characters (such as with *\$1*)
+ Leading wildcard use is limited to 1 query term per query. For example, you cannot search for `thingName:*my` and `thingGroupNames:*echo` in the same query. Queries that include a leading wildcard have a max query length of 100 characters.
**Note**
Accounts with a Fleet Indexing query API limit exceeding 30 requests per second are subject to throttling when using leading-wildcard query terms. This 30 requests-per-second limit applies at the account level across all Fleet Indexing query APIs.
## Unsupported features
The query syntax doesn't support the following features:
+ Regular expressions
+ Boosting
+ Ranking
+ Fuzzy searches
+ Proximity search
+ Sorting
+ Aggregation
+ Special characters: ```, `@`, `#`, `%`, `\`, `/`, `'`, `;`, and `,`. Note that `,` is only supported in geoqueries.
## Notes
A few things to note about the query language:
+ The default operator is AND. A query for `"thingName:abc thingType:xyz"` is equivalent to `"thingName:abc AND thingType:xyz"`.
+ If a field isn't specified, AWS IoT searches for the term in all the registry, Device Shadow, and Device Defender fields.
+ All field names are case sensitive.
+ Search is case insensitive. Words are separated by white-space characters as defined by Java's `Character.isWhitespace(int)`.
+ Indexing of Device Shadow data (unnamed shadows and named shadows) includes reported, desired, delta, and metadata sections.
+ Device shadow and registry versions aren't searchable, but are present in the response.
+ The maximum number of terms in a query is twelve.
+ The special character `,` is only supported in geoqueries.
# Example thing queries
Specify queries in a query string using a query syntax. The queries are passed to the [https://docs.aws.amazon.com/iot/latest/apireference/API_SearchIndex.html](https://docs.aws.amazon.com/iot/latest/apireference/API_SearchIndex.html) API. The following table lists some example query strings.
| Query string | Result |
| --- | --- |
| abc | Queries for "abc" in any registry, shadow (classic unnamed shadow and named shadow), or Device Defender violations field. |
| thingName:myThingName | Queries for a thing with name "myThingName". |
| thingName:my\$1 | Queries for things with names that begin with "my". |
| thingName:\$1my | Queries for things with names that end in "my". Leading wildcard use is limited to 1 query term per query. Queries that include a leading wildcard have a max query length of 100 characters. |
| thingName:\$1my\$1 | Queries for things that contain the substring "my". Leading wildcard use is limited to 1 query term per query. Queries that include a leading wildcard have a max query length of 100 characters. |
| thingName:ab? | Queries for things with names that have "ab" plus one additional character (for example, "aba", "abb", "abc", and so on. |
| thingTypeName:aa | Queries for things that are associated with type "aa". |
| thingGroupNames:a | Queries for things with a parent thing group or billing group name "a". |
| thingGroupNames:a\$1 | Queries for things with a parent thing group or billing group name matching the pattern "a\$1". |
| attributes.myAttribute:75 | Queries for things with an attribute named "myAttribute" that has the value 75. |
| attributes.myAttribute:[75 TO 80] | Queries for things with an attribute named "myAttribute" that has a value that falls within a numeric range (75–80, inclusive). |
| attributes.myAttribute:\$175 TO 80] | Queries for things with an attribute named "myAttribute" that has a value that falls within the numeric range (>75 and <=80). |
| attributes.serialNumber:["abcd" TO "abcf"] | Queries for things with an attribute named "serialNumber" that has a value within an alphanumeric string range. This query returns things with a "serialNumber" attribute with values "abcd", "abce", or "abcf". |
| attributes.myAttribute:i\$1t | Queries for things with an attribute named "myAttribute" where the value is 'i', followed by any number of characters, followed by 't'. |
| attributes.attr1:abc AND attributes.attr2<5 NOT attributes.attr3>10 | Queries for things that combine terms using Boolean expressions. This query returns things that have an attribute named "attr1" with a value "abc", an attribute named "attr2" that's less than 5, and an attribute named "attr3" that' not greater than 10. |
| shadow.hasDelta:true | Queries for things with an unnamed shadow that has a delta element. |
| NOT attributes.model:legacy | Queries for things where the attribute named "model" is not "legacy". |
| shadow.reported.stats.battery:\$170 TO 100\$1 (v2 OR v3) NOT attributes.model:legacy | Queries for things with the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/example-queries.html) |
| shadow.reported.myvalues:2 | Queries for things where the `myvalues` array in the shadow's reported section contains a value of 2. |
| shadow.reported.location:\$1 NOT shadow.desired.stats.battery:\$1 | Queries for things with the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/example-queries.html) |
| shadow.name..hasDelta:true | Queries for things that have a shadow with the given name and also a delta element. |
| shadow.name..desired.filament:\$1 | Queries for things that have a shadow with the given name and also a desired filament property. |
| shadow.name..reported.location:\$1 | Queries for things that have a shadow with the given name and where the `location` attribute exists in the named shadow's reported section. |
| connectivity.connected:true | Queries for all connected devices. |
| connectivity.connected:false | Queries for all disconnected devices. |
| connectivity.connected:true AND connectivity.timestamp : [1557651600000 TO 1557867600000] | Queries for all connected devices with a connect timestamp >= 1557651600000 and <= 1557867600000. Timestamps are given in milliseconds since epoch. |
| connectivity.connected:false AND connectivity.timestamp : [1557651600000 TO 1557867600000] | Queries for all disconnected devices with a disconnect timestamp >= 1557651600000 and <= 1557867600000. Timestamps are given in milliseconds since epoch. |
| connectivity.connected:true AND connectivity.timestamp > 1557651600000 | Queries for all connected devices with a connect timestamp > 1557651600000. Timestamps are given in milliseconds since epoch. |
| connectivity.connected:\$1 | Queries for all devices with connectivity information present. |
| connectivity.disconnectReason:\$1 | Queries for all devices with connectivity disconnectReason present. |
| connectivity.disconnectReason:CLIENT\$1INITIATED\$1DISCONNECT | Queries for all devices disconnected due to CLIENT\$1INITIATED\$1DISCONNECT. |
| deviceDefender.violationCount:[0 TO 100] | Queries for things with a Device Defender violations count value that falls within the numeric range (0-100, inclusive). |
| deviceDefender..disconnectBehavior.inViolation:true | Queries for things that are in violation for the behavior disconnectBehavior as defined in the security profile device-SecurityProfile. Note that inViolation:false is not a valid query. |
| deviceDefender..disconnectBehavior.lastViolationValue.number>2 | Queries for things that are in violation for the behavior disconnectBehavior as defined in the security profile device-SecurityProfile with a last violation event value greater than 2. |
| deviceDefender..disconnectBehavior.lastViolationTime>1634227200000 | Queries for things that are in violation for the behavior `disconnectBehavior` as defined in the security profile device-SecurityProfile with a last violation event after a specified epoch time. |
| shadow.name.gps-tracker.reported.coordinates:geo\$1distance,47.6204,-122.3491,15.5km | Queries for things that are within the radial distance of 15.5 km from the coordinates of 47.6204,-122.3491. This query string applies to when your location data is stored in a named shadow. |
| shadow.reported.coordinates:geo\$1distance,47.6204,-122.3491,15.5km | Queries for things that are within the radial distance of 15.5 km from the coordinates of 47.6204,-122.3491. This query string applies to when your location data is stored in a classic shadow. |
# Example thing group queries
Queries are specified in a query string using a query syntax and passed to the [https://docs.aws.amazon.com/iot/latest/apireference/API_SearchIndex.html](https://docs.aws.amazon.com/iot/latest/apireference/API_SearchIndex.html) API. The following table lists some example query strings.
| Query string | Result |
| --- | --- |
| abc | Queries for "abc" in any field. |
| thingGroupName:myGroupThingName | Queries for a thing group with name "myGroupThingName". |
| thingGroupName:my\$1 | Queries for thing groups with names that begin with "my". |
| thingGroupName:ab? | Queries for thing groups with names that have "ab" plus one additional character (for example: "aba", "abb", "abc", and so on). |
| attributes.myAttribute:75 | Queries for thing groups with an attribute named "myAttribute" that has the value 75. |
| attributes.myAttribute:[75 TO 80] | Queries for thing groups with an attribute named "myAttribute" whose value falls within a numeric range (75–80, inclusive). |
| attributes.myAttribute:[75 TO 80] | Queries for thing groups with an attribute named "myAttribute" whose value falls within the numeric range (>75 and <=80). |
| attributes.myAttribute:["abcd" TO "abcf"] | Queries for thing groups with an attribute named "myAttribute" whose value is within an alphanumeric string range. This query returns thing groups with a "serialNumber" attribute with values "abcd", "abce", or "abcf". |
| attributes.myAttribute:i\$1t | Queries for thing groups with an attribute named "myAttribute" whose value is 'i', followed by any number of characters, followed by 't'. |
| attributes.attr1:abc AND attributes.attr2<5 NOT attributes.attr3>10 | Queries for thing groups that combine terms using Boolean expressions. This query returns thing groups that have an attribute named "attr1" with a value "abc", an attribute named "attr2" that's less than 5, and an attribute named "attr3" that's not greater than 10. |
| NOT attributes.myAttribute:cde | Queries for thing groups where the attribute named "myAttribute" is not "cde". |
| parentGroupNames:(myParentThingGroupName) | Queries for thing groups whose parent group name matches "myParentThingGroupName". |
| parentGroupNames:(myParentThingGroupName OR myRootThingGroupName) | Queries for thing groups whose parent group name matches "myParentThingGroupName" or "myRootThingGroupName". |
| parentGroupNames:(myParentThingGroupNa\$1) | Queries for thing groups whose parent group name begins with "myParentThingGroupNa". |
# Indexing location data
You can use [AWS IoT fleet indexing](https://docs.aws.amazon.com//iot/latest/developerguide/iot-indexing.html) to index your devices' last sent location data and search for devices using geoqueries. This feature resolves device monitoring and management use cases such as location tracking and proximity search. Location indexing works similarly to other fleet indexing features, and with additional configurations to specify in your [thing indexing](managing-fleet-index.md).
Common use cases include: search and aggregate devices located within desired geographic boundaries, get location specific insights using query terms related to device metadata and state from indexed data sources, provide a granular view such as filtering results to a specific geographic area to reduce rendering lags within your fleet monitoring maps and track last reported device location, and identify devices that are outside of the desired boundary limits and generate alarms using [fleet metrics](iot-fleet-metrics.md). To get started with location indexing and geoqueries, see [Getting started tutorial](location-indexing-tutorial.md).
## Supported data formats
AWS IoT fleet indexing supports the following location data formats:
1.
**Well-known text representation of coordinate reference systems**
A string that follows the [Geographic information - Well-known text representation of coordinate reference systems](https://docs.ogc.org/is/12-063r5/12-063r5.html) format. An example can be `"POINT(long lat)"`.
1.
**A string that represents the coordinates**
A string with the format of `"latitude, longitude"` or `"longitude, latitude"` . If you use `"longitude, latitude"`, you must also specify `order` in `geoLocations`. An example can be `"41.12,-71.34"`.
1.
**An object of lat(latitude), lon(longitude) keys**
This format is applicable to classic shadow and named shadow. Supported keys: `lat`, `latitude`, `lon`, `long`, `longitude`. An example can be `{"lat": 41.12, "lon": -71.34}`.
1.
**An array that represents the coordinates**
An array with the format `[lat,lon]` or `[lon,lat]`. If you use the format `[lon,lat]`, which is the same as the coordinates in [GeoJSON](https://geojson.org/) (applicable to classic shadow and named shadow), you must also specify `order` in `geoLocations`.
An example can be:
```
{
"location": {
"coordinates": [
**Longitude**,
**Latitude**
],
"type": "Point",
"properties": {
"country": "United States",
"city": "New York",
"postalCode": "*****",
"horizontalAccuracy": 20,
"horizontalConfidenceLevel": 0.67,
"state": "New York",
"timestamp": "2023-01-04T20:59:13.024Z"
}
}
}
```
## How to index location data
The following steps show how to update indexing configuration for your location data and use geoqueries to search for devices.
1.
**Know where your location data is stored**
Fleet indexing currently supports indexing location data stored in classic shadows or named shadows.
1.
**Use supported location data formats**
Make sure your location data format follows one of the [Supported data formats](#location-indexing-format).
1.
**Update indexing configuration**
At a minimum need, enable thing (registry) indexing configuration. You must also enable indexing on classic shadow or named shadow that contain your location data. When updating your thing indexing, you should include your location data in the indexing configuration.
1.
**Create and run geoqueries**
Depending on your use cases, create geoqueries and run them to search for devices. The geoqeury you compose must follow the [Query syntax](https://docs.aws.amazon.com//iot/latest/developerguide/query-syntax.html). You can find some examples in [Example geoqueries](#location-indexing-geoqueries).
## Update thing indexing configuration
To index location data, you must update indexing configuration and include your location data. Depending on where your location data is stored, follow the steps to update your indexing configuration:
### Location data stored in classic shadows
If your location data is stored in a classic shadow, you must set `thingIndexingMode` to be `REGISTRY_AND_SHADOW` and specify your location data in the `geoLocations` fields (`name` and `order`) in [https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html](https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html).
In the following thing indexing configuration example, you specify the location data path `shadow.reported.coordinates` as `name` and `LonLat` as `order`.
```
{
"thingIndexingMode": "REGISTRY_AND_SHADOW",
"filter": {
"geoLocations": [
{
"name": "shadow.reported.coordinates",
"order": "LonLat"
}
]
}
}
```
+ `thingIndexingMode`
The indexing mode controls if registry or shadow is indexed. When `thingIndexingMode` is set to be `OFF`, thing indexing is disabled.
To index location data stored in a classic shadow, you must set `thingIndexingMode` to be `REGISTRY_AND_SHADOW`. For more information, see [Thing indexing modes](managing-index.md#index-mode).
+ `filter`
Indexing filter provides additional selections for named shadows and geolocation data. For more information, see [Indexing filter](managing-index.md#thing-indexing-filter).
+ `geoLocations`
The list of geolocation targets that you select to index. The default maximum number of geolocation targets for indexing is `1`. To increase the limit, see [AWS IoT Device Management Quotas](https://docs.aws.amazon.com//general/latest/gr/iot_device_management.html#fleet-indexing-limits).
+ `name`
The name of the geolocation target field. An example value of `name` can be the location data path of your shadow: `shadow.reported.coordinates`.
+ `order`
The order of the geolocation target field. Valid values: `LatLon` and `LonLat`. `LatLon` means latitude and longitude. `LonLat` means longitude and latitude. This field is optional. The default value is `LatLon`.
### Location data stored in named shadows
If your location data is stored in a named shadow, set `namedShadowIndexingMode` to be `ON`, add your named shadow name(s) to the `namedShadowNames` field in [https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html](https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html), and specify your location data path in the `geoLocations` field in [https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html](https://docs.aws.amazon.com//iot/latest/apireference/API_IndexingFilter.html).
In the following thing indexing configuration example, you specify the location data path `shadow.name.namedShadow1.reported.coordinates` as `name` and `LonLat` as `order`.
```
{
"thingIndexingMode": "REGISTRY",
"namedShadowIndexingMode": "ON",
"filter": {
"namedShadowNames": [
"namedShadow1"
],
"geoLocations": [
{
"name": "shadow.name.namedShadow1.reported.coordinates",
"order": "LonLat"
}
]
}
}
```
+ `thingIndexingMode`
The indexing mode controls if registry or shadow is indexed. When `thingIndexingMode` is set to be `OFF`, thing indexing is disabled.
To index location data stored in a named shadow, you must set `thingIndexingMode` to be `REGISTRY` (or `REGISTRY_AND_SHADOW`). For more information, see [Thing indexing modes](managing-index.md#index-mode).
+ `filter`
Indexing filter provides additional selections for named shadows and geolocation data. For more information, see [Indexing filter](managing-index.md#thing-indexing-filter).
+ `geoLocations`
The list of geolocation targets that you select to index. The default maximum number of geolocation targets for indexing is `1`. To increase the limit, see [AWS IoT Device Management Quotas](https://docs.aws.amazon.com//general/latest/gr/iot_device_management.html#fleet-indexing-limits).
+ `name`
The name of the geolocation target field. An example value of `name` can be the location data path of your shadow: `shadow.name.namedShadow1.reported.coordinates`.
+ `order`
The order of the geolocation target field. Valid values: `LatLon` and `LonLat`. `LatLon` means latitude and longitude. `LonLat` means longitude and latitude. This field is optional. The default value is `LatLon`.
## Example geoqueries
After you complete the indexing configuration for your location data, run geoqueries to search for devices. You can also combine your geoqueries with other query strings. For more information, see [Query syntax](query-syntax.md) and [Example thing queries](example-queries.md).
**Example query 1**
This example assumes the location data is stored in a named shadow `gps-tracker`. The output of this command is the list of devices that are within the radial distance of 15.5 km from the center point with coordinates (47.6204,-122.3491).
```
aws iot search-index --query-string \
"shadow.name.gps-tracker.reported.coordinates:geo_distance,47.6204,-122.3491,15.5km"
```
**Example query 2**
This example assumes the location data is stored in a classic shadow. The output of this command is the list of devices that are within the radial distance of 15.5 km from the center point with coordinates (47.6204,-122.3491).
```
aws iot search-index --query-string \
"shadow.reported.coordinates:geo_distance,47.6204,-122.3491,15.5km"
```
**Example query 3**
This example assumes the location data is stored in a classic shadow. The output of this command is the list of devices that are not connected and outside the radial distance of 15.5 km from the center point with coordinates (47.6204,-122.3491).
```
aws iot search-index --query-string \
"connectivity.connected:false AND (NOT shadow.reported.coordinates:geo_distance,47.6204,-122.3491,15.5km)"
```
# Getting started tutorial
This tutorial demonstrates how to use [fleet indexing](iot-indexing.md) to [index your location data](location-indexing-geoquery.md). For simplicity, you create a thing to represent your device and store the location data in a named shadow, update thing indexing configuration for location indexing, and run example geoqueries to search for devices within a radial boundary.
This tutorial takes about 15 minutes to complete.
**Topics**
+ [
## Prerequisites
](#location-indexing-tutorial-prerequisites)
+ [
## Create thing and shadow
](#location-indexing-create-resources)
+ [
## Update thing indexing configuration
](#location-indexing-update-configuration)
+ [
## Run geoquery
](#location-indexing-run-geoquery)
## Prerequisites
+ Install the latest version of [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html).
+ Familiarize yourself with [Location indexing and geoqueries](https://docs.aws.amazon.com/iot/latest/developerguide/location-indexing-geoquery.html), [Manage thing indexing](https://docs.aws.amazon.com/iot/latest/developerguide/managing-index.html), and [Query syntax](https://docs.aws.amazon.com/iot/latest/developerguide/query-syntax.html).
## Create thing and shadow
You create a thing to represent your device, and a named shadow to store its location data (coordinates 47.61564, -122.33584).
1. Run the following command to create your thing that represents your bike named Bike-1. For more information about how to create a thing using AWS CLI, see [create-thing](https://docs.aws.amazon.com//cli/latest/reference/iot/create-thing.html) from *AWS CLI** Reference*.
```
aws iot create-thing --thing-name "Bike-1" \
--attribute-payload '{"attributes": {"model":"OEM-2302-12", "battery":"35", "acqDate":"06/09/23"}}'
```
The output of this command can look like the following:
```
{
"thingName": "Bike-1",
"thingArn": "arn:aws:iot:us-east-1:123456789012:thing/Bike-1",
"thingId": "df9cf01d-b0c8-48fe-a2e2-e16cff6b23df"
}
```
1. Run the following command to create a named shadow to store Bike-1's location data (coordinates 47.61564, -122.33584). For more information about how to create a named shadow using AWS CLI, see [update-thing-shadow](https://docs.aws.amazon.com//cli/latest/reference/iot-data/update-thing-shadow.html) from *AWS CLI** Reference*.
```
aws iot-data update-thing-shadow \
--thing-name Bike-1 \
--shadow-name Bike1-shadow \
--cli-binary-format raw-in-base64-out \
--payload '{"state":{"reported":{"coordinates":{"lat": 47.6153, "lon": -122.3333}}}}' \
"output.txt" \
```
This command doesn't produce any output. To view the named shadow you created, you can run the [list-named-shadows-for-thing](https://docs.aws.amazon.com//cli/latest/reference/iot-data/list-named-shadows-for-thing.html) CLI command.
```
aws iot-data list-named-shadows-for-thing --thing-name Bike-1
```
The output of this command can look like the following:
```
{
"results": [
"Bike1-shadow"
],
"timestamp": 1699574309
}
```
## Update thing indexing configuration
To index your location data, you must update your thing indexing configuration to include the location data. Because your location data is stored in a named shadow in this tutorial, set `thingIndexingMode` to be `REGISTRY` (at a minimum requirement), set `namedShadowIndexingMode` to be `ON`, and add your location data to the configuration. In this example, you must add the name of your named shadow and the shadow's location data path to `filter`.
1. Run the command to update your indexing configuration for location indexing.
```
aws iot update-indexing-configuration --cli-input-json '{
"thingIndexingConfiguration": { "thingIndexingMode": "REGISTRY",
"thingConnectivityIndexingMode": "OFF",
"deviceDefenderIndexingMode": "OFF",
"namedShadowIndexingMode": "ON",
"filter": {
"namedShadowNames": ["Bike1-shadow"],
"geoLocations":[{
"name":"shadow.name.Bike1-shadow.reported.coordinates"
}]
},
"customFields": [
{ "name":"attributes.battery",
"type":"Number"}] } }'
```
The command doesn't produce any output. You may need to wait for a moment until the update is complete. To check the status, run the [describe-index](https://docs.aws.amazon.com//cli/latest/reference/iot/describe-index.html) CLI command. If you see `indexStatus` shows: `ACTIVE`, your thing indexing update is complete.
1. Run the command to verify your indexing configuration. This step is optional.
```
aws iot get-indexing-configuration
```
The output can look like the following:
```
{
"thingIndexingConfiguration": {
"thingIndexingMode": "REGISTRY",
"thingConnectivityIndexingMode": "OFF",
"deviceDefenderIndexingMode": "OFF",
"namedShadowIndexingMode": "ON",
"managedFields": [
{
"name": "shadow.name.*.hasDelta",
"type": "Boolean"
},
{
"name": "registry.version",
"type": "Number"
},
{
"name": "registry.thingTypeName",
"type": "String"
},
{
"name": "registry.thingGroupNames",
"type": "String"
},
{
"name": "shadow.name.*.version",
"type": "Number"
},
{
"name": "thingName",
"type": "String"
},
{
"name": "thingId",
"type": "String"
}
],
"customFields": [
{
"name": "attributes.battery",
"type": "Number"
}
],
"filter": {
"namedShadowNames": [
"Bike1-shadow"
],
"geoLocations": [
{
"name": "shadow.name.Bike1-shadow.reported.coordinates",
"order": "LatLon"
}
]
}
},
"thingGroupIndexingConfiguration": {
"thingGroupIndexingMode": "OFF"
}
}
```
## Run geoquery
Now you have updated your thing indexing configuration to include the location data. Try to create some geoqueries and run them to see if you can get your desired search results. A geoquery must follow the [Query syntax](query-syntax.md). You can find some useful example geoqueries in [Example geoqueries](location-indexing-geoquery.md#location-indexing-geoqueries).
In the following example command, you use the geoquery `shadow.name.Bike1-shadow.reported.coordinates:geo_distance,47.6204,-122.3491,15.5km` to search for devices that are within the radial distance of 15.5 km from the center point with coordinates (47.6204,-122.3491).
```
aws iot search-index --query-string "shadow.name.Bike1-shadow.reported.coordinates:geo_distance,47.6204,-122.3491,15.5km"
```
Because you have a device located at the coordinates "lat": 47.6153, "lon": -122.3333, which falls within the distance of 15.5 km of the center point, you should be able to see this device (Bike-1) in the output. The output can look like the following:
```
{
"things": [
{
"thingName": "Bike-1",
"thingId": "df9cf01d-b0c8-48fe-a2e2-e16cff6b23df",
"attributes": {
"acqDate": "06/09/23",
"battery": "35",
"model": "OEM-2302-12"
},
"shadow": "{\"reported\":{\"coordinates\":{\"lat\":47.6153,\"lon\":-122.3333}},\"metadata\":{\"reported\":{\"coordinates\":{\"lat\":{\"timestamp\":1699572906},\"lon\":{\"timestamp\":1699572906}}}},\"hasDelta\":false,\"version\":1}"
}
]
}
```
For more information, see [Indexing location data](location-indexing-geoquery.md).
# Fleet metrics
Fleet metrics is a feature of [fleet indexing](iot-indexing.md), a managed service that allows you to index, search, and aggregate your devices' data in AWS IoT. You can use fleet metrics, to monitor your fleet devices' aggregate state in [CloudWatch](https://console.aws.amazon.com/cloudwatch/) over time, including reviewing your fleet devices' disconnection rate or average battery level changes of a specified period.
Using fleet metrics, you can build [aggregation queries](index-aggregate.md) whose results are continually emitted to [CloudWatch](https://console.aws.amazon.com/cloudwatch/) as metrics for analyzing trends and creating alarms. For your monitoring tasks, you can specify the aggregation queries of different aggregation types (**Statistics**, **Cardinality**, and **Percentile**). You can save all of your aggregation queries to create fleet metrics for reuse in the future.
# Getting started tutorial
In this tutorial, you create a [fleet metric](iot-fleet-metrics.md) to monitor your sensors' temperatures to detect potential anomalies. When creating the fleet metric, you define an [aggregation query](index-aggregate.md) that detects the number of sensors with temperatures exceeding 80 degrees Fahrenheit. You specify the query to run every 60 seconds and the query results are emitted to CloudWatch, where you can view the number of sensors that have potential high-temperature risks, and set alarms. To complete this tutorial, you'll use [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html).
In this tutorial, you'll learn how to:
+ [Set up](#fleet-metrics-tutorial-setup)
+ [Create fleet metrics](#fleet-metrics-tutorial-create)
+ [View metrics in CloudWatch](#fleet-metrics-tutorial-view-data)
+ [Clean up resources](#fleet-metrics-tutorial-delete-fleet-metrics)
This tutorial takes about 15 minutes to complete.
## Prerequisites
+ Install the latest version of [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html)
+ Familiarize yourself with [Querying for aggregate data](https://docs.aws.amazon.com/iot/latest/developerguide/index-aggregate.html)
+ Familiarize yourself with [Using Amazon CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html)
## Set up
To use fleet metrics, enable fleet indexing. To enable fleet indexing for your things or thing groups with specified data sources and associated configurations, follow the instructions in [Managing thing indexing](managing-index.md#enable-index) and [Managing thing group indexing](thinggroup-index.md#enable-group-index).
**To set up**
1. Run the following command to enable fleet indexing and specify the data sources to search from.
```
aws iot update-indexing-configuration \
--thing-indexing-configuration "thingIndexingMode=REGISTRY_AND_SHADOW,customFields=[{name=attributes.temperature,type=Number},{name=attributes.rackId,type=String},{name=attributes.stateNormal,type=Boolean}],thingConnectivityIndexingMode=STATUS" \
```
The preceding example CLI command enables fleet indexing to support searching registry data, shadow data, and thing connectivity status using the `AWS_Things` index.
The configuration change can take a few minutes to complete. Verify that your fleet indexing is enabled before you create fleet metrics.
To check if your fleet indexing has been enabled, run the following CLI command:
```
aws --region us-east-1 iot describe-index --index-name "AWS_Things"
```
For more information, see [Enable thing indexing](managing-index.md#enable-index).
1. Run the following bash script to create ten things and describe them.
```
# Bash script. Type `bash` before running in other shells.
Temperatures=(70 71 72 73 74 75 47 97 98 99)
Racks=(Rack1 Rack1 Rack2 Rack2 Rack3 Rack4 Rack5 Rack6 Rack6 Rack6)
IsNormal=(true true true true true true false false false false)
for ((i=0; i < 10; i++))
do
thing=$(aws iot create-thing --thing-name "TempSensor$i" --attribute-payload attributes="{temperature=${Temperatures[@]:$i:1},rackId=${Racks[@]:$i:1},stateNormal=${IsNormal[@]:$i:1}}")
aws iot describe-thing --thing-name "TempSensor$i"
done
```
This script creates ten things to represent ten sensors. Each thing has attributes of `temperature`, `rackId`, and `stateNormal` as described in the following table:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/fleet-metrics-get-started.html)
The output of this script contains ten JSON files. One of the JSON file looks like the following:
```
{
"version": 1,
"thingName": "TempSensor0",
"defaultClientId": "TempSensor0",
"attributes": {
"rackId": "Rack1",
"stateNormal": "true",
"temperature": "70"
},
"thingArn": "arn:aws:iot:region:account:thing/TempSensor0",
"thingId": "example-thing-id"
}
```
For more information, see [Create a thing](https://docs.aws.amazon.com/iot/latest/developerguide/thing-registry.html#create-thing).
## Create fleet metrics
**To create a fleet metric**
1. Run the following command to create a fleet metric named *high\$1temp\$1FM*. You create the fleet metric to monitor the number of sensors with temperatures exceeding 80 degrees Fahrenheit in CloudWatch.
```
aws iot create-fleet-metric --metric-name "high_temp_FM" --query-string "thingName:TempSensor* AND attributes.temperature >80" --period 60 --aggregation-field "attributes.temperature" --aggregation-type name=Statistics,values=count
```
--metric-name
Data type: string. The `--metric-name` parameter specifies a fleet metric name. In this example, you're creating a fleet metric named *high\$1temp\$1FM*.
--query-string
Data type: string. The `--query-string` parameter specifies the query string. In this example, the query string means to query all the things with names starting with *TempSensor* and with temperatures higher than 80 degrees Fahrenheit. For more information, see [Query syntax](query-syntax.md).
--period
Data type: integer. The `--period` parameter specifies the time to retrieve the aggregated data in seconds. In this example, you specify that the fleet metric you're creating retrieves the aggregated data every 60 seconds.
--aggregation-field
Data type: string. The `--aggregation-field` parameter specifies the attribute to evaluate. In this example, the temperature attribute is to be evaluated.
--aggregation-type
The `--aggregation-type` parameter specifies the statistical summary to display in the fleet metric. For your monitoring tasks, you can customize the aggregation query properties for the different aggregation types (**Statistics**, **Cardinality**, and **Percentile**). In this example, you specify **count** for the aggregation type and **Statistics** to return the count of devices that have attributes that match the query, in other words, to return the count of the devices with names starting with *TempSensor* and with temperatures higher than 80 degrees Fahrenheit. For more information, see [Querying for aggregate data](index-aggregate.md).
The output of this command looks like the following:
```
{
"metricArn": "arn:aws:iot:region:111122223333:fleetmetric/high_temp_FM",
"metricName": "high_temp_FM"
}
```
**Note**
It can take a moment for the data points to display in CloudWatch.
To learn more about how to create a fleet metric, read [Managing fleet metrics](managing-fleet-metrics.md).
If you can't create a fleet metric, read [Troubleshooting fleet metrics](fleet-indexing-troubleshooting.md#fleet-metrics-troubleshooting).
1. (Optional) Run the following command to describe your fleet metric named *high\$1temp\$1FM*:
```
aws iot describe-fleet-metric --metric-name "high_temp_FM"
```
The output of this command looks like the following:
```
{
"queryVersion": "2017-09-30",
"lastModifiedDate": 1625249775.834,
"queryString": "*",
"period": 60,
"metricArn": "arn:aws:iot:region:111122223333:fleetmetric/high_temp_FM",
"aggregationField": "registry.version",
"version": 1,
"aggregationType": {
"values": [
"count"
],
"name": "Statistics"
},
"indexName": "AWS_Things",
"creationDate": 1625249775.834,
"metricName": "high_temp_FM"
}
```
## View fleet metrics in CloudWatch
After creating the fleet metric, you can view the metric data in CloudWatch. In this tutorial, you will see the metric that shows the number of sensors with names starting with *TempSensor* and with temperatures higher than 80 degrees Fahrenheit.
**To view data points in CloudWatch**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. On the CloudWatch menu on the left panel, choose **Metrics** to expand the submenu and then choose **All metrics**. This opens the page with the upper half to display the graph and the lower half containing four tabbed sections.
1. The first tabbed section **All metrics** lists all the metrics that you can view in groups, choose **IoTFleetMetrics**. This contains all of your fleet metrics.
1. On the **Aggregation type** section of the **All metrics** tab, choose **Aggregation type** to view all the fleet metrics you created.
1. Choose the fleet metric to display graph on the left of the **Aggregation type** section. You will see the value *count* to the left of your **Metric name**, and this is the value of the aggregation type that you specified in the [Create fleet metrics](#fleet-metrics-tutorial-create) section of this tutorial.
1. Choose the second tab named **Graphed metrics** to the right of the **All metrics** tab to view the fleet metric you chose from the previous step.
You should be able to see a graph that displays the number of sensors with temperatures higher than 80 degrees Fahrenheit like the following:
![\[AWS IoT fleet metrics\]](http://docs.aws.amazon.com/iot/latest/developerguide/images/fm-metric-in-cloudwatch.png)
**Note**
The **Period** attribute in CloudWatch defaults to 5 minutes. It's the time interval between data points displaying in CloudWatch. You can change the **Period** setting based on your needs.
1. (Optional) You can set a metric alarm.
1. On the CloudWatch menu on the left panel, choose **Alarms** to expand the submenu and then choose **All alarms**.
1. On the **Alarms** page, choose **Create alarm** on the upper right corner. Follow the **Create alarm** instructions in console to create an alarm as needed. For more information, see [Using Amazon CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html).
To learn more, read [Using Amazon CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html).
If you can't see data points in CloudWatch, read [Troubleshooting fleet metrics](fleet-indexing-troubleshooting.md#fleet-metrics-troubleshooting).
## Clean up
**To delete fleet metrics**
You use the **delete-fleet-metric** CLI command to delete fleet metrics.
To delete the fleet metric named *high\$1temp\$1FM*, run the following command.
```
aws iot delete-fleet-metric --metric-name "high_temp_FM"
```
**To clean up things**
You use the **delete-thing** CLI command to delete things.
To delete the ten things that you created, run the following script:
```
# Bash script. Type `bash` before running in other shells.
for ((i=0; i < 10; i++))
do
thing=$(aws iot delete-thing --thing-name "TempSensor$i")
done
```
**To clean up metrics in CloudWatch**
CloudWatch doesn't support metrics deletion. Metrics expire based on their retention schedules. To learn more, [Using Amazon CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html).
# Managing fleet metrics
This topic shows how to use the AWS IoT console and AWS CLI to manage your fleet metrics.
**Topics**
+ [
## Managing fleet metrics (Console)
](#managing-fleet-metrics-console)
+ [
## Managing fleet metrics (CLI)
](#managing-fleet-metrics-cli)
+ [
## Authorize tagging of IoT resources
](#managing-fleet-metrics-policy)
## Managing fleet metrics (Console)
The following sections show how to use the AWS IoT console to manage your fleet metrics. Make sure you've enabled fleet indexing with associated data sources and configurations before creating fleet metrics.
### Enable fleet indexing
If you've already enabled fleet indexing, skip this section.
If you haven't enabled fleet indexing, follow these instructions.
1. Open your AWS IoT console at [https://console.aws.amazon.com/iot/](https://console.aws.amazon.com/iot/).
1. On the AWS IoT menu, choose **Settings**.
1. To view the detailed settings, on the **Settings** page, scroll down to the **Fleet indexing** section.
1. To update your fleet indexing settings, to the right of the **Fleet indexing** section, select **Manage indexing**.
1. On the **Manage fleet indexing** page, update your fleet indexing settings based on your needs.
+ **Configuration**
To turn on thing indexing, toggle **Thing indexing** on, and then select the data sources you want to index from.
To turn on thing group indexing, toggle **Thing group indexing** on.
+ **Custom fields for aggregation - *optional***
Custom fields are a list of field name and field type pairs.
To add a custom field pair, choose **Add new field**. Enter a custom field name such as `attributes.temperature`, then select a field type from the **Field type** menu. Note that a custom field name begins with `attributes.` and will be saved as an attribute to run [thing aggregations queries](https://docs.aws.amazon.com/iot/latest/developerguide/index-aggregate.html).
To update and save the setting, choose **Update**.
### Create a fleet metric
1. Open your AWS IoT console at [https://console.aws.amazon.com/iot/](https://console.aws.amazon.com/iot/).
1. On the AWS IoT menu, choose **Manage**, and then choose **Fleet metrics**.
1. On the **Fleet metrics** page, choose **Create fleet metric** and complete the creation steps.
1. In step 1 **Configure fleet metrics**
+ In **Query** section, enter a query string to specify the things or thing groups you want to perform the aggregate search. The query string consists of an attribute and a value. For **Properties**, choose the attribute you want, or, if it doesn't appear in the list, enter the attribute in the field. Enter the value after `:`. An example query string can be `thingName:TempSensor*`. For each query string you enter, press **enter** in your keyboard. If you enter multiple query strings, specify their relationship by selecting **and**, **or**, **and not**, or **or not** between them.
+ In **Report properties**, choose **Index name**, **Aggregation type**, and **Aggregation field** from their respective lists. Next, select the data you want to aggregate in **Select data**, where you can select multiple data values.
+ Choose **Next**.
1. In step 2 **Specify fleet metric properties**
+ In **Fleet metric name** field, enter a name for the fleet metric you're creating.
+ In **Description - *optional*** field, enter a description for the fleet metric you're creating. This field is optional.
+ In **Hours** and **Minutes** fields, enter the time (how often) you want the fleet metric to emit data to CloudWatch.
+ Choose **Next**.
1. In step 3 **Review and create**
+ Review the settings of step 1 and step 2. To edit the settings, choose **Edit**.
+ Choose **Create fleet metric**.
After successful creation, the fleet metric is listed on the **Fleet metric** page.
### Update a fleet metric
1. On the **Fleet metric** page, choose the fleet metric that you want to update.
1. On the fleet metric **Details** page, choose **Edit**. This opens the creation steps where you can update your fleet metric in any of the three steps.
1. After you finish updating the fleet metric, choose **Update fleet metric**.
### Delete a fleet metric
1. On the **Fleet metric** page, choose the fleet metric that you want to delete.
1. On the next page that shows details of your fleet metric, choose **Delete**.
1. In the dialog box, enter the name of your fleet metric to confirm deletion.
1. Choose **Delete**. This step deletes your fleet metric permanently.
## Managing fleet metrics (CLI)
The following sections show how to use the AWS CLI to manage your fleet metrics. Make sure you've enabled fleet indexing with associated data sources and configurations before creating fleet metrics. To enable fleet indexing for your things or thing groups, follow the instructions in [Managing thing indexing](managing-index.md#enable-index) or [Managing thing group indexing](thinggroup-index.md#enable-group-index).
### Create a fleet metric
You can use the create-fleet-metric CLI command to create a fleet metric.
```
aws iot create-fleet-metric --metric-name "YourFleetMetricName" --query-string "*" --period 60 --aggregation-field "registry.version" --aggregation-type name=Statistics,values=sum
```
The output of this command contains the name and Amazon Resource Name (ARN) of your fleet metric. The output looks like the following:
```
{
"metricArn": "arn:aws:iot:us-east-1:111122223333:fleetmetric/YourFleetMetricName",
"metricName": "YourFleetMetricName"
}
```
### List fleet metrics
You can use the list-fleet-metric CLI command to list all the fleet metrics in your account.
```
aws iot list-fleet-metrics
```
The output of this command contains all your fleet metrics. The output looks like the following:
```
{
"fleetMetrics": [
{
"metricArn": "arn:aws:iot:us-east-1:111122223333:fleetmetric/YourFleetMetric1",
"metricName": "YourFleetMetric1"
},
{
"metricArn": "arn:aws:iot:us-east-1:111122223333:fleetmetric/YourFleetMetric2",
"metricName": "YourFleetMetric2"
}
]
}
```
### Describe a fleet metric
You can use the describe-fleet-metric CLI command to display more detailed information about a fleet metric.
```
aws iot describe-fleet-metric --metric-name "YourFleetMetricName"
```
The output of command contains the detailed information about the specified fleet metric. The output looks like the following:
```
{
"queryVersion": "2017-09-30",
"lastModifiedDate": 1625790642.355,
"queryString": "*",
"period": 60,
"metricArn": "arn:aws:iot:us-east-1:111122223333:fleetmetric/YourFleetMetricName",
"aggregationField": "registry.version",
"version": 1,
"aggregationType": {
"values": [
"sum"
],
"name": "Statistics"
},
"indexName": "AWS_Things",
"creationDate": 1625790642.355,
"metricName": "YourFleetMetricName"
}
```
### Update a fleet metric
You can use the update-fleet-metric CLI command to update a fleet metric.
```
aws iot update-fleet-metric --metric-name "YourFleetMetricName" --query-string "*" --period 120 --aggregation-field "registry.version" --aggregation-type name=Statistics,values=sum,count --index-name AWS_Things
```
The update-fleet-metric command doesn't produce any output. You can use the describe-fleet-metric CLI command to see the result.
```
{
"queryVersion": "2017-09-30",
"lastModifiedDate": 1625792300.881,
"queryString": "*",
"period": 120,
"metricArn": "arn:aws:iot:us-east-1:111122223333:fleetmetric/YourFleetMetricName",
"aggregationField": "registry.version",
"version": 2,
"aggregationType": {
"values": [
"sum",
"count"
],
"name": "Statistics"
},
"indexName": "AWS_Things",
"creationDate": 1625792300.881,
"metricName": "YourFleetMetricName"
}
```
### Delete a fleet metric
Use the delete-fleet-metric CLI command to delete a fleet metric.
```
aws iot delete-fleet-metric --metric-name "YourFleetMetricName"
```
This command doesn't produce any output if the deletion is successful or if you specify a fleet metric that doesn't exist.
For more information, see [Troubleshooting fleet metrics](fleet-indexing-troubleshooting.md#fleet-metrics-troubleshooting).
## Authorize tagging of IoT resources
For better control over fleet metrics that you can create, modify, or use, you can attach tags to the fleet metrics.
To tag fleet metrics that you create by using AWS Management Console or AWS CLI, you must include the `iot:TagResource` action in your IAM policy to grant the user permissions. If your IAM policy doesn't include `iot:TagResource`, any actions to create a fleet metric with a tag will return an `AccessDeniedException` error.
For general information about tagging your resources, see [Tagging your AWS IoT resources](https://docs.aws.amazon.com//iot/latest/developerguide/tagging-iot.html).
### IAM policy example
Refer to the following IAM policy example granting tagging permissions when you create a fleet metric:
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"iot:TagResource"
],
"Effect": "Allow",
"Resource": [
"arn:aws:iot:*:*:fleetmetric/*"
]
},
{
"Action": [
"iot:CreateFleetMetric"
],
"Effect": "Allow",
"Resource": [
"arn:aws:iot:*:*:index/*",
"arn:aws:iot:*:*:fleetmetric/*"
]
}
]
}
```
For more information, see [Actions, resources, and condition keys for AWS IoT](https://docs.aws.amazon.com//service-authorization/latest/reference/list_awsiot.html).