# Rules for AWS IoT
Rules
Rules give your devices the ability to interact with AWS services. Rules are analyzed and actions are performed based on the MQTT topic stream. You can use rules to support the following tasks:
+ Augment or filter data received from a device.
+ Write data received from a device to an Amazon DynamoDB database.
+ Save a file to Amazon S3.
+ Send a push notification to all users who are using Amazon SNS.
+ Publish data to an Amazon SQS queue.
+ Invoke a Lambda function to extract data.
+ Process messages from a large number of devices using Amazon Kinesis.
+ Send data to Amazon OpenSearch Service.
+ Capture a CloudWatch metric.
+ Change a CloudWatch alarm.
+ Send the data from an MQTT message to Amazon SageMaker AI to make predictions based on a machine learning (ML) model.
+ Send a message to a Salesforce IoT Input Stream.
+ Start process of a Step Functions state machine.
+ Send message data to an AWS IoT Events input.
+ Send message data to an asset property in AWS IoT SiteWise.
+ Send message data to a web application or service.
Your rules can use MQTT messages that pass through the publish/subscribe protocol supported by the [Device communication protocols](protocols.md). You can also use the [Basic Ingest](iot-basic-ingest.md) feature to securely send device data to the AWS services listed previously, without incurring [messaging costs](https://aws.amazon.com/iot-core/pricing/). The [Basic Ingest](iot-basic-ingest.md) feature optimizes data flow by removing the publish/subscribe message broker from the ingestion path. This makes it cost effective while still keeping the security and data processing features of AWS IoT.
Before AWS IoT can perform these actions, you must grant it permission to access your AWS resources on your behalf. When the actions are performed, you incur the standard charges for the AWS services that you use.
**Topics**
+ [
# Granting an AWS IoT rule the access it requires
](iot-create-role.md)
+ [
# Passing role permissions
](pass-role.md)
+ [
# Creating an AWS IoT rule
](iot-create-rule.md)
+ [
# Managing an AWS IoT rule
](iot-managae-rule.md)
+ [
# AWS IoT rule actions
](iot-rule-actions.md)
+ [
## Troubleshooting a rule
](#iot-troubleshoot-rule)
+ [
# Accessing cross-account resources using AWS IoT rules
](accessing-cross-account-resources-using-rules.md)
+ [
# Error handling (error action)
](rule-error-handling.md)
+ [
# Reducing messaging costs with Basic Ingest
](iot-basic-ingest.md)
+ [
# AWS IoT SQL reference
](iot-sql-reference.md)
# Granting an AWS IoT rule the access it requires
Grant access
Use IAM roles to control the AWS resources to which each rule has access. Before you create a rule, you must create an IAM role with a policy that allows access to the required AWS resources. AWS IoT assumes this role when implementing a rule.
**Complete the following steps to create the IAM role and AWS IoT policy that grant an AWS IoT rule the access it requires (AWS CLI).**
1. Save the following trust policy document, which grants AWS IoT permission to assume the role, to a file named `iot-role-trust.json`.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "iot.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "123456789012"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:iot:us-east-1:123456789012:rule/rulename"
}
}
}
]
}
```
Use the [create-role](https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html) command to create an IAM role specifying the `iot-role-trust.json` file:
```
aws iam create-role --role-name my-iot-role --assume-role-policy-document file://iot-role-trust.json
```
The output of this command looks like the following:
```
{
"Role": {
"AssumeRolePolicyDocument": "url-encoded-json",
"RoleId": "AKIAIOSFODNN7EXAMPLE",
"CreateDate": "2015-09-30T18:43:32.821Z",
"RoleName": "my-iot-role",
"Path": "/",
"Arn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}
```
1. Save the following JSON into a file named `my-iot-policy.json`.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "dynamodb:*",
"Resource": "*"
}
]
}
```
This JSON is an example policy document that grants AWS IoT administrator access to DynamoDB.
Use the [create-policy](https://docs.aws.amazon.com/cli/latest/reference/iam/create-policy.html) command to grant AWS IoT access to your AWS resources upon assuming the role, passing in the `my-iot-policy.json` file:
```
aws iam create-policy --policy-name my-iot-policy --policy-document file://my-iot-policy.json
```
For more information about how to grant access to AWS services in policies for AWS IoT, see [Creating an AWS IoT rule](iot-create-rule.md).
The output of the [create-policy](https://docs.aws.amazon.com/cli/latest/reference/iam/create-policy.html) command contains the ARN of the policy. Attach the policy to a role.
```
{
"Policy": {
"PolicyName": "my-iot-policy",
"CreateDate": "2015-09-30T19:31:18.620Z",
"AttachmentCount": 0,
"IsAttachable": true,
"PolicyId": "ZXR6A36LTYANPAI7NJ5UV",
"DefaultVersionId": "v1",
"Path": "/",
"Arn": "arn:aws:iam::123456789012:policy/my-iot-policy",
"UpdateDate": "2015-09-30T19:31:18.620Z"
}
}
```
1. Use the [attach-role-policy](https://docs.aws.amazon.com/cli/latest/reference/iam/attach-role-policy.html) command to attach your policy to your role:
```
aws iam attach-role-policy --role-name my-iot-role --policy-arn "arn:aws:iam::123456789012:policy/my-iot-policy"
```
## Revoke rule engine access
To immediately revoke rule engine access, do the following
1. Remove iot.amazonaws.com from the [trust policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-role.html)
1. Follow the steps to [revoke iot role sessions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_revoke-sessions.html)
# Passing role permissions
Pass role permissions
Part of a rule definition is an IAM role that grants permission to access resources specified in the rule's action. The rules engine assumes that role when the rule's action is invoked. The role must be defined in the same AWS account as the rule.
When creating or replacing a rule you are, in effect, passing a role to the rules engine. The `iam:PassRole` permission is required to perform this operation. To verify that you have this permission, create a policy that grants the `iam:PassRole` permission and attach it to your IAM user. The following policy shows how to allow `iam:PassRole` permission for a role.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "Stmt1",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::123456789012:role/myRole"
]
}
]
}
```
In this policy example, the `iam:PassRole` permission is granted for the role `myRole`. The role is specified using the role's ARN. Attach this policy to your IAM user or role that your user belongs to. For more information, see [Working with Managed Policies](https://docs.aws.amazon.com/service-authorization/latest/reference/access_policies_managed-using.html).
**Note**
Lambda functions use resource-based policy, where the policy is attached directly to the Lambda function itself. When you create a rule that invokes a Lambda function, you don't pass a role, so the user creating the rule doesn't need the `iam:PassRole` permission. For more information about Lambda function authorization, see [Granting Permissions Using a Resource Policy](https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html#intro-permission-model-access-policy).
# Creating an AWS IoT rule
Create a rule
You can create AWS IoT rules to route data from your connected things to interact with other AWS services. An AWS IoT rule consists of the following components:
**Components of a rule**
| Component | Description | Required or Optional |
| --- | --- | --- |
| Rule name | The name of the rule. Note that we do not recommend the use of personally identifiable information in your rule names. | Required. |
| Rule description | A textual description of the rule. Note that we do not recommend the use of personally identifiable information in your rule descriptions. | Optional. |
| SQL statement | A simplified SQL syntax to filter messages received on an MQTT topic and push the data elsewhere. For more information, see [AWS IoT SQL reference](iot-sql-reference.md). | Required. |
| SQL version | The version of the SQL rules engine to use when evaluating the rule. Although this property is optional, we strongly recommend that you specify the SQL version. The AWS IoT Core console sets this property to `2016-03-23` by default. If this property is not set, such as in an AWS CLI command or an CloudFormation template, `2015-10-08` is used. For more information, see [SQL versions](iot-rule-sql-version.md). | Required. |
| One or more actions | The actions AWS IoT performs when enacting the rule. For example, you can insert data into a DynamoDB table, write data to an Amazon S3 bucket, publish to an Amazon SNS topic, or invoke a Lambda function. | Required. |
| An error action | The action AWS IoT performs when it's unable to perform a rule's action. | Optional. |
Before you create an AWS IoT rule, you must create an IAM role with a policy that allows access to the required AWS resources. AWS IoT assumes this role when implementing a rule. For more information, see [Granting an AWS IoT rule the access it requires](https://docs.aws.amazon.com//iot/latest/developerguide/iot-create-role.html) and [Passing role permissions](https://docs.aws.amazon.com//iot/latest/developerguide/pass-role.html).
When you create a rule, be aware of how much data you're publishing on topics. If you create rules that include a wildcard topic pattern, they might match a large percentage of your messages. If this is the case, you might need to increase the capacity of the AWS resources used by the target actions. We recommend avoiding wildcard topic patterns in republish rules to prevent duplicate processing and reduce costs.
**Note**
Creating and updating rules are administrator-level actions. Any user who has permission to create or update rules is able to access data processed by the rules.
## Create a rule (Console)
**To create a rule (AWS Management Console)**
Use the [AWS Management Console](https://console.aws.amazon.com//iot/home#/home) command to create a rule:
1. Open the [AWS IoT console](https://console.aws.amazon.com//iot/home#/home).
1. On the left navigation, choose **Message routing** from **Manage** section. Then choose **Rules**.
1. On the **Rules** page, choose **Create rule**.
1. On the **Specify rule properties** page, enter a name for your rule. **Rule description** and **Tags** are optional. Choose **Next**.
1. On the **Configure SQL statement** page, choose a SQL version and enter a SQL statement. An example SQL statement can be `SELECT temperature FROM 'iot/topic' WHERE temperature > 50`. For more information, see [SQL versions](https://docs.aws.amazon.com//iot/latest/developerguide/iot-rule-sql-version.html) and [AWS IoT SQL reference](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-reference.html).
1. On the **Attach rule actions** page, add rule actions to route data to other AWS services.
1. In **Rule actions**, select a rule action from the drop down list. For example, you can choose **Kinesis Stream**. For more information about rule actions, see [AWS IoT rule actions](https://docs.aws.amazon.com//iot/latest/developerguide/iot-rule-actions.html).
1. Depending on the rule action you choose, enter related configuration details. For example, if you choose **Kinesis Stream**, you will need to choose or create a data stream resource, and optionally enter configuration details such as **Partition key**, which is used to group data by shard in a steam.
1. In **IAM role**, choose or create a role to grant AWS IoT access to your endpoint. Note that AWS IoT will automatically create a policy with a prefix of `aws-iot-rule` under your IAM role selected. You can choose **View** to view your IAM role and the policy from the IAM console. **Error action** is optional. You can find more information in [Error handling (error action)](https://docs.aws.amazon.com//iot/latest/developerguide/rule-error-handling.html). For more information about creating an IAM role for your rule, see [Grant a rule the access it requires](https://docs.aws.amazon.com//iot/latest/developerguide/iot-create-role.html). Choose **Next**.
1. On the **Review and create** page, review all the configuration and make edits if needed. Choose **Create**.
After you create a rule successfully, you will see the rule listed on the **Rules** page. You can select a rule to open the **Details** page where you can view a rule, edit a rule, deactivate a rule, and delete a rule.
## Create a rule (CLI)
**To create a rule (AWS CLI)**
Use the [create-topic-rule](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html) command to create a rule:
```
aws iot create-topic-rule --rule-name myrule --topic-rule-payload file://myrule.json
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified DynamoDB table. The SQL statement filters the messages and the role ARN grants AWS IoT permission to write to the DynamoDB table.
```
{
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"dynamoDB": {
"tableName": "my-dynamodb-table",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role",
"hashKeyField": "topic",
"hashKeyValue": "${topic(2)}",
"rangeKeyField": "timestamp",
"rangeKeyValue": "${timestamp()}"
}
}
]
}
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified S3 bucket. The SQL statement filters the messages, and the role ARN grants AWS IoT permission to write to the Amazon S3 bucket.
```
{
"awsIotSqlVersion": "2016-03-23",
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"bucketName": "amzn-s3-demo-bucket",
"key": "myS3Key"
}
}
]
}
```
The following is an example payload file with a rule that pushes data to Amazon OpenSearch Service:
```
{
"sql": "SELECT *, timestamp() as timestamp FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"OpenSearch": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_es",
"endpoint": "https://my-endpoint",
"index": "my-index",
"type": "my-type",
"id": "${newuuid()}"
}
}
]
}
```
The following is an example payload file with a rule that invokes a Lambda function:
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-west-2:123456789012:function:my-lambda-function"
}
}
]
}
```
The following is an example payload file with a rule that publishes to an Amazon SNS topic:
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sns": {
"targetArn": "arn:aws:sns:us-west-2:123456789012:my-sns-topic",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}
]
}
```
The following is an example payload file with a rule that republishes on a different MQTT topic:
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"topic": "my-mqtt-topic",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}
]
}
```
The following is an example payload file with a rule that pushes data to an Amazon Data Firehose stream:
```
{
"sql": "SELECT * FROM 'my-topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"firehose": {
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role",
"deliveryStreamName": "my-stream-name"
}
}
]
}
```
The following is an example payload file with a rule that uses the Amazon SageMaker AI `machinelearning_predict` function to republish to a topic if the data in the MQTT payload is classified as a 1.
```
{
"sql": "SELECT * FROM 'iot/test' where machinelearning_predict('my-model', 'arn:aws:iam::123456789012:role/my-iot-aml-role', *).predictedLabel=1",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role",
"topic": "my-mqtt-topic"
}
}
]
}
```
The following is an example payload file with a rule that publishes messages to a Salesforce IoT Cloud input stream.
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"salesforce": {
"token": "ABCDEFGHI123456789abcdefghi123456789",
"url": "https://ingestion-cluster-id.my-env.sfdcnow.com/streams/stream-id/connection-id/my-event"
}
}
]
}
```
The following is an example payload file with a rule that starts an execution of a Step Functions state machine.
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"stepFunctions": {
"stateMachineName": "myCoolStateMachine",
"executionNamePrefix": "coolRunning",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}
]
}
```
# Managing an AWS IoT rule
Manage a rule
You can use the following actions to manage your AWS IoT rules.
**Topics**
+ [
## Tagging a rule
](#iot-create-rule-tagging)
+ [
## Viewing a rule
](#iot-view-rules)
+ [
## Deleting a rule
](#iot-delete-rule)
## Tagging a rule
To add another layer of specificity to your new or existing rules, you can apply tagging. Tagging leverages key-value pairs in your rules to provide you with greater control over how and where your rules are applied to your AWS IoT resources and services. For example, you can limit the scope of your rule to only apply in your beta environment for pre release testing (`Key=environment, Value=beta`) or capturing all messages sent to the `iot/test` topic from a specific endpoint only and storing them in an Amazon S3 bucket.
### IAM policy example
For an example that shows how to grant tagging permissions for a rule, consider a user that runs the following command to create a rule and tag it to apply only to their beta environment.
In the example, replace:
+ *MyTopicRuleName* with the name of the rule.
+ *myrule.json* with the name of the policy document.
```
aws iot create-topic-rule
--rule-name MyTopicRuleName
--topic-rule-payload file://myrule.json
--tags "environment=beta"
```
For this example, you must use the following IAM policy:
****
```
{
"Version":"2012-10-17",
"Statement":
{
"Action": [ "iot:CreateTopicRule", "iot:TagResource" ],
"Effect": "Allow",
"Resource": [
"arn:aws:iot:us-east-1:123456789012:rule/MyTopicRuleName"
]
}
}
```
The above example shows a newly created rule called `MyTopicRuleName` that applies only to your beta environment. The `iot:TagResource` in the policy statement with `MyTopicRuleName` specifically called out allows tagging when creating or updating `MyTopicRuleName`. The parameter `--tags "environment=beta"` used when creating the rule limits the scope of `MyTopicRuleName` to only your beta environment. If you remove the parameter `--tags "environment=beta"`, then `MyTopicRuleName` will apply to all environments.
For more information on creating IAM roles and policies specific to an AWS IoT rule, see [Granting an AWS IoT rule the access it requires](iot-create-role.md)
For general information about tagging your resources, see [Tagging your AWS IoT resources](tagging-iot.md).
## Viewing a rule
Use the [list-topic-rules](https://docs.aws.amazon.com/cli/latest/reference/iot/list-topic-rules.html) command to list your rules:
```
aws iot list-topic-rules
```
Use the [get-topic-rule](https://docs.aws.amazon.com/cli/latest/reference/iot/get-topic-rule.html) command to get information about a rule:
```
aws iot get-topic-rule --rule-name myrule
```
## Deleting a rule
When you are finished with a rule, you can delete it.
**To delete a rule (AWS CLI)**
Use the [delete-topic-rule](https://docs.aws.amazon.com/cli/latest/reference/iot/delete-topic-rule.html) command to delete a rule:
```
aws iot delete-topic-rule --rule-name myrule
```
# AWS IoT rule actions
AWS IoT rule actions specify what to do when a rule is invoked. You can define actions to send data to an Amazon DynamoDB database, send data to Amazon Kinesis Data Streams, invoke an AWS Lambda function, and so on. AWS IoT supports the following actions in AWS Regions where the action's service is available.
| Rule action | Description | Name in API |
| --- | --- | --- |
| [Apache Kafka](apache-kafka-rule-action.md) | Sends a message to an Apache Kafka cluster. | kafka |
| [CloudWatch alarms](cloudwatch-alarms-rule-action.md) | Changes the state of an Amazon CloudWatch alarm. | cloudwatchAlarm |
| [CloudWatch Logs](cloudwatch-logs-rule-action.md) | Sends a message to Amazon CloudWatch Logs. | cloudwatchLogs |
| [CloudWatch metrics](cloudwatch-metrics-rule-action.md) | Sends a message to a CloudWatch metric. | cloudwatchMetric |
| [DynamoDB](dynamodb-rule-action.md) | Sends a message to a DynamoDB table. | dynamoDB |
| [DynamoDBv2](dynamodb-v2-rule-action.md) | Sends message data to multiple columns in a DynamoDB table. | dynamoDBv2 |
| [Elasticsearch](elasticsearch-rule-action.md) | Sends a message to an OpenSearch endpoint. | OpenSearch |
| [HTTP](https-rule-action.md) | Posts a message to an HTTPS endpoint. | http |
| [AWS IoT Events](iotevents-rule-action.md) | Sends a message to an AWS IoT Events input. | iotEvents |
| [AWS IoT SiteWise](iotsitewise-rule-action.md) | Sends message data to AWS IoT SiteWise asset properties. | iotSiteWise |
| [Firehose](kinesis-firehose-rule-action.md) | Sends a message to a Firehose delivery stream. | firehose |
| [Kinesis Data Streams](kinesis-rule-action.md) | Sends a message to a Kinesis data stream. | kinesis |
| [Lambda](lambda-rule-action.md) | Invokes a Lambda function with message data as input. | lambda |
| [Location](location-rule-action.md) | Sends location data to Amazon Location Service. | location |
| [OpenSearch](opensearch-rule-action.md) | Sends a message to an Amazon OpenSearch Service endpoint. | OpenSearch |
| [Republish](republish-rule-action.md) | Republishes a message to another MQTT topic. | republish |
| [S3](s3-rule-action.md) | Stores a message in an Amazon Simple Storage Service (Amazon S3) bucket. | s3 |
| [Salesforce IoT](salesforce-iot-rule-action.md) | Sends a message to a Salesforce IoT input stream. | salesforce |
| [SNS](sns-rule-action.md) | Publishes a message as an Amazon Simple Notification Service (Amazon SNS) push notification. | sns |
| [SQS](sqs-rule-action.md) | Sends a message to an Amazon Simple Queue Service (Amazon SQS) queue. | sqs |
| [Step Functions](stepfunctions-rule-action.md) | Starts an AWS Step Functions state machine. | stepFunctions |
| [Timestream](timestream-rule-action.md) | Sends a message to an Amazon Timestream database table. | timestream |
**Notes**
Define the rule in the same AWS Region as another service's resource so that the rule action can interact with that resource.
The AWS IoT rules engine might make multiple attempts to perform an action if intermittent errors occur. If all attempts fail, the message is discarded and the error is available in your CloudWatch Logs. You can specify an error action for each rule that is invoked after a failure occurs. For more information, see [Error handling (error action)](rule-error-handling.md).
Some rule actions activate actions in services that integrate with AWS Key Management Service (AWS KMS) to support data encryption at rest. If you use a customer-managed AWS KMS key (KMS key) to encrypt data at rest, the service must have permission to use the KMS key on the caller's behalf. To learn how to manage permissions for your customer managed KMS key, see the data encryption topics in the appropriate service guide. For more information about customer managed KMS keys, see [AWS Key Management Service concepts](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html) in the *AWS Key Management Service Developer Guide*.
You can use any [function](iot-sql-functions.md) or [substitution template](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html) in an error action's SQL statement including the external functions: [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-secret](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-secret), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-machine-learning](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-machine-learning), and [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64). If an error action requires to call an external function, then invoking the error action can result in additional bill for the external function.
# Apache Kafka
The Apache Kafka (Kafka) action sends messages directly to your [Amazon Managed Streaming for Apache Kafka](https://docs.aws.amazon.com//msk/latest/developerguide/what-is-msk.html) (Amazon MSK), Apache Kafka clusters managed by third-party providers such as [Confluent Cloud](https://www.confluent.io/), or self-managed Apache Kafka clusters. With Kafka rule action, you can route your IoT data to Kafka clusters. This enables you to build high-performance data pipelines for various purposes, such as streaming analytics, data integration, visualization, and mission-critical business applications.
**Note**
This topic assumes familiarity with the Apache Kafka platform and related concepts. For more information about Apache Kafka, see [Apache Kafka](https://kafka.apache.org/). [MSK Serverless](https://docs.aws.amazon.com//msk/latest/developerguide/serverless.html) is not supported. MSK Serverless clusters can only be done via IAM authentication, which Apache Kafka rule action doesn't currently support. For more information about how to configure AWS IoT Core with Confluent, see [Leveraging Confluent and AWS to Solve IoT Device and Data Management Challenges](https://aws.amazon.com/blogs/apn/leveraging-confluent-and-aws-to-solve-iot-device-and-data-management-challenges/).
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `ec2:CreateNetworkInterface`, `ec2:DescribeNetworkInterfaces`, `ec2:CreateNetworkInterfacePermission`, `ec2:DeleteNetworkInterface`, `ec2:DescribeSubnets`, `ec2:DescribeVpcs`, `ec2:DescribeVpcAttribute`, and `ec2:DescribeSecurityGroups` operations. This role creates and manages elastic network interfaces to your Amazon Virtual Private Cloud to reach your Kafka broker. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT Core to perform this rule action.
For more information about network interfaces, see [Elastic network interfaces](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) in the *Amazon EC2 User Guide*.
The policy attached to the role that you specify should look like the following example.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateNetworkInterfacePermission",
"ec2:DeleteNetworkInterface",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs",
"ec2:DescribeVpcAttribute",
"ec2:DescribeSecurityGroups"
],
"Resource": "*"
}
]
}
```
+ If you use AWS Secrets Manager to store the credentials required to connect to your Kafka broker, you must create an IAM role that AWS IoT Core can assume to perform the `secretsmanager:GetSecretValue` and `secretsmanager:DescribeSecret` operations.
The policy attached to the role that you specify should look like the following example.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": [
"arn:aws:secretsmanager:us-east-1:123456789012:secret:kafka_client_truststore-*",
"arn:aws:secretsmanager:us-east-1:123456789012:secret:kafka_keytab-*"
]
}
]
}
```
+ You can run your Apache Kafka clusters inside Amazon Virtual Private Cloud (Amazon VPC). You must create an Apache Kafka Virtual Private Cloud (VPC) destination and use an NAT gateway in your subnets to forward messages from AWS IoT to a public Kafka cluster. The AWS IoT rules engine creates a network interface in each of the subnets listed in the destination to route traffic directly to the VPC. When you destination, the AWS IoT rules engine automatically creates a VPC rule action. For more information about VPC rule actions, see [Apache Kafka Virtual Private Cloud (VPC) destinations](kafka-vpc-destination.md).
+ If you use a customer managed AWS KMS key (KMS key) to encrypt data at rest, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Amazon MSK encryption](https://docs.aws.amazon.com/msk/latest/developerguide/msk-encryption.html) in the *Amazon Managed Streaming for Apache Kafka Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
destinationArn
The Amazon Resource Name (ARN) of the Apache Kafka Virtual Private Cloud (VPC) destination. For information about creating a destination, see [Apache Kafka Virtual Private Cloud (VPC) destinations](kafka-vpc-destination.md).
topic
The Kafka topic for messages to be sent to the Kafka broker.
You can substitute this field using a substitution template. For more information, see [Substitution templates](iot-substitution-templates.md).
key (optional)
The Kafka message key.
You can substitute this field using a substitution template. For more information, see [Substitution templates](iot-substitution-templates.md).
headers (optional)
The list of Kafka headers that you specify. Each header is a key-value pair that you can specify when you create a Kafka action. You can use these headers to route data from IoT clients to downstream Kafka clusters without modifying your message payload.
You can substitute this field using a substitution template. To understand how to pass an inline Rule's function as a substitution template in Kafka Action's header, see [Examples](#apache-kafka-rule-action-examples). For more information, see [Substitution templates](iot-substitution-templates.md).
Headers in binary format are not supported.
partition (optional)
The Kafka message partition.
You can substitute this field using a substitution template. For more information, see [Substitution templates](iot-substitution-templates.md).
clientProperties
An object that defines the properties of the Apache Kafka producer client.
acks (optional)
The number of acknowledgments the producer requires the server to have received before considering a request complete.
If you specify 0 as the value, the producer won't wait for any acknowledgment from the server. If the server doesn't receive the message, the producer won't retry to send the message.
Valid values: `-1`, `0`, `1`, `all`. The default value is `1`.
bootstrap.servers
A list of host and port pairs (for example, `host1:port1`, `host2:port2`) used to establish the initial connection to your Kafka cluster.
compression.type (optional)
The compression type for all data generated by the producer.
Valid values: `none`, `gzip`, `snappy`, `lz4`, `zstd`. The default value is `none`.
security.protocol
The security protocol used to attach to your Kafka broker.
Valid values: `SSL`, `SASL_SSL`. The default value is `SSL`.
key.serializer
Specifies how to turn the key objects that you provide with the`ProducerRecord` into bytes.
Valid value: `StringSerializer`.
value.serializer
Specifies how to turn value objects that you provide with the `ProducerRecord` into bytes.
Valid value: `ByteBufferSerializer`.
ssl.truststore
The truststore file in base64 format or the location of the truststore file in [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/). This value isn't required if your truststore is trusted by Amazon certificate authorities (CA).
This field supports substitution templates. If you use Secrets Manager to store the credentials required to connect to your Kafka broker, you can use the `get_secret` SQL function to retrieve the value for this field. For more information about substitution templates, see [Substitution templates](iot-substitution-templates.md). For more information about the `get_secret` SQL function, see [get\$1secret(secretId, secretType, key, roleArn)](iot-sql-functions.md#iot-sql-function-get-secret). If the truststore is in the form of a file, use the `SecretBinary` parameter. If the truststore is in the form of a string, use the `SecretString` parameter.
The maximum size of this value is 65 KB.
ssl.truststore.password
The password for the truststore. This value is required only if you've created a password for the truststore.
ssl.keystore
The keystore file. This value is required when you specify `SSL` as the value for `security.protocol`.
This field supports substitution templates. Use Secrets Manager to store the credentials required to connect to your Kafka broker. To retrieve the value for this field, use the `get_secret` SQL function. For more information about substitution templates, see [Substitution templates](iot-substitution-templates.md). For more information about the `get_secret` SQL function, see [get\$1secret(secretId, secretType, key, roleArn)](iot-sql-functions.md#iot-sql-function-get-secret). Use the `SecretBinary` parameter.
ssl.keystore.password
The store password for the keystore file. This value is required if you specify a value for `ssl.keystore`.
The value of this field can be plaintext . This field also supports substitution templates. Use Secrets Manager to store the credentials required to connect to your Kafka broker. To retrieve the value for this field, use the `get_secret` SQL function. For more information about substitution templates, see [Substitution templates](iot-substitution-templates.md). For more information about the `get_secret` SQL function, see [get\$1secret(secretId, secretType, key, roleArn)](iot-sql-functions.md#iot-sql-function-get-secret). Use the `SecretString` parameter.
ssl.key.password
The password of the private key in your keystore file.
This field supports substitution templates. Use Secrets Manager to store the credentials required to connect to your Kafka broker. To retrieve the value for this field, use the `get_secret` SQL function. For more information about substitution templates, see [Substitution templates](iot-substitution-templates.md). For more information about the `get_secret` SQL function, see [get\$1secret(secretId, secretType, key, roleArn)](iot-sql-functions.md#iot-sql-function-get-secret). Use the `SecretString` parameter.
sasl.mechanism
The security mechanism used to connect to your Kafka broker. This value is required when you specify `SASL_SSL` for `security.protocol`.
Valid values: `PLAIN`, `SCRAM-SHA-512`, `GSSAPI`.
`SCRAM-SHA-512` is the only supported security mechanism in the cn-north-1, cn-northwest-1, us-gov-east-1, and us-gov-west-1 Regions.
sasl.plain.username
The username used to retrieve the secret string from Secrets Manager. This value is required when you specify `SASL_SSL` for `security.protocol` and `PLAIN` for `sasl.mechanism`.
sasl.plain.password
The password used to retrieve the secret string from Secrets Manager. This value is required when you specify `SASL_SSL` for `security.protocol` and `PLAIN` for `sasl.mechanism`.
sasl.scram.username
The username used to retrieve the secret string from Secrets Manager. This value is required when you specify `SASL_SSL` for `security.protocol` and `SCRAM-SHA-512` for `sasl.mechanism`.
sasl.scram.password
The password used to retrieve the secret string from Secrets Manager. This value is required when you specify `SASL_SSL` for `security.protocol` and `SCRAM-SHA-512` for `sasl.mechanism`.
sasl.kerberos.keytab
The keytab file for Kerberos authentication in Secrets Manager. This value is required when you specify `SASL_SSL` for `security.protocol` and `GSSAPI` for `sasl.mechanism`.
This field supports substitution templates. Use Secrets Manager to store the credentials required to connect to your Kafka broker. To retrieve the value for this field, use the `get_secret` SQL function. For more information about substitution templates, see [Substitution templates](iot-substitution-templates.md). For more information about the `get_secret` SQL function, see [get\$1secret(secretId, secretType, key, roleArn)](iot-sql-functions.md#iot-sql-function-get-secret). Use the `SecretBinary` parameter.
sasl.kerberos.service.name
The Kerberos principal name under which Apache Kafka runs. This value is required when you specify `SASL_SSL` for `security.protocol` and `GSSAPI` for `sasl.mechanism`.
sasl.kerberos.krb5.kdc
The hostname of the key distribution center (KDC) to which your Apache Kafka producer client connects. This value is required when you specify `SASL_SSL` for `security.protocol` and `GSSAPI` for `sasl.mechanism`.
sasl.kerberos.krb5.realm
The realm to which your Apache Kafka producer client connects. This value is required when you specify `SASL_SSL` for `security.protocol` and `GSSAPI` for `sasl.mechanism`.
sasl.kerberos.principal
The unique Kerberos identity to which Kerberos can assign tickets to access Kerberos-aware services. This value is required when you specify `SASL_SSL` for `security.protocol` and `GSSAPI` for `sasl.mechanism`.
## Examples
The following JSON example defines an Apache Kafka action in an AWS IoT rule. The following example passes the [ sourceIp()](iot-sql-functions.md#iot-function-sourceip) inline function as a [substitution template](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html) in the Kafka Action header.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"kafka": {
"destinationArn": "arn:aws:iot:region:123456789012:ruledestination/vpc/VPCDestinationARN",
"topic": "TopicName",
"clientProperties": {
"bootstrap.servers": "kafka.com:9092",
"security.protocol": "SASL_SSL",
"ssl.truststore": "${get_secret('kafka_client_truststore', 'SecretBinary','arn:aws:iam::123456789012:role/kafka-get-secret-role-name')}",
"ssl.truststore.password": "kafka password",
"sasl.mechanism": "GSSAPI",
"sasl.kerberos.service.name": "kafka",
"sasl.kerberos.krb5.kdc": "kerberosdns.com",
"sasl.kerberos.keytab": "${get_secret('kafka_keytab','SecretBinary', 'arn:aws:iam::123456789012:role/kafka-get-secret-role-name')}",
"sasl.kerberos.krb5.realm": "KERBEROSREALM",
"sasl.kerberos.principal": "kafka-keytab/kafka-keytab.com"
},
"headers": [
{
"key": "static_header_key",
"value": "static_header_value"
},
{
"key": "substitutable_header_key",
"value": "${value_from_payload}"
},
{
"key": "source_ip",
"value": "${sourceIp()}"
}
]
}
}
]
}
}
```
**Important notes about your Kerberos setup**
+ Your key distribution center (KDC) must be resolvable through private Domain Name System (DNS) within your target VPC. One possible approach is to add the KDC DNS entry to a private hosted zone. For more information about this approach, see [Working with private hosted zones](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/hosted-zones-private.html).
+ Each VPC must have DNS resolution enabled. For more information, see [Using DNS with your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html).
+ Network interface security groups and instance-level security groups in the VPC destination must allow traffic from within your VPC on the following ports.
+ TCP traffic on the bootstrap broker listener port (often 9092, but must be within the 9000–9100 range)
+ TCP and UDP traffic on port 88 for the KDC
+ `SCRAM-SHA-512` is the only supported security mechanism in the cn-north-1, cn-northwest-1, us-gov-east-1, and us-gov-west-1 Regions.
# Apache Kafka Virtual Private Cloud (VPC) destinations
The Apache Kafka rule action routes data to an Apache Kafka cluster in an Amazon Virtual Private Cloud (Amazon VPC). The VPC configuration used by the Apache Kafka rule action is automatically enabled when you specify the VPC destination for your rule action.
An Apache Kafka Virtual Private Cloud (VPC) destination contains a list of subnets inside the VPC. The rules engine creates an elastic network interface in each subnet that you specify in this list. For more information about network interfaces, see [Elastic network interfaces](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) in the Amazon EC2 User Guide.
## Requirements and considerations
+ If you're using a self-managed Apache Kafka cluster that will be accessed using a public endpoint across the internet:
+ Create a NAT gateway for instances in your subnets. The NAT gateway has a public IP address that can connect to the internet, which allows the rules engine to forward your messages to the public Kafka cluster.
+ Allocate an Elastic IP address with the elastic network interfaces (ENIs) that are created by the Apache Kafka Virtual Private Cloud (VPC) destination. The security groups that you use must be configured to block incoming traffic.
**Note**
If the Apache Kafka Virtual Private Cloud (VPC) destination is disabled and then re-enabled, you must re-associate the elastic IPs with the new ENIs.
+ If an Apache Kafka Virtual Private Cloud (VPC) destination doesn't receive any traffic for 30 days in a row, it will be disabled.
+ If any resources used by the Apache Kafka Virtual Private Cloud (VPC) destination change, the destination will be disabled and unable to be used.
+ Some changes that can disable a Apache Kafka Virtual Private Cloud (VPC) destination include:
+ Deleting the VPC, subnets, security groups, or the role used.
+ Modifying the role to no longer have the necessary permissions.
+ Reaching near subnet capacity which makes us unable to apply [ FedRAMP](https://aws.amazon.com/compliance/fedramp/) patching.
+ Disabling the destination.
## Pricing
For pricing purposes, a VPC rule action is metered in addition to the action that sends a message to a resource when the resource is in your VPC. For pricing information, see [AWS IoT Core pricing](https://aws.amazon.com/iot-core/pricing/).
## Creating Apache Kafka Virtual Private Cloud (VPC) destinations
You create a Apache Kafka Virtual Private Cloud (VPC) destination by using the [CreateTopicRuleDestination](https://docs.aws.amazon.com/iot/latest/apireference/API_CreateTopicRuleDestination.html) API or the AWS IoT Core console.
When you create a destination, you must specify the following information.
vpcId
The unique ID of the Amazon VPC.
subnetIds
A list of subnets in which the rules engine creates elastic network interfaces. The rules engine allocates a single network interface for each subnet in the list.
securityGroups (optional)
A list of security groups to apply to the network interfaces.
roleArn
The Amazon Resource Name (ARN) of a role that has permission to create network interfaces on your behalf.
This ARN should have a policy attached to it that looks like the following example.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeVpcs",
"ec2:DeleteNetworkInterface",
"ec2:DescribeSubnets",
"ec2:DescribeVpcAttribute",
"ec2:DescribeSecurityGroups"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:CreateNetworkInterfacePermission",
"Resource": "*",
"Condition": {
"StringEquals": {
"ec2:ResourceTag/VPCDestinationENI": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"ec2:CreateAction": "CreateNetworkInterface",
"aws:RequestTag/VPCDestinationENI": "true"
}
}
}
]
}
```
### Creating an Apache Kafka Virtual Private Cloud (VPC) destination by using AWS CLI
The following example shows how to create a destination by using AWS CLI.
```
aws --region regions iot create-topic-rule-destination --destination-configuration 'vpcConfiguration={subnetIds=["subnet-123456789101230456"],securityGroups=[],vpcId="vpc-123456789101230456",roleArn="arn:aws:iam::123456789012:role/role-name"}'
```
After you run this command, the destination status will be `IN_PROGRESS`. After a few minutes, its status will change to either `ERROR` (if the command isn't successful) or `ENABLED`. When the destination status is `ENABLED`, it's ready to use.
You can use the following command to get the status of your Apache Kafka Virtual Private Cloud (VPC) destination.
```
aws --region region iot get-topic-rule-destination --arn "VPCDestinationARN"
```
### Creating a Apache Kafka Virtual Private Cloud (VPC) destination by using the AWS IoT Core console
The following steps describe how to create a destination by using the AWS IoT Core console.
1. Navigate to the AWS IoT Core console. In the left pane, on the **Act** tab, choose **Destinations**.
1. Enter values for the following fields.
+ **VPC ID**
+ **Subnet IDs**
+ **Security Group**
1. Select a role that has the permissions required to create network interfaces. The preceding example policy contains these permissions.
When the Apache Kafka Virtual Private Cloud (VPC) destination status is **ENABLED**, it's ready to use.
# CloudWatch alarms
The CloudWatch alarm (`cloudWatchAlarm`) action changes the state of an Amazon CloudWatch alarm. You can specify the state change reason and value in this call.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `cloudwatch:SetAlarmState` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`alarmName`
The CloudWatch alarm name.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`stateReason`
Reason for the alarm change.
Supports [substitution templates](iot-substitution-templates.md): Yes
`stateValue`
The value of the alarm state. Valid values: `OK`, `ALARM`, `INSUFFICIENT_DATA`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The IAM role that allows access to the CloudWatch alarm. For more information, see [Requirements](#cloudwatch-alarms-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a CloudWatch alarm action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"cloudwatchAlarm": {
"alarmName": "IotAlarm",
"stateReason": "Temperature stabilized.",
"stateValue": "OK",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_cw"
}
}
]
}
}
```
## See also
+ [What is Amazon CloudWatch?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/) in the *Amazon CloudWatch User Guide*
+ [Using Amazon CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html) in the *Amazon CloudWatch User Guide*
# CloudWatch Logs
The CloudWatch Logs (`cloudwatchLogs`) action sends data to Amazon CloudWatch Logs. You can use `batchMode` to upload and timestamp multiple device log records in one message. You can also specify the log group where the action sends data.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `logs:CreateLogStream`, `logs:DescribeLogStreams`, and `logs:PutLogEvents` operations. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use a customer managed AWS KMS key (KMS key) to encrypt log data in CloudWatch Logs, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Encrypt log data in CloudWatch Logs using AWS KMS](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html) in the *Amazon CloudWatch Logs User Guide*.
## MQTT message format requirements for `batchMode`
If you use the CloudWatch Logs rule action with `batchMode` turned off, there are no MQTT message formatting requirements. (Note: the `batchMode` parameter's default value is `false`.) However, if you use the CloudWatch Logs rule action with `batchMode` turned on (the parameter value is `true`), MQTT messages containing device-side logs must be formatted to contain a timestamp and a message payload. **Note:** `timestamp` represents the time that the event occurred and is expressed as a number of milliseconds after January 1, 1970 00:00:00 UTC.
The following is an example of the publish format:
```
[
{"timestamp": 1673520691093, "message": "Test message 1"},
{"timestamp": 1673520692879, "message": "Test message 2"},
{"timestamp": 1673520693442, "message": "Test message 3"}
]
```
Depending on how the device-side logs are generated, they might need to be filtered and reformatted before they're sent to comply with this requirement. For more information, see [MQTT Message payload](https://docs.aws.amazon.com/iot/latest/developerguide/topicdata.html).
Independent of the `batchMode` parameter, `message` contents must comply with AWS IoT message size limitations. For more information, see [AWS IoT Core endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/iot-core.html).
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`logGroupName`
The CloudWatch log group where the action sends data.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`roleArn`
The IAM role that allows access to the CloudWatch log group. For more information, see [Requirements](#cloudwatch-logs-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
(optional) `batchMode`
Indicates whether batches of log records will be extracted and uploaded into CloudWatch. Values include `true` or `false` (default). For more information, see [Requirements](#cloudwatch-logs-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a CloudWatch Logs action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"cloudwatchLogs": {
"logGroupName": "IotLogs",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_cw",
"batchMode": false
}
}
]
}
}
```
## See also
+ [What is Amazon CloudWatch Logs?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/) in the *Amazon CloudWatch Logs User Guide*
# CloudWatch metrics
The CloudWatch metric (`cloudwatchMetric`) action captures an Amazon CloudWatch metric. You can specify the metric namespace, name, value, unit, and timestamp.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `cloudwatch:PutMetricData` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`metricName`
The CloudWatch metric name.
Supports [substitution templates](iot-substitution-templates.md): Yes
`metricNamespace`
The CloudWatch metric namespace name.
Supports [substitution templates](iot-substitution-templates.md): Yes
`metricUnit`
The metric unit supported by CloudWatch.
Supports [substitution templates](iot-substitution-templates.md): Yes
`metricValue`
A string that contains the CloudWatch metric value.
Supports [substitution templates](iot-substitution-templates.md): Yes
`metricTimestamp`
(Optional) A string that contains the timestamp, expressed in seconds in Unix epoch time. Defaults to the current Unix epoch time.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The IAM role that allows access to the CloudWatch metric. For more information, see [Requirements](#cloudwatch-metrics-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a CloudWatch metric action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"cloudwatchMetric": {
"metricName": "IotMetric",
"metricNamespace": "IotNamespace",
"metricUnit": "Count",
"metricValue": "1",
"metricTimestamp": "1456821314",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_cw"
}
}
]
}
}
```
The following JSON example defines a CloudWatch metric action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"cloudwatchMetric": {
"metricName": "${topic()}",
"metricNamespace": "${namespace}",
"metricUnit": "${unit}",
"metricValue": "${value}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_cw"
}
}
]
}
}
```
## See also
+ [What is Amazon CloudWatch?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/) in the *Amazon CloudWatch User Guide*
+ [Using Amazon CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html) in the *Amazon CloudWatch User Guide*
# DynamoDB
The DynamoDB (`dynamoDB`) action writes all or part of an MQTT message to an Amazon DynamoDB table.
You can follow a tutorial that shows you how to create and test a rule with a DynamoDB action. For more information, see [Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md).
**Note**
This rule writes non-JSON data to DynamoDB as binary data. The DynamoDB console displays the data as base64-encoded text.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `dynamodb:PutItem` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use a customer managed AWS KMS key (KMS key) to encrypt data at rest in DynamoDB, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Customer Managed KMS key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.howitworks.html#managed-cmk-customer-managed) in the *Amazon DynamoDB Getting Started Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`tableName`
The name of the DynamoDB table.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`hashKeyField`
The name of the hash key (also called the partition key).
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`hashKeyType`
(Optional) The data type of the hash key (also called the partition key). Valid values: `STRING`, `NUMBER`.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`hashKeyValue`
The value of the hash key. Consider using a substitution template such as `${topic()}` or `${timestamp()}`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`rangeKeyField`
(Optional) The name of the range key (also called the sort key).
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`rangeKeyType`
(Optional) The data type of the range key (also called the sort key). Valid values: `STRING`, `NUMBER`.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`rangeKeyValue`
(Optional) The value of the range key. Consider using a substitution template such as `${topic()}` or `${timestamp()}`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`payloadField`
(Optional) The name of the column where the payload is written. If you omit this value, the payload is written to the column named `payload`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`operation`
(Optional) The type of operation to be performed. Valid values: `INSERT`, `UPDATE`, `DELETE`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleARN`
The IAM role that allows access to the DynamoDB table. For more information, see [Requirements](#dynamodb-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
The data written to the DynamoDB table is the result from the SQL statement of the rule.
## Examples
The following JSON example defines a DynamoDB action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * AS message FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"dynamoDB": {
"tableName": "my_ddb_table",
"hashKeyField": "key",
"hashKeyValue": "${topic()}",
"rangeKeyField": "timestamp",
"rangeKeyValue": "${timestamp()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_dynamoDB"
}
}
]
}
}
```
## See also
+ [What is Amazon DynamoDB?](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/) in the *Amazon DynamoDB Developer Guide*
+ [Getting started with DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStartedDynamoDB.html) in the *Amazon DynamoDB Developer Guide*
+ [Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md)
# DynamoDBv2
The DynamoDBv2 (`dynamoDBv2`) action writes all or part of an MQTT message to an Amazon DynamoDB table. Each attribute in the payload is written to a separate column in the DynamoDB database.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `dynamodb:PutItem` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ The MQTT message payload must contain a root-level key that matches the table's primary partition key and a root-level key that matches the table's primary sort key, if one is defined.
+ If you use a customer managed AWS KMS key (KMS key) to encrypt data at rest in DynamoDB, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Customer Managed KMS key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.howitworks.html#managed-cmk-customer-managed) in the *Amazon DynamoDB Getting Started Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`putItem`
An object that specifies the DynamoDB table to which the message data will be written. This object must contain the following information:
`tableName`
The name of the DynamoDB table.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`roleARN`
The IAM role that allows access to the DynamoDB table. For more information, see [Requirements](#dynamodb-v2-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
The data written to the DynamoDB table is the result from the SQL statement of the rule.
## Examples
The following JSON example defines a DynamoDBv2 action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * AS message FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"dynamoDBv2": {
"putItem": {
"tableName": "my_ddb_table"
},
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_dynamoDBv2",
}
}
]
}
}
```
The following JSON example defines a DynamoDB action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2015-10-08",
"actions": [
{
"dynamoDBv2": {
"putItem": {
"tableName": "${topic()}"
},
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_dynamoDBv2"
}
}
]
}
}
```
## See also
+ [What is Amazon DynamoDB?](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/) in the *Amazon DynamoDB Developer Guide*
+ [Getting started with DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStartedDynamoDB.html) in the *Amazon DynamoDB Developer Guide*
# Elasticsearch
The Elasticsearch (`elasticsearch`) action writes data from MQTT messages to an Amazon OpenSearch Service domain. You can then use tools like OpenSearch Dashboards to query and visualize data in OpenSearch Service.
**Warning**
The `Elasticsearch` action can only be used by existing rule actions. To create a new rule action or to update an existing rule action, use the `OpenSearch` rule action instead. For more information, see [OpenSearch](opensearch-rule-action.md).
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `es:ESHttpPut` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use a customer managed AWS KMS key (KMS key) to encrypt data at rest in OpenSearch, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Encryption of data at rest for Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/encryption-at-rest.html) in the *Amazon OpenSearch Service Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`endpoint`
The endpoint of your service domain.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`index`
The index where you want to store your data.
Supports [substitution templates](iot-substitution-templates.md): Yes
`type`
The type of document you are storing.
Supports [substitution templates](iot-substitution-templates.md): Yes
`id`
The unique identifier for each document.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleARN`
The IAM role that allows access to the OpenSearch Service domain. For more information, see [Requirements](#elasticsearch-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines an Elasticsearch action in an AWS IoT rule and how you can specify the fields for the `elasticsearch` action. For more information, see [ElasticsearchAction](https://docs.aws.amazon.com/iot/latest/apireference/API_ElasticsearchAction.html).
```
{
"topicRulePayload": {
"sql": "SELECT *, timestamp() as timestamp FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"elasticsearch": {
"endpoint": "https://my-endpoint",
"index": "my-index",
"type": "my-type",
"id": "${newuuid()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_es"
}
}
]
}
}
```
The following JSON example defines an Elasticsearch action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"elasticsearch": {
"endpoint": "https://my-endpoint",
"index": "${topic()}",
"type": "${type}",
"id": "${newuuid()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_es"
}
}
]
}
}
```
## See also
+ [OpenSearch](opensearch-rule-action.md)
+ [What is Amazon OpenSearch Service?](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/)
# HTTP
The HTTPS (`http`) action sends data from an MQTT message to an HTTPS endpoint, which can point to a web application or service.
## Requirements
This rule action has the following requirements:
+ You must confirm and enable HTTPS endpoints before the rules engine can use them. For more information, see [HTTP action destinations](http-action-destination.md).
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`url`
The HTTPS endpoint where the message is sent using the HTTP POST method. If you use an IP address in place of a hostname, it must be an IPv4 address. IPv6 addresses are not supported.
Supports [substitution templates](iot-substitution-templates.md): Yes
`confirmationUrl`
(Optional) If specified, AWS IoT uses the confirmation URL to create a matching topic rule destination. You must enable the HTTP action destination before using it in an HTTP action. For more information, see [HTTP action destinations](http-action-destination.md). If you use substitution templates, you must manually create an HTTP action destination before the `http` action can be used. `confirmationUrl` must be a prefix of `url`.
The relationship between `url` and `confirmationUrl` is described by the following:
+ If `url` is hardcoded and `confirmationUrl` is not provided, we implicitly treat the `url` field as the `confirmationUrl`. AWS IoT creates a topic rule destination for `url`.
+ If `url` and `confirmationUrl` are hardcoded, `url` must begin with `confirmationUrl`. AWS IoT creates a topic rule destination for `confirmationUrl`.
+ If `url` contains a substitution template, you must specify `confirmationUrl` and `url` must begin with `confirmationUrl`. If `confirmationUrl` contains substitution templates, you must manually create an HTTP action destination before the `http` action can be used. If `confirmationUrl` does not contain substitution templates, AWS IoT creates a topic rule destination for `confirmationUrl`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`headers`
(Optional) The list of headers to include in HTTP requests to the endpoint. Each header must contain the following information:
`key`
The key of the header.
Supports [substitution templates](iot-substitution-templates.md): No
`value`
The value of the header.
Supports [substitution templates](iot-substitution-templates.md): Yes
The default content type is application/json when the payload is in JSON format. Otherwise, it is application/octet-stream. You can overwrite it by specifying the exact content type in the header with the key content-type (case insensitive).
`auth`
(Optional) The authentication used by the rules engine to connect to the endpoint URL specified in the `url` argument. Currently, Signature Version 4 is the only supported authentication type. For more information, see [HTTP Authorization](https://docs.aws.amazon.com/iot/latest/apireference/API_HttpAuthorization.html).
Supports [substitution templates](iot-substitution-templates.md): No
`enableBatching`
(Optional) Whether to process the HTTP action messages into a single request for a given url. Value can be true or false. For more information on batching, see [Batching HTTP action messages](http_batching.md).
Boolean value
Supports [substitution templates](iot-substitution-templates.md): No
`batchConfig`
(Optional) Configuration settings for batching. Once enabled, `batchConfig` parameters must be specified. If `batchConfig` parameters are not specified, the default values will be used.
`maxBatchOpenMs`
The maximum amount of time (in milliseconds) an outgoing message waits for other messages to create the batch. The higher the setting, the longer the latency of the batched HTTP action.
Minimum Value: 5 ms. Maximum Value: 200 ms.
Default Value: 20 ms
Supports [substitution templates](iot-substitution-templates.md): No
`maxBatchSize`
The maximum number of messages that are batched together in a single action execution.
Supports [substitution templates](iot-substitution-templates.md): No
Minimum Value: 2 messages. Maximum Value: 10 messages
Default Value: 10 messages
`maxBatchSizeBytes`
Maximum size of a message batch, in bytes.
Minimum Value: 100 bytes. Maximum Value: 131,072 bytes
Default Value: 5,120 bytes
Supports [substitution templates](iot-substitution-templates.md): No
The default content type is application/json when the payload is in JSON format. Otherwise, it is application/octet-stream. You can overwrite it by specifying the exact content type in the header with the key content-type (case insensitive).
## Examples
The following JSON example defines an AWS IoT rule with an HTTP action.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"http": {
"url": "https://www.example.com/subpath",
"confirmationUrl": "https://www.example.com",
"headers": [
{
"key": "static_header_key",
"value": "static_header_value"
},
{
"key": "substitutable_header_key",
"value": "${value_from_payload}"
}
]
}
}
]
}
}
```
```
"http": {
"url": "https://www.example.com/subpath",
"confirmationUrl": "https://www.example.com",
"headers": [
{
"key": "Content-Type",
"value": "application/json"
}
],
"enableBatching": true,
"batchConfig": {
"maxBatchOpenMs": 123,
"maxBatchSize": 5,
"maxBatchSizeBytes": 131072,
}
},
"errorAction": {
"http": {
"url": "https://www.example.com/subpath",
"confirmationUrl": "https://www.example.com"
// batchConfig is not allowed here
}
}
```
## HTTP action retry logic
The AWS IoT rules engine retries the HTTP action according to these rules:
+ The rules engine tries to send a message at least once.
+ The rules engine retries at most twice. The maximum number of tries is three.
+ The rules engine does not attempt a retry if:
+ The previous try provided a response larger than 16,384 bytes.
+ The downstream web service or application closes the TCP connection after the try.
+ The total time to complete a request with retries exceeded the request timeout limit.
+ The request returns an HTTP status code other than 429, 500-599.
**Note**
[Standard data transfer costs](https://aws.amazon.com/ec2/pricing/on-demand/) apply to retries.
## See also
+ [Batching HTTP action messages](http_batching.md)
+ [HTTP action destinations](http-action-destination.md)
+ [Route data directly from AWS IoT Core to your web services](https://aws.amazon.com/blogs/iot/route-data-directly-from-iot-core-to-your-web-services/) in the *Internet of Things on AWS* blog
# Batching HTTP action messages
You can use batching to send multiple HTTP action messages in a single request.
## Overview
Batching enables you to send messages from AWS IoT Core Rules Engine to your HTTP endpoints in batches. This functionality can help reduce your costs by lowering the number of HTTP action executions as well as improve efficiency by reducing the overhead associated with establishing new connections.
**Note**
The batched HTTP action is metered as a single action. You are metered in increments of 5 kiB, based on the size of outbound batched payload emitted by the AWS IoT Core Rules Engine to the downstream service. For more information, see the [AWS IoT Core pricing page](https://aws.amazon.com/iot-core/pricing/).
When you enable batching in the definition of your IoT Rule Action, the following parameters will be available for configuration:
`maxBatchOpenMs`
The maximum amount of time (in milliseconds) an outgoing message waits for other messages to create the batch. The higher the setting, the longer the latency of the batched HTTP action.
Minimum Value: 5 ms. Maximum Value: 200 ms.
Default Value: 20 ms
Supports [substitution templates](iot-substitution-templates.md): No
`maxBatchSize`
The maximum number of messages that are batched together in a single IoT rule action execution.
Minimum Value: 2 messages. Maximum Value: 10 messages
Default Value: 10 messages
Supports [substitution templates](iot-substitution-templates.md): No
`maxBatchSizeBytes`
Maximum size of a message batch, in bytes.
Minimum Value: 100 bytes. Maximum Value: 131,072 bytes
Default Value: 5120 bytes
Supports [substitution templates](iot-substitution-templates.md): No
**Important**
When you specify multiple batch parameters, batching completes when the first limit is reached. For example, if you specify 100 ms as the Maximum Batch Open Time and 5 kiB as the Maximum Batch Size, and Rules Engine batches only 2 kiB within 100 ms, then a 2 kiB batch will be created and sent.
## Using HTTP headers in a batch
When you use headers in your HTTP action, the batched request uses the header value from the last message that was added to the batch (not necessarily the last message you published). We recommend using header values that are either:
+ Identical across all messages in the batch
+ Applicable to all messages (for example, authentication credentials)
The headers are sent with the HTTP request and are not part of the message body.
**Note**
When batching is enabled:
The batched request automatically includes the `Content-Type: application/json` header, as the batch is sent as a JSON array.
We can't guarantee that the last message in the batch is the last message that you published. It is the last message that made it into the batch.
## Payload Example
The following example shows the structure of a batched message payload sent to your HTTP endpoint:
```
[
{
"user_id": "user1",
"steps_today": 1000
},
{
"user_id": "user2",
"steps_today": 21000
},
{
"user_id": "user8",
"steps_today": 1500
},
...
]
```
## Limitations
The following are limitations on batching:
+ AWS IoT Core does not guarantee overall message ordering. Batching is performed locally on each host, which may result in messages within a batch being processed in a different order than they were received.
+ AWS IoT Core does not provide message processing support on the receiver side. You are responsible for ensuring that your downstream service is configured to accept and process data in batches.
+ Cross-account batching is not supported, even if messages are destined for the same resource identifier (HTTP URL or resource ARN).
+ AWS IoT Core does not guarantee that the batch size will meet the configuration you specified. Batches may be smaller than your configured limits based on timing and message flow.
+ When batching is enabled, binary payloads (non-UTF-8 data) are not supported. Only UTF-8 text payloads (such as JSON) are accepted. To send binary data, base64 encode it before sending it to the HTTP action, and then decode it at your receiving endpoint. For example, you can use the [encode function](iot-sql-functions.html#iot-function-encode) in IoT rules to encode the binary payload. Alternatively, you can encode the binary payload in your IoT device and publish it to AWS IoT Core.
## Error Actions for Batching
You will not be able to define a separate batching logic in your Error Action definition. However, your Error Action will support batching if you have defined batching logic in your primary Action.
When a batch request fails, AWS IoT Core Rules engine will follow the [HTTP action retry logic](https-rule-action.md#https-rule-action-retry-logic). After the final retry attempt, an error action will be invoked for the entire failed batch.
The following is an example of an error action message with batching enabled:
```
{
"ruleName": "FailedTopicRule",
"topic": "topic/rulesengine",
"payloadsWithMetadata": [
{
"id": 1,
"cloudwatchTraceId": "bebd6d93-6d4a-899e-9e40-56e82252d2be",
"clientId": "Test",
"sourceIp": "10.0.0.0",
"base64OriginalPayload": "eyJ1c2VyX2lkIjogInVzZXI1NjQ3IiwgInN0ZXBzX3RvZGF5IjogMTMzNjUsICJ0aW1lc3RhbXAiOiAiMjAyNS0xMC0wOVQwNzoyMjo1OC45ODQ3OTAxNzZaIn0="
},
{
"id": 2,
"cloudwatchTraceId": "af94d3b8-0b18-1dbf-2c7d-513f5cb9e2e1",
"clientId": "Test",
"sourceIp": "10.0.0.0",
"base64OriginalPayload": "eyJ1c2VyX2lkIjogInVzZXI1NjQ3IiwgInN0ZXBzX3RvZGF5IjogMTMzNjUsICJ0aW1lc3RhbXAiOiAiMjAyNS0xMC0wOVQwNzoyMjo1OC45ODQ3OTAxNzZaIn0="
},
{
"id": 3,
"cloudwatchTraceId": "ca441266-c2ce-c916-6aee-b9e5c7831675",
"clientId": "Test",
"sourceIp": "10.0.0.0",
"base64OriginalPayload": "eyJ1c2VyX2lkIjogInVzZXI1NjQ3IiwgInN0ZXBzX3RvZGF5IjogMTMzNjUsICJ0aW1lc3RhbXAiOiAiMjAyNS0xMC0wOVQwNzoyMjo1OC45ODQ3OTAxNzZaIn0="
}
],
"failures": [
{
"affectedIds": [
1,
2,
3
],
"failedAction": "HttpAction",
"failedResource": "https://example.foobar.com/HttpAction",
"errorMessage": "HttpAction failed to make a request to the specified endpoint. StatusCode: 500. Reason: Internal Server Error."
},
{
"affectedIds": [
3
],
"failedAction": "S3Action",
"failedResource": "amzn-s3-demo-bucket",
"errorMessage": "Failed to put S3 object. The error received was The specified bucket does not exist"
},
{
"affectedIds": [
3
],
"failedAction": "LambdaAction",
"failedResource": "arn:aws:lambda:us-west-2:123456789012:function:dummy",
"errorMessage": "Failed to invoke lambda function. Received Server error from Lambda. The error code is 403"
}
]
}
```
**Note**
Batched action failures also generate larger error action payloads which can increase the probability of error action failures due to size. You can monitor error action failures using the `ErrorActionFailure` metric. See [Rule action metrics](metrics_dimensions.md#rule-action-metrics) for more information.
## Batching HTTP action messages with the AWS CLI
### Creating or updating a rule action with batching
1. Use the appropriate AWS CLI command to create or update a rule:
+ To create a new rule, use the [create-topic-rule](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html) command:
```
aws iot create-topic-rule --rule-name myrule --topic-rule-payload file://myrule.json
```
+ To update an existing rule, use the [replace-topic-rule](https://docs.aws.amazon.com/cli/latest/reference/iot/replace-topic-rule.html) command:
```
aws iot replace-topic-rule --rule-name myrule --topic-rule-payload file://myrule.json
```
1. Enable batching capabilities by setting the enableBatching parameter to true in your topic rule payload:
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"http": {
"url": "https://www.example.com/subpath",
"confirmationUrl": "https://www.example.com",
"headers": [
{
"key": "static_header_key",
"value": "static_header_value"
},
{
"key": "substitutable_header_key",
"value": "${value_from_payload}"
}
],
"enableBatching": true,
"batchConfig": {
"maxBatchOpenMs": 100,
"maxBatchSize": 5,
"maxBatchSizeBytes": 1024
}
}
}
]
}
```
1. Configure the batching parameters. You do not need to specify all batch parameters. You can choose to specify 1, 2, or all 3 batch parameters. If you do not specify a batch parameter, Rules Engine will update that parameter with the default values. For more information on batching parameters and their default values, see [HTTP parameters](https-rule-action.md#https-rule-action-parameters).
# HTTP action destinations
An HTTP action destination is a web service to which the rules engine can route data from a topic rule. An AWS IoT Core resource describes the web service for AWS IoT. Destination resources can be shared by different rules.
Before AWS IoT Core can send data to another web service, it must confirm that it can access the service's endpoint.
## Overview
An HTTP action destination refers to a web service that supports a confirmation URL and one or more data collection URLs. The destination resource contains the confirmation URL of your web service. When you configure an HTTP action, you specify the actual URL of the endpoint that should receive the data along with the web service's confirmation URL. After your destination is confirmed, the topic rule sends the result of the SQL statement to the HTTPS endpoint (and not to the confirmation URL).
An HTTP action destination can be in one of the following states:
ENABLED
The destination has been confirmed and can be used by a rule action. A destination must be in the `ENABLED` state for it to be used in a rule. You can only enable a destination that's in DISABLED status.
DISABLED
The destination has been confirmed but it can't be used by a rule action. This is useful if you want to temporarily prevent traffic to your endpoint without having to go through the confirmation process again. You can only disable a destination that's in ENABLED status.
IN\$1PROGRESS
Confirmation of the destination is in progress.
ERROR
Destination confirmation timed out.
After an HTTP action destination has been confirmed and enabled, it can be used with any rule in your account.
## Managing HTTP action destinations
You can use the following operations to manage your HTTP action destinations.
### Creating HTTP action destinations
You create an HTTP action destination by calling the `CreateTopicRuleDestination` operation or by using the AWS IoT console.
After you create a destination, AWS IoT sends a confirmation request to the confirmation URL. The confirmation request has the following format:
```
HTTP POST {confirmationUrl}/?confirmationToken={confirmationToken}
Headers:
x-amz-rules-engine-message-type: DestinationConfirmation
x-amz-rules-engine-destination-arn:"arn:aws:iot:us-east-1:123456789012:ruledestination/http/7a280e37-b9c6-47a2-a751-0703693f46e4"
Content-Type: application/json
Body:
{
"arn":"arn:aws:iot:us-east-1:123456789012:ruledestination/http/7a280e37-b9c6-47a2-a751-0703693f46e4",
"confirmationToken": "AYADeMXLrPrNY2wqJAKsFNn-…NBJndA",
"enableUrl": "https://iot.us-east-1.amazonaws.com/confirmdestination/AYADeMXLrPrNY2wqJAKsFNn-…NBJndA",
"messageType": "DestinationConfirmation"
}
```
The content of the confirmation request includes the following information:
arn
The Amazon Resource Name (ARN) for the HTTP action destination to confirm.
confirmationToken
The confirmation token sent by AWS IoT Core. The token in the example is truncated. Your token will be longer. You'll need this token to confirm your destination with AWS IoT Core.
enableUrl
The URL to which you browse to confirm a topic rule destination.
messageType
The type of message.
### Confirming HTTP action destinations
To complete the endpoint confirmation process, if you're using the AWS CLI, you must perform the following steps after your confirmation URL receives the confirmation request.
1.
**Confirm that the destination is ready to receive messages**
To confirm that the HTTP action destination is ready to receive IoT messages, either call the `enableUrl` in the confirmation request, or perform the `ConfirmTopicRuleDestination` API operation and pass the `confirmationToken` from the confirmation request.
1.
**Set topic rule status to enabled**
After you've confirmed that the destination can receive messages, you must perform the `UpdateTopicRuleDestination` API operation to set the status of the topic rule to `ENABLED`.
If you're using the AWS IoT console, copy the `confirmationToken` and paste it into the destination's confirmation dialog in the AWS IoT console. You can then enable the topic rule.
### Sending a new confirmation request
To activate a new confirmation message for a destination, call `UpdateTopicRuleDestination` and set the topic rule destination's status to `IN_PROGRESS`.
Repeat the confirmation process after you send a new confirmation request.
### Disabling and deleting an HTTP action destination
To disable a destination, call `UpdateTopicRuleDestination` and set the topic rule destination's status to `DISABLED`. A topic rule in the DISABLED state can be enabled again without the need to send a new confirmation request.
To delete an HTTP action destination, call `DeleteTopicRuleDestination`.
## Certificate Authority Support
**Note**
Self-signed certificates are not supported.
HTTPS Endpoints in an HTTP action destination support certificates issued by both [AWS Private Certificate Authority ](https://www.amazontrust.com/repository/) and [Lets Encrypt](https://letsencrypt.org/certificates/).
# AWS IoT Events
The AWS IoT Events (`iotEvents`) action sends data from an MQTT message to an AWS IoT Events input.
**Important**
If the payload is sent to AWS IoT Core without the `Input attribute Key`, or if the key isn't in the same JSON path specified in the key, it will cause the IoT rule to fail with the error `Failed to send message to Iot Events`.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `iotevents:BatchPutMessage` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`batchMode`
(Optional) Whether to process the event actions as a batch. The default value is `false`.
When `batchMode` is `true` and the rule SQL statement evaluates to an Array, each Array element is treated as a separate message when it's sent to AWS IoT Events by calling [https://docs.aws.amazon.com/iotevents/latest/apireference/API_iotevents-data_BatchPutMessage.html](https://docs.aws.amazon.com/iotevents/latest/apireference/API_iotevents-data_BatchPutMessage.html). The resulting array can't have more than 10 messages.
When `batchMode` is `true`, you can't specify a `messageId`.
Supports [substitution templates](iot-substitution-templates.md): No
`inputName`
The name of the AWS IoT Events input.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`messageId`
(Optional) Use this to verify that only one input (message) with a given `messageId` is processed by an AWS IoT Events detector. You can use the `${newuuid()}` substitution template to generate a unique ID for each request.
When `batchMode` is `true`, you can't specify a `messageId`--a new UUID value will be assigned.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The IAM role that allows AWS IoT to send an input to an AWS IoT Events detector. For more information, see [Requirements](#iotevents-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines an IoT Events action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"iotEvents": {
"inputName": "MyIoTEventsInput",
"messageId": "${newuuid()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_events"
}
}
]
}
}
```
## See also
+ [What is AWS IoT Events?](https://docs.aws.amazon.com/iotevents/latest/developerguide/) in the *AWS IoT Events Developer Guide*
# AWS IoT SiteWise
The AWS IoT SiteWise (`iotSiteWise`) action sends data from an MQTT message to asset properties in AWS IoT SiteWise.
You can follow a tutorial that shows you how to ingest data from AWS IoT things. For more information, see the [Ingesting data to AWS IoT SiteWise from AWS IoT things](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/ingest-data-from-iot-things.html) tutorial or the [Ingesting data using AWS IoT Core rules](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/iot-rules.html) section in the *AWS IoT SiteWise User Guide*.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `iotsitewise:BatchPutAssetPropertyValue` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
You can attach the following example trust policy to the role.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iotsitewise:BatchPutAssetPropertyValue",
"Resource": "*"
}
]
}
```
To improve security, you can specify an AWS IoT SiteWise asset hierarchy path in the `Condition` property. The following example is a trust policy that specifies an asset hierarchy path.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iotsitewise:BatchPutAssetPropertyValue",
"Resource": "*",
"Condition": {
"StringLike": {
"iotsitewise:assetHierarchyPath": [
"/root node asset ID",
"/root node asset ID/*"
]
}
}
}
]
}
```
+ When you send data to AWS IoT SiteWise with this action, your data must meet the requirements of the `BatchPutAssetPropertyValue` operation. For more information, see [BatchPutAssetPropertyValue](https://docs.aws.amazon.com/iot-sitewise/latest/APIReference/API_BatchPutAssetPropertyValue.html) in the *AWS IoT SiteWise API Reference*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`putAssetPropertyValueEntries`
A list of asset property value entries that each contain the following information:
`propertyAlias`
(Optional) The property alias associated with your asset property. Specify either a `propertyAlias` or both an `assetId` and a `propertyId`. For more information about property aliases, see [Mapping industrial data streams to asset properties](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/connect-data-streams.html) in the *AWS IoT SiteWise User Guide*.
Supports [substitution templates](iot-substitution-templates.md): Yes
`assetId`
(Optional) The ID of the AWS IoT SiteWise asset. Specify either a `propertyAlias` or both an `assetId` and a `propertyId`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`propertyId`
(Optional) The ID of the asset's property. Specify either a `propertyAlias` or both an `assetId` and a `propertyId`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`entryId`
(Optional) A unique identifier for this entry. Define the `entryId` to better track which message caused an error if failure occurs. Defaults to a new UUID.
Supports [substitution templates](iot-substitution-templates.md): Yes
`propertyValues`
A list of property values to insert that each contain timestamp, quality, and value (TQV) in the following format:
`timestamp`
A timestamp structure that contains the following information:
`timeInSeconds`
A string that contains the time in seconds in Unix epoch time. If your message payload doesn't have a timestamp, you can use [timestamp()](iot-sql-functions.md#iot-function-timestamp), which returns the current time in milliseconds. To convert that time to seconds, you can use the following substitution template: **\$1\$1floor(timestamp() / 1E3)\$1**.
Supports [substitution templates](iot-substitution-templates.md): Yes
`offsetInNanos`
(Optional) A string that contains the nanosecond time offset from the time in seconds. If your message payload doesn't have a timestamp, you can use [timestamp()](iot-sql-functions.md#iot-function-timestamp), which returns the current time in milliseconds. To calculate the nanosecond offset from that time, you can use the following substitution template: **\$1\$1(timestamp() % 1E3) \$1 1E6\$1**.
Supports [substitution templates](iot-substitution-templates.md): Yes
Regarding Unix epoch time, AWS IoT SiteWise accepts only entries that have a timestamp of up to 7 days in the past up to 5 minutes in the future.
`quality`
(Optional) A string that describes the quality of the value. Valid values: `GOOD`, `BAD`, `UNCERTAIN`.
Supports [substitution templates](iot-substitution-templates.md): Yes
`value`
A value structure that contains one of the following value fields, depending on the asset property's data type:
`booleanValue`
(Optional) A string that contains the Boolean value of the value entry.
Supports [substitution templates](iot-substitution-templates.md): Yes
`doubleValue`
(Optional) A string that contains the double value of the value entry.
Supports [substitution templates](iot-substitution-templates.md): Yes
`integerValue`
(Optional) A string that contains the integer value of the value entry.
Supports [substitution templates](iot-substitution-templates.md): Yes
`stringValue`
(Optional) The string value of the value entry.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The ARN of the IAM role that grants AWS IoT permission to send an asset property value to AWS IoT SiteWise. For more information, see [Requirements](#iotsitewise-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a basic IoT SiteWise action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"iotSiteWise": {
"putAssetPropertyValueEntries": [
{
"propertyAlias": "/some/property/alias",
"propertyValues": [
{
"timestamp": {
"timeInSeconds": "${my.payload.timeInSeconds}"
},
"value": {
"integerValue": "${my.payload.value}"
}
}
]
}
],
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sitewise"
}
}
]
}
}
```
The following JSON example defines an IoT SiteWise action in an AWS IoT rule. This example uses the topic as the property alias and the `timestamp()` function. For example, if you publish data to `/company/windfarm/3/turbine/7/rpm`, this action sends the data to the asset property with a property alias that's the same as the topic that you specified.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM '/company/windfarm/+/turbine/+/+'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"iotSiteWise": {
"putAssetPropertyValueEntries": [
{
"propertyAlias": "${topic()}",
"propertyValues": [
{
"timestamp": {
"timeInSeconds": "${floor(timestamp() / 1E3)}",
"offsetInNanos": "${(timestamp() % 1E3) * 1E6}"
},
"value": {
"doubleValue": "${my.payload.value}"
}
}
]
}
],
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sitewise"
}
}
]
}
}
```
## See also
+ [What is AWS IoT SiteWise?](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/what-is-sitewise.html) in the *AWS IoT SiteWise User Guide*
+ [Ingesting data using AWS IoT Core rules](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/iot-rules.html) in the *AWS IoT SiteWise User Guide*
+ [Ingesting data to AWS IoT SiteWise from AWS IoT things](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/ingest-data-from-iot-things.html) in the *AWS IoT SiteWise User Guide*
+ [Troubleshooting an AWS IoT SiteWise rule action](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/troubleshoot-rule.html) in the *AWS IoT SiteWise User Guide*
# Firehose
The Firehose(`firehose`) action sends data from an MQTT message to an Amazon Data Firehose stream.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `firehose:PutRecord` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use Firehose to send data to an Amazon S3 bucket, and you use an AWS KMS customer managed AWS KMS key to encrypt data at rest in Amazon S3, Firehose must have access to your bucket and permission to use the AWS KMS key on the caller's behalf. For more information, see [Grant Firehose access to an Amazon S3 destination](https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html#using-iam-s3) in the *Amazon Data Firehose Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`batchMode`
(Optional) Whether to deliver the Firehose stream as a batch by using [https://docs.aws.amazon.com/firehose/latest/APIReference/API_PutRecordBatch.html](https://docs.aws.amazon.com/firehose/latest/APIReference/API_PutRecordBatch.html) . The default value is `false`.
When `batchMode` is `true` and the rule's SQL statement evaluates to an Array, each Array element forms one record in the `PutRecordBatch` request. The resulting array can't have more than 500 records.
Supports [substitution templates](iot-substitution-templates.md): No
`deliveryStreamName`
The Firehose stream to which to write the message data.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`separator`
(Optional) A character separator that is used to separate records written to the Firehose stream. If you omit this parameter, the stream uses no separator. Valid values: `,` (comma), `\t` (tab), `\n` (newline), `\r\n` (Windows newline).
Supports [substitution templates](iot-substitution-templates.md): No
`roleArn`
The IAM role that allows access to the Firehose stream. For more information, see [Requirements](#kinesis-firehose-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a Firehose action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"firehose": {
"deliveryStreamName": "my_firehose_stream",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_firehose"
}
}
]
}
}
```
The following JSON example defines a Firehose action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"firehose": {
"deliveryStreamName": "${topic()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_firehose"
}
}
]
}
}
```
## See also
+ [What is Amazon Data Firehose?](https://docs.aws.amazon.com/firehose/latest/dev/) in the *Amazon Data Firehose Developer Guide*
# Kinesis Data Streams
The Kinesis Data Streams (`kinesis`) action writes data from an MQTT message to Amazon Kinesis Data Streams.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `kinesis:PutRecord` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use an AWS KMS customer-managed AWS KMS key (KMS key) to encrypt data at rest in Kinesis Data Streams, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [Permissions to use user-generated AWS KMS keys](https://docs.aws.amazon.com/streams/latest/dev/permissions-user-key-KMS.html) in the *Amazon Kinesis Data Streams Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`stream`
The Kinesis data stream to which to write data.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`partitionKey`
The partition key used to determine to which shard the data is written. The partition key is usually composed of an expression (for example, `${topic()}` or `${timestamp()}`).
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The ARN of the IAM role that grants AWS IoT permission to access the Kinesis data stream. For more information, see [Requirements](#kinesis-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a Kinesis Data Streams action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"kinesis": {
"streamName": "my_kinesis_stream",
"partitionKey": "${topic()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_kinesis"
}
}
]
}
}
```
The following JSON example defines a Kinesis action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"kinesis": {
"streamName": "${topic()}",
"partitionKey": "${timestamp()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_kinesis"
}
}
]
}
}
```
## See also
+ [What is Amazon Kinesis Data Streams?](https://docs.aws.amazon.com/streams/latest/dev/) in the *Amazon Kinesis Data Streams Developer Guide*
# Lambda
A Lambda (`lambda`) action invokes an AWS Lambda function, passing in an MQTT message. AWS IoT invokes Lambda functions asynchronously.
You can follow a tutorial that shows you how to create and test a rule with a Lambda action. For more information, see [Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md).
## Requirements
This rule action has the following requirements:
+ For AWS IoT to invoke a Lambda function, you must configure a policy that grants the `lambda:InvokeFunction` permission to AWS IoT. You can only invoke a Lambda function defined in the same AWS Region where your Lambda policy exists. Lambda functions use resource-based policies, so you must attach the policy to the Lambda function itself.
Use the following AWS CLI command to attach a policy that grants the `lambda:InvokeFunction` permission. In this command, replace:
+ *function\$1name* with the name of the Lambda function. You add a new permission to update the function's resource policy.
+ *region* with the AWS Region of the function.
+ *account-id* with the AWS account number where the rule is defined.
+ *rule-name* with the name of the AWS IoT rule for which you are defining the Lambda action.
+ *unique\$1id* with a unique statement identifier.
**Important**
If you add a permission for an AWS IoT principal without providing the `source-arn` or `source-account`, any AWS account that creates a rule with your Lambda action can activate rules to invoke your Lambda function from AWS IoT.
For more information, see [AWS Lambda permissions](https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html).
```
aws lambda add-permission \
--function-name function_name \
--region region \
--principal iot.amazonaws.com \
--source-arn arn:aws:iot:region:account-id:rule/rule_name \
--source-account account-id
--statement-id unique_id
--action "lambda:InvokeFunction"
```
+ If you use the AWS IoT console to create a rule for the Lambda rule action, the Lambda function is triggered automatically. If you use AWS CloudFormation instead with the [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iot-topicrule-lambdaaction.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iot-topicrule-lambdaaction.html), you must add an [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-permission.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-permission.html) resource. The resource then grants you permission to trigger the Lambda function.
The following code shows an example of how to add this resource. In this example, replace:
+ *function\$1name* with the name of the Lambda function.
+ *region* with the AWS Region of the function.
+ *account-id* with the AWS account number where the rule is defined.
+ *rule-name* with the name of the AWS IoT rule for which you are defining the Lambda action.
```
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref function_name
Principal: "iot.amazonaws.com"
SourceAccount: account-id
SourceArn: arn:aws:iot:region:account-id:rule/rule_name
```
+ If you use an AWS KMS customer managed AWS KMS key to encrypt data at rest in Lambda, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [Encryption at rest](https://docs.aws.amazon.com/lambda/latest/dg/security-dataprotection.html#security-privacy-atrest) in the *AWS Lambda Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`functionArn`
The ARN of the Lambda function to invoke. AWS IoT must have permission to invoke the function. For more information, see [Requirements](#lambda-rule-action-requirements).
If you don't specify a version or alias for your Lambda function, the most recent version of the function is shut down. You can specify a version or alias if you want to shut down a specific version of your Lambda function. To specify a version or alias, append the version or alias to the ARN of the Lambda function.
```
arn:aws:lambda:us-east-2:123456789012:function:myLambdaFunction:someAlias
```
For more information about versioning and aliases, and see [AWS Lambda function versioning and aliases](https://docs.aws.amazon.com/lambda/latest/dg/versioning-aliases.html).
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
## Examples
The following JSON example defines a Lambda action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-2:123456789012:function:myLambdaFunction"
}
}
]
}
}
```
The following JSON example defines a Lambda action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-1:123456789012:function:${topic()}"
}
}
]
}
}
```
## See also
+ [What is AWS Lambda?](https://docs.aws.amazon.com/lambda/latest/dg/) in the *AWS Lambda Developer Guide*
+ [Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md)
# Location
The Location (`location`) action sends your geographical location data to [Amazon Location Service](https://docs.aws.amazon.com//location/latest/developerguide/welcome.html).
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `geo:BatchUpdateDevicePosition` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`deviceId`
The unique ID of the device providing the location data. For more information, see [https://docs.aws.amazon.com//location/latest/APIReference/API_DevicePositionUpdate.html](https://docs.aws.amazon.com//location/latest/APIReference/API_DevicePositionUpdate.html) from the *Amazon Location Service API Reference*.
Supports [substitution templates](iot-substitution-templates.md): Yes
`latitude`
A string that evaluates to a double value that represents the latitude of the device's location.
Supports [substitution templates](iot-substitution-templates.md): Yes
`longitude`
A string that evaluates to a double value that represents the longitude of the device's location.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The IAM role that allows access to the Amazon Location Service domain. For more information, see [Requirements](#location-rule-action-requirements).
`timestamp`
The time that the location data was sampled. The default value is the time that the MQTT message was processed.
The `timestamp` value consists of the following two values:
+ `value`: An expression that returns a long epoch time value. You can use the [time\$1to\$1epoch(String, String)](iot-sql-functions.md#iot-sql-function-time-to-epoch) function to create a valid timestamp from a date or time value passed in the message payload. Supports [substitution templates](iot-substitution-templates.md): Yes.
+ `unit`: (Optional) The precision of the timestamp value that results from the expression described in `value`. Valid values: `SECONDS` \$1 `MILLISECONDS` \$1 `MICROSECONDS` \$1 `NANOSECONDS`. The default is `MILLISECONDS`. Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only.
`trackerName`
The name of the tracker resource in Amazon Location in which the location is updated. For more information, see [Tracker](https://docs.aws.amazon.com//location/latest/developerguide/geofence-tracker-concepts.html#tracking-overview) from the *Amazon Location Service Developer Guide*.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
## Examples
The following JSON example defines a Location action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"location": {
"roleArn": "arn:aws:iam::123454962127:role/service-role/ExampleRole",
"trackerName": "MyTracker",
"deviceId": "001",
"sampleTime": {
"value": "${timestamp()}",
"unit": "MILLISECONDS"
},
"latitude": "-12.3456",
"longitude": "65.4321"
}
}
]
}
}
```
The following JSON example defines a Location action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"location": {
"roleArn": "arn:aws:iam::123456789012:role/service-role/ExampleRole",
"trackerName": "${TrackerName}",
"deviceId": "${DeviceID}",
"timestamp": {
"value": "${timestamp()}",
"unit": "MILLISECONDS"
},
"latitude": "${get(position, 0)}",
"longitude": "${get(position, 1)}"
}
}
]
}
}
```
The following MQTT payload example shows how substitution templates in the preceding example accesses data. You can use the [https://docs.aws.amazon.com/cli/latest/reference/location/get-device-position-history.html](https://docs.aws.amazon.com/cli/latest/reference/location/get-device-position-history.html) CLI command to verify that the MQTT payload data is delivered in your location tracker.
```
{
"TrackerName": "mytracker",
"DeviceID": "001",
"position": [
"-12.3456",
"65.4321"
]
}
```
```
aws location get-device-position-history --device-id 001 --tracker-name mytracker
```
```
{
"DevicePositions": [
{
"DeviceId": "001",
"Position": [
-12.3456,
65.4321
],
"ReceivedTime": "2022-11-11T01:31:54.464000+00:00",
"SampleTime": "2022-11-11T01:31:54.308000+00:00"
}
]
}
```
## See also
+ [What is Amazon Location Service?](https://docs.aws.amazon.com//location/latest/developerguide/welcome.html) in the *Amazon Location Service Developer Guide*.
# OpenSearch
The OpenSearch (`openSearch`) action writes data from MQTT messages to an Amazon OpenSearch Service domain. You can then use tools like OpenSearch Dashboards to query and visualize data in OpenSearch Service.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `es:ESHttpPut` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use a customer managed AWS KMS key to encrypt data at rest in OpenSearch Service, the service must have permission to use the KMS key on the caller's behalf. For more information, see [Encryption of data at rest for Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/encryption-at-rest.html) in the *Amazon OpenSearch Service Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`endpoint`
The endpoint of your Amazon OpenSearch Service domain.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`index`
The OpenSearch index where you want to store your data.
Supports [substitution templates](iot-substitution-templates.md): Yes
`type`
The type of document you are storing.
For OpenSearch versions later than 1.0, the value of the `type` parameter must be `_doc`. For more information, see the [OpenSearch documentation](https://opensearch.org/docs/1.0/opensearch/rest-api/document-apis/index-document/#response-body-fields).
Supports [substitution templates](iot-substitution-templates.md): Yes
`id`
The unique identifier for each document.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleARN`
The IAM role that allows access to the OpenSearch Service domain. For more information, see [Requirements](#opensearch-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Limitations
The OpenSearch (`openSearch`) action cannot be used to deliver data to VPC Elasticsearch clusters.
## Examples
The following JSON example defines an OpenSearch action in an AWS IoT rule and how you can specify the fields for the `OpenSearch` action. For more information, see [OpenSearchAction](https://docs.aws.amazon.com/iot/latest/apireference/API_OpenSearchAction.html).
```
{
"topicRulePayload": {
"sql": "SELECT *, timestamp() as timestamp FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"openSearch": {
"endpoint": "https://my-endpoint",
"index": "my-index",
"type": "_doc",
"id": "${newuuid()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_os"
}
}
]
}
}
```
The following JSON example defines an OpenSearch action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"openSearch": {
"endpoint": "https://my-endpoint",
"index": "${topic()}",
"type": "${type}",
"id": "${newuuid()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_os"
}
}
]
}
}
```
**Note**
The the substituted `type` field works for OpenSearch version 1.0. For any versions later than 1.0, the value of `type` must be `_doc`.
## See also
[What is Amazon OpenSearch Service?](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/) in the *Amazon OpenSearch Service Developer Guide*
# Republish
The republish (`republish`) action republishes an MQTT message to another MQTT topic.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `iot:Publish` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`headers`
MQTT Version 5.0 headers information.
For more information, see [RepublishAction](https://docs.aws.amazon.com//iot/latest/apireference/API_RepublishAction.html) and [MqttHeaders](https://docs.aws.amazon.com//iot/latest/apireference/API_MqttHeaders.html) in the *AWS API Reference*.
`topic`
The MQTT topic to which to republish the message.
To republish to a reserved topic, which begins with `$`, use `$$` instead. For example, to republish to the device shadow topic `$aws/things/MyThing/shadow/update`, specify the topic as `$$aws/things/MyThing/shadow/update`.
Republishing to [reserved job topics](reserved-topics.md#reserved-topics-job) is not supported.
AWS IoT Device Defender reserve topics don't support HTTP publish.
Supports [substitution templates](iot-substitution-templates.md): Yes
`qos`
(Optional) The Quality of Service (QoS) level to use when republishing messages. Valid values: `0`, `1`. The default value is `0`. For more information about MQTT QoS, see [MQTT](mqtt.md).
Supports [substitution templates](iot-substitution-templates.md): No
`roleArn`
The IAM role that allows AWS IoT to publish to the MQTT topic. For more information, see [Requirements](#republish-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a republish action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"topic": "another/topic",
"qos": 1,
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_republish"
}
}
]
}
}
```
The following JSON example defines a republish action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"topic": "${topic()}/republish",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_republish"
}
}
]
}
}
```
The following JSON example defines a republish action with `headers` in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"topic": "${topic()}/republish",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_republish",
"headers": {
"payloadFormatIndicator": "UTF8_DATA",
"contentType": "rule/contentType",
"correlationData": "cnVsZSBjb3JyZWxhdGlvbiBkYXRh",
"userProperties": [
{
"key": "ruleKey1",
"value": "ruleValue1"
},
{
"key": "ruleKey2",
"value": "ruleValue2"
}
]
}
}
}
]
}
}
```
**Note**
The original source IP won't be passed though [Republish action](#republish-rule-action).
# S3
The S3 (`s3`) action writes the data from an MQTT message to an Amazon Simple Storage Service (Amazon S3) bucket.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `s3:PutObject` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use an AWS KMS customermanaged AWS KMS key to encrypt data at rest in Amazon S3, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [AWS managed AWS KMS keys and customer managed AWS KMS keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html#aws-managed-customer-managed-cmks) in the *Amazon Simple Storage Service Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`bucket`
The Amazon S3 bucket to which to write data.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`cannedacl`
(Optional) The Amazon S3 canned ACL that controls access to the object identified by the object key. For more information, including allowed values, see [Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl).
Supports [substitution templates](iot-substitution-templates.md): No
`key`
The path to the file where the data is written.
Consider an example where this parameter is `${topic()}/${timestamp()}` and the rule receives a message where the topic is `some/topic`. If the current timestamp is `1460685389`, then this action writes the data to a file called `1460685389` in the `some/topic` folder of the S3 bucket.
If you use a static key, AWS IoT overwrites a single file each time the rule invokes. We recommend that you use the message timestamp or another unique message identifier so that a new file is saved in Amazon S3 for each message received.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The IAM role that allows access to the Amazon S3 bucket. For more information, see [Requirements](#s3-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines an S3 action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"s3": {
"bucketName": "amzn-s3-demo-bucket",
"cannedacl": "public-read",
"key": "${topic()}/${timestamp()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3"
}
}
]
}
}
```
## See also
+ [What is Amazon S3?](https://docs.aws.amazon.com/AmazonS3/latest/userguide/) in the *Amazon Simple Storage Service User Guide*
# Salesforce IoT
The Salesforce IoT (`salesforce`) action sends data from the MQTT message that triggered the rule to a Salesforce IoT input stream.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`url`
The URL exposed by the Salesforce IoT input stream. The URL is available from the Salesforce IoT platform when you create an input stream. For more information, see the Salesforce IoT documentation.
Supports [substitution templates](iot-substitution-templates.md): No
`token`
The token used to authenticate access to the specified Salesforce IoT input stream. The token is available from the Salesforce IoT platform when you create an input stream. For more information, see the Salesforce IoT documentation.
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a Salesforce IoT action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"salesforce": {
"token": "ABCDEFGHI123456789abcdefghi123456789",
"url": "https://ingestion-cluster-id.my-env.sfdcnow.com/streams/stream-id/connection-id/my-event"
}
}
]
}
}
```
# SNS
The SNS (`sns`) action sends the data from an MQTT message as an Amazon Simple Notification Service (Amazon SNS) push notification.
You can follow a tutorial that shows you how to create and test a rule with an SNS action. For more information, see [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md).
**Note**
The SNS action doesn't support [Amazon SNS FIFO (First-In-First-Out) topics](https://docs.aws.amazon.com/sns/latest/dg/sns-fifo-topics.html). Because the rules engine is a fully distributed service, there is no guarantee of message order when the SNS action is invoked.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `sns:Publish` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use an AWS KMS customer managed-managed AWS KMS key to encrypt data at rest in Amazon SNS, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [Key management](https://docs.aws.amazon.com/sns/latest/dg/sns-key-management.html) in the *Amazon Simple Notification Service Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`targetArn`
The SNS topic or individual device to which the push notification is sent.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`messageFormat`
(Optional) The message format. Amazon SNS uses this setting to determine if the payload should be parsed and if relevant platform-specific parts of the payload should be extracted. Valid values: `JSON`, `RAW`. Defaults to `RAW`.
Supports [substitution templates](iot-substitution-templates.md): No
`roleArn`
The IAM role that allows access to SNS. For more information, see [Requirements](#sns-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines an SNS action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sns": {
"targetArn": "arn:aws:sns:us-east-2:123456789012:my_sns_topic",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sns"
}
}
]
}
}
```
The following JSON example defines an SNS action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sns": {
"targetArn": "arn:aws:sns:us-east-1:123456789012:${topic()}",
"messageFormat": "JSON",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sns"
}
}
]
}
}
```
## See also
+ [What is Amazon Simple Notification Service?](https://docs.aws.amazon.com/sns/latest/dg/) in the *Amazon Simple Notification Service Developer Guide*
+ [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md)
# SQS
The SQS (`sqs`) action sends data from an MQTT message to an Amazon Simple Queue Service (Amazon SQS) queue.
**Note**
The SQS action doesn't support [Amazon SQS FIFO (First-In-First-Out) queues](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html). Because the rules engine is a fully distributed service, there is no guarantee of message order when the SQS action is triggered.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `sqs:SendMessage` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
+ If you use an AWS KMS customer managed AWS KMS key to encrypt data at rest in Amazon SQS, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [Key management](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-key-management.html) in the *Amazon Simple Queue Service Developer Guide*.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`queueUrl`
The URL of the Amazon SQS queue to which to write the data. The region in this URL doesn't need to be the same AWS Region as your [AWS IoT rule](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html).
There can be additional charges for data transfer cross AWS Regions using the SQS rule action. For more information, see [Amazon SQS pricing](https://aws.amazon.com/sqs/pricing/).
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`useBase64`
Set this parameter to `true` to configure the rule action to base64-encode the message data before it writes the data to the Amazon SQS queue. Defaults to `false`.
Supports [substitution templates](iot-substitution-templates.md): No
`roleArn`
The IAM role that allows access to the Amazon SQS queue. For more information, see [Requirements](#sqs-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines an SQS action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sqs": {
"queueUrl": "https://sqs.us-east-2.amazonaws.com/123456789012/my_sqs_queue",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sqs"
}
}
]
}
}
```
The following JSON example defines an SQS action with substitution templates in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sqs": {
"queueUrl": "https://sqs.us-east-2.amazonaws.com/123456789012/${topic()}",
"useBase64": true,
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sqs"
}
}
]
}
}
```
## See also
+ [What is Amazon Simple Queue Service?](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/) in the *Amazon Simple Queue Service Developer Guide*
# Step Functions
The Step Functions (`stepFunctions`) action starts an AWS Step Functions state machine.
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `states:StartExecution` operation. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`stateMachineName`
The name of the Step Functions state machine to start.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`executionNamePrefix`
(Optional) The name given to the state machine execution consists of this prefix followed by a UUID. Step Functions creates a unique name for each state machine execution if one is not provided.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The ARN of the role that grants AWS IoT permission to start the state machine. For more information, see [Requirements](#stepfunctions-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
## Examples
The following JSON example defines a Step Functions action in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"stepFunctions": {
"stateMachineName": "myStateMachine",
"executionNamePrefix": "myExecution",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_step_functions"
}
}
]
}
}
```
## See also
+ [What is AWS Step Functions?](https://docs.aws.amazon.com/step-functions/latest/dg/) in the *AWS Step Functions Developer Guide*
# Timestream
The Timestream rule action writes attributes (measures) from an MQTT message into an Amazon Timestream table. For more information about Amazon Timestream, see [What Is Amazon Timestream?](https://docs.aws.amazon.com/timestream/latest/developerguide/what-is-timestream.html).
**Note**
Amazon Timestream is not available in all AWS Regions. If Amazon Timestream is not available in your Region, it won't appear in the list of rule actions.
The attributes that this rule stores in the Timestream database are those that result from the rule's query statement. The value of each attribute in the query statement's result is parsed to infer its data type (as in a [DynamoDBv2](dynamodb-v2-rule-action.md) action). Each attribute's value is written to its own record in the Timestream table. To specify or change an attribute's data type, use the [`cast()`](iot-sql-functions.md#iot-sql-function-cast) function in the query statement. For more information about the contents of each Timestream record, see [Timestream record content](#timestream-rule-action-data).
**Note**
With SQL V2 (2016-03-23), numeric values that are whole numbers, such as `10.0`, are converted their Integer representation (`10`). Explicitly casting them to a `Decimal` value, such as by using the [cast()](iot-sql-functions.md#iot-sql-function-cast) function, does not prevent this behavior—the result is still an `Integer` value. This can cause type mismatch errors that prevent data from being recorded in the Timestream database. To process whole number numeric values as `Decimal` values, use SQL V1 (2015-10-08) for the rule query statement.
**Note**
The maximum number of values that a Timestream rule action can write into an Amazon Timestream table is 100. For more information, see [Amazon Timestream Quota's Reference](https://docs.aws.amazon.com//timestream/latest/developerguide/ts-limits.html#limits.default).
## Requirements
This rule action has the following requirements:
+ An IAM role that AWS IoT can assume to perform the `timestream:DescribeEndpoints` and `timestream:WriteRecords` operations. For more information, see [Granting an AWS IoT rule the access it requires](iot-create-role.md).
In the AWS IoT console, you can choose, update, or create a role to allow AWS IoT to perform this rule action.
+ If you use a customer- AWS KMS to encrypt data at rest in Timestream, the service must have permission to use the AWS KMS key on the caller's behalf. For more information, see [How AWS services use AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/service-integration.html).
## Parameters
When you create an AWS IoT rule with this action, you must specify the following information:
`databaseName`
The name of an Amazon Timestream database that has the table to receive the records this action creates. See also `tableName`.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`dimensions`
Metadata attributes of the time series that are written in each measure record. For example, the name and Availability Zone of an EC2 instance or the name of the manufacturer of a wind turbine are dimensions.
`name`
The metadata dimension name. This is the name of the column in the database table record.
Dimensions can't be named: `measure_name`, `measure_value`, or `time`. These names are reserved. Dimension names can't start with `ts_` or `measure_value` and they can't contain the colon (`:`) character.
Supports [substitution templates](iot-substitution-templates.md): No
`value`
The value to write in this column of the database record.
Supports [substitution templates](iot-substitution-templates.md): Yes
`roleArn`
The Amazon Resource Name (ARN) of the role that grants AWS IoT permission to write to the Timestream database table. For more information, see [Requirements](#timestream-rule-action-requirements).
Supports [substitution templates](iot-substitution-templates.md): No
`tableName`
The name of the database table into which to write the measure records. See also `databaseName`.
Supports [substitution templates](iot-substitution-templates.md): API and AWS CLI only
`timestamp`
The value to use for the entry's timestamp. If blank, the time that the entry was processed is used.
`unit`
The precision of the timestamp value that results from the expression described in `value`.
Valid values: `SECONDS` \$1 `MILLISECONDS` \$1 `MICROSECONDS` \$1 `NANOSECONDS`. The default is `MILLISECONDS`.
`value`
An expression that returns a long epoch time value.
You can use the [time\$1to\$1epoch(String, String)](iot-sql-functions.md#iot-sql-function-time-to-epoch) function to create a valid timestamp from a date or time value passed in the message payload.
## Timestream record content
The data written to the Amazon Timestream table by this action include a timestamp, metadata from the Timestream rule action, and the result of the rule's query statement.
For each attribute (measure) in the result of the query statement, this rule action writes a record to the specified Timestream table with these columns.
| Column name | Attribute type | Value | Comments |
| --- | --- | --- | --- |
| *dimension-name* | DIMENSION | The value specified in the Timestream rule action entry. | Each **Dimension** specified in the rule action entry creates a column in the Timestream database with the dimension's name. |
| measure\$1name | MEASURE\$1NAME | The attribute's name | The name of the attribute in the result of the query statement whose value is specified in the `measure_value::data-type` column. |
| measure\$1value::*data-type* | MEASURE\$1VALUE | The value of the attribute in the result of the query statement. The attribute's name is in the `measure_name` column. | The value is interpreted\$1 and cast as the most suitable match of: `bigint`, `boolean`, `double`, or `varchar`. Amazon Timestream creates a separate column for each data type. The value in the message can be cast to another data type by using the [`cast()`](iot-sql-functions.md#iot-sql-function-cast) function in the rule's query statement. |
| time | TIMESTAMP | The date and time of the record in the database. | This value is assigned by rules engine or the `timestamp` property, if it is defined. |
\$1 The attribute value read from the message payload is interpreted as follows. See the [Examples](#timestream-rule-action-examples) for an illustration of each of these cases.
+ An unquoted value of `true` or `false` is interpreted as a `boolean` type.
+ A decimal numeric is interpreted as a `double` type.
+ A numeric value without a decimal point is interpreted as a `bigint` type.
+ A quoted string is interpreted as a `varchar` type.
+ Objects and array values are converted to JSON strings and stored as a `varchar` type.
## Examples
The following JSON example defines a Timestream rule action with a substitution template in an AWS IoT rule.
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'iot/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"timestream": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_timestream",
"tableName": "devices_metrics",
"dimensions": [
{
"name": "device_id",
"value": "${clientId()}"
},
{
"name": "device_firmware_sku",
"value": "My Static Metadata"
}
],
"databaseName": "record_devices"
}
}
]
}
}
```
Using the Timestream topic rule action defined in the previous example with the following message payload results in the Amazon Timestream records written in the table that follows.
```
{
"boolean_value": true,
"integer_value": 123456789012,
"double_value": 123.456789012,
"string_value": "String value",
"boolean_value_as_string": "true",
"integer_value_as_string": "123456789012",
"double_value_as_string": "123.456789012",
"array_of_integers": [23,36,56,72],
"array of strings": ["red", "green","blue"],
"complex_value": {
"simple_element": 42,
"array_of_integers": [23,36,56,72],
"array of strings": ["red", "green","blue"]
}
}
```
The following table displays the database columns and records that using the specified topic rule action to process the previous message payload creates. The `device_firmware_sku` and `device_id` columns are the DIMENSIONS defined in the topic rule action. The Timestream topic rule action creates the `time` column and the `measure_name` and `measure_value::*` columns, which it fills with the values from the result of the topic rule action's query statement.
| device\$1firmware\$1sku | device\$1id | measure\$1name | measure\$1value::bigint | measure\$1value::varchar | measure\$1value::double | measure\$1value::boolean | time |
| --- | --- | --- | --- | --- | --- | --- | --- |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | complex\$1value | - | \$1"simple\$1element":42,"array\$1of\$1integers":[23,36,56,72],"array of strings":["red","green","blue"]\$1 | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | integer\$1value\$1as\$1string | - | 123456789012 | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | boolean\$1value | - | - | - | TRUE | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | integer\$1value | 123456789012 | - | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | string\$1value | - | String value | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | array\$1of\$1integers | - | [23,36,56,72] | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | array of strings | - | ["red","green","blue"] | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | boolean\$1value\$1as\$1string | - | TRUE | - | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | double\$1value | - | - | 123.456789012 | - | 2020-08-26 22:42:16.423000000 |
| My Static Metadata | iotconsole-159EXAMPLE738-0 | double\$1value\$1as\$1string | - | 123.45679 | - | - | 2020-08-26 22:42:16.423000000 |
## Troubleshooting a rule
If you have an issue with your rules, we recommend that you activate CloudWatch Logs. You can analyze your logs to determine whether the issue is authorization or whether, for example, a WHERE clause condition didn't match. For more information, see [Setting Up CloudWatch Logs](https://docs.aws.amazon.com/iot/latest/developerguide/cloud-watch-logs.html).
# Accessing cross-account resources using AWS IoT rules
Access cross-account resources
You can configure AWS IoT rules for cross-account access so that data ingested on MQTT topics of one account can be routed into the AWS services, such as Amazon SQS and Lambda, of another account. The following explains how to set up AWS IoT rules for cross-account data ingestion, from an MQTT topic in one account, to a destination in another account.
Cross-account rules can be configured using [resource-based permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_controlling.html#TypesPermissions) on the destination resource. Therefore, only destinations that support resource-based permissions can be enabled for the cross-account access with AWS IoT rules. The supported destinations include Amazon SQS, Amazon SNS, Amazon S3, and AWS Lambda.
**Note**
For the supported destinations, except for Amazon SQS, you must define the rule in the same AWS Region as another service's resource so that the rule action can interact with that resource. For more information about AWS IoT rule actions, see [AWS IoT rule actions](iot-rule-actions.md). For more information about rule's SQS action, see [SQS](sqs-rule-action.md).
## Prerequisites
+ Familiarity with [AWS IoT rules](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html)
+ An understanding of [IAM users](http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_identity-management.html), [roles](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), and [resource-based permission](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_permissions.html#TypesPermissions)
+ Having [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) installed
## Cross-account setup for Amazon SQS
Scenario: Account A sends data from an MQTT message to account B's Amazon SQS queue.
| AWS account | Account referred to as | Description |
| --- | --- | --- |
| 1111-1111-1111 | Account A | Rule action: sqs:SendMessage |
| 2222-2222-2222 | Account B | Amazon SQS queue [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/accessing-cross-account-resources-using-rules.html) |
**Note**
Your destination Amazon SQS queue doesn't have to be in the same AWS Region as your [AWS IoT rule](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html). For more information about rule's SQS action, see [SQS](sqs-rule-action.md).
**Do the Account A tasks**
**Note**
To run the following commands, your IAM user should have permissions to `iot:CreateTopicRule` with the rule's Amazon Resource Name (ARN) as a resource, and permissions to `iam:PassRole` action with a resource as the role's ARN.
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account A’s IAM user.
1. Create an IAM role that trusts AWS IoT rules engine, and attaches a policy that allows access to account B's Amazon SQS queue. See example commands and policy documents in [Granting AWS IoT the required access](https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-role.html).
1. To create a rule that is attached to a topic, run the [create-topic-rule command](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html).
```
aws iot create-topic-rule --rule-name myRule --topic-rule-payload file://./my-rule.json
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified Amazon SQS queue. The SQL statement filters the messages and the role ARN grants AWS IoT permissions to add the message to the Amazon SQS queue.
```
{
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sqs": {
"queueUrl": "https://sqs.region.amazonaws.com/2222-2222-2222/ExampleQueue",
"roleArn": "arn:aws:iam::1111-1111-1111:role/my-iot-role",
"useBase64": false
}
}
]
}
```
For more information about how to define an Amazon SQS action in an AWS IoT rule, see [AWS IoT rule actions - Amazon SQS](https://docs.aws.amazon.com/iot/latest/developerguide/sqs-rule-action.html).
**Do the Account B tasks**
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account B’s IAM user.
1. To give permissions for the Amazon SQS queue resource to account A, run the [add-permission command](https://docs.aws.amazon.com/cli/latest/reference/sqs/add-permission.html).
```
aws sqs add-permission --queue-url https://sqs.region.amazonaws.com/2222-2222-2222/ExampleQueue --label SendMessagesToMyQueue --aws-account-ids 1111-1111-1111 --actions SendMessage
```
## Cross-account setup for Amazon SNS
Scenario: Account A sends data from an MQTT message to an Amazon SNS topic of account B.
| AWS account | Account referred to as | Description |
| --- | --- | --- |
| 1111-1111-1111 | Account A | Rule action: sns:Publish |
| 2222-2222-2222 | Account B | Amazon SNS topic ARN: arn:aws:sns:region:2222-2222-2222:ExampleTopic |
**Do the Account A tasks**
**Notes**
To run the following commands, your IAM user should have permissions to `iot:CreateTopicRule` with rule ARN as a resource and permissions to the `iam:PassRole` action with a resource as role ARN.
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account A’s IAM user.
1. Create an IAM role that trusts AWS IoT rules engine, and attaches a policy that allows access to account B's Amazon SNS topic. For example commands and policy documents, see [Granting AWS IoT the required access](https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-role.html).
1. To create a rule that is attached to a topic, run the [create-topic-rule command](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html).
```
aws iot create-topic-rule --rule-name myRule --topic-rule-payload file://./my-rule.json
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified Amazon SNS topic. The SQL statement filters the messages, and the role ARN grants AWS IoT permissions to send the message to the Amazon SNS topic.
```
{
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"sns": {
"targetArn": "arn:aws:sns:region:2222-2222-2222:ExampleTopic",
"roleArn": "arn:aws:iam::1111-1111-1111:role/my-iot-role"
}
}
]
}
```
For more information about how to define an Amazon SNS action in an AWS IoT rule, see [AWS IoT rule actions - Amazon SNS](https://docs.aws.amazon.com/iot/latest/developerguide/sns-rule-action.html).
**Do the Account B tasks**
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account B’s IAM user.
1. To give permission on the Amazon SNS topic resource to account A, run the [add-permission command](https://docs.aws.amazon.com/cli/latest/reference/sns/add-permission.html).
```
aws sns add-permission --topic-arn arn:aws:sns:region:2222-2222-2222:ExampleTopic --label Publish-Permission --aws-account-id 1111-1111-1111 --action-name Publish
```
## Cross-account setup for Amazon S3
Scenario: Account A sends data from an MQTT message to an Amazon S3 bucket of account B.
| AWS account | Account referred to as | Description |
| --- | --- | --- |
| 1111-1111-1111 | Account A | Rule action: s3:PutObject |
| 2222-2222-2222 | Account B | Amazon S3 bucket ARN: arn:aws:s3:::amzn-s3-demo-bucket |
**Do the Account A tasks**
**Note**
To run the following commands, your IAM user should have permissions to `iot:CreateTopicRule` with the rule ARN as a resource and permissions to `iam:PassRole` action with a resource as role ARN.
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account A’s IAM user.
1. Create an IAM role that trusts AWS IoT rules engine and attaches a policy that allows access to account B's Amazon S3 bucket. For example commands and policy documents, see [Granting AWS IoT the required access](https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-role.html).
1. To create a rule that is attached to your target S3 bucket, run the [create-topic-rule command](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html).
```
aws iot create-topic-rule --rule-name my-rule --topic-rule-payload file://./my-rule.json
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified Amazon S3 bucket. The SQL statement filters the messages, and the role ARN grants AWS IoT permissions to add the message to the Amazon S3 bucket.
```
{
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"s3": {
"bucketName": "amzn-s3-demo-bucket",
"key": "${topic()}/${timestamp()}",
"roleArn": "arn:aws:iam::1111-1111-1111:role/my-iot-role"
}
}
]
}
```
For more information about how to define an Amazon S3 action in an AWS IoT rule, see [AWS IoT rule actions - Amazon S3](https://docs.aws.amazon.com/iot/latest/developerguide/s3-rule-action.html).
**Do the Account B tasks**
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account B’s IAM user.
1. Create a bucket policy that trusts account A's principal.
The following is an example payload file that defines a bucket policy that trusts the principal of another account.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AddCannedAcl",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:root"
]
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::amzn-s3-demo-bucket/*"
}
]
}
```
For more information, see [bucket policy examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-1).
1. To attach the bucket policy to the specified bucket, run the [put-bucket-policy command](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-policy.html).
```
aws s3api put-bucket-policy --bucket amzn-s3-demo-bucket --policy file://./amzn-s3-demo-bucket-policy.json
```
1. To make the cross-account access work, make sure you have the correct **Block all public access** settings. For more information, see [Security Best Practices for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html).
## Cross-account setup for AWS Lambda
Scenario: Account A invokes an AWS Lambda function of account B, passing in an MQTT message.
| AWS account | Account referred to as | Description |
| --- | --- | --- |
| 1111-1111-1111 | Account A | Rule action: lambda:InvokeFunction |
| 2222-2222-2222 | Account B | Lambda function ARN: arn:aws:lambda:region:2222-2222-2222:function:example-function |
**Do the Account A tasks**
**Notes**
To run the following commands, your IAM user should have permissions to `iot:CreateTopicRule` with rule ARN as a resource, and permissions to `iam:PassRole` action with resource as role ARN.
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account A’s IAM user.
1. Run the [create-topic-rule command ](https://docs.aws.amazon.com/cli/latest/reference/iot/create-topic-rule.html)to create a rule that defines cross-account access to account B's Lambda function.
```
aws iot create-topic-rule --rule-name my-rule --topic-rule-payload file://./my-rule.json
```
The following is an example payload file with a rule that inserts all messages sent to the `iot/test` topic into the specified Lambda function. The SQL statement filters the messages and the role ARN grants AWS IoT permission to pass in the data to the Lambda function.
```
{
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:region:2222-2222-2222:function:example-function"
}
}
]
}
```
For more information about how to define an AWS Lambda action in an AWS IoT rule, read [AWS IoT rule actions - Lambda](https://docs.aws.amazon.com/iot/latest/developerguide/lambda-rule-action.html).
**Do the Account B tasks**
1. [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) using account B’s IAM user.
1. Run [Lambda's add-permission command](https://docs.aws.amazon.com/cli/latest/reference/lambda/add-permission.html) to give AWS IoT rules permission to activate the Lambda function. To run the following command, your IAM user should have permission to `lambda:AddPermission` action.
```
aws lambda add-permission --function-name example-function --region us-east-1 --principal iot.amazonaws.com --source-arn arn:aws:iot:region:1111-1111-1111:rule/example-rule --source-account 1111-1111-1111 --statement-id "unique_id" --action "lambda:InvokeFunction"
```
**Options:**
**--principal**
This field gives permission to AWS IoT (represented by `iot.amazonaws.com`) to call the Lambda function.
**--source-arn**
This field confirms that only `arn:aws:iot:region:1111-1111-1111:rule/example-rule` in AWS IoT triggers this Lambda function and no other rule in the same or different account can activate this Lambda function.
**--source-account**
This field confirms that AWS IoT activates this Lambda function only on behalf of the `1111-1111-1111` account.
**Notes**
If you see an error message "The rule could not be found" from your AWS Lambda function’s console under **Configuration**, ignore the error message and proceed to test the connection.
# Error handling (error action)
When AWS IoT receives a message from a device, the rules engine checks to see if the message matches a rule. If so, the rule's query statement is evaluated and the rule's actions are activated, passing the query statement's result.
If a problem occurs when activating an action, the rules engine activates an error action, if one is specified for the rule. This might happen when:
+ A rule doesn't have permission to access an Amazon S3 bucket.
+ A user error causes DynamoDB provisioned throughput to be exceeded.
**Note**
The error handling covered in this topic is for [rule actions](iot-rule-actions.md). To debug SQL issues, including external functions, you can set up AWS IoT logging. For more information, see [Configure AWS IoT logging](configure-logging.md).
## Error action message format
A single message is generated per rule and message. For example, if two rule actions in the same rule fail, the error action receives one message that contains both errors.
The error action message looks like the following example.
```
{
"ruleName": "TestAction",
"topic": "testme/action",
"cloudwatchTraceId": "7e146a2c-95b5-6caf-98b9-50e3969734c7",
"clientId": "iotconsole-1511213971966-0",
"base64OriginalPayload": "ewogICJtZXNzYWdlIjogIkhlbGxvIHZyb20gQVdTIElvVCBjb25zb2xlIgp9",
"failures": [
{
"failedAction": "S3Action",
"failedResource": "us-east-1-s3-verify-user",
"errorMessage": "Failed to put S3 object. The error received was The specified bucket does not exist (Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Request ID: 9DF5416B9B47B9AF; S3 Extended Request ID: yMah1cwPhqTH267QLPhTKeVPKJB8BO5ndBHzOmWtxLTM6uAvwYYuqieAKyb6qRPTxP1tHXCoR4Y=). Message arrived on: error/action, Action: s3, Bucket: us-east-1-s3-verify-user, Key: \"aaa\". Value of x-amz-id-2: yMah1cwPhqTH267QLPhTKeVPKJB8BO5ndBHzOmWtxLTM6uAvwYYuqieAKyb6qRPTxP1tHXCoR4Y="
}
]
}
```
ruleName
The name of the rule that triggered the error action.
topic
The topic in which the original message was received.
cloudwatchTraceId
A unique identity referring to the error logs in CloudWatch.
clientId
The client ID of the message publisher.
base64OriginalPayload
The original message payload Base64-encoded.
failures
failedAction
The name of the action that failed to complete (for example, "S3Action").
failedResource
The name of the resource (for example, the name of an S3 bucket).
errorMessage
The description and explanation of the error.
## Error action example
Here is an example of a rule with an added error action. The following rule has an action that writes message data to a DynamoDB table and an error action that writes data to an Amazon S3 bucket:
```
{
"sql" : "SELECT * FROM ..."
"actions" : [{
"dynamoDB" : {
"table" : "PoorlyConfiguredTable",
"hashKeyField" : "AConstantString",
"hashKeyValue" : "AHashKey"}}
],
"errorAction" : {
"s3" : {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"bucketName" : "message-processing-errors",
"key" : "${replace(topic(), '/', '-') + '-' + timestamp() + '-' + newuuid()}"
}
}
}
```
You can use any [function](iot-sql-functions.md) or [substitution template](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html) in an error action's SQL statement including the external functions: [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-secret](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-secret), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-machine-learning](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-machine-learning), and [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64). If an error action requires to call an external function, then invoking the error action can result in additional bill for the external function.
For more information about rules and how to specify an error action, see [Creating an AWS IoT Rule](https://docs.aws.amazon.com//iot/latest/developerguide/iot-create-rule.html).
For more information about using CloudWatch to monitor the success or failure of rules, see [AWS IoT metrics and dimensions](metrics_dimensions.md).
# Reducing messaging costs with Basic Ingest
Basic Ingest
You can use Basic Ingest, to securely send device data to the AWS services supported by [AWS IoT rule actions](iot-rule-actions.md), without incurring [messaging costs](https://aws.amazon.com/iot-core/pricing/). Basic Ingest optimizes data flow by removing the publish/subscribe message broker from the ingestion path.
Basic Ingest can send messages from your devices or applications. The messages have topic names that start with `$aws/rules/rule_name` for their first three levels, where `rule_name` is the name of the AWS IoT rule that you want to invoke.
You can use an existing rule with Basic Ingest by adding the Basic Ingest prefix (`$aws/rules/rule_name`) to the message topic that you'd use to invoke the rule. For example, if you have a rule named `BuildingManager` that's invoked by messages with topics like `Buildings/Building5/Floor2/Room201/Lights` (`"sql": "SELECT * FROM 'Buildings/#'"`), you can invoke the same rule with Basic Ingest by sending a message with topic `$aws/rules/BuildingManager/Buildings/Building5/Floor2/Room201/Lights`.
**Note**
Your devices and rules can't subscribe to Basic Ingest reserved topics. For example, the AWS IoT Device Defender metric `num-messages-received` metrics is not emitted as it doesn't support subscribing to topics. For more information, see [Reserved topics](reserved-topics.md).
If you need a publish/subscribe broker to distribute messages to multiple subscribers (for example, to deliver messages to other devices and the rules engine), you should continue to use the AWS IoT message broker to handle the message distribution. However, make sure that you publish your messages on topics other than Basic Ingest topics.
## Using Basic Ingest
Before you use Basic Ingest, verify that your device or application is using a [policy](iot-policies.md) that has publish permissions on `$aws/rules/*`. Or, you can specify permission for individual rules with `$aws/rules/rule_name/*` in the policy. Otherwise, your devices and applications can continue to use their existing connections with AWS IoT Core.
When the message reaches the rules engine, there's no difference in implementation or error handling between rules invoked from Basic Ingest and those invoked through message broker subscriptions.
You can create rules for use with Basic Ingest. Keep in mind the following:
+ The initial prefix of a Basic Ingest topic (`$aws/rules/rule_name`) isn't available to the [topic(Decimal)](iot-sql-functions.md#iot-function-topic) function.
+ If you define a rule that's invoked only with Basic Ingest, the `FROM` clause is optional in the `sql` field of the `rule` definition. It's still required if the rule is also invoked by other messages that must be sent through the message broker (for example, because those other messages must be distributed to multiple subscribers). For more information, see [AWS IoT SQL reference](iot-sql-reference.md).
+ The first three levels of the Basic Ingest topic (`$aws/rules/rule_name`) aren't counted toward the 8-segment length limit or toward the 256-total character limit for a topic. Otherwise, the same restrictions apply as documented in [AWS IoT Limits](https://docs.aws.amazon.com/general/latest/gr/iot-core.html#limits_iot).
+ If a message is received with a Basic Ingest topic that specifies an inactive rule or a rule that doesn't exist, an error log is created in an Amazon CloudWatch log to help you with debugging. For more information, see [Rules engine log entries](cwl-format.md#rule-engine-logs). A `RuleNotFound` metric is indicated and you can create alarms on this metric. For more information, see Rule Metrics in [Rule metrics](metrics_dimensions.md#rulemetrics).
+ You can still publish with QoS 1 on Basic Ingest topics. You receive a PUBACK after the message is successfully delivered to the rules engine. Receiving a PUBACK doesn't mean that your rule actions were completed successfully. You can configure an error action to handle errors when an action is run. For more information, see [Error handling (error action)](rule-error-handling.md).
# AWS IoT SQL reference
In AWS IoT, rules are defined using an SQL-like syntax. SQL statements are composed of three types of clauses:
**SET**
(Optional) Defines variables that you can reuse throughout SQL statements and substitution templates. Assign values to variables using expressions. Reference these variables in SELECT and WHERE clauses, and in action substitution templates.
The SET clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Variables](iot-sql-set.md#iot-sql-set-usage), and [Nested object queries](iot-sql-nested-queries.md).
**SELECT**
(Required) Extracts information from the payload of an incoming message and performs transformations on the information. The messages to use are identified by the [topic filter](topics.md#topicfilters) specified in the FROM clause.
The SELECT clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Substitution templates](iot-substitution-templates.md), [Variables](iot-sql-set.md#iot-sql-set-usage), [Nested object queries](iot-sql-nested-queries.md), and [Binary payloads](binary-payloads.md).
**FROM**
The MQTT message [topic filter](topics.md#topicfilters) that identifies the messages to extract data from. The rule is activated for each message sent to an MQTT topic that matches the topic filter specified here. Required for rules that are activated by messages that pass through the message broker. Optional for rules that are only activated using the [Basic Ingest](iot-basic-ingest.md) feature.
**WHERE**
(Optional) Adds conditional logic that determines whether the actions specified by a rule are carried out.
The WHERE clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Variables](iot-sql-set.md#iot-sql-set-usage), and [Nested object queries](iot-sql-nested-queries.md).
An example SQL statement looks like this:
```
SELECT color AS rgb FROM 'topic/subtopic' WHERE temperature > 50
```
An example MQTT message (also called an incoming payload) looks like this:
```
{
"color":"red",
"temperature":100
}
```
If this message is published on the `'topic/subtopic'` topic, the rule is triggered and the SQL statement is evaluated. The SQL statement extracts the value of the `color` property if the `"temperature"` property is greater than 50. The WHERE clause specifies the condition `temperature > 50`. The `AS` keyword renames the `"color"` property to `"rgb"`. The result (also called an *outgoing payload*) looks like this:
```
{
"rgb":"red"
}
```
This data is then forwarded to the rule's action, which sends the data for more processing. For more information about rule actions, see [AWS IoT rule actions](iot-rule-actions.md).
**Note**
Comments are not currently supported in AWS IoT SQL syntax.
Attribute names with spaces in them can't be used as field names in the SQL statement. While the incoming payload can have attribute names with spaces in them, such names can't be used in the SQL statement. They will, however, be passed through to the outgoing payload if you use a wildcard (\$1) field name specification.
# SELECT clause
The AWS IoT SELECT clause is essentially the same as the ANSI SQL SELECT clause, with some minor differences.
The SELECT clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Variables](iot-sql-set.md#iot-sql-set-usage), [Nested object queries](iot-sql-nested-queries.md), and [Binary payloads](binary-payloads.md).
You can use the SELECT clause to extract information from incoming MQTT messages. You can also use `SELECT *` to retrieve the entire incoming message payload. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL statement: SELECT * FROM 'topic/subtopic'
Outgoing payload: {"color":"red", "temperature":50}
```
If the payload is a JSON object, you can reference keys in the object. Your outgoing payload contains the key-value pair. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL statement: SELECT color FROM 'topic/subtopic'
Outgoing payload: {"color":"red"}
```
You can use the AS keyword to rename keys. For example:
```
Incoming payload published on topic 'topic/subtopic':{"color":"red", "temperature":50}
SQL:SELECT color AS my_color FROM 'topic/subtopic'
Outgoing payload: {"my_color":"red"}
```
You can select multiple items by separating them with a comma. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL: SELECT color as my_color, temperature as fahrenheit FROM 'topic/subtopic'
Outgoing payload: {"my_color":"red","fahrenheit":50}
```
You can select multiple items including '\$1' to add items to the incoming payload. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL: SELECT *, 15 as speed FROM 'topic/subtopic'
Outgoing payload: {"color":"red", "temperature":50, "speed":15}
```
You can use the `"VALUE"` keyword to produce outgoing payloads that are not JSON objects. With SQL version `2015-10-08`, you can select only one item. With SQL version `2016-03-23` or later, you can also select an array to output as a top-level object.
**Example**
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL: SELECT VALUE color FROM 'topic/subtopic'
Outgoing payload: "red"
```
You can use `'.'` syntax to drill into nested JSON objects in the incoming payload. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":{"red":255,"green":0,"blue":0}, "temperature":50}
SQL: SELECT color.red as red_value FROM 'topic/subtopic'
Outgoing payload: {"red_value":255}
```
For information about how to use JSON object and property names that include reserved characters, such as numbers or the hyphen (minus) character, see [JSON extensions](iot-sql-json.md)
You can use functions (see [Functions](iot-sql-functions.md)) to transform the incoming payload. You can use parentheses for grouping. For example:
```
Incoming payload published on topic 'topic/subtopic': {"color":"red", "temperature":50}
SQL: SELECT (temperature - 32) * 5 / 9 AS celsius, upper(color) as my_color FROM 'topic/subtopic'
Outgoing payload: {"celsius":10,"my_color":"RED"}
```
# FROM clause
The FROM clause subscribes your rule to a [topic](topics.md#topicnames) or [topic filter](topics.md#topicfilters). Enclose the topic or topic filter in single quotes ('). The rule is triggered for each message sent to an MQTT topic that matches the topic filter specified here. You can subscribe to a group of similar topics using a topic filter.
**Example:**
Incoming payload published on topic `'topic/subtopic'`: `{temperature: 50}`
Incoming payload published on topic `'topic/subtopic-2'`: `{temperature: 50}`
SQL: `"SELECT temperature AS t FROM 'topic/subtopic'"`.
The rule is subscribed to `'topic/subtopic'`, so the incoming payload is passed to the rule. The outgoing payload, passed to the rule actions, is: `{t: 50}`. The rule is not subscribed to `'topic/subtopic-2'`, so the rule is not triggered for the message published on `'topic/subtopic-2'`.
**\$1 Wildcard Example:**
You can use the '\$1' (multi-level) wildcard character to match one or more particular path elements:
Incoming payload published on topic `'topic/subtopic'`: `{temperature: 50}`.
Incoming payload published on topic `'topic/subtopic-2'`: `{temperature: 60}`.
Incoming payload published on topic `'topic/subtopic-3/details'`: `{temperature: 70}`.
Incoming payload published on topic `'topic-2/subtopic-x'`: `{temperature: 80}`.
SQL: `"SELECT temperature AS t FROM 'topic/#'"`.
The rule is subscribed to any topic that begins with `'topic'`, so it's executed three times, sending outgoing payloads of `{t: 50}` (for topic/subtopic), `{t: 60}` (for topic/subtopic-2), and `{t: 70}` (for topic/subtopic-3/details) to its actions. It's not subscribed to `'topic-2/subtopic-x'`, so the rule isn't triggered for the `{temperature: 80}` message.
**\$1 Wildcard Example:**
You can use the '\$1' (single-level) wildcard character to match any one particular path element:
Incoming payload published on topic `'topic/subtopic'`: `{temperature: 50}`.
Incoming payload published on topic `'topic/subtopic-2'`: `{temperature: 60}`.
Incoming payload published on topic `'topic/subtopic-3/details'`: `{temperature: 70}`.
Incoming payload published on topic `'topic-2/subtopic-x'`: `{temperature: 80}`.
SQL: `"SELECT temperature AS t FROM 'topic/+'"`.
The rule is subscribed to all topics with two path elements where the first element is `'topic'`. The rule is executed for the messages sent to `'topic/subtopic'` and `'topic/subtopic-2'`, but not `'topic/subtopic-3/details'` (it has more levels than the topic filter) or `'topic-2/subtopic-x'` (it doesn't start with `topic`).
# SET clause
Use the SET clause to define variables that store expression results. You can reuse these variables in SELECT and WHERE clauses, and in substitution templates. This helps you avoid duplicating complex expressions and reduce the number of function calls in your SQL statement.
The SET clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Variables](#iot-sql-set-usage), and [Nested object queries](iot-sql-nested-queries.md).
## SET clause syntax
The SET clause must appear before the SELECT clause in your SQL statement. Use the following syntax:
```
SET @variable_name = expression [, @variable_name2 = expression2]
```
Syntax rules:
+ Start variable names with `@`
+ Variable names can contain letters, numbers, and underscores
+ Variable names can be up to 64 characters long
+ Multiple variables can be set in a single SET clause, separated by commas
+ Each variable can only be assigned once (variables are immutable)
+ The SET keyword can only be used once per SQL statement
## Using variables
After you define variables, you can use them in:
+ SELECT clauses
+ WHERE clauses
+ Other SET variable assignments
+ Action substitution templates
+ Error action substitution templates
+ Nested SELECT queries
+ Function parameters (certain parameters such as roleArn parameters and parameters that switch the mode of a function similar to `transform("enrichArray", attributes, values)` do not support variables)
Variables are referenced using the same `@variable_name` syntax used in the SET clause. You can also use JSON extension syntax to access properties of variables that contain objects, such as `@variable_name.property`.
## SET clause examples
**Basic variable usage**
The following example shows a payload published on topic `device/data`: `{"temp_fahrenheit": 75, "humidity": 60}`
SQL statement:
```
SET @temp_celsius = (temp_fahrenheit - 32) * 5 / 9
SELECT @temp_celsius AS celsius, humidity FROM 'device/data'
```
Outgoing payload: `{"celsius": 23.89, "humidity": 60}`
**Access members in embedded JSON objects **
The following example shows a payload published on topic `device/data`: `{"device1": {"deviceId":"weather_sensor", "deviceData": {"sensors": {"temp_fahrenheit": 75, "humidity": 60}, "location": [47.606,-122.332]}}}`
SQL statement:
```
SET @device_sensor_data = device1.deviceData.sensors
SELECT @device_sensor_data.temp_fahrenheit AS temp_fahrenheit, @device_sensor_data.humidity as humidity, device1.deviceId as deviceId FROM 'device/data'
```
Outgoing payload: `{"temp_fahrenheit":75,"humidity":60,"deviceId":"weather_sensor"}`
for more information on how to work with JSON extensions, reference [JSON extensions](iot-sql-json.md)
**Avoiding duplicate function calls**
SET variables help avoid duplicating complex decode operations:
```
SET @decoded_data = decode(encode(*, 'base64'), 'proto', 'schema', 'schema.desc', 'message.proto', 'Message')
SELECT @decoded_data.sensor_id, @decoded_data.reading FROM 'device/protobuf'
WHERE @decoded_data.reading > 100
```
Without SET variables, you would need to repeat the decode function three times, which exceeds the function call limits.
**Multiple variables**
You can define multiple variables in a single SET clause by separating them with commas:
```
SET @user_data = get_user_properties(device_id), @threshold = 50
SELECT @user_data.name, temp_fahrenheit FROM 'sensors/+'
WHERE temp_fahrenheit > @threshold AND @user_data.active = true
```
**Using variables in substitution templates**
Variables can also be used in action substitution templates, allowing you to reuse computed values across both the SQL statement and rule actions.
SQL statement:
```
SET @temp_celsius = (temp_fahrenheit - 32) * 5 / 9
SELECT @temp_celsius AS celsius, humidity FROM 'device/data'
```
Action configuration:
```
{
"s3": {
"roleArn": "arn:aws:iam::123456789012:role/testRuleRole",
"bucketName": "bucket",
"key": "temperature-data/${device_id}/temp-${@temp_celsius}C.json"
}
}
```
In this example, the SET variable `@temp_celsius` is used in a substitution template to construct the key field of the S3 action.
**Non-JSON payload usage**
SET variables does not support non-JSON payloads directly, so the payload must be encoded or decoded first:
```
SET @encoded_payload = encode(*, 'base64')
SELECT @encoded_payload AS raw_data FROM 'device/binary'
```
for more information on how to work with non-JSON payloads, reference [Working with binary payloads](binary-payloads.md)
## SET clause limits
The following limits apply to SET variables:
+ Maximum of 10 unique variables per SQL statement
+ Maximum variable value size of 128 KiB (minified UTF-8 JSON string)
+ Maximum total value size of 128 KiB for all variables
+ Variable names limited to 64 characters
+ Variables can accept JSON payloads directly as is (non-JSON payloads must first be encoded/decoded)
# WHERE clause
The WHERE clause determines if the actions specified by a rule are carried out. If the WHERE clause evaluates to true, the rule actions are performed. Otherwise, the rule actions are not performed.
The WHERE clause supports [Data types](iot-sql-data-types.md), [Operators](iot-sql-operators.md), [Functions](iot-sql-functions.md), [Literals](iot-sql-literals.md), [Case statements](iot-sql-case.md), [JSON extensions](iot-sql-json.md), [Variables](iot-sql-set.md#iot-sql-set-usage), and [Nested object queries](iot-sql-nested-queries.md).
**Example:**
Incoming payload published on `topic/subtopic`: `{"color":"red", "temperature":40}`.
SQL: `SELECT color AS my_color FROM 'topic/subtopic' WHERE temperature > 50 AND color <> 'red'`.
In this case, the rule will be triggered, but the actions specified by the rule will not be performed. There will be no outgoing payload.
You can use functions and operators in the WHERE clause. However, you cannot reference any aliases created with the AS keyword in the SELECT. The WHERE clause is evaluated first, to determine if SELECT is evaluated.
**Example with non-JSON payload:**
Incoming non-JSON payload published on `topic/subtopic`: `80`
SQL: ``SELECT decode(encode(*, 'base64'), 'base64') AS value FROM 'topic/subtopic' WHERE decode(encode(*, 'base64'), 'base64') > 50`
In this case, the rule will be triggered, and the actions specified by the rule will be performed. The outgoing payload will be transformed by the SELECT clause as a JSON payload `{"value":80}`.
# Data types
The AWS IoT rules engine supports all JSON data types.
**Supported data types**
| Type | Meaning |
| --- | --- |
| Int | A discrete Int. 34 digits maximum. |
| Decimal | A `Decimal` with a precision of 34 digits, with a minimum non-zero magnitude of 1E-999 and a maximum magnitude 9.999...E999. Some functions return `Decimal` values with double precision rather than 34-digit precision. With SQL V2 (2016-03-23), numeric values that are whole numbers, such as `10.0`, are processed as an `Int` value (`10`) instead of the expected `Decimal` value (`10.0`). To reliably process whole number numeric values as `Decimal` values, use SQL V1 (2015-10-08) for the rule query statement. |
| Boolean | True or False. |
| String | A UTF-8 string. |
| Array | A series of values that don't have to have the same type. |
| Object | A JSON value consisting of a key and a value. Keys must be strings. Values can be any type. |
| Null | Null as defined by JSON. It's an actual value that represents the absence of a value. You can explicitly create a Null value by using the Null keyword in your SQL statement. For example: "SELECT NULL AS n FROM 'topic/subtopic'" |
| Undefined | Not a value. This isn't explicitly representable in JSON except by omitting the value. For example, in the object `{"foo": null}`, the key "foo" returns NULL, but the key "bar" returns `Undefined`. Internally, the SQL language treats `Undefined` as a value, but it isn't representable in JSON, so when serialized to JSON, the results are `Undefined`. {"foo":null, "bar":undefined} is serialized to JSON as: {"foo":null} Similarly, `Undefined` is converted to an empty string when serialized by itself. Functions called with invalid arguments (for example, wrong types, wrong number of arguments, and so on) return `Undefined`. |
## Conversions
The following table lists the results when a value of one type is converted to another type (when a value of the incorrect type is given to a function). For example, if the absolute value function "abs" (which expects an `Int` or `Decimal`) is given a `String`, it attempts to convert the `String` to a `Decimal`, following these rules. In this case, 'abs("-5.123")' is treated as 'abs(-5.123)'.
**Note**
There are no attempted conversions to `Array`, `Object`, `Null`, or `Undefined`.
**To decimal**
| Argument type | Result |
| --- | --- |
| Int | A Decimal with no decimal point. |
| Decimal | The source value. |
| Boolean | Undefined. (You can explicitly use the cast function to transform true = 1.0, false = 0.0.) |
| String | The SQL engine tries to parse the string as a Decimal. AWS IoT attempts to parse strings matching the regular expression:^-?\$1d\$1(\$1.\$1d\$1)?((?i)E-?\$1d\$1)?\$1. "0", "-1.2", "5E-12" are all examples of strings that are converted automatically to Decimals. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Null. |
| Undefined | Undefined. |
**To int**
| Argument type | Result |
| --- | --- |
| Int | The source value. |
| Decimal | The source value rounded to the nearest Int. |
| Boolean | Undefined. (You can explicitly use the cast function to transform true = 1.0, false = 0.0.) |
| String | The SQL engine tries to parse the string as a Decimal. AWS IoT attempts to parse strings matching the regular expression:^-?\$1d\$1(\$1.\$1d\$1)?((?i)E-?\$1d\$1)?\$1. "0", "-1.2", "5E-12" are all examples of strings that are converted automatically to Decimals. AWS IoT attempts to convert the String to a Decimal, and then truncates the decimal places of that Decimal to make an Int. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Null. |
| Undefined | Undefined. |
**To Boolean**
| Argument type | Result |
| --- | --- |
| Int | Undefined. (You can explicitly use the cast function to transform 0 = False, any\$1nonzero\$1value = True.) |
| Decimal | Undefined. (You can explicitly use the cast function to transform 0 = False, any\$1nonzero\$1value = True.) |
| Boolean | The original value. |
| String | "true"=True and "false"=False (case insensitive). Other string values are Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
**To string**
| Argument type | Result |
| --- | --- |
| Int | A string representation of the Int in standard notation. |
| Decimal | A string representing the Decimal value, possibly in scientific notation. |
| Boolean | "true" or "false". All lowercase. |
| String | The original value. |
| Array | The Array serialized to JSON. The resultant string is a comma-separated list, enclosed in square brackets. A String is quoted. A Decimal, Int, Boolean, and Null is not. |
| Object | The object serialized to JSON. The resultant string is a comma-separated list of key-value pairs and begins and ends with curly braces. A String is quoted. A Decimal, Int, Boolean, and Null is not. |
| Null | Undefined. |
| Undefined | Undefined. |
# Operators
The following operators can be used in SELECT and WHERE clauses.
## AND operator
Returns a `Boolean` result. Performs a logical AND operation. Returns true if left and right operands are true. Otherwise, returns false. `Boolean` operands or case insensitive "true" or "false" string operands are required.
*Syntax:* ` expression AND expression`.
**AND operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Boolean | Boolean | Boolean. True if both operands are true. Otherwise, false. |
| String/Boolean | String/Boolean | If all strings are "true" or "false" (case insensitive), they are converted to Boolean and processed normally as boolean AND boolean. |
| Other value | Other value | Undefined. |
## OR operator
Returns a `Boolean` result. Performs a logical OR operation. Returns true if either the left or the right operands are true. Otherwise, returns false. `Boolean` operands or case insensitive "true" or "false" string operands are required.
*Syntax:* ` expression OR expression`.
**OR operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Boolean | Boolean | Boolean. True if either operand is true. Otherwise, false. |
| String/Boolean | String/Boolean | If all strings are "true" or "false" (case insensitive), they are converted to Booleans and processed normally as boolean OR boolean. |
| Other value | Other value | Undefined. |
## NOT operator
Returns a `Boolean` result. Performs a logical NOT operation. Returns true if the operand is false. Otherwise, returns true. A `Boolean` operand or case insensitive "true" or "false" string operand is required.
*Syntax:* `NOT expression`.
**NOT operator**
| Operand | Output |
| --- | --- |
| Boolean | Boolean. True if operand is false. Otherwise, true. |
| String | If string is "true" or "false" (case insensitive), it is converted to the corresponding Boolean value, and the opposite value is returned. |
| Other value | Undefined. |
## IN operator
Returns a `Boolean` result. You can use the IN operator in a WHERE clause to check if a value matches any value in an array. It returns true if the match is found, and false otherwise.
*Syntax:* ` expression IN expression`.
**IN operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int/Decimal/String/Array/Object | Array | True if the Integer/Decimal/String/Array/Object element is found in the array. Otherwise, false. |
*Example:*
```
SQL: "select * from 'a/b' where 3 in arr"
JSON: {"arr":[1, 2, 3, "three", 5.7, null]}
```
In this example, the condition clause `where 3 in arr` will evaluate to true because 3 is present in the array named `arr`. Hence in the SQL statement, `select * from 'a/b'` will execute. This example also shows that the array can be heterogeneous.
## EXISTS operator
Returns a `Boolean` result. You can use the EXISTS operator in a conditional clause to test for the existence of elements in a subquery. It returns true if the subquery returns one or more elements and false if the subquery returns no elements.
*Syntax:* ` expression`.
*Example:*
```
SQL: "select * from 'a/b' where exists (select * from arr as a where a = 3)"
JSON: {"arr":[1, 2, 3]}
```
In this example, the condition clause `where exists (select * from arr as a where a = 3)` will evaluate to true because 3 is present in the array named `arr`. Hence in the SQL statement, `select * from 'a/b'` will execute.
*Example:*
```
SQL: select * from 'a/b' where exists (select * from e as e where foo = 2)
JSON: {"foo":4,"bar":5,"e":[{"foo":1},{"foo":2}]}
```
In this example, the condition clause `where exists (select * from e as e where foo = 2)` will evaluate to true because the array `e` within the JSON object contains the object `{"foo":2}`. Hence in the SQL statement, `select * from 'a/b'` will execute.
## > operator
Returns a `Boolean` result. Returns true if the left operand is greater than the right operand. Both operands are converted to a `Decimal`, and then compared.
*Syntax:* `expression > expression`.
**> operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | Boolean. True if the left operand is greater than the right operand. Otherwise, false. |
| String/Int/Decimal | String/Int/Decimal | If all strings can be converted to Decimal, then Boolean. Returns true if the left operand is greater than the right operand. Otherwise, false. |
| Other value | Undefined. | Undefined. |
## >= operator
Returns a `Boolean` result. Returns true if the left operand is greater than or equal to the right operand. Both operands are converted to a `Decimal`, and then compared.
*Syntax:* `expression >= expression`.
**>= operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | Boolean. True if the left operand is greater than or equal to the right operand. Otherwise, false. |
| String/Int/Decimal | String/Int/Decimal | If all strings can be converted to Decimal, then Boolean. Returns true if the left operand is greater than or equal to the right operand. Otherwise, false. |
| Other value | Undefined. | Undefined. |
## < operator
Returns a `Boolean` result. Returns true if the left operand is less than the right operand. Both operands are converted to a `Decimal`, and then compared.
*Syntax:* `expression < expression`.
**< operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | Boolean. True if the left operand is less than the right operand. Otherwise, false. |
| String/Int/Decimal | String/Int/Decimal | If all strings can be converted to Decimal, then Boolean. Returns true if the left operand is less than the right operand. Otherwise, false. |
| Other value | Undefined | Undefined |
## <= operator
Returns a `Boolean` result. Returns true if the left operand is less than or equal to the right operand. Both operands are converted to a `Decimal`, and then compared.
*Syntax:* `expression <= expression`.
**<= operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | Boolean. True if the left operand is less than or equal to the right operand. Otherwise, false. |
| String/Int/Decimal | String/Int/Decimal | If all strings can be converted to Decimal, then Boolean. Returns true if the left operand is less than or equal to the right operand. Otherwise, false. |
| Other value | Undefined | Undefined |
## <> operator
Returns a `Boolean` result. Returns true if both left and right operands are not equal. Otherwise, returns false.
*Syntax:* ` expression <> expression`.
**<> operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | True if left operand is not equal to right operand. Otherwise, false. |
| Decimal | Decimal | True if left operand is not equal to right operand. Otherwise, false.Int is converted to Decimal before being compared. |
| String | String | True if left operand is not equal to right operand. Otherwise, false. |
| Array | Array | True if the items in each operand are not equal and not in the same order. Otherwise, false |
| Object | Object | True if the keys and values of each operand are not equal. Otherwise, false. The order of keys/values is unimportant. |
| Null | Null | False. |
| Any value | Undefined | Undefined. |
| Undefined | Any value | Undefined. |
| Mismatched type | Mismatched type | True. |
## = operator
Returns a `Boolean` result. Returns true if both left and right operands are equal. Otherwise, returns false.
*Syntax:* ` expression = expression`.
**= operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | True if left operand is equal to right operand. Otherwise, false. |
| Decimal | Decimal | True if left operand is equal to right operand. Otherwise, false.Int is converted to Decimal before being compared. |
| String | String | True if left operand is equal to right operand. Otherwise, false. |
| Array | Array | True if the items in each operand are equal and in the same order. Otherwise, false. |
| Object | Object | True if the keys and values of each operand are equal. Otherwise, false. The order of keys/values is unimportant. |
| Any value | Undefined | Undefined. |
| Undefined | Any value | Undefined. |
| Mismatched type | Mismatched type | False. |
## \$1 operator
The "\$1" is an overloaded operator. It can be used for string concatenation or addition.
*Syntax:* ` expression + expression`.
**\$1 operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| String | Any value | Converts the right operand to a string and concatenates it to the end of the left operand. |
| Any value | String | Converts the left operand to a string and concatenates the right operand to the end of the converted left operand. |
| Int | Int | Int value. Adds operands together. |
| Int/Decimal | Int/Decimal | Decimal value. Adds operands together. |
| Other value | Other value | Undefined. |
## - operator
Subtracts the right operand from the left operand.
*Syntax:* ` expression - expression`.
**- operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int value. Subtracts right operand from left operand. |
| Int/Decimal | Int/Decimal | Decimal value. Subtracts right operand from left operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals correctly, a Decimal value is returned. Subtracts right operand from left operand. Otherwise, returns Undefined. |
| Other value | Other value | Undefined. |
| Other value | Other value | Undefined. |
## \$1 operator
Multiplies the left operand by the right operand.
*Syntax:* ` expression * expression`.
**\$1 operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int value. Multiplies the left operand by the right operand. |
| Int/Decimal | Int/Decimal | Decimal value. Multiplies the left operand by the right operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals correctly, a Decimal value is returned. Multiplies the left operand by the right operand. Otherwise, returns Undefined. |
| Other value | Other value | Undefined. |
## / operator
Divides the left operand by the right operand.
*Syntax:* ` expression / expression`.
**/ operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int value. Divides the left operand by the right operand. |
| Int/Decimal | Int/Decimal | Decimal value. Divides the left operand by the right operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals correctly, a Decimal value is returned. Divides the left operand by the right operand. Otherwise, returns Undefined. |
| Other value | Other value | Undefined. |
## % operator
Returns the remainder from dividing the left operand by the right operand.
*Syntax:* ` expression % expression`.
**% operator**
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int value. Returns the remainder from dividing the left operand by the right operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals correctly, a Decimal value is returned. Returns the remainder from dividing the left operand by the right operand. Otherwise, Undefined. |
| Other value | Other value | Undefined. |
# Functions
You can use the following built-in functions in the SELECT or WHERE clauses of your SQL expressions.
The following external functions are billed equivalent to that of a rule action: [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-func-aws-lambda), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-dynamodb), [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-registry_data), and [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-function-get-thing-shadow). You also get billed for the [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-sql-decode-base64) function only when you are [decoding a Protobuf message to JSON](https://docs.aws.amazon.com//iot/latest/developerguide/binary-payloads.html#binary-payloads-protobuf). For more details, refer to the [AWS IoT Core pricing page](https://aws.amazon.com/iot-core/pricing/).
## abs(Decimal)
Returns the absolute value of a number. Supported by SQL version 2015-10-08 and later.
Example: `abs(-5)` returns 5.
****
| Argument type | Result |
| --- | --- |
| Int | Int, the absolute value of the argument. |
| Decimal | Decimal, the absolute value of the argument. |
| Boolean | Undefined. |
| String | Decimal. The result is the absolute value of the argument. If the string cannot be converted, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## accountid()
Returns the ID of the account that owns this rule as a `String`. Supported by SQL version 2015-10-08 and later.
Example:
`accountid() ` = "123456789012"
## acos(Decimal)
Returns the inverse cosine of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `acos(0)` = 1.5707963267948966
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the inverse cosine of the argument. Imaginary results are returned as Undefined. |
| Decimal | Decimal (with double precision), the inverse cosine of the argument. Imaginary results are returned as Undefined. |
| Boolean | Undefined. |
| String | Decimal, the inverse cosine of the argument. If the string cannot be converted, the result is Undefined. Imaginary results are returned as Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## asin(Decimal)
Returns the inverse sine of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `asin(0)` = 0.0
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the inverse sine of the argument. Imaginary results are returned as Undefined. |
| Decimal | Decimal (with double precision), the inverse sine of the argument. Imaginary results are returned as Undefined. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the inverse sine of the argument. If the string cannot be converted, the result is Undefined. Imaginary results are returned as Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## atan(Decimal)
Returns the inverse tangent of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `atan(0)` = 0.0
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the inverse tangent of the argument. Imaginary results are returned as Undefined. |
| Decimal | Decimal (with double precision), the inverse tangent of the argument. Imaginary results are returned as Undefined. |
| Boolean | Undefined. |
| String | Decimal, the inverse tangent of the argument. If the string cannot be converted, the result is Undefined. Imaginary results are returned as Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## atan2(Decimal, Decimal)
Returns the angle, in radians, between the positive x-axis and the (x, y) point defined in the two arguments. The angle is positive for counter-clockwise angles (upper half-plane, y > 0), and negative for clockwise angles (lower half-plane, y < 0). `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `atan2(1, 0)` = 1.5707963267948966
****
| Argument type | Argument type | Result |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | Decimal (with double precision), the angle between the x-axis and the specified (x,y) point. |
| Int/Decimal/String | Int/Decimal/String | Decimal, the inverse tangent of the point described. If a string cannot be converted, the result is Undefined. |
| Other value | Other value | Undefined. |
## aws\$1lambda(functionArn, inputJson)
Calls the specified Lambda function passing `inputJson` to the Lambda function and returns the JSON generated by the Lambda function.
**Arguments**
| Argument | Description |
| --- | --- |
| functionArn | The ARN of the Lambda function to call. The Lambda function must return JSON data. |
| inputJson | The JSON input passed to the Lambda function. To pass nested object queries and literals, you must use SQL version 2016-03-23. |
You must grant AWS IoT `lambda:InvokeFunction` permissions to invoke the specified Lambda function. The following example shows how to grant the `lambda:InvokeFunction` permission using the AWS CLI:
```
aws lambda add-permission --function-name "function_name"
--region "region"
--principal iot.amazonaws.com
--source-arn arn:aws:iot:us-east-1:account_id:rule/rule_name
--source-account "account_id"
--statement-id "unique_id"
--action "lambda:InvokeFunction"
```
The following are the arguments for the **add-permission** command:
--function-name
Name of the Lambda function. You add a new permission to update the function's resource policy.
--region
The AWS Region of your account.
--principal
The principal who is getting the permission. This should be `iot.amazonaws.com` to allow AWS IoT permission to call a Lambda function.
--source-arn
The ARN of the rule. You can use the **get-topic-rule** AWS CLI command to get the ARN of a rule.
--source-account
The AWS account where the rule is defined.
--statement-id
A unique statement identifier.
--action
The Lambda action that you want to allow in this statement. To allow AWS IoT to invoke a Lambda function, specify `lambda:InvokeFunction`.
**Important**
If you add a permission for an AWS IoT principal without providing the `source-arn` or `source-account`, any AWS account that creates a rule with your Lambda action can trigger rules to invoke your Lambda function from AWS IoT. For more information, see [Lambda Permission Model](https://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html).
Given a JSON message payload like:
```
{
"attribute1": 21,
"attribute2": "value"
}
```
The `aws_lambda` function can be used to call Lambda function as follows.
```
SELECT
aws_lambda("arn:aws:lambda:us-east-1:account_id:function:lambda_function", {"payload":attribute1}) as output FROM 'topic-filter'
```
If you want to pass the full MQTT message payload, you can specify the JSON payload using '\$1', such as the following example.
```
SELECT
aws_lambda("arn:aws:lambda:us-east-1:account_id:function:lambda_function", *) as output FROM 'topic-filter'
```
`payload.inner.element` selects data from messages published on topic 'topic/subtopic'.
`some.value` selects data from the output that's generated by the Lambda function.
**Note**
The rules engine limits the execution duration of Lambda functions. Lambda function calls from rules should be completed within 2000 milliseconds.
## bitand(Int, Int)
Performs a bitwise AND on the bit representations of the two `Int`(-converted) arguments. Supported by SQL version 2015-10-08 and later.
Example: `bitand(13, 5)` = 5
****
| Argument type | Argument type | Result |
| --- | --- | --- |
| Int | Int | Int, a bitwise AND of the two arguments. |
| Int/Decimal | Int/Decimal | Int, a bitwise AND of the two arguments. All non-Int numbers are rounded down to the nearest Int. If any of the arguments cannot be converted to an Int, the result is Undefined. |
| Int/Decimal/String | Int/Decimal/String | Int, a bitwise AND of the two arguments. All strings are converted to decimals and are rounded down to the nearest Int. If the conversion fails, the result is Undefined. |
| Other value | Other value | Undefined. |
## bitor(Int, Int)
Performs a bitwise OR of the bit representations of the two arguments. Supported by SQL version 2015-10-08 and later.
Example: `bitor(8, 5)` = 13
****
| Argument type | Argument type | Result |
| --- | --- | --- |
| Int | Int | Int, the bitwise OR of the two arguments. |
| Int/Decimal | Int/Decimal | Int, the bitwise OR of the two arguments. All non-Int numbers are rounded down to the nearest Int. If the conversion fails, the result is Undefined. |
| Int/Decimal/String | Int/Decimal/String | Int, the bitwise OR on the two arguments. All strings are converted to decimals and rounded down to the nearest Int. If the conversion fails, the result is Undefined. |
| Other value | Other value | Undefined. |
## bitxor(Int, Int)
Performs a bitwise XOR on the bit representations of the two `Int`(-converted) arguments. Supported by SQL version 2015-10-08 and later.
Example:`bitor(13, 5)` = 8
****
| Argument type | Argument type | Result |
| --- | --- | --- |
| Int | Int | Int, a bitwise XOR on the two arguments. |
| Int/Decimal | Int/Decimal | Int, a bitwise XOR on the two arguments. Non-Int numbers are rounded down to the nearest Int. |
| Int/Decimal/String | Int/Decimal/String | Int, a bitwise XOR on the two arguments. strings are converted to decimals and rounded down to the nearest Int. If any conversion fails, the result is Undefined. |
| Other value | Other value | Undefined. |
## bitnot(Int)
Performs a bitwise NOT on the bit representations of the `Int`(-converted) argument. Supported by SQL version 2015-10-08 and later.
Example: `bitnot(13)` = 2
****
| Argument type | Result |
| --- | --- |
| Int | Int, a bitwise NOT of the argument. |
| Decimal | Int, a bitwise NOT of the argument. The Decimal value is rounded down to the nearest Int. |
| String | Int, a bitwise NOT of the argument. Strings are converted to decimals and rounded down to the nearest Int. If any conversion fails, the result is Undefined. |
| Other value | Other value. |
## cast()
Converts a value from one data type to another. Cast behaves mostly like the standard conversions, with the addition of the ability to cast numbers to or from Booleans. If AWS IoT cannot determine how to cast one type to another, the result is `Undefined`. Supported by SQL version 2015-10-08 and later. Format: cast(*value* as *type*).
Example:
`cast(true as Int) ` = 1
The following keywords might appear after "as" when calling `cast`:
**For SQL version 2015-10-08 and 2016-03-23**
| Keyword | Result |
| --- | --- |
| String | Casts value to String. |
| Nvarchar | Casts value to String. |
| Text | Casts value to String. |
| Ntext | Casts value to String. |
| varchar | Casts value to String. |
| Int | Casts value to Int. |
| Integer | Casts value to Int. |
| Double | Casts value to Decimal (with double precision). |
**Additionally, for SQL version 2016-03-23**
| Keyword | Result |
| --- | --- |
| Decimal | Casts value to Decimal. |
| Bool | Casts value to Boolean. |
| Boolean | Casts value to Boolean. |
Casting rules:
**Cast to decimal**
| Argument type | Result |
| --- | --- |
| Int | A Decimal with no decimal point. |
| Decimal | The source value. With SQL V2 (2016-03-23), numeric values that are whole numbers, such as `10.0`, return an `Int` value (`10`) instead of the expected `Decimal` value (`10.0`). To reliably cast whole number numeric values as `Decimal` values, use SQL V1 (2015-10-08) for the rule query statement. |
| Boolean | true = 1.0, false = 0.0. |
| String | Tries to parse the string as a Decimal. AWS IoT attempts to parse strings matching the regex: ^-?\$1d\$1(\$1.\$1d\$1)?((?i)E-?\$1d\$1)?\$1. "0", "-1.2", "5E-12" are all examples of strings that are converted automatically to decimals. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
**Cast to int**
| Argument type | Result |
| --- | --- |
| Int | The source value. |
| Decimal | The source value, rounded down to the nearest Int. |
| Boolean | true = 1.0, false = 0.0. |
| String | Tries to parse the string as a Decimal. AWS IoT attempts to parse strings matching the regex: ^-?\$1d\$1(\$1.\$1d\$1)?((?i)E-?\$1d\$1)?\$1. "0", "-1.2", "5E-12" are all examples of strings that are converted automatically to decimals. AWS IoT attempts to convert the string to a Decimal and round down to the nearest Int. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
**Cast to `Boolean`**
| Argument type | Result |
| --- | --- |
| Int | 0 = False, any\$1nonzero\$1value = True. |
| Decimal | 0 = False, any\$1nonzero\$1value = True. |
| Boolean | The source value. |
| String | "true" = True and "false" = False (case insensitive). Other string values = Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
**Cast to string**
| Argument type | Result |
| --- | --- |
| Int | A string representation of the Int, in standard notation. |
| Decimal | A string representing the Decimal value, possibly in scientific notation. |
| Boolean | "true" or "false", all lowercase. |
| String | The source value. |
| Array | The array serialized to JSON. The result string is a comma-separated list enclosed in square brackets. String is quoted. Decimal, Int, and Boolean are not. |
| Object | The object serialized to JSON. The JSON string is a comma-separated list of key-value pairs and begins and ends with curly braces. String is quoted. Decimal, Int, Boolean, and Null are not. |
| Null | Undefined. |
| Undefined | Undefined. |
## ceil(Decimal)
Rounds the given `Decimal` up to the nearest `Int`. Supported by SQL version 2015-10-08 and later.
Examples:
`ceil(1.2)` = 2
`ceil(-1.2)` = -1
****
| Argument type | Result |
| --- | --- |
| Int | Int, the argument value. |
| Decimal | Int, the Decimal value rounded up to the nearest Int. |
| String | Int. The string is converted to Decimal and rounded up to the nearest Int. If the string cannot be converted to a Decimal, the result is Undefined. |
| Other value | Undefined. |
## chr(String)
Returns the ASCII character that corresponds to the given `Int` argument. Supported by SQL version 2015-10-08 and later.
Examples:
`chr(65)` = "A".
`chr(49)` = "1".
****
| Argument type | Result |
| --- | --- |
| Int | The character corresponding to the specified ASCII value. If the argument is not a valid ASCII value, the result is Undefined. |
| Decimal | The character corresponding to the specified ASCII value. The Decimal argument is rounded down to the nearest Int. If the argument is not a valid ASCII value, the result is Undefined. |
| Boolean | Undefined. |
| String | If the String can be converted to a Decimal, it is rounded down to the nearest Int. If the argument is not a valid ASCII value, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Other value | Undefined. |
## clientid()
Returns the ID of the MQTT client sending the message, or `n/a` if the message wasn't sent over MQTT. Supported by SQL version 2015-10-08 and later.
Example:
`clientid() ` = "123456789012"
## concat()
Concatenates arrays or strings. This function accepts any number of arguments and returns a `String` or an `Array`. Supported by SQL version 2015-10-08 and later.
Examples:
`concat() ` = `Undefined`.
`concat(1) ` = "1".
`concat([1, 2, 3], 4)` = [1, 2, 3, 4].
`concat([1, 2, 3], "hello")` = [1, 2, 3, "hello"]
`concat("con", "cat")` = "concat"
`concat(1, "hello")` = "1hello"
`concat("he","is","man")` = "heisman"
`concat([1, 2, 3], "hello", [4, 5, 6])` = [1, 2, 3, "hello", 4, 5, 6]
****
| Number of arguments | Result |
| --- | --- |
| 0 | Undefined. |
| 1 | The argument is returned unmodified. |
| 2\$1 | If any argument is an `Array`, the result is a single array containing all of the arguments. If no arguments are arrays, and at least one argument is a `String`, the result is the concatenation of the `String` representations of all the arguments. Arguments are converted to strings using the standard conversions previously listed. |
## cos(Decimal)
Returns the cosine of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example:
`cos(0)` = 1.
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the cosine of the argument. Imaginary results are returned as Undefined. |
| Decimal | Decimal (with double precision), the cosine of the argument. Imaginary results are returned as Undefined. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the cosine of the argument. If the string cannot be converted to a Decimal, the result is Undefined. Imaginary results are returned as Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## cosh(Decimal)
Returns the hyperbolic cosine of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `cosh(2.3)` = 5.037220649268761.
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the hyperbolic cosine of the argument. Imaginary results are returned as Undefined. |
| Decimal | Decimal (with double precision), the hyperbolic cosine of the argument. Imaginary results are returned as Undefined. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the hyperbolic cosine of the argument. If the string cannot be converted to a Decimal, the result is Undefined. Imaginary results are returned as Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## decode(value, decodingScheme)
Use the `decode` function to decode an encoded value. If the decoded string is a JSON document, an addressable object is returned. Otherwise, the decoded string is returned as a string. The function returns NULL if the string cannot be decoded. This function supports decoding base64-encoded strings and Protocol Buffer (protobuf) message format.
Supported by SQL version 2016-03-23 and later.
value
A string value or any of the valid expressions, as defined in [AWS IoT SQL reference](iot-sql-reference.md), that return a string.
decodingScheme
A literal string representing the scheme used to decode the value. Currently, only `'base64'` and `'proto'` are supported.
### Decoding base64-encoded strings
In this example, the message payload includes an encoded value.
```
{
encoded_temp: "eyAidGVtcGVyYXR1cmUiOiAzMyB9Cg=="
}
```
The `decode` function in this SQL statement decodes the value in the message payload.
```
SELECT decode(encoded_temp,"base64").temperature AS temp from 'topic/subtopic'
```
Decoding the `encoded_temp` value results in the following valid JSON document, which allows the SELECT statement to read the temperature value.
```
{ "temperature": 33 }
```
The result of the SELECT statement in this example is shown here.
```
{ "temp": 33 }
```
If the decoded value was not a valid JSON document, the decoded value would be returned as a string.
### Decoding protobuf message payload
You can use the decode SQL function to configure a Rule that can decode your protobuf message payload. For more information, see [Decoding protobuf message payloads](binary-payloads.md#binary-payloads-protobuf).
**Important**
If you omit the `source‐arn` or `source‐account` when setting permissions for an AWS IoT principal, any AWS account can invoke your Decode function through other AWS IoT rules. To secure your function, see [Bucket policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html) in the *Amazon Simple Storage Service User Guide*.
The function signature looks like the following:
```
decode(, 'proto', '', '', '', '')
```
`ENCODED DATA`
Specifies the protobuf-encoded data to be decoded. If the entire message sent to the Rule is protobuf-encoded data, you can reference the raw binary incoming payload using `*`. Otherwise, this field must be a base-64 encoded JSON string and a reference to the string can be passed in directly.
1) To decode a raw binary protobuf incoming payload:
```
decode(*, 'proto', ...)
```
2) To decode a protobuf-encoded message represented by a base64-encoded string 'a.b':
```
decode(a.b, 'proto', ...)
```
`proto`
Specifies the data to be decoded in a protobuf message format. If you specify `base64` instead of `proto`, this function will decode base64-encoded strings as JSON.
`S3 BUCKET NAME`
The name of the Amazon S3 bucket where you’ve uploaded your `FileDescriptorSet` file.
`S3 OBJECT KEY`
The object key that specifies the `FileDescriptorSet` file within the Amazon S3 bucket.
`PROTO NAME`
The name of the `.proto` file (excluding the extension) from which the `FileDescriptorSet` file was generated.
`MESSAGE TYPE`
The name of the protobuf message structure within the `FileDescriptorSet` file, to which the data to be decoded should conform.
An example SQL expression using the decode SQL function can look like the following:
```
SELECT VALUE decode(*, 'proto', 's3-bucket', 'messageformat.desc', 'myproto', 'messagetype') FROM 'some/topic'
```
+ `*`
Represents a binary incoming payload, which conforms to the protobuf message type called `mymessagetype`.
+ `messageformat.desc`
The `FileDescriptorSet` file stored in an Amazon S3 bucket named `s3-bucket`.
+ `myproto`
The original `.proto` file used to generate the `FileDescriptorSet` file named `myproto.proto`.
+ `messagetype`
The message type called `messagetype` (along with any imported dependencies) as defined in `myproto.proto`.
## encode(value, encodingScheme)
Use the `encode` function to encode the payload, which potentially might be non-JSON data, into its string representation based on the encoding scheme. Supported by SQL version 2016-03-23 and later.
value
Any of the valid expressions, as defined in [AWS IoT SQL reference](iot-sql-reference.md). You can specify \$1 to encode the entire payload, regardless of whether it's in JSON format. If you supply an expression, the result of the evaluation is converted to a string before it is encoded.
encodingScheme
A literal string representing the encoding scheme you want to use. Currently, only `'base64'` is supported.
## endswith(String, String)
Returns a `Boolean` indicating whether the first `String` argument ends with the second `String` argument. If either argument is `Null` or `Undefined`, the result is `Undefined`. Supported by SQL version 2015-10-08 and later.
Example: `endswith("cat","at")` = true.
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| String | String | True if the first argument ends in the second argument. Otherwise, false. |
| Other value | Other value | Both arguments are converted to strings using the standard conversion rules. True if the first argument ends in the second argument. Otherwise, false. If either argument is Null or Undefined, the result is Undefined. |
## exp(Decimal)
Returns e raised to the `Decimal` argument. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `exp(1)` = e.
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), e ^ argument. |
| Decimal | Decimal (with double precision), e ^ argument. |
| String | Decimal (with double precision), e ^ argument. If the String cannot be converted to a Decimal, the result is Undefined. |
| Other value | Undefined. |
## floor(Decimal)
Rounds the given `Decimal` down to the nearest `Int`. Supported by SQL version 2015-10-08 and later.
Examples:
`floor(1.2)` = 1
`floor(-1.2)` = -2
****
| Argument type | Result |
| --- | --- |
| Int | Int, the argument value. |
| Decimal | Int, the Decimal value rounded down to the nearest Int. |
| String | Int. The string is converted to Decimal and rounded down to the nearest Int. If the string cannot be converted to a Decimal, the result is Undefined. |
| Other value | Undefined. |
## get
Extracts a value from a collection-like type (Array, String, Object). No conversion is applied to the first argument. Conversion applies as documented in the table to the second argument. Supported by SQL version 2015-10-08 and later.
Examples:
`get(["a", "b", "c"], 1) ` = "b"
`get({"a":"b"}, "a")` = "b"
`get("abc", 0)` = "a"
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| Array | Any Type (converted to Int) | The item at the 0-based index of the Array provided by the second argument (converted to Int). If the conversion is unsuccessful, the result is Undefined. If the index is outside the bounds of the Array (negative or >= array.length), the result is Undefined. |
| String | Any Type (converted to Int) | The character at the 0-based index of the string provided by the second argument (converted to Int). If the conversion is unsuccessful, the result is Undefined. If the index is outside the bounds of the string (negative or >= string.length), the result is Undefined. |
| Object | String (no conversion is applied) | The value stored in the first argument object corresponding to the string key provided as the second argument. |
| Other value | Any value | Undefined. |
## get\$1dynamodb(tableName, partitionKeyName, partitionKeyValue, sortKeyName, sortKeyValue, roleArn)
Retrieves data from a DynamoDB table. `get_dynamodb()` allows you to query a DynamoDB table while a rule is evaluated. You can filter or augment message payloads using data retrieved from DynamoDB. Supported by SQL version 2016-03-23 and later.
`get_dynamodb()` takes the following parameters:
tableName
The name of the DynamoDB table to query.
partitionKeyName
The name of the partition key. For more information, see [ DynamoDB Keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey).
partitionKeyValue
The value of the partition key used to identify a record. For more information, see [ DynamoDB Keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey).
sortKeyName
(Optional) The name of the sort key. This parameter is required only if the DynamoDB table queried uses a composite key. For more information, see [ DynamoDB Keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey).
sortKeyValue
(Optional) The value of the sort key. This parameter is required only if the DynamoDB table queried uses a composite key. For more information, see [ DynamoDB Keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey).
roleArn
The ARN of an IAM role that grants access to the DynamoDB table. The rules engine assumes this role to access the DynamoDB table on your behalf. Avoid using an overly permissive role. Grant the role only those permissions required by the rule. The following is an example policy that grants access to one DynamoDB table.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "dynamodb:GetItem",
"Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/table-name"
}
]
}
```
As an example of how to use `get_dynamodb()`, say you have a DynamoDB table that contains device ID and location information for all of your devices connected to AWS IoT. The following SELECT statement uses the `get_dynamodb()` function to retrieve the location for the specified device ID:
`SELECT *, get_dynamodb("InServiceDevices", "deviceId", id, "arn:aws:iam::12345678910:role/getdynamo").location AS location FROM 'some/topic' `
**Note**
You can call `get_dynamodb()` a maximum of one time per SQL statement. Calling `get_dynamodb()` multiple times in a single SQL statement causes the rule to terminate without invoking any actions.
## get\$1mqtt\$1property(name)
References any of the following MQTT5 headers: `contentType`, `payLoadFormatIndicator`, `responseTopic`, and `correlationData`. This function takes any of the following literal strings as an argument: `content_type`, `format_indicator`, `response_topic`, and `correlation_data`. For more information, see the following **Function arguments** table.
contentType
String: A UTF-8 encoded string that describes the content of the publishing message.
payLoadFormatIndicator
String: An Enum string value that indicates whether the payload is formatted as UTF-8. Valid values are `UNSPECIFIED_BYTES` and `UTF8_DATA`.
responseTopic
String: A UTF-8 encoded string that's used as the topic name for a response message. The response topic is used to describe the topic that the receiver should publish to as part of the request-response flow. The topic must not contain wildcard characters.
correlationData
String: The base64-encoded binary data used by the sender of the Request Message to identify which request the Response Message is for when it's received.
The following table shows the acceptable function arguments and their associated return types for the `get_mqtt_property` function:
**Function arguments**
| SQL | Returned data type (if present) | Returned data type (if not present) |
| --- | --- | --- |
| get\$1mqtt\$1property("format\$1indicator") | String (UNSPECIFIED\$1BYTES or UTF8\$1DATA) | String (UNSPECIFIED\$1BYTES) |
| get\$1mqtt\$1property("content\$1type") | String | Undefined |
| get\$1mqtt\$1property("response\$1topic") | String | Undefined |
| get\$1mqtt\$1property("correlation\$1data") | base64 encoded String | Undefined |
| get\$1mqtt\$1property("some\$1invalid\$1name") | Undefined | Undefined |
The following example Rules SQL references any of the following MQTT5 headers: `contentType`, `payLoadFormatIndicator`, `responseTopic`, and `correlationData`.
```
SELECT *, get_mqtt_property('content_type') as contentType,
get_mqtt_property('format_indicator') as payloadFormatIndicator,
get_mqtt_property('response_topic') as responseTopic,
get_mqtt_property('correlation_data') as correlationData
FROM 'some/topic'
```
## get\$1or\$1default(expression, defaultValue)
Returns the default value in the second parameter if specified or else returns undefined, when the expression in the first parameter returns null, undefined, or fails. Supported by SQL version 2016-03-23 and later.
**Important**
`get_or_default` does not support non-JSON payloads directly as is. If you are using a non-JSON payload, use the `encode` or `decode` functions.
`get_or_default()` takes the following parameters:
expression
Any valid expression containing [Data types](iot-sql-data-types.md), [Functions](#iot-sql-functions), [Literals](iot-sql-literals.md), [Variables](iot-sql-set.md#iot-sql-set-usage), [Nested object queries](iot-sql-nested-queries.md), or [JSON extensions](iot-sql-json.md).
defaultValue
(Optional) Any valid expression containing [Data types](iot-sql-data-types.md), [Functions](#iot-sql-functions), [Literals](iot-sql-literals.md), [Variables](iot-sql-set.md#iot-sql-set-usage), [Nested object queries](iot-sql-nested-queries.md), or [JSON extensions](iot-sql-json.md). This is the value to be returned whenever the first argument returns null, undefined, or fails.
Functions that fetch data from customer owned resources, such as get\$1secret, get\$1dynamodb, aws\$1lambda, get\$1thing\$1shadow, decode-protobuf, and machinelearning\$1predict, are not allowed for the defaultValue parameter.
The following table shows acceptable function arguments for each argument and their associated outputs:
| First argument | Second argument | Output |
| --- | --- | --- |
| Successful evaluation | Any value or not specified | The first argument value. |
| Undefined, Null, or failure | Any value including Undefined or Null | The second argument value. |
| Undefined, Null, or failure | not specified | Undefined |
**Examples:**
Example 1:
The following example provides a defaultValue value if a DynamoDB table or query fails:
```
SELECT
device_id,
get_or_default(
get_dynamodb("DeviceConfig", "deviceId", nonExistentId, "arn:aws:iam::123456789012:role/ROLE_NAME"),
{"mode": "standard", "timeout": 30, "enabled": true }
) as config
FROM 'device/telemetry'
```
Example 2:
The following example provides a safe default value "UNKNOWN" if status is undefined:
```
SELECT
get_or_default( CASE status
WHEN 'active' THEN 'GOOD'
WHEN 'inactive' THEN 'BAD'/
ELSE 'UNKNOWN'
END, 'UNKNOWN') as status_category
FROM 'topic/subtopic'
```
Example 3:
The following example shows how you can also use get\$1or\$1default with a single parameter. This is useful in scenarios where you may not have a clear default value, but you do not want your rule execution to fail.
```
SELECT
get_dynamodb("DeviceConfig", "deviceId", nonExistentId, "arn:aws:iam::123456789012:role/ROLE_NAME") as config
FROM 'device/telemetry'
```
If the DynamoDB lookup fails, the rule execution will fail, and no actions will be executed. If the following SQL is used instead:
```
SELECT
get_or_default(get_dynamodb("DeviceConfig", "deviceId", nonExistentId, "arn:aws:iam::123456789012:role/ROLE_NAME")) as config
FROM 'device/telemetry'
```
The get\$1or\$1default statement will evaluate to `Undefined`,so in this example the SELECT statement overall will evaluate to `{}` and any rule actions will be attempted.
**Important**
We recommend following these best practices to maintain security when using this function:
Avoid using hardcoded secrets in rule definitions including default values
Use AWS Secrets Manager for managing sensitive information
## get\$1registry\$1data(registryAPI, thingName, roleArn)
Retrieves AWS IoT thing registry data in an AWS IoT rule. You can read registry data (such as attributes, thing type, and thing groups a device belongs to) and use this information to filter, enrich, or dynamically route messages. Supported by SQL version 2016-03-23 and later.
`get_registry_data()` takes the following parameters:
registryAPI
The registry API being called. Valid values are `DescribeThing` and `ListThingGroupsForThing`. These values must be constant strings.
thingName
String: The name of the thing whose registry data you want to retrieve.
roleArn
String: A role ARN with `iot:DescribeThing` permission and/or `iot:ListThingGroupsForThing` permission based on the API being called.
The response format of the `get_registry_data` function is the same as the registry API called. For more information, see the [DescribeThing](https://docs.aws.amazon.com//iot/latest/apireference/API_DescribeThing.html) and [ListThingGroupsForThing](https://docs.aws.amazon.com//iot/latest/apireference/API_ListThingGroupsForThing.html) APIs.
Example:
You can retrieve thing type information to allow filtering the AWS IoT Core lifecycle event messages for things (with the thing name matching the MQTT client id) where thing type is `testenv`.
```
SELECT *
FROM '$aws/events/lifecycle/+'
WHERE
get_registry_data("DescribeThing",clientId,[roleArn]).thingTypeName='testenv'
```
Example:
You can retrieve thing attributes for a device with thing name `sensor1` for all messages sent by its gateway device `gateway1`.
```
SELECT *, get_registry_data("DescribeThing","sensor1",[roleArn]).attributes.temperature_threhold AS device1_tempthreshold
FROM home1/gateway1/sensor1/#
```
**Note**
You can call `get_registry_data()` a maximum of one time per SQL statement and substitution templates for actions and error actions.
## get\$1secret(secretId, secretType, key, roleArn)
Retrieves the value of the encrypted `SecretString` or `SecretBinary` field of the current version of a secret in [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/). For more information about creating and maintaining secrets, see [CreateSecret](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_CreateSecret.html), [UpdateSecret](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_UpdateSecret.html), and [PutSecretValue](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_PutSecretValue.html).
`get_secret()` takes the following parameters:
secretId
String: The Amazon Resource Name (ARN) or the friendly name of the secret to retrieve.
secretType
String: The secret type. Valid values: `SecretString` \$1 `SecretBinary`.
SecretString
+ For secrets that you create as JSON objects by using the APIs, the AWS CLI, or the AWS Secrets Manager console:
+ If you specify a value for the `key` parameter, this function returns the value of the specified key.
+ If you don't specify a value for the `key` parameter, this function returns the entire JSON object.
+ For secrets that you create as non-JSON objects by using the APIs or the AWS CLI:
+ If you specify a value for the `key` parameter, this function fails with an exception.
+ If you don't specify a value for the `key` parameter, this function returns the contents of the secret.
SecretBinary
+ If you specify a value for the `key` parameter, this function fails with an exception.
+ If you don't specify a value for the `key` parameter, this function returns the secret value as a base64-encoded UTF-8 string.
key
(Optional) String: The key name inside a JSON object stored in the `SecretString` field of a secret. Use this value when you want to retrieve only the value of a key stored in a secret instead of the entire JSON object.
If you specify a value for this parameter and the secret doesn't contain a JSON object inside its `SecretString` field, this function fails with an exception.
roleArn
String: A role ARN with `secretsmanager:GetSecretValue` and `secretsmanager:DescribeSecret` permissions.
**Note**
This function always returns the current version of the secret (the version with the `AWSCURRENT` tag). The AWS IoT rules engine caches each secret for up to 15 minutes. As a result, the rules engine can take up to 15 minutes to update a secret. This means that if you retrieve a secret up to 15 minutes after an update with AWS Secrets Manager, this function might return the previous version.
This function is not metered, but AWS Secrets Manager charges apply. Because of the secret caching mechanism, the rules engine occasionally calls AWS Secrets Manager. Because the rules engine is a fully distributed service, you might see multiple Secrets Manager API calls from the rules engine during the 15-minute caching window.
Examples:
You can use the `get_secret` function in an authentication header in an HTTPS rule action, as in the following API key authentication example.
```
"API_KEY": "${get_secret('API_KEY', 'SecretString', 'API_KEY_VALUE', 'arn:aws:iam::12345678910:role/getsecret')}"
```
For more information about the HTTPS rule action, see [HTTP](https-rule-action.md).
## get\$1thing\$1shadow(thingName, shadowName, roleArn)
Returns the specified shadow of the specified thing. Supported by SQL version 2016-03-23 and later.
thingName
String: The name of the thing whose shadow you want to retrieve.
shadowName
(Optional) String: The name of the shadow. This parameter is required only when referencing named shadows.
roleArn
String: A role ARN with `iot:GetThingShadow` permission.
Examples:
When used with a named shadow, provide the `shadowName` parameter.
```
SELECT * from 'topic/subtopic'
WHERE
get_thing_shadow("MyThing","MyThingShadow","arn:aws:iam::123456789012:role/AllowsThingShadowAccess")
.state.reported.alarm = 'ON'
```
When used with an unnamed shadow, omit the `shadowName` parameter.
```
SELECT * from 'topic/subtopic'
WHERE
get_thing_shadow("MyThing","arn:aws:iam::123456789012:role/AllowsThingShadowAccess")
.state.reported.alarm = 'ON'
```
## get\$1user\$1properties(userPropertyKey)
References User Properties, which is one type of property headers supported in MQTT5.
userProperty
String: A user property is a key-value pair. This function takes the key as an argument and returns an array of all values that match the associated key.
**Function arguments**
For the following User Properties in the message headers:
| Key | Value |
| --- | --- |
| some key | some value |
| a different key | a different value |
| some key | value with duplicate key |
The following table shows the expected SQL behavior:
| SQL | Returned data type | Returned data value |
| --- | --- | --- |
| get\$1user\$1properties('some key') | Array of String | ['some value', 'value with duplicate key'] |
| get\$1user\$1properties('other key') | Array of String | ['a different value'] |
| get\$1user\$1properties( ) | Array of key-value pair Objects | [\$1'"some key": "some value"'\$1, \$1"other key": "a different value"\$1, \$1"some key": "value with duplicate key"\$1] |
| get\$1user\$1properties('non-existent key') | Undefined | |
The following example Rules SQL references User Properties (a type of MQTT5 property header) into the payload:
```
SELECT *, get_user_properties('user defined property key') as userProperty
FROM 'some/topic'
```
## Hashing functions
AWS IoT provides the following hashing functions:
+ md2
+ md5
+ sha1
+ sha224
+ sha256
+ sha384
+ sha512
All hash functions expect one string argument. The result is the hashed value of that string. Standard string conversions apply to non-string arguments. All hash functions are supported by SQL version 2015-10-08 and later.
Examples:
`md2("hello")` = "a9046c73e00331af68917d3804f70655"
`md5("hello")` = "5d41402abc4b2a76b9719d911017c592"
## indexof(String, String)
Returns the first index (0-based) of the second argument as a substring in the first argument. Both arguments are expected as strings. Arguments that are not strings are subjected to standard string conversion rules. This function does not apply to arrays, only to strings. Supported by SQL version 2016-03-23 and later.
Examples:
`indexof("abcd", "bc") ` = 1
## isNull()
Returns true if the argument is the `Null` value. Supported by SQL version 2016-03-23 and later.
Examples:
`isNull(5) ` = false.
`isNull(Null) ` = true.
****
| Argument type | Result |
| --- | --- |
| Int | false |
| Decimal | false |
| Boolean | false |
| String | false |
| Array | false |
| Object | false |
| Null | true |
| Undefined | false |
## isUndefined()
Returns true if the argument is `Undefined`. Supported by SQL version 2016-03-23 and later.
Examples:
`isUndefined(5) ` = false.
`isUndefined(floor([1,2,3]))) ` = true.
****
| Argument type | Result |
| --- | --- |
| Int | false |
| Decimal | false |
| Boolean | false |
| String | false |
| Array | false |
| Object | false |
| Null | false |
| Undefined | true |
## length(String)
Returns the number of characters in the provided string. Standard conversion rules apply to non-`String` arguments. Supported by SQL version 2016-03-23 and later.
Examples:
`length("hi")` = 2
`length(false)` = 5
## ln(Decimal)
Returns the natural logarithm of the argument. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `ln(e)` = 1.
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the natural log of the argument. |
| Decimal | Decimal (with double precision), the natural log of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the natural log of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## log(Decimal)
Returns the base 10 logarithm of the argument. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `log(100)` = 2.0.
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the base 10 log of the argument. |
| Decimal | Decimal (with double precision), the base 10 log of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the base 10 log of the argument. If the String cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## lower(String)
Returns the lowercase version of the given `String`. Non-string arguments are converted to strings using the standard conversion rules. Supported by SQL version 2015-10-08 and later.
Examples:
`lower("HELLO")` = "hello".
`lower(["HELLO"])` = "[\$1"hello\$1"]".
## lpad(String, Int)
Returns the `String` argument, padded on the left side with the number of spaces specified by the second argument. The `Int` argument must be between 0 and 1000. If the provided value is outside of this valid range, the argument is set to the nearest valid value (0 or 1000). Supported by SQL version 2015-10-08 and later.
Examples:
`lpad("hello", 2)` = "` hello`".
`lpad(1, 3)` = "` 1`"
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| String | Int | String, the provided String padded on the left side with a number of spaces equal to the provided Int. |
| String | Decimal | The Decimal argument is rounded down to the nearest Int and the String is padded on the left with the specified number of spaces. |
| String | String | The second argument is converted to a Decimal, which is rounded down to the nearest Int, and the String is padded with the specified number spaces on the left. If the second argument cannot be converted to an Int, the result is Undefined. |
| Other value | Int/Decimal/String | The first value is converted to a String using the standard conversions, and then the LPAD function is applied on that String. If it cannot be converted, the result is Undefined. |
| Any value | Other value | Undefined. |
## ltrim(String)
Removes all leading white space (tabs and spaces) from the provided `String`. Supported by SQL version 2015-10-08 and later.
Example:
`Ltrim(" h i ")` = "hi ".
****
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int with all leading white space removed. |
| Decimal | The String representation of the Decimal with all leading white space removed. |
| Boolean | The String representation of the Boolean ("true" or "false") with all leading white space removed. |
| String | The argument with all leading white space removed. |
| Array | The String representation of the Array (using standard conversion rules) with all leading white space removed. |
| Object | The String representation of the Object (using standard conversion rules) with all leading white space removed. |
| Null | Undefined. |
| Undefined | Undefined. |
## machinelearning\$1predict(modelId, roleArn, record)
Use the `machinelearning_predict` function to make predictions using the data from an MQTT message based on an Amazon SageMaker AI model. Supported by SQL version 2015-10-08 and later. The arguments for the `machinelearning_predict` function are:
modelId
The ID of the model against which to run the prediction. The real-time endpoint of the model must be enabled.
roleArn
The IAM role that has a policy with `machinelearning:Predict` and `machinelearning:GetMLModel` permissions and allows access to the model against which the prediction is run.
record
The data to be passed into the SageMaker AI Predict API. This should be represented as a single layer JSON object. If the record is a multi-level JSON object, the record is flattened by serializing its values. For example, the following JSON:
```
{ "key1": {"innerKey1": "value1"}, "key2": 0}
```
would become:
```
{ "key1": "{\"innerKey1\": \"value1\"}", "key2": 0}
```
The function returns a JSON object with the following fields:
predictedLabel
The classification of the input based on the model.
details
Contains the following attributes:
PredictiveModelType
The model type. Valid values are REGRESSION, BINARY, MULTICLASS.
Algorithm
The algorithm used by SageMaker AI to make predictions. The value must be SGD.
predictedScores
Contains the raw classification score corresponding to each label.
predictedValue
The value predicted by SageMaker AI.
## mod(Decimal, Decimal)
Returns the remainder of the division of the first argument by the second argument. Equivalent to [remainder(Decimal, Decimal)](#iot-func-remainder). You can also use "%" as an infix operator for the same modulo functionality. Supported by SQL version 2015-10-08 and later.
Example: `mod(8, 3)` = 2.
****
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int, the first argument modulo the second argument. |
| Int/Decimal | Int/Decimal | Decimal, the first argument modulo the second operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals, the result is the first argument modulo the second argument. Otherwise, Undefined. |
| Other value | Other value | Undefined. |
## nanvl(AnyValue, AnyValue)
Returns the first argument if it is a valid `Decimal`. Otherwise, the second argument is returned. Supported by SQL version 2015-10-08 and later.
Example: `Nanvl(8, 3)` = 8.
****
| Argument type 1 | Argument type 2 | Output |
| --- | --- | --- |
| Undefined | Any value | The second argument. |
| Null | Any value | The second argument. |
| Decimal (NaN) | Any value | The second argument. |
| Decimal (not NaN) | Any value | The first argument. |
| Other value | Any value | The first argument. |
## newuuid()
Returns a random 16-byte UUID. Supported by SQL version 2015-10-08 and later.
Example: `newuuid()` = `123a4567-b89c-12d3-e456-789012345000`
## numbytes(String)
Returns the number of bytes in the UTF-8 encoding of the provided string. Standard conversion rules apply to non-`String` arguments. Supported by SQL version 2016-03-23 and later.
Examples:
`numbytes("hi")` = 2
`numbytes("€") ` = 3
## parse\$1time(String, Long[, String])
Use the `parse_time` function to format a timestamp into a human-readable date/time format. Supported by SQL version 2016-03-23 and later. To convert a timestamp string into milliseconds, see [time\$1to\$1epoch(String, String)](#iot-sql-function-time-to-epoch).
The `parse_time` function expects the following arguments:
pattern
(String) A date/time pattern that follows [Joda-Time formats](http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html).
timestamp
(Long) The time to be formatted in milliseconds since Unix epoch. See function [timestamp()](#iot-function-timestamp).
timezone
(String) The time zone of the formatted date/time. The default is "UTC". The function supports [Joda-Time time zones](http://joda-time.sourceforge.net/timezones.html). This argument is optional.
Examples:
When this message is published to the topic 'A/B', the payload `{"ts": "1970.01.01 AD at 21:46:40 CST"}` is sent to the S3 bucket:
```
{
"ruleArn": "arn:aws:iot:us-east-2:ACCOUNT_ID:rule/RULE_NAME",
"topicRulePayload": {
"sql": "SELECT parse_time(\"yyyy.MM.dd G 'at' HH:mm:ss z\", 100000000, 'America/Belize' ) as ts FROM 'A/B'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::ACCOUNT_ID:rule:role/ROLE_NAME",
"bucketName": "BUCKET_NAME",
"key": "KEY_NAME"
}
}
],
"ruleName": "RULE_NAME"
}
}
```
When this message is published to the topic 'A/B', a payload similar to `{"ts": "2017.06.09 AD at 17:19:46 UTC"}` (but with the current date/time) is sent to the S3 bucket:
```
{
"ruleArn": "arn:aws:iot:us-east-2:ACCOUNT_ID:rule/RULE_NAME",
"topicRulePayload": {
"sql": "SELECT parse_time(\"yyyy.MM.dd G 'at' HH:mm:ss z\", timestamp() ) as ts FROM 'A/B'",
"awsIotSqlVersion": "2016-03-23",
"ruleDisabled": false,
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::ACCOUNT_ID:rule:role/ROLE_NAME",
"bucketName": "BUCKET_NAME",
"key": "KEY_NAME"
}
}
],
"ruleName": "RULE_NAME"
}
}
```
`parse_time()` can also be used as a substitution template. For example, when this message is published to the topic 'A/B', the payload is sent to the S3 bucket with key = "2017":
```
{
"ruleArn": "arn:aws:iot:us-east-2:ACCOUNT_ID:rule/RULE_NAME",
"topicRulePayload": {
"sql": "SELECT * FROM 'A/B'",
"awsIotSqlVersion": "2016-03-23",
"ruleDisabled": false,
"actions": [{
"s3": {
"roleArn": "arn:aws:iam::ACCOUNT_ID:rule:role/ROLE_NAME",
"bucketName": "BUCKET_NAME",
"key": "${parse_time('yyyy', timestamp(), 'UTC')}"
}
}],
"ruleName": "RULE_NAME"
}
}
```
## power(Decimal, Decimal)
Returns the first argument raised to the second argument. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later. Supported by SQL version 2015-10-08 and later.
Example: `power(2, 5)` = 32.0.
****
| Argument type 1 | Argument type 2 | Output |
| --- | --- | --- |
| Int/Decimal | Int/Decimal | A Decimal (with double precision), the first argument raised to the second argument's power. |
| Int/Decimal/String | Int/Decimal/String | A Decimal (with double precision), the first argument raised to the second argument's power. Any strings are converted to decimals. If any String fails to be converted to Decimal, the result is Undefined. |
| Other value | Other value | Undefined. |
## principal()
Returns the principal that the device uses for authentication, based on how the triggering message was published. The following table describes the principal returned for each publishing method and protocol.
****
| How the message is published | Protocol | Credential type | Principal |
| --- | --- | --- | --- |
| MQTT client | MQTT | X.509 device certificate | X.509 certificate thumbprint |
| AWS IoT console MQTT client | MQTT | IAM user or role | iam-role-id:session-name |
| AWS CLI | HTTP | IAM user or role | userid |
| AWS IoT Device SDK | MQTT | X.509 device certificate | X.509 certificate thumbprint |
| AWS IoT Device SDK | MQTT over WebSocket | IAM user or role | userid |
The following examples show the different types of values that `principal()` can return:
+ X.509 certificate thumbprint: `ba67293af50bf2506f5f93469686da660c7c844e7b3950bfb16813e0d31e9373`
+ IAM role ID and session name: `ABCD1EFG3HIJK2LMNOP5:my-session-name`
+ Returns a user ID: `ABCD1EFG3HIJK2LMNOP5`
## rand()
Returns a pseudorandom, uniformly distributed double between 0.0 and 1.0. Supported by SQL version 2015-10-08 and later.
Example:
`rand()` = 0.8231909191640703
## regexp\$1matches(String, String)
Returns true if the string (first argument) contains a match for the regular expression (second argument). If you use `|` in the regular expression, use it with `()`.
Examples:
`regexp_matches("aaaa", "a{2,}") ` = true.
`regexp_matches("aaaa", "b")` = false.
`regexp_matches("aaa", "(aaa|bbb)") ` = true.
`regexp_matches("bbb", "(aaa|bbb)") ` = true.
`regexp_matches("ccc", "(aaa|bbb)") ` = false.
**First argument:**
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int. |
| Decimal | The String representation of the Decimal. |
| Boolean | The String representation of the Boolean ("true" or "false"). |
| String | The String. |
| Array | The String representation of the Array (using standard conversion rules). |
| Object | The String representation of the Object (using standard conversion rules). |
| Null | Undefined. |
| Undefined | Undefined. |
*Second argument:*
Must be a valid regex expression. Non-string types are converted to `String` using the standard conversion rules. Depending on the type, the resultant string might not be a valid regular expression. If the (converted) argument is not valid regex, the result is `Undefined`.
## regexp\$1replace(String, String, String)
Replaces all occurrences of the second argument (regular expression) in the first argument with the third argument. Reference capture groups with "\$1". Supported by SQL version 2015-10-08 and later.
Example:
`regexp_replace("abcd", "bc", "x")` = "axd".
`regexp_replace("abcd", "b(.*)d", "$1")` = "ac".
**First argument:**
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int. |
| Decimal | The String representation of the Decimal. |
| Boolean | The String representation of the Boolean ("true" or "false"). |
| String | The source value. |
| Array | The String representation of the Array (using standard conversion rules). |
| Object | The String representation of the Object (using standard conversion rules). |
| Null | Undefined. |
| Undefined | Undefined. |
*Second argument:*
Must be a valid regex expression. Non-string types are converted to `String` using the standard conversion rules. Depending on the type, the resultant string might not be a valid regular expression. If the (converted) argument is not a valid regex expression, the result is `Undefined`.
*Third argument:*
Must be a valid regex replacement string. (Can reference capture groups.) Non-string types are converted to `String` using the standard conversion rules. If the (converted) argument is not a valid regex replacement string, the result is `Undefined`.
## regexp\$1substr(String, String)
Finds the first match of the second parameter (regex) in the first parameter. Reference capture groups with "\$1". Supported by SQL version 2015-10-08 and later.
Example:
`regexp_substr("hihihello", "hi")` = "hi"
`regexp_substr("hihihello", "(hi)*")` = "hihi"
**First argument:**
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int. |
| Decimal | The String representation of the Decimal. |
| Boolean | The String representation of the Boolean ("true" or "false"). |
| String | The String argument. |
| Array | The String representation of the Array (using standard conversion rules). |
| Object | The String representation of the Object (using standard conversion rules). |
| Null | Undefined. |
| Undefined | Undefined. |
*Second argument:*
Must be a valid regex expression. Non-string types are converted to `String` using the standard conversion rules. Depending on the type, the resultant string might not be a valid regular expression. If the (converted) argument is not a valid regex expression, the result is `Undefined`.
## remainder(Decimal, Decimal)
Returns the remainder of the division of the first argument by the second argument. Equivalent to [mod(Decimal, Decimal)](#iot-func-mod). You can also use "%" as an infix operator for the same modulo functionality. Supported by SQL version 2015-10-08 and later.
Example: `remainder(8, 3)` = 2.
****
| Left operand | Right operand | Output |
| --- | --- | --- |
| Int | Int | Int, the first argument modulo the second argument. |
| Int/Decimal | Int/Decimal | Decimal, the first argument modulo the second operand. |
| String/Int/Decimal | String/Int/Decimal | If all strings convert to decimals, the result is the first argument modulo the second argument. Otherwise, Undefined. |
| Other value | Other value | Undefined. |
## replace(String, String, String)
Replaces all occurrences of the second argument in the first argument with the third argument. Supported by SQL version 2015-10-08 and later.
Example:
`replace("abcd", "bc", "x")` = `"axd"`.
`replace("abcdabcd", "b", "x")` = `"axcdaxcd"`.
**All arguments**
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int. |
| Decimal | The String representation of the Decimal. |
| Boolean | The String representation of the Boolean ("true" or "false"). |
| String | The source value. |
| Array | The String representation of the Array (using standard conversion rules). |
| Object | The String representation of the Object (using standard conversion rules). |
| Null | Undefined. |
| Undefined | Undefined. |
## rpad(String, Int)
Returns the string argument, padded on the right side with the number of spaces specified in the second argument. The `Int` argument must be between 0 and 1000. If the provided value is outside of this valid range, the argument is set to the nearest valid value (0 or 1000). Supported by SQL version 2015-10-08 and later.
Examples:
`rpad("hello", 2)` = "`hello `".
`rpad(1, 3)` = "`1 `".
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| String | Int | The String is padded on the right side with a number of spaces equal to the provided Int. |
| String | Decimal | The Decimal argument is rounded down to the nearest Int and the string is padded on the right side with a number of spaces equal to the provided Int. |
| String | String | The second argument is converted to a Decimal, which is rounded down to the nearest Int. The String is padded on the right side with a number of spaces equal to the Int value. |
| Other value | Int/Decimal/String | The first value is converted to a String using the standard conversions, and the rpad function is applied on that String. If it cannot be converted, the result is Undefined. |
| Any value | Other value | Undefined. |
## round(Decimal)
Rounds the given `Decimal` to the nearest `Int`. If the `Decimal` is equidistant from two `Int` values (for example, 0.5), the `Decimal` is rounded up. Supported by SQL version 2015-10-08 and later.
Example: `Round(1.2)` = 1.
`Round(1.5)` = 2.
`Round(1.7)` = 2.
`Round(-1.1)` = -1.
`Round(-1.5)` = -2.
****
| Argument type | Result |
| --- | --- |
| Int | The argument. |
| Decimal | Decimal is rounded down to the nearest Int. |
| String | Decimal is rounded down to the nearest Int. If the string cannot be converted to a Decimal, the result is Undefined. |
| Other value | Undefined. |
## rtrim(String)
Removes all trailing white space (tabs and spaces) from the provided `String`. Supported by SQL version 2015-10-08 and later.
Examples:
`rtrim(" h i ")` = " h i"
****
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int. |
| Decimal | The String representation of the Decimal. |
| Boolean | The String representation of the Boolean ("true" or "false"). |
| Array | The String representation of the Array (using standard conversion rules). |
| Object | The String representation of the Object (using standard conversion rules). |
| Null | Undefined. |
| Undefined | Undefined |
## sign(Decimal)
Returns the sign of the given number. When the sign of the argument is positive, 1 is returned. When the sign of the argument is negative, -1 is returned. If the argument is 0, 0 is returned. Supported by SQL version 2015-10-08 and later.
Examples:
`sign(-7)` = -1.
`sign(0)` = 0.
`sign(13)` = 1.
****
| Argument type | Result |
| --- | --- |
| Int | Int, the sign of the Int value. |
| Decimal | Int, the sign of the Decimal value. |
| String | Int, the sign of the Decimal value. The string is converted to a Decimal value, and the sign of the Decimal value is returned. If the String cannot be converted to a Decimal, the result is Undefined. Supported by SQL version 2015-10-08 and later. |
| Other value | Undefined. |
## sin(Decimal)
Returns the sine of a number in radians. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `sin(0)` = 0.0
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the sine of the argument. |
| Decimal | Decimal (with double precision), the sine of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the sine of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## sinh(Decimal)
Returns the hyperbolic sine of a number. `Decimal` values are rounded to double precision before function application. The result is a `Decimal` value of double precision. Supported by SQL version 2015-10-08 and later.
Example: `sinh(2.3)` = 4.936961805545957
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the hyperbolic sine of the argument. |
| Decimal | Decimal (with double precision), the hyperbolic sine of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the hyperbolic sine of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## sourceip()
Retrieves the IP address of a device or the router that connects to it. If your device is connected to the internet directly, the function will return the source IP address of the device. If your device is connected to a router that connects to the internet, the function will return the source IP address of the router. Supported by SQL version 2016-03-23. `sourceip()` doesn't take any parameters.
**Important**
A device's public source IP address is often the IP address of the last Network Address Translation (NAT) Gateway such as your internet service provider's router or cable modem.
Examples:
`sourceip()="192.158.1.38"`
`sourceip()="1.102.103.104"`
`sourceip()="2001:db8:ff00::12ab:34cd"`
SQL example:
`SELECT *, sourceip() as deviceIp FROM 'some/topic'`
Examples of how to use the sourceip() function in AWS IoT Core rule actions:
**Example 1**
The following example shows how to call the () function as a [substitution template](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html) in a [DynamoDB action](https://docs.aws.amazon.com//iot/latest/developerguide/dynamodb-rule-action.html).
```
{
"topicRulePayload": {
"sql": "SELECT * AS message FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"dynamoDB": {
"tableName": "my_ddb_table",
"hashKeyField": "key",
"hashKeyValue": "${sourceip()}",
"rangeKeyField": "timestamp",
"rangeKeyValue": "${timestamp()}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_dynamoDB"
}
}
]
}
}
```
**Example 2**
The following example shows how to add the sourceip() function as an MQTT user property using [substitution templates](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html).
```
{
"topicRulePayload": {
"sql": "SELECT * FROM 'some/topic'",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [
{
"republish": {
"topic": "${topic()}/republish",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_republish",
"headers": {
"payloadFormatIndicator": "UTF8_DATA",
"contentType": "rule/contentType",
"correlationData": "cnVsZSBjb3JyZWxhdGlvbiBkYXRh",
"userProperties": [
{
"key": "ruleKey1",
"value": "ruleValue1"
},
{
"key": "sourceip",
"value": "${sourceip()}"
}
]
}
}
}
]
}
}
```
You can retrieve the source IP address from messages passing to AWS IoT Core rules from both Message Broker and [Basic Ingest](https://docs.aws.amazon.com//iot/latest/developerguide/iot-basic-ingest.html) pathways. You can also retrieve the source IP for both IPv4 and IPv6 messages. The source IP will be displayed like the following:
IPv6: `yyyy:yyyy:yyyy::yyyy:yyyy`
IPv4: `xxx.xxx.xxx.xxx`
**Note**
The original source IP won't be passed though [Republish action](republish-rule-action.md).
## substring(String, Int[, Int])
Expects a `String` followed by one or two `Int` values. For a `String` and a single `Int` argument, this function returns the substring of the provided `String` from the provided `Int` index (0-based, inclusive) to the end of the `String`. For a `String` and two `Int` arguments, this function returns the substring of the provided `String` from the first `Int` index argument (0-based, inclusive) to the second `Int` index argument (0-based, exclusive). Indices that are less than zero are set to zero. Indices that are greater than the `String` length are set to the `String` length. For the three argument version, if the first index is greater than (or equal to) the second index, the result is the empty `String`.
If the arguments provided are not (*String*, *Int*), or (*String*, *Int*, *Int*), the standard conversions are applied to the arguments to attempt to convert them into the correct types. If the types cannot be converted, the result of the function is `Undefined`. Supported by SQL version 2015-10-08 and later.
Examples:
`substring("012345", 0)` = "012345".
`substring("012345", 2)` = "2345".
`substring("012345", 2.745)` = "2345".
`substring(123, 2)` = "3".
`substring("012345", -1)` = "012345".
`substring(true, 1.2)` = "rue".
`substring(false, -2.411E247)` = "false".
`substring("012345", 1, 3)` = "12".
`substring("012345", -50, 50)` = "012345".
`substring("012345", 3, 1)` = "".
## sql\$1version()
Returns the SQL version specified in this rule. Supported by SQL version 2015-10-08 and later.
Example:
`sql_version()` = "2016-03-23"
## sqrt(Decimal)
Returns the square root of a number. `Decimal` arguments are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `sqrt(9)` = 3.0.
****
| Argument type | Result |
| --- | --- |
| Int | The square root of the argument. |
| Decimal | The square root of the argument. |
| Boolean | Undefined. |
| String | The square root of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## startswith(String, String)
Returns `Boolean`, whether the first string argument starts with the second string argument. If either argument is `Null` or `Undefined`, the result is `Undefined`. Supported by SQL version 2015-10-08 and later.
Example:
`startswith("ranger","ran")` = true
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| String | String | Whether the first string starts with the second string. |
| Other value | Other value | Both arguments are converted to strings using the standard conversion rules. Returns true if the first string starts with the second string. If either argument is Null or Undefined, the result is Undefined. |
## tan(Decimal)
Returns the tangent of a number in radians. `Decimal` values are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `tan(3)` = -0.1425465430742778
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the tangent of the argument. |
| Decimal | Decimal (with double precision), the tangent of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the tangent of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## tanh(Decimal)
Returns the hyperbolic tangent of a number in radians. `Decimal` values are rounded to double precision before function application. Supported by SQL version 2015-10-08 and later.
Example: `tanh(2.3)` = 0.9800963962661914
****
| Argument type | Result |
| --- | --- |
| Int | Decimal (with double precision), the hyperbolic tangent of the argument. |
| Decimal | Decimal (with double precision), the hyperbolic tangent of the argument. |
| Boolean | Undefined. |
| String | Decimal (with double precision), the hyperbolic tangent of the argument. If the string cannot be converted to a Decimal, the result is Undefined. |
| Array | Undefined. |
| Object | Undefined. |
| Null | Undefined. |
| Undefined | Undefined. |
## time\$1to\$1epoch(String, String)
Use the `time_to_epoch` function to convert a timestamp string into a number of milliseconds in Unix epoch time. Supported by SQL version 2016-03-23 and later. To convert milliseconds to a formatted timestamp string, see [parse\$1time(String, Long[, String])](#iot-sql-function-parse-time).
The `time_to_epoch` function expects the following arguments:
timestamp
(String) The timestamp string to be converted to milliseconds since Unix epoch. If the timestamp string doesn't specify a timezone, the function uses the UTC timezone.
pattern
(String) A date/time pattern that follows [JDK11 Time Formats](http://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/format/DateTimeFormatter.html).
Examples:
`time_to_epoch("2020-04-03 09:45:18 UTC+01:00", "yyyy-MM-dd HH:mm:ss VV")` = 1585903518000
`time_to_epoch("18 December 2015", "dd MMMM yyyy")` = 1450396800000
`time_to_epoch("2007-12-03 10:15:30.592 America/Los_Angeles", "yyyy-MM-dd HH:mm:ss.SSS z")` = 1196705730592
## timestamp()
Returns the current timestamp in milliseconds from 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, as observed by the AWS IoT rules engine. Supported by SQL version 2015-10-08 and later.
Example: `timestamp()` = `1481825251155`
## topic(Decimal)
Returns the topic to which the message that triggered the rule was sent. If no parameter is specified, the entire topic is returned. The `Decimal` parameter is used to specify a specific topic segment, with 1 designating the first segment. For the topic `foo/bar/baz`, topic(1) returns `foo`, topic(2) returns `bar`, and so on. Supported by SQL version 2015-10-08 and later.
Examples:
`topic()` = "things/myThings/thingOne"
`topic(1)` = "things"
When [Basic Ingest](iot-basic-ingest.md) is used, the initial prefix of the topic (`$aws/rules/rule-name`) is not available to the topic() function. For example, given the topic:
`$aws/rules/BuildingManager/Buildings/Building5/Floor2/Room201/Lights`
`topic()` = "Buildings/Building5/Floor2/Room201/Lights"
`topic(3)` = "Floor2"
## traceid()
Returns the trace ID (UUID) of the MQTT message, or `Undefined` if the message wasn't sent over MQTT. Supported by SQL version 2015-10-08 and later.
Example:
`traceid() ` = "12345678-1234-1234-1234-123456789012"
## transform(String, Object, Array)
Returns an array of objects that contains the result of the specified transformation of the `Object` parameter on the `Array` parameter.
Supported by SQL version 2016-03-23 and later.
String
The transformation mode to use. Refer to the following table for the supported transformation modes and how they create the `Result` from the `Object` and `Array` parameters.
Object
An object that contains the attributes to apply to each element of the `Array`.
Array
An array of objects into which the attributes of `Object` are applied.
Each object in this Array corresponds to an object in the function's response. Each object in the function's response contains the attributes present in the original object and the attributes provided by `Object` as determined by the transformation mode specified in `String`.
| `String` parameter | `Object` parameter | `Array` parameter | Result |
| --- | --- | --- | --- |
| `enrichArray` | Object | Array of objects | An Array of objects in which each object contains the attributes of an element from the `Array` parameter and the attributes of the `Object` parameter. |
| Any other value | Any value | Any value | Undefined |
**Note**
The array returned by this function is limited to 128 KiB.
### Transform function example 1
This example shows how the **transform()** function produces a single array of objects from a data object and an array.
In this example, the following message is published to the MQTT topic `A/B`.
```
{
"attributes": {
"data1": 1,
"data2": 2
},
"values": [
{
"a": 3
},
{
"b": 4
},
{
"c": 5
}
]
}
```
This SQL statement for a topic rule action uses the **transform()** function with a `String` value of `enrichArray`. In this example, `Object` is the `attributes` property from the message payload and `Array` is the `values` array, which contains three objects.
```
select value transform("enrichArray", attributes, values) from 'A/B'
```
Upon receiving the message payload, the SQL statement evaluates to the following response.
```
[
{
"a": 3,
"data1": 1,
"data2": 2
},
{
"b": 4,
"data1": 1,
"data2": 2
},
{
"c": 5,
"data1": 1,
"data2": 2
}
]
```
### Transform function example 2
This example shows how the **transform()** function can use literal values to include and rename individual attributes from the message payload.
In this example, the following message is published to the MQTT topic `A/B`. This is the same message that was used in [Transform function example 1](#iot-func-transform-example1).
```
{
"attributes": {
"data1": 1,
"data2": 2
},
"values": [
{
"a": 3
},
{
"b": 4
},
{
"c": 5
}
]
}
```
This SQL statement for a topic rule action uses the **transform()** function with a `String` value of `enrichArray`. The `Object` in the **transform()** function has a single attribute named `key` with the value of `attributes.data1` in the message payload and `Array` is the `values` array, which contains the same three objects used in the previous example.
```
select value transform("enrichArray", {"key": attributes.data1}, values) from 'A/B'
```
Upon receiving the message payload, this SQL statement evaluates to the following response. Notice how the `data1` property is named `key` in the response.
```
[
{
"a": 3,
"key": 1
},
{
"b": 4,
"key": 1
},
{
"c": 5,
"key": 1
}
]
```
### Transform function example 3
This example shows how the **transform()** function can be used in nested SELECT clauses to select multiple attributes and create new objects for subsequent processing.
In this example, the following message is published to the MQTT topic `A/B`.
```
{
"data1": "example",
"data2": {
"a": "first attribute",
"b": "second attribute",
"c": [
{
"x": {
"someInt": 5,
"someString": "hello"
},
"y": true
},
{
"x": {
"someInt": 10,
"someString": "world"
},
"y": false
}
]
}
}
```
The `Object` for this transform function is the object returned by the SELECT statement, which contains the `a` and `b` elements of the message's `data2` object. The `Array` parameter consists of the two objects from the `data2.c` array in the original message.
```
select value transform('enrichArray', (select a, b from data2), (select value c from data2)) from 'A/B'
```
With the preceding message, the SQL statement evaluates to the following response.
```
[
{
"x": {
"someInt": 5,
"someString": "hello"
},
"y": true,
"a": "first attribute",
"b": "second attribute"
},
{
"x": {
"someInt": 10,
"someString": "world"
},
"y": false,
"a": "first attribute",
"b": "second attribute"
}
]
```
The array returned in this response could be used with topic rule actions that support `batchMode`.
## trim(String)
Removes all leading and trailing white space from the provided `String`. Supported by SQL version 2015-10-08 and later.
Example:
`Trim(" hi ") ` = "hi"
****
| Argument type | Result |
| --- | --- |
| Int | The String representation of the Int with all leading and trailing white space removed. |
| Decimal | The String representation of the Decimal with all leading and trailing white space removed. |
| Boolean | The String representation of the Boolean ("true" or "false") with all leading and trailing white space removed. |
| String | The String with all leading and trailing white space removed. |
| Array | The String representation of the Array using standard conversion rules. |
| Object | The String representation of the Object using standard conversion rules. |
| Null | Undefined. |
| Undefined | Undefined. |
## trunc(Decimal, Int)
Truncates the first argument to the number of `Decimal` places specified by the second argument. If the second argument is less than zero, it is set to zero. If the second argument is greater than 34, it is set to 34. Trailing zeroes are stripped from the result. Supported by SQL version 2015-10-08 and later.
Examples:
`trunc(2.3, 0)` = 2.
`trunc(2.3123, 2)` = 2.31.
`trunc(2.888, 2)` = 2.88.
`trunc(2.00, 5)` = 2.
****
| Argument type 1 | Argument type 2 | Result |
| --- | --- | --- |
| Int | Int | The source value. |
| Int/Decimal | Int/Decimal | The first argument is truncated to the length described by the second argument. The second argument, if not an Int, is rounded down to the nearest Int. |
| Int/Decimal/String | Int/Decimal | The first argument is truncated to the length described by the second argument. The second argument, if not an Int, is rounded down to the nearest Int. A String is converted to a Decimal value. If the string conversion fails, the result is Undefined. |
| Other value | | Undefined. |
## upper(String)
Returns the uppercase version of the given `String`. Non-`String` arguments are converted to `String` using the standard conversion rules. Supported by SQL version 2015-10-08 and later.
Examples:
`upper("hello")` = "HELLO"
`upper(["hello"])` = "[\$1"HELLO\$1"]"
# Literals
You can directly specify literal objects in the SELECT and WHERE clauses of your rule SQL, which can be useful for passing information.
**Note**
Literals are available only when using SQL version 2016-03-23 or later.
JSON object syntax is used (key-value pairs, comma-separated, where keys are strings and values are JSON values, wrapped in curly brackets \$1\$1). For example:
Incoming payload published on topic `topic/subtopic`: `{"lat_long": [47.606,-122.332]}`
SQL statement: `SELECT {'latitude': get(lat_long, 0),'longitude':get(lat_long, 1)} as lat_long FROM 'topic/subtopic'`
The resulting outgoing payload would be: `{"lat_long":{"latitude":47.606,"longitude":-122.332}}`.
You can also directly specify arrays in the SELECT and WHERE clauses of your rule SQL, which allows you to group information. JSON syntax is used (wrap comma-separated items in square brackets [] to create an array literal). For example:
Incoming payload published on topic `topic/subtopic`: `{"lat": 47.696, "long": -122.332}`
SQL statement: `SELECT [lat,long] as lat_long FROM 'topic/subtopic'`
The resulting output payload would be: `{"lat_long": [47.606,-122.332]}`.
# Case statements
Case statements can be used for branching execution, like a switch statement.
Syntax:
```
CASE v WHEN t[1] THEN r[1]
WHEN t[2] THEN r[2] ...
WHEN t[n] THEN r[n]
ELSE r[e] END
```
The expression *`v`* is evaluated and matched for equality against the *`t[i]`* value of each `WHEN` clause. If a match is found, the corresponding *`r[i]`* expression becomes the result of the `CASE` statement. The `WHEN` clauses are evaluated in order so that if there's more than one matching clause, the result of the first matching clause becomes the result of the `CASE` statement. If there are no matches, *`r[e]`* of the `ELSE` clause is the result. If there's no match and no `ELSE` clause, the result is `Undefined`.
`CASE` statements require at least one `WHEN` clause. An `ELSE` clause is optional.
For example:
Incoming payload published on topic `topic/subtopic`:
```
{
"color":"yellow"
}
```
SQL statement:
```
SELECT CASE color
WHEN 'green' THEN 'go'
WHEN 'yellow' THEN 'caution'
WHEN 'red' THEN 'stop'
ELSE 'you are not at a stop light' END as instructions
FROM 'topic/subtopic'
```
The resulting output payload would be:
```
{
"instructions":"caution"
}
```
**Note**
If *`v`* is `Undefined`, the result of the case statement is `Undefined`.
# JSON extensions
You can use the following extensions to ANSI SQL syntax to facilitate work with nested JSON objects.
"." Operator
This operator accesses members in embedded JSON objects and functions identically to ANSI SQL and JavaScript. For example:
```
SELECT foo.bar AS bar.baz FROM 'topic/subtopic'
```
selects the value of the `bar` property in the `foo` object from the following message payload sent to the `topic/subtopic` topic.
```
{
"foo": {
"bar": "RED",
"bar1": "GREEN",
"bar2": "BLUE"
}
}
```
If a JSON property name includes a hyphen character or numeric characters, the 'dot' notation will not work. Instead, you must use the [get function](iot-sql-functions.md#iot-sql-function-get) to extract the property's value.
In this example, the following message is sent to the `iot/rules` topic.
```
{
"mydata": {
"item2": {
"0": {
"my-key": "myValue"
}
}
}
}
```
Normally, the value of `my-key` would be identified as in this query.
```
SELECT * from iot/rules WHERE mydata.item2.0.my-key= "myValue"
```
However, because the property name `my-key` contains a hyphen and `item2` contains a numeric character, the [get function](iot-sql-functions.md#iot-sql-function-get) must be used as the following query shows.
```
SELECT * from 'iot/rules' WHERE get(get(get(mydata,"item2"),"0"),"my-key") = "myValue"
```
`*` Operator
This functions in the same way as the `*` wildcard in ANSI SQL. It's used in the SELECT clause only and creates a new JSON object containing the message data. If the message payload is not in JSON format, `*` returns the entire message payload as raw bytes. For example:
```
SELECT * FROM 'topic/subtopic'
```
**Applying a Function to an Attribute Value**
The following is an example JSON payload that might be published by a device:
```
{
"deviceid" : "iot123",
"temp" : 54.98,
"humidity" : 32.43,
"coords" : {
"latitude" : 47.615694,
"longitude" : -122.3359976
}
}
```
The following example applies a function to an attribute value in a JSON payload:
```
SELECT temp, md5(deviceid) AS hashed_id FROM topic/#
```
The result of this query is the following JSON object:
```
{
"temp": 54.98,
"hashed_id": "e37f81fb397e595c4aeb5645b8cbbbd1"
}
```
# Substitution templates
You can use a substitution template to augment the JSON data returned when a rule is triggered and AWS IoT performs an action. The syntax for a substitution template is `${`*expression*`}`, where *expression* can be any expression supported by AWS IoT in SELECT clauses, WHERE clauses, and [AWS IoT rule actions](iot-rule-actions.md). This expression can be plugged into an action field on a rule, allowing you to dynamically configure an action. In effect, this feature substitutes a piece of information in an action. This includes functions, operators, and information present in the original message payload.
**Important**
Because an expression in a substitution template is evaluated separately from the "SELECT ..." statement, you can't reference an alias created using the AS clause. You can only reference information present in the original payload, [functions](iot-sql-functions.md), and [operators](iot-sql-operators.md).
For more information about supported expressions, see [AWS IoT SQL reference](iot-sql-reference.md).
The following rule actions support substitution templates. Each action supports different fields that can be substituted.
+ [Apache Kafka](apache-kafka-rule-action.md)
+ [CloudWatch alarms](cloudwatch-alarms-rule-action.md)
+ [CloudWatch Logs](cloudwatch-logs-rule-action.md)
+ [CloudWatch metrics](cloudwatch-metrics-rule-action.md)
+ [DynamoDB](dynamodb-rule-action.md)
+ [DynamoDBv2](dynamodb-v2-rule-action.md)
+ [Elasticsearch](elasticsearch-rule-action.md)
+ [HTTP](https-rule-action.md)
+ [AWS IoT Events](iotevents-rule-action.md)
+ [AWS IoT SiteWise](iotsitewise-rule-action.md)
+ [Kinesis Data Streams](kinesis-rule-action.md)
+ [Firehose](kinesis-firehose-rule-action.md)
+ [Lambda](lambda-rule-action.md)
+ [Location](location-rule-action.md)
+ [OpenSearch](opensearch-rule-action.md)
+ [Republish](republish-rule-action.md)
+ [S3](s3-rule-action.md)
+ [SNS](sns-rule-action.md)
+ [SQS](sqs-rule-action.md)
+ [Step Functions](stepfunctions-rule-action.md)
+ [Timestream](timestream-rule-action.md)
Substitution templates appear in the action parameters within a rule:
```
{
"sql": "SELECT *, timestamp() AS timestamp FROM 'my/iot/topic'",
"ruleDisabled": false,
"actions": [{
"republish": {
"topic": "${topic()}/republish",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}]
}
```
If this rule is triggered by the following JSON published to `my/iot/topic`:
```
{
"deviceid": "iot123",
"temp": 54.98,
"humidity": 32.43,
"coords": {
"latitude": 47.615694,
"longitude": -122.3359976
}
}
```
Then this rule publishes the following JSON to `my/iot/topic/republish`, which AWS IoT substitutes from `${topic()}/republish`:
```
{
"deviceid": "iot123",
"temp": 54.98,
"humidity": 32.43,
"coords": {
"latitude": 47.615694,
"longitude": -122.3359976
},
"timestamp": 1579637878451
}
```
# Nested object queries
You can use nested SELECT clauses to query for attributes within arrays and inner JSON objects. Supported by SQL version 2016-03-23 and later.
Consider the following MQTT message:
```
{
"e": [
{ "n": "temperature", "u": "Cel", "t": 1234, "v": 22.5 },
{ "n": "light", "u": "lm", "t": 1235, "v": 135 },
{ "n": "acidity", "u": "pH", "t": 1235, "v": 7 }
]
}
```
**Example**
You can convert values to a new array with the following rule.
```
SELECT (SELECT VALUE n FROM e) as sensors FROM 'my/topic'
```
The rule generates the following output.
```
{
"sensors": [
"temperature",
"light",
"acidity"
]
}
```
**Example**
Using the same MQTT message, you can also query a specific value within a nested object with the following rule.
```
SELECT (SELECT v FROM e WHERE n = 'temperature') as temperature FROM 'my/topic'
```
The rule generates the following output.
```
{
"temperature": [
{
"v": 22.5
}
]
}
```
**Example**
You can also flatten the output with a more complicated rule.
```
SELECT get((SELECT v FROM e WHERE n = 'temperature'), 0).v as temperature FROM 'topic'
```
The rule generates the following output.
```
{
"temperature": 22.5
}
```
# Working with binary payloads
Binary payloads
To handle your message payload as raw binary data (rather than a JSON object), you can use the \$1 operator to refer to it in a SELECT clause.
**Topics**
+ [
## Binary payload examples
](#binary-payloads-examples)
+ [
## Decoding protobuf message payloads
](#binary-payloads-protobuf)
## Binary payload examples
When you use \$1 to refer to the message payload as raw binary data, you can add data to the rule. If you have an empty or a JSON payload, the resulting payload can have data added using the rule. The following shows examples of supported `SELECT` clauses.
+ You can use the following `SELECT` clauses with only a \$1 for binary payloads.
+
```
SELECT * FROM 'topic/subtopic'
```
+
```
SELECT * FROM 'topic/subtopic' WHERE timestamp() % 12 = 0
```
+ You can also add data and use the following `SELECT` clauses.
+
```
SELECT *, principal() as principal, timestamp() as time FROM 'topic/subtopic'
```
+
```
SELECT encode(*, 'base64') AS data, timestamp() AS ts FROM 'topic/subtopic'
```
+ You can also use these `SELECT` clauses with binary payloads.
+ The following refers to `device_type` in the WHERE clause.
```
SELECT * FROM 'topic/subtopic' WHERE device_type = 'thermostat'
```
+ The following is also supported.
```
{
"sql": "SELECT * FROM 'topic/subtopic'",
"actions": [
{
"republish": {
"topic": "device/${device_id}"
}
}
]
}
```
The following rule actions don't support binary payloads so you must decode them.
+ Some rule actions don't support binary payload input, such as a [Lambda action](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rule-actions.html#lambda-rule), so you must decode binary payloads. The Lambda rule action can receive binary data, if it's base64 encoded and in a JSON payload. You can do this by changing the rule to the following.
```
SELECT encode(*, 'base64') AS data FROM 'my_topic'
```
+ The SQL statement doesn't support string as input. To convert a string input to JSON, you can run the following command.
```
SELECT decode(encode(*, 'base64'), 'base64') AS payload FROM 'topic'
```
## Decoding protobuf message payloads
[Protocol Buffers (protobuf)](https://developers.google.com/protocol-buffers) is an open-source data format used to serialize structured data in a compact, binary form. It's used for transmitting data over networks or storing it in files. Protobuf allows you to send data in small packet sizes and at a faster rate than other messaging formats. AWS IoT Core Rules support protobuf by providing the [decode(value, decodingScheme)](iot-sql-functions.md#iot-sql-decode-base64) SQL function, which allows you to decode protobuf-encoded message payloads to JSON format and route them to downstream services. This section details the step-by-step process to configure protobuf decoding in AWS IoT Core Rules.
**Topics**
+ [
### Prerequisites
](#binary-payloads-protobuf-prerequisites)
+ [
### Create descriptor files
](#binary-payloads-protobuf-descriptor-steps)
+ [
### Upload descriptor files to S3 bucket
](#binary-payloads-protobuf-s3-steps)
+ [
### Configure protobuf decoding in Rules
](#binary-payloads-protobuf-steps)
+ [
### Limitations
](#binary-payloads-protobuf-limitations)
+ [
### Best practices
](#binary-payloads-protobuf-bestpractices)
### Prerequisites
+ A basic understanding of [Protocol Buffers (protobuf)](https://developers.google.com/protocol-buffers)
+ The [`.proto` files](https://developers.google.com/protocol-buffers/docs/proto3) that define message types and related dependencies
+ Installing [Protobuf Compiler (protoc)](https://github.com/protocolbuffers/protobuf/releases) on your system
### Create descriptor files
If you already have your descriptor files, you can skip this step. A descriptor file (`.desc`) is a compiled version of a `.proto` file, which is a text file that defines the data structures and message types to be used in a protobuf serialization. To generate a descriptor file, you must define a `.proto` file and use the [protoc](https://github.com/protocolbuffers/protobuf/releases) compiler to compile it.
1. Create `.proto` files that define the message types. An example `.proto` file can look like the following:
```
syntax = "proto3";
message Person {
optional string name = 1;
optional int32 id = 2;
optional string email = 3;
}
```
In this example `.proto` file, you use proto3 syntax and define message type `Person`. The `Person` message definition specifies three fields (name, id, and email). For more information about `.proto` file message formats, see [Language Guide (proto3)](https://developers.google.com/protocol-buffers/docs/proto3).
1. Use the [protoc](https://github.com/protocolbuffers/protobuf/releases) compiler to compile the `.proto` files and generate a descriptor file. An example command to create a descriptor (`.desc`) file can be the following:
```
protoc --descriptor_set_out=.desc \
--proto_path= \
--include_imports \
.proto
```
This example command generates a descriptor file `.desc`, which AWS IoT Core Rules can use to decode protobuf payloads that conform to the data structure defined in `.proto`.
+ `--descriptor_set_out`
Specifies the name of the descriptor file (`.desc` ) that should be generated.
+ `--proto_path`
Specifies the locations of any imported `.proto` files that are referenced by the file being compiled. You can specify the flag multiple times if you have multiple imported `.proto` files with different locations.
+ `--include_imports`
Specifies that any imported `.proto` files should also be compiled and included in the `.desc` descriptor file.
+ `.proto`
Specifies the name of the `.proto` file that you want to compile.
For more information about the protoc reference, see [API Reference](https://developers.google.com/protocol-buffers/docs/reference/overview).
### Upload descriptor files to S3 bucket
After you create your descriptor files `.desc`, upload the descriptor files `.desc` to an Amazon S3 bucket, using the AWS API, AWS SDK, or the AWS Management Console.
**Important considerations**
+ Make sure that you upload the descriptor files to an Amazon S3 bucket in your AWS account in the same AWS Region where you intend to configure your Rules.
+ Make sure that you grant AWS IoT Core access to read the `FileDescriptorSet` from S3. If your S3 bucket has server-side encryption (SSE) disabled or if your S3 bucket is encrypted using Amazon S3-managed keys (SSE-S3), no additional policy configurations are required. This can be accomplished with the example bucket policy:
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": "iot.amazonaws.com"
},
"Action": "s3:Get*",
"Resource": "arn:aws:s3:::/.desc"
}
]
}
```
+ If your S3 bucket is encrypted using an AWS Key Management Service key (SSE-KMS), make sure that you grant AWS IoT Core permission to use the key when accessing your S3 bucket. You can do this by adding this statement to your key policy:
```
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": "iot.amazonaws.com"
},
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
}
```
### Configure protobuf decoding in Rules
After you upload the descriptor files to your Amazon S3 bucket, configure a [Rule](https://docs.aws.amazon.com//iot/latest/developerguide/iot-create-rule.html) that can decode your protobuf message payload format using the [decode(value, decodingScheme)](iot-sql-functions.md#iot-sql-decode-base64) SQL function. A detailed function signature and example can be found in the [decode(value, decodingScheme)](iot-sql-functions.md#iot-sql-decode-base64) SQL function of the *AWS IoT SQL reference*.
The following is an example SQL expression using the [decode(value, decodingScheme)](iot-sql-functions.md#iot-sql-decode-base64) function:
```
SELECT VALUE decode(*, 'proto', '', '.desc', '', '') FROM ''
```
In this example expression:
+ You use the [decode(value, decodingScheme)](iot-sql-functions.md#iot-sql-decode-base64) SQL function to decode the binary message payload referenced by `*`. This can be a binary protobuf-encoded payload or a JSON string that represents a base64-encoded protobuf payload.
+ The message payload provided is encoded using the `Person` message type defined in `PROTO_FILENAME.proto`.
+ The Amazon S3 bucket named `BUCKET NAME` contains the `FILENAME.desc` generated from `PROTO_FILENAME.proto`.
After you complete the configuration, publish a message to AWS IoT Core on the topic to which the Rule is subscribed.
### Limitations
AWS IoT Core Rules support protobuf with the following limitations:
+ Decoding protobuf message payloads within [substitution templates](https://docs.aws.amazon.com//iot/latest/developerguide/iot-substitution-templates.html) is not supported.
+ When decoding protobuf message payloads, you can use the [decode SQL function](iot-sql-functions.md#iot-sql-decode-base64) within a single SQL expression up to two times.
+ The maximum inbound payload size is 128 KiB (1KiB =1024 bytes), the maximum outbound payload size is 128 KiB, and the maximum size for a `FileDescriptorSet` object stored in an Amazon S3 bucket is 32 KiB.
+ Amazon S3 buckets encrypted with SSE-C encryption are not supported.
### Best practices
Here are some best practices and troubleshooting tips.
+ Back up your proto files in the Amazon S3 bucket.
It's a good practice to back up your proto files in case something goes wrong. For example, if you incorrectly modify the proto files without backups when running protoc, this can cause issues in your production stack. There are multiple ways to back up your files in an Amazon S3 bucket. For example, you can [use versioning in S3 buckets](https://docs.aws.amazon.com//AmazonS3/latest/userguide/Versioning.html). For more information about how to back up files in Amazon S3 buckets, refer to the *[Amazon S3 Developer Guide](https://docs.aws.amazon.com//aws-backup/latest/devguide/recovery-points.html)*.
+ Configure AWS IoT logging to view log entries.
It's a good practice to configure AWS IoT logging so that you can check AWS IoT logs for your account in CloudWatch. When a rule's SQL query calls an external function, AWS IoT Core Rules generates a log entry with an `eventType` of `FunctionExecution`, which contains the reason field that will help you troubleshoot failures. Possible errors include an Amazon S3 object not found, or invalid protobuf file descriptor. For more information about how to configure AWS IoT logging and see the log entries, see [Configure AWS IoT logging](https://docs.aws.amazon.com//iot/latest/developerguide/configure-logging.html) and [Rules engine log entries](https://docs.aws.amazon.com//iot/latest/developerguide/cwl-format.html#log-rules-fn-exec).
+ Update `FileDescriptorSet` using a new object key and update the object key in your Rule.
You can update `FileDescriptorSet` by uploading an updated descriptor file to your Amazon S3 bucket. Your updates to `FileDescriptorSet` can take up to 15 minutes to be reflected. To avoid this delay, it's a good practice to upload your updated `FileDescriptorSet` using a new object key, and update the object key in your Rule.
# SQL versions
The AWS IoT rules engine uses an SQL-like syntax to select data from MQTT messages. The SQL statements are interpreted based on an SQL version specified with the `awsIotSqlVersion` property in a JSON document that describes the rule. For more information about the structure of JSON rule documents, see [Creating a Rule](iot-create-rule.md). The `awsIotSqlVersion` property lets you specify which version of the AWS IoT SQL rules engine that you want to use. When a new version is deployed, you can continue to use an earlier version or change your rule to use the new version. Your current rules continue to use the version with which they were created.
The following JSON example shows you how to specify the SQL version using the `awsIotSqlVersion` property.
```
{
"sql": "expression",
"ruleDisabled": false,
"awsIotSqlVersion": "2016-03-23",
"actions": [{
"republish": {
"topic": "my-mqtt-topic",
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role"
}
}]
}
```
AWS IoT currently supports the following SQL versions:
+ `2016-03-23` – The SQL version built on 2016-03-23 (recommended).
+ `2015-10-08` – The original SQL version built on 2015-10-08.
+ `beta` – The most recent beta SQL version. This version could introduce breaking changes to your rules.
## What's new in the 2016-03-23 SQL rules engine version
+ Fixes for selecting nested JSON objects.
+ Fixes for array queries.
+ Intra-object query support. For more information, see [Nested object queries](iot-sql-nested-queries.md).
+ Support to output an array as a top-level object.
+ Addition of the `encode(value, encodingScheme)` function, which can be applied on JSON and non-JSON format data. For more information, see the [encode function](iot-sql-functions.md#iot-sql-encode-payload).
### Output an `Array` as a top-level object
This feature allows a rule to return an array as a top-level object. For example, given the following MQTT message:
```
{
"a": {"b":"c"},
"arr":[1,2,3,4]
}
```
And the following rule:
```
SELECT VALUE arr FROM 'topic'
```
The rule generates the following output.
```
[1,2,3,4]
```