Amazon Monitron is no longer open to new customers. Existing customers can continue to use the service as normal. For capabilities similar to Amazon Monitron, see our [blog post](https://aws.amazon.com/blogs/machine-learning/maintain-access-and-consider-alternatives-for-amazon-monitron).
# Amazon Monitron Kinesis data export v1
**Note**
Amazon Monitron Kinesis data export schema v1 has been deprecated. Learn more about the [v2 data export schema](monitron-kinesis-export-v2.md).
You can export incoming measurement data and the corresponding inference results from Amazon Monitron and perform real-time analysis. Data export streams live data to Kinesis.
**Topics**
+ [
## Exporting your data to a Kinesis stream
](#exporting-stream-procedure)
+ [
## Editing live data export settings
](#edit-live-export)
+ [
## Stopping a live data export
](#stop-kinesis-export)
+ [
## Viewing data export errors
](#viewing-kinesis-export-errors)
+ [
## Using server-side encryption for the Kinesis stream
](#data-export-server-side-encryption)
+ [
# Monitoring with Amazon CloudWatch Logs
](data-export-cloudwatch-logs.md)
+ [
# Storing exported data in Amazon S3
](kinesis-store-S3.md)
+ [
# Processing data with Lambda
](data-export-lambda.md)
+ [
# Understanding the v1 data export schema
](data-export-schema.md)
## Exporting your data to a Kinesis stream
1. From your project's main page, near the bottom of the page, on the right, choose **Start live data export**.
1. Under **Select Amazon Kinesis data stream**, do one of the following:
+ Enter the name of an existing stream in the search box. Then skip to Step 5.
+ Choose **Create a new data stream**.
1. On the **Create data stream** page, under **Data stream configuration**, enter your data stream name.
1. Under Data stream capacity, choose your capacity mode:
+ If your data stream’s throughput requirements are unpredictable and variable, choose **On-demand**.
+ If you can reliably estimate the throughput requirements of your data stream, choose **Provisioned**. Then, under provisioned shards, enter the number of shards you want to create, or choose the **Shard estimator**.
1. Choose **Create data stream**.
## Editing live data export settings
To edit your live data export settings:
1. Open the Amazon Monitron console.
1. Choose **Projects** from the navigation pane.
1. If you have multiple projects, choose the project for which you want to edit the export settings.
1. From the main page for your project, under **Live data export**, from the **Actions** dropdown menu, choose **Edit live data export settings**.
## Stopping a live data export
1. Open the Amazon Monitron console.
1. Choose **Projects** from the navigation pane.
1. If you have multiple projects, choose the project for which you want to edit the export settings.
1. From the main page for your project, under **Live data export**, from the **Actions** dropdown menu, choose **Stop live data export**.
1. In the pop-up window, choose **Stop**.
## Viewing data export errors
To view the error messages in the CloudWatch Logs interface:
+ On the Amazon Monitron console, from the main page for your project, under **Live data export**, choose **CloudWatch log group**.
## Using server-side encryption for the Kinesis stream
You can enable server-side encryption for your Kinesis stream before setting up Kinesis data export. However, if server-side encryption is enabled after Kinesis data export is set up, Amazon Monitron will not be able to publish to the stream. That's because Amazon Monitron will not have permission to call [kms:GenerateDataKey](https://docs.aws.amazon.com/kms/latest/APIReference/API_GenerateDataKey.html) so that it can encrypt data sent to Kinesis.
To work around this, follow the instructions under [Editing live data export settings](#edit-live-export), but without changing the configuration. This will associate the encryption you have set up with your export configuration.
# Monitoring with Amazon CloudWatch Logs
You can monitor Amazon Monitron live data export using Amazon CloudWatch Logs. When a measurement fails to export, Amazon Monitron will send a log event to your CloudWatch Logs. You can also set up a metric filter on the error log to generate metrics and set up alarms. An alarm can watch for certain thresholds and send notifications or take actions when those thresholds are met. For more information, see [the CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html).
Amazon Monitron sends log events to the /aws/monitron/data-export/\$1HASH\$1ID\$1 log group.
The log event has the following JSON format:
```
{
"assetDisplayName": "string",
"destination": "string",
"errorCode": "string",
"errorMessage": "string",
"eventId": "string",
"positionDisplayName": "string",
"projectDisplayName": "string",
"projectName": "string",
"sensorId": "string",
"siteDisplayName": "string",
"timestamp": "string"
}
```
assetDisplayName
+ The asset name displayed in the App
+ Type: String
destination
+ The ARN of the Kinesis data stream
+ Type: String
+ Pattern: arn:aws:kinesis:\$1\$1REGION\$1\$1:\$1\$1AWS\$1ACCOUNT\$1ID\$1\$1:stream/\$1\$1STREAM\$1NAME\$1\$1
errorCode
+ The error code
+ Type: String
+ Valid Values: `INTERNAL_SEVER_ERROR | KINESIS_RESOURCE_NOT_FOUND | KINESIS_PROVISIONED_THROUGHPUT_EXCEEDED | KMS_ACCESS_DENIED | KMS_NOT_FOUND | KMS_DISABLED | KMS_INVALID_STATE | KMS_THROTTLING`
errorMessage
+ The detailed error message
+ Type: String
eventId
+ The unique event ID corresponding to each measurement export
+ Type: String
positionDisplayName
+ The sensor position name displayed in the App
+ Type: String
sensorId
+ The physical ID of the sensor from which the measurement is sent
+ Type: String
siteDisplayName
+ The site name displayed in the App
+ Type: String
timestamp
+ The timestamp when the measurement is received by Amazon Monitron service in UTC
+ Type: String
+ Pattern: yyyy-mm-dd hh:mm:ss.SSS
# Storing exported data in Amazon S3
**Topics**
+ [
## Using a predefined CloudFormation template
](#kinesis-cloudfront-makestack)
+ [
## Configuring Kinesis manually in the console
](#kinesis-configure-console)
## Using a predefined CloudFormation template
Amazon Monitron provides a predefined AWS CloudFormation template to help quickly set up the Firehose to deliver data from a Kinesis data stream to the Amazon S3 bucket. This template enables dynamic partitioning and the Amazon S3 objects delivered will use the following key format recommended by Amazon Monitron: `/project={projectName}/site={siteName}/time={yyyy-mm-dd 00:00:00}/{filename}`
1. Sign into your AWS account.
1. Open a new browser tab with the following URL:
```
https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://s3.us-east-1.amazonaws.com/monitron-cloudformation-templates-us-east-1/monitron_kinesis_data_export.yaml&stackName=monitron-kinesis-live-data-export
```
1. On the CloudFormation page that opens, in the upper right corner, select the region in which you are using Amazon Monitron.
1. By default, the template will create a new Kinesis data stream and S3 bucket along with other resources needed to deliver data to Amazon S3. You can change the parameters to use existing resources.
1. Check the box that says *I acknowledge that AWS CloudFormation might create IAM resources.*
1. Choose **Create stack**.
1. On the next page, choose the refresh icon as often as you like until the status of the stack is CREATE\$1COMPLETE.
## Configuring Kinesis manually in the console
1. Sign in to the AWS Management Console and open the Kinesis console at https://console.aws.amazon.com/kinesis.
1. Choose **Delivery streams** in the navigation pane.
1. Choose **Create delivery stream**.
1. For Source, select **Amazon Kinesis Data Streams**.
1. For Destination, select **Amazon S3**.
1. Under **Source settings, Kinesis data stream**, enter the ARN of your Kinesis data stream.
1. Under **delivery stream name**, enter the name of your Kinesis data stream.
1. Under **Desination settings**, choose an Amazon S3 bucket or enter a bucket URI.
1. (optional) Enable dynamic partitioning using inline parsing for JSON. This option is appropriate if you want to partition streaming measurement data based on source information and timestamp. For example:
+ Choose **Enabled** for **Dynamic partitioning**.
+ Choose **Enabled** for **New line delimiter**.
+ Choose **Enabled** for **Inline parsing for JSON**.
+ Under **Dynamic partitioning keys**, add:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/kinesis-store-S3.html)
1. Choose **Apply dynamic partitioning keys** and confirm the generated Amazon S3 bucket prefix is `!{partitionKeyFromQuery:project}/!{partitionKeyFromQuery:site}/!{partitionKeyFromQuery:time}/`.
1. In Amazon S3, objects will use the following key format: `/project={projectName}/site={siteName}/time={yyyy-mm-dd 00:00:00}/{filename}`.
1. Choose **Create delivery stream**.
1. (optional) Use a more granular path.
If you chose a dynamic partition, use the preceeding Amazon S3 key format if you plan to use AWS Glue and Athena to query the data. You can also choose a finer key format, but the Amazon Athena query will not be efficient. Here is an example of setting up a finer Amazon S3 key path.
Under **Dynamic partitioning keys**, add:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/kinesis-store-S3.html)
In Amazon S3, objects will use the following key format: ` /project={projectName}/site={siteName}/asset={assetName}/position={positionName}/sensor={sensorId}/date={yyyy-mm-dd}/time={HH:MM:SS}/{filename}`
# Processing data with Lambda
**Topics**
+ [
## Step 1: Create the [IAM role](https://docs.aws.amazon.com//lambda/latest/dg/lambda-intro-execution-role.html) that gives your function permission to access AWS resources
](#create-iam-role)
+ [
## Step 2: Create the Lambda function
](#create-lambda-function)
+ [
## Step 3: Configure the Lambda function
](#configure-lambda-function)
+ [
## Step 4: Enable Kinesis trigger in AWS Lambda console
](#enable-lambda-trigger)
## Step 1: Create the [IAM role](https://docs.aws.amazon.com//lambda/latest/dg/lambda-intro-execution-role.html) that gives your function permission to access AWS resources
1. Open the [roles page](https://console.aws.amazon.com/iam/home?#/roles) in the IAM console.
1. Choose **Create role**.
1. Create a role with the following properties.
+ Trusted entity: Lambda
+ Permissions: AWSLambdaKinesisExecutionRole (and AWSKeyManagementServicePowerUser if the Kinesis stream is encrypted)
+ Role name: lambda-kinesis-role
![\[IAM role creation interface showing name, review, and create steps with role details and permissions.\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/images/lambda-create-role.png)
## Step 2: Create the Lambda function
1. Open the **Functions** page in the Lambda console.
1. Choose **Create function**.
1. Choose **Use a blueprint**.
1. In the **Blueprints** search bar, search and choose **kinesis-process-record (nodejs)** or **kinesis-process-record-python**.
1. Choose **Configure**.
![\[Create function interface with options to author from scratch, use a blueprint, or select container image.\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/images/lambda-create-function.png)
## Step 3: Configure the Lambda function
1. Choose **Function name**
1. Choose the role created in the first step as the **Execution role**.
1. Configure Kinesis trigger.
1. Choose your Kinesis stream.
1. Click **Create function**.
![\[Lambda function configuration form with basic information and Kinesis trigger settings.\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/images/lambda-kinesis-trigger.png)
## Step 4: Enable Kinesis trigger in AWS Lambda console
1. On the **Configuration** tab, choose **Triggers**.
1. Check the box next to the name of the Kinesis stream and choose **Enable**.
![\[Lambda function configuration page with Triggers tab and Kinesis stream trigger highlighted.\]](http://docs.aws.amazon.com/Monitron/latest/user-guide/images/kinesis-process-record-lambda.png)
The blueprint used in this example only consumes log data from the selected stream. You can further edit Lambda function code later to complete a more complicated task.
# Understanding the v1 data export schema
**Note**
Amazon Monitron Kinesis data export schema v1 has been deprecated. Learn more about the [v2 data export schema](monitron-kinesis-export-v2.md).
Each measurement data and its corresponding inference result are exported as one Kinesis data stream record in JSON format.
**Topics**
+ [
## v1 schema format
](#data-export-schema-format)
+ [
## v1 schema parameters
](#data-export-schema-parameters)
## v1 schema format
```
{
"timestamp": "string",
"eventId": "string",
"version": "string",
"projectDisplayName": "string",
"siteDisplayName": "string",
"assetDisplayName": "string",
"sensorPositionDisplayName": "string",
"sensor": {
"physicalId": "string",
"rssi": number
},
"gateway": {
"physicalId": "string"
},
"measurement": {
"features": {
"acceleration": {
"band0To6000Hz": {
"xAxis": {
"rms": number
},
"yAxis": {
"rms": number
},
"zAxis": {
"rms": number
}
},
"band10To1000Hz": {
"resultantVector": {
"absMax": number,
"absMin": number,
"crestFactor": number,
"rms": number
},
"xAxis": {
"rms": number
},
"yAxis": {
"rms": number
},
"zAxis": {
"rms": number
}
}
},
"temperature": number,
"velocity": {
"band10To1000Hz": {
"resultantVector": {
"absMax": number,
"absMin": number,
"crestFactor": number,
"rms": number
},
"xAxis": {
"rms": number
},
"yAxis": {
"rms": number
},
"zAxis": {
"rms": number
}
}
}
},
"sequenceNo": number
},
"models": {
"temperatureML": {
"persistentClassificationOutput": "string",
"pointwiseClassificationOutput": "string"
},
"vibrationISO": {
"isoClass": "string",
"mutedThreshold": "string",
"persistentClassificationOutput": "string",
"pointwiseClassificationOutput": "string"
},
"vibrationML": {
"persistentClassificationOutput": "string",
"pointwiseClassificationOutput": "string"
}
},
"assetState": {
"newState": "string",
"previousState": "string"
}
}
```
## v1 schema parameters
timestamp
+ The timestamp when the measurement is received by Monitron service in UTC
+ Type: String
+ Pattern: yyyy-mm-dd hh:mm:ss.SSS
eventId
+ The unique data export event ID assigned for each measurement. Can be used to deduplicate the Kinesis stream records received.
+ Type: String
version
+ Schema version
+ Type: String
+ Current Value: 1.0
projectDisplayName
+ The project name displayed in the App and console
+ Type: String
siteDisplayName
+ The site name displayed in the App
+ Type: String
assetDisplayName
+ The asset name displayed in the App
+ Type: String
sensorPositionDisplayName
+ The sensor position name displayed in the App
+ Type: String
sensor.physicalId
+ The physical ID of the sensor from which the measurement is sent
+ Type: String
sensor.rssi
+ The sensor bluetooth received signal strength indicator value
+ Type: Number
+ Unit: dBm
gateway.physicalId
+ The physical ID of the gateway used to transmit data to Amazon Monitron service
+ Type: String
measurement.features.acceleration.band0To6000Hz.xAxis.rms
+ The root mean square of the acceleration observed in the frequency band 0–6000 Hz in the x axis
+ Type: Number
+ Unit: m/s^2
measurement.features.acceleration.band0To6000Hz.yAxis.rms
+ The root mean square of the acceleration observed in the frequency band 0–6000 Hz in the y axis
+ Type: Number
+ Unit: m/s^2
measurement.features.acceleration.band0To6000Hz.zAxis.rms
+ The root mean square of the acceleration observed in the frequency band 0–6000 Hz in the y axis
+ Type: Number
+ Unit: m/s^2
measurement.features.acceleration.band10To1000Hz.resultantVector.absMax
+ The absolute maximum acceleration observed in the frequency band 10–1000 Hz
+ Type: Number
+ Unit: m/s^2
measurement.features.acceleration.band10To1000Hz.resultantVector.absMin
+ The absolute minimum acceleration observed in the frequency band 10–1000 Hz
+ Type: Number
+ Unit: m/s^2
measurement.features.acceleration.band10To1000Hz.resultantVector.crestFactor
+ The acceleration crest factor observed in the frequency band 10–1000 Hz
+ Type: Number
measurement.features.acceleration.band10To1000Hz.resultantVector.rms
+ The root mean square of the acceleration observed in the frequency band 10–1000 Hz
+ Type: Number
+ m/s^2
measurement.features.acceleration.band10To1000Hz.xAxis.rms
+ The root mean square of the acceleration observed in the frequency band 10–1000 Hz in the x axis
+ Type: Number
+ m/s^2
measurement.features.acceleration.band10To1000Hz.yAxis.rms
+ The root mean square of the acceleration observed in the frequency band 10–1000 Hz in the y axis
+ Type: Number
+ m/s^2
measurement.features.acceleration.band10To1000Hz.zAxis.rms
+ The root mean square of the acceleration observed in the frequency band 10–1000 Hz in the z axis
+ Type: Number
+ m/s^2
measurement.features.temperature
+ The temperature observed
+ Type: Number
+ °C/degC
measurement.features.velocity.band10To1000Hz.resultantVector.absMax
+ The absolute maximum velocity observed in the frequency band 10–1000 Hz
+ Type: Number
+ mm/s
measurement.features.velocity.band10To1000Hz.resultantVector.absMin
+ The absolute minimum velocity observed in the frequency band 10–1000 Hz
+ Type: Number
+ mm/s
measurement.features.velocity.band10To1000Hz.resultantVector.crestFactor
+ The velocity crest factor observed in the frequency band 10–1000 Hz
+ Type: Number
measurement.features.velocity.band10To1000Hz.resultantVector.rms
+ The root mean square of the velocity observed in the frequency band 10–1000 Hz
+ Type: Number
+ mm/s
measurement.features.velocity.band10To1000Hz.xAxis.rms
+ The root mean square of the velocity observed in the frequency band 10–1000 Hz in the x axis
+ Type: Number
+ mm/s
measurement.features.velocity.band10To1000Hz.yAxis.rms
+ The root mean square of the velocity observed in the frequency band 10–1000 Hz in the y axis
+ Type: Number
+ mm/s
measurement.features.velocity.band10To1000Hz.zAxis.rms
+ The root mean square of the velocity observed in the frequency band 10–1000 Hz in the z axis
+ Type: Number
+ mm/s
measurement.sequenceNo
+ The measurement sequence number
+ Type: Number
models.temperatureML.persistentClassificationOutput
+ The persistent classification output from the machine learning based temperature model
+ Type: Number
+ Valid Values: `UNKNOWN | HEALTHY | WARNING | ALARM`
models.temperatureML.pointwiseClassificationOutput
+ The point–wise classification output from the machine learning based temperature model
+ Type: String
+ Valid Values: `UNKNOWN | INITIALIZING | HEALTHY | WARNING | ALARM`
models.vibrationISO.isoClass
+ The ISO 20816 class (a standard for measurement and evaluation of machine vibration) used by the ISO based vibration model
+ Type: String
+ Valid Values: `CLASS1 | CLASS2 | CLASS3 | CLASS4 | FAN_BV2`
models.vibrationISO.mutedThreshold
+ The threshold to mute the notification from the ISO based vibration model
+ Type: String
+ Valid Values: `WARNING | ALARM`
models.vibrationISO.persistentClassificationOutput
+ The persistent classification output from the ISO based vibration model
+ Type: String
+ Valid Values: `UNKNOWN | HEALTHY | WARNING | ALARM`
models.vibrationISO.pointwiseClassificationOutput
+ The point–wise classification output from the the ISO based vibration model
+ Type: String
+ Valid Values: `UNKNOWN | HEALTHY | WARNING | ALARM | MUTED_WARNING | MUTED_ALARM`
models.vibrationML.persistentClassificationOutput
+ The persistent classification output from the machine learning based vibration model
+ Type: String
+ Valid Values: `UNKNOWN | HEALTHY | WARNING | ALARM`
models.vibrationML.pointwiseClassificationOutput
+ The point–wise classification output from the machine learning based vibration model
+ Type: String
+ Valid Values: `UNKNOWN | INITIALIZING | HEALTHY | WARNING | ALARM`
assetState.newState
+ The machine status after processing the measurement
+ Type: String
+ Valid Values: `UNKNOWN | HEALTHY | NEEDS_MAINTENANCE | WARNING | ALARM`
assetState.previousState
+ The machine status before processing the measurement
+ Type: String
+ Valid Values: `UNKNOWN | HEALTHY | NEEDS_MAINTENANCE | WARNING | ALARM`