# Classic streams, V2 gateways for AWS IoT SiteWise Edge Classic streams, V2 gateways Understand the features and limitations of Classic streams, V2 gateways for AWS IoT SiteWise Edge. The Classic streams, V2 gateway maintains traditional functionality familiar from earlier AWS IoT SiteWise deployments before the introduction of MQTT-enabled, V3 gateways. These SiteWise Edge gateways are considered Classic streams, V2 gateways. They maintain backward compatibility and are functional with the data processing pack. While the Classic streams, V2 gateway offers reliable performance for existing setups, it has limitations compared to newer gateway options. Specifically, this gateway type is not fully compatible with the advanced features available in the MQTT-enabled, V3 gateway destination. To use the MQTT messaging protocol, you can create a new MQTT-enabled, V3 gateway. For more information, see [MQTT-enabled, V3 gateways for AWS IoT SiteWise Edge](mqtt-enabled-v3-gateway.md). **Topics** + [ # Use packs to collect and process data in SiteWise Edge ](data-packs.md) + [ # Configure the AWS IoT SiteWise publisher component ](configure-publisher-component.md) + [ # Destinations and AWS IoT Greengrass stream manager ](destinations-gg-stream-manager.md) + [ # Configure edge capabilities on AWS IoT SiteWise Edge ](edge-data-collection-and-processing.md) + [ # Configure edge data processing for AWS IoT SiteWise models and assets ](edge-processing.md) # Use packs to collect and process data in SiteWise Edge Use packs **Note** The data processing pack (DPP) feature is no longer availabke to new customers. Existing customers can continue to use the service as normal. For more information, see [Data processing pack availability change](https://docs.aws.amazon.com/iot-sitewise/latest/appguide/iotsitewise-dpp-availability-change.html). AWS IoT SiteWise Edge gateways use different packs to determine how to collect and process your data. Currently, the following packs are available: + **Data collection pack** – Use this pack to collect your industrial data and route it to AWS Cloud destinations. By default, this pack is enabled automatically for your SiteWise Edge gateway. + **Data processing pack** – Use this pack to enable SiteWise Edge gateway communication with edge-configured asset models and assets. You can use edge configuration to control what asset data to compute and process on-site. You can then send your data to AWS IoT SiteWise or other AWS services. For more information about the data processing pack, see [Configure edge data processing for AWS IoT SiteWise models and assets](edge-processing.md). ## Upgrading packs **Important** Upgrading data processing pack versions from before (and including) 2.0.x to version 2.1.x will result in data loss of locally stored measurements. SiteWise Edge gateways use different packs to determine how to collect and process your data. You can use the AWS IoT SiteWise console to upgrade packs. **To upgrade packs (console)** 1. Navigate to the [AWS IoT SiteWise console](https://console.aws.amazon.com/iotsitewise/). 1. In the left navigation, choose **Edge gateways** in the **Edge** section. 1. In the **Gateways** list, choose the SiteWise Edge gateway with the packs you want to upgrade. 1. In the **Gateway configuration** section, choose **Software updates available**. 1. On the Edit software versions page, choose **Updates**. **Note** You can only upgrade packs that are enabled. To find the list of packs that are enabled for this SiteWise Edge gateway, choose **Overview**, and then see the **Edge capabilities** section. 1. On the edit software versions page, in the **Gateway component updates** section, do the following: + To update the **OPC UA collector**, choose a version, and then choose **Deploy**. + To update the **Publisher**, choose a version, and then choose **Deploy**. + To update the **Data processing pack**, choose a version, and then choose **Deploy**. 1. When you're done deploying new versions, choose **Done**. If you're experiencing problems upgrading the packs, see [Unable to deploy packs to SiteWise Edge gateways](troubleshooting-gateway.md#gateway-issue-ggv2-packs). # Data processing pack availability change **Note** The data processing pack (DPP) feature is no longer availabke to new customers. Existing customers can continue to use the service as normal. For more information, see [Data processing pack availability change](https://docs.aws.amazon.com/iot-sitewise/latest/appguide/iotsitewise-dpp-availability-change.html). For capabilities similar to AWS IoT data processing pack feature explore either [open-source alternatives](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/open-source-edge-integrations.html) or our [partner integrations](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/cpa-add-source.html). The AWS IoT SiteWise Data Processing Pack is a feature of AWS IoT SiteWise that provides data transformations, metrics, filtering, local storage and visualization at the edge. **Note** AWS IoT SiteWise and the AWS IoT SiteWise Edge data collection pack feature continues to be available, but the data processing pack feature is entering maintenance mode. ## Migration options Explore these migration options for replacing the data processing pack functionality. **Open-source alternatives** Create local data processing pipelines using Node-RED for data transformation, InfluxDB for time-series storage, and Grafana for visualization. These tools work with MQTT-enabled, V3 gateways through MQTT to provide edge processing and local insights while synchronizing data with the AWS Cloud. For more information, see [Process and visualize data with SiteWise Edge and open-source tools](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/open-source-edge-integrations.html). **Partner integrations** Connect industrial equipment and sensors through third-party partner data sources like CloudRail, EasyEdge, and Litmus Edge. These Greengrass components are developed in partnership with AWS and support over 200 industrial protocols for comprehensive data collection and processing. For more information, see [Add a data source](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/cpa-add-source.html). **MQTT-enabled, V3 gateways** MQTT-enabled, V3 gateways use lightweight MQTT protocol for efficient data communication and offer flexible data destinations including real-time ingestion and buffered Amazon S3 ingestion. You can implement path filters for precise data collection and benefit from improved scalability and IoT standards alignment. MQTT-enabled, V3 gateways provide cloud-based data processing through AWS IoT SiteWise core services including asset models, computed properties, alarms, and historical data queries. For more information, see [MQTT-enabled, V3 gateways for AWS IoT SiteWise Edge](mqtt-enabled-v3-gateway.md). ## Frequently asked questions ### Can I migrate gradually? Yes, you can migrate gradually using any combination of the migration options. You can deploy MQTT-enabled, V3 gateways, open-source alternatives, or partner integrations alongside existing Classic streams, V2 gateways with the data processing pack. All options can send data to the same AWS IoT SiteWise environment. ### How long can I continue using the data processing pack? The data processing pack remains available to existing customers in maintenance mode. You'll receive advance notice if any changes to availability are planned. Monitor AWS service announcements and your account notifications for updates. # Configure the AWS IoT SiteWise publisher component Configure the publisher After you create an AWS IoT SiteWise Edge gateway and install the software, you can set up the publisher component so your SiteWise Edge gateway can export data to the AWS Cloud. Use the publisher component to enable additional features or configure default settings. For more information, see [AWS IoT SiteWise publisher](https://docs.aws.amazon.com/greengrass/v2/developerguide/iotsitewise-publisher-component.html) in the *AWS IoT Greengrass Version 2 Developer Guide*. **Note** The publisher configuration differs based on the type of gateway you're using. For Classic stream, V2 gateways, use the `iotsitewise:publisher:2` namespace. For MQTT-enabled, V3 gateways, use the `iotsitewise:publisher:3` namespace. ------ #### [ Console ] 1. Navigate to the [AWS IoT SiteWise console](https://console.aws.amazon.com/iotsitewise/). 1. In the navigation pane, choose **Edge gateways**. 1. Select the SiteWise Edge gateway for which you want to configure the publisher. 1. In the **Publisher configuration** section, choose **Edit** 1. For **Publishing order**, choose one of the following: + **Publish oldest data first** – The SiteWise Edge gateway publishes the oldest data to the cloud first by default. + **Publish newest data first** – The SiteWise Edge gateway publishes the newest data to the cloud first. 1. (Optional) If you don't want the SiteWise Edge gateway to compress your data, unselect **Activate compression when uploading data**. 1. (Optional) If you don't want to publish old data, choose **Exclude expired data** and do the following: 1. For **Cutoff period**, enter a value and choose a unit. The cutoff period must be between five minutes and seven days. For example, if the cutoff period is three days, data that's older than three days isn't published to the cloud. 1. (Optional) To set custom settings about how data is handled on your local device, choose **Local storage settings** and do the following: 1. For **Retention period**, enter a number and choose a unit. The retention period must be between one minute and 30 days, and greater than or equal to the rotation period. For example, if the retention period is 14 days, the SiteWise Edge gateway deletes any data at the edge that's older than the specified cutoff period after it's stored for 14 days. 1. For **Rotation period**, enter a number and choose a unit. The rotation period must be greater than one minute, and equal to, or less than, the retention period. For example, say the rotation period is two days, the SiteWise Edge gateway batches up and saves data that is older than the cutoff period to a single file. For self-hosted gateways through AWS IoT Greengrass V2, the SiteWise Edge gateway transfers a batch of data to the following local directory once every two days: `/greengrass/v2/work/aws.iot.SiteWiseEdgePublisher/exports`. 1. For **Storage capacity**, enter a value that is greater than or equal to 1. If the storage capacity is 2 GB, the SiteWise Edge gateway starts deleting data when more than 2 GB of data is stored locally. 1. Choose **Save**. ------ #### [ AWS CLI ] Use the [UpdateGatewayCapabilityConfiguration](https://docs.aws.amazon.com/iot-sitewise/latest/APIReference/API_UpdateGatewayCapabilityConfiguration.html) API to configure the publisher. Set the `capabilityNamespace` parameter to `iotsitewise:publisher:2`. **Example : Publisher configuration for Classic Stream, V2 gateways** The publisher namespace: `iotsitewise:publisher:2` ``` { "SiteWisePublisherConfiguration": { "publishingOrder": "TIME_ORDER", "enableCompression": true, "dropPolicy": { "cutoffAge": "7d", "exportPolicy": { "retentionPeriod": "7d", "rotationPeriod": "6h", "exportSizeLimitGB": 10 } } }, "SiteWiseS3PublisherConfiguration": { "accessRoleArn": "arn:aws:iam:123456789012:role/roleName", "streamToS3ConfigMapping": [ { "streamName": "S3_OPC-UA_Data_Collector", "targetBucketArn": "arn:aws:s3:::amzn-s3-demo-bucket/dataCollector", "publishPolicy": { "publishFrequency": "10m", "localSizeLimitGB": 10 }, "siteWiseImportPolicy": { "enableSiteWiseStorageImport": true, "enableDeleteAfterImport": true } } ] } } ``` The publisher provides the following configuration parameters that you can customize: `SiteWisePublisherConfiguration` `publishingOrder` The order in which data is published to the cloud. The value of this parameter can be one of the following: + `TIME_ORDER` (**Publish oldest data first**) – The earliest data is published to the cloud first, by default. + `RECENT_DATA` (**Publish newest data first**) – The newest data is published to the cloud first. `enableCompression` Set this to `true` to compress data before publishing. Data compression can reduce bandwidth usage. `dropPolicy` (Optional) A policy that controls what data is published to the cloud. `cutoffAge` The maximum age of data to be published specified in days, hours, and minutes. For example, `7d` or `1d7h16m`. Data older than what you specify is not sent to AWS IoT SiteWise. Data that is earlier than the cutoff period is not published to the cloud. The cutoff age must be between five minutes and seven days. You can use `m`, `h`, and `d` when you specify a cutoff age. Note that `m` represents minutes, `h` represents hours, and `d` represents days. `exportPolicy` (Optional) A policy that manages data storage at the edge. This policy applies to data that is earlier than the cutoff age. `retentionPeriod` Your SiteWise Edge gateway deletes any data at the edge that is earlier than the cutoff period from the local storage after it is stored for the specified retention period. The retention period must be between one minute and 30 days, and greater than or equal to the rotation period. You can use `m`, `h`, and `d` when you specify a retention period. Note that `m` represents minutes, `h` represents hours, and `d` represents days. `rotationPeriod` The time interval over which to batch up and save data that is earlier than the cutoff period to a single file. The SiteWise Edge gateway transfers one batch of data to the following local directory at the end of each rotation period: `/greengrass/v2/work/aws.iot.SiteWiseEdgePublisher/exports`. The rotation period must be greater than one minute, and equal to or less than the retention period. You can use `m`, `h`, and `d` when you specify a rotation period. Note that `m` represents minutes, `h` represents hours, and `d` represents days. `exportSizeLimitGB` The maximum allowed size of data stored locally, in GB. If this quota is breached, the SiteWise Edge gateway starts deleting the earliest data until the size of data stored locally is equal to or less than the quota. The value of this parameter must be greater than or equal to 1. `SiteWiseS3PublisherConfiguration` `accessRoleArn` The access role that gives AWS IoT SiteWise permission to manage the Amazon S3 bucket that you are publishing to. `streamToS3ConfigMapping` An array of configurations that maps a stream to an Amazon S3 configuration. `streamName` The stream to read from and publish to the Amazon S3 configuration. `targetBucketArn` The bucket ARN to publish to. `publishPolicy` `publishFrequency` The frequency with which the SiteWise Edge gateway publishes to the Amazon S3 bucket. `localSizeLimitGB` The maximum size of the files written to local disk. If this threshold is breached, the publisher publishes all buffered data to its destination. `siteWiseImportPolicy` `enableSiteWiseStorageImport` Set this to `true` to import data from an Amazon S3 bucket to AWS IoT SiteWise storage. `enableDeleteAfterImport` Set this to `true` to delete the file in the Amazon S3 bucket after ingestion into the AWS IoT SiteWise storage. ------ # Destinations and AWS IoT Greengrass stream manager AWS IoT Greengrass stream manager AWS IoT Greengrass stream manager allows you to send data to the following AWS Cloud destinations: channels in AWS IoT Analytics, streams in Amazon Kinesis Data Streams, asset properties in AWS IoT SiteWise, or objects in Amazon Simple Storage Service (Amazon S3). For more information, see [Manage data streams on the AWS IoT Greengrass Core](https://docs.aws.amazon.com/greengrass/v2/developerguide/manage-data-streams.html) in *AWS IoT Greengrass Version 2 Developer Guide*. **Example : Data stream message structure** The following example shows the required data stream message structure transmitted by the AWS IoT Greengrass stream manager. ``` { "assetId": "string", "propertyAlias": "string", "propertyId": "string", "propertyValues": [ { "quality": "string", "timestamp": { "offsetInNanos": number, "timeInSeconds": number }, "value": { "booleanValue": boolean, "doubleValue": number, "integerValue": number, "stringValue": "string" } } ] } ``` **Note** The data stream message must include either (`assetId` and `propertyId`) or `propertyAlias` in its structure. `assetId` (Optional) The ID of the asset to update. `propertyAlias` (Optional) The alias that identifies the property, such as an OPC UA server data stream path. For example: ``` /company/windfarm/3/turbine/7/temperature ``` For more information, see [ Manage data streams](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/manage-data-streams.html) in the *AWS IoT SiteWise User Guide*. `propertyId` (Optional) The ID of the asset property for this entry. `propertyValues` (Required) The list of property values to upload. You can specify up to 10 `propertyValues` array elements. `quality` (Optional) The quality of the asset property value. `timestamp` (Required) The timestamp of the asset property value. `offsetInNanos` (Optional) The nanosecond offset from `timeInSeconds`. `timeInSeconds` (Required) The timestamp date, in seconds, in the Unix epoch format. Fractional nanosecond data is provided by `offsetInNanos`. `value` (Required) The value of the asset property. Only one of the following values can exist in the `value` field. `booleanValue` (Optional) Asset property data of type Boolean (`true` or `false`). `doubleValue` (Optional) Asset property data of type double (floating point number). `integerValue` (Optional) Asset property data of type integer (whole number). `stringValue` (Optional) Asset property data of type string (sequence of characters). # Configure edge capabilities on AWS IoT SiteWise Edge Configure edge capabilities **Note** The data processing pack (DPP) feature is no longer availabke to new customers. Existing customers can continue to use the service as normal. For more information, see [Data processing pack availability change](https://docs.aws.amazon.com/iot-sitewise/latest/appguide/iotsitewise-dpp-availability-change.html). You can use AWS IoT SiteWise Edge to collect and temporarily store data so that you can organize and process device data locally. By enabling edge processing, you can choose to send only aggregated data to the AWS Cloud to optimize your bandwidth usage and cloud storage costs. Using AWS IoT SiteWise components with AWS IoT Greengrass, you can collect and process data at the edge before sending it to the AWS Cloud, or manage it on-premises using SiteWise Edge APIs. Data collection happens through data packs and AWS IoT SiteWise components that run on AWS IoT Greengrass. **Note** AWS IoT SiteWise retains your edge data on your SiteWise Edge gateways up to 30 days. The retention period of your data is dependent on the available disk space of your device. If your SiteWise Edge gateway has been disconnected from the AWS Cloud for 30 days, the [Data Processing Pack](configure-opcua-source.md) is automatically disabled. **Topics** + [ ## Set up edge capability in SiteWise Edge ](#using-sitewise-edge) ## Set up edge capability in SiteWise Edge Set up edge capability AWS IoT SiteWise provides the following packs that your SiteWise Edge gateway can use to determine how to collect and process your data. Select packs to enable edge capabilities for your SiteWise Edge gateway. + **Data collection pack** enables your SiteWise Edge gateway to collect data from multiple OPC UA servers, and then export the data from the edge to the AWS Cloud. It becomes active once you have added data sources to your SiteWise Edge gateway. + **Data processing pack** enables your SiteWise Edge gateway to process your equipment data at the edge. For example, you can use asset models to compute metrics and transforms. For more information about asset models and assets, see [Model industrial assets](industrial-asset-models.md). **Note** The data processing pack is only available on x86 platforms. **To configure edge capabilities** 1. Navigate to the [AWS IoT SiteWise console](https://console.aws.amazon.com/iotsitewise/). 1. In the navigation pane, choose **Edge gateways**. 1. Select the SiteWise Edge gateway for which you want to activate edge capabilities. 1. In the **Edge capabilities** section, choose **Edit** 1. In the **Edge capabilities** section, select **Enable data processing pack (incurs additional charges)**. 1. (Optional) In the **Edge LDAP connection** section, you can grant user groups in your corporate directory access to this SiteWise Edge gateway. The user groups can use the Lightweight Directory Access Protocol (LDAP) credentials to access the SiteWise Edge gateway. Then they can use the AWS OpsHub for AWS IoT SiteWise application, AWS IoT SiteWise API operations, or other tools to manage the SiteWise Edge gateway. For more information, see [Manage SiteWise Edge gateways](manage-gateways-ggv2.md). **Note** You can also use the Linux or Microsoft Windows credentials to access the SiteWise Edge gateway. For more information, see [Access your SiteWise Edge gateway using Linux operating system credentials](manage-gateways-ggv2.md#linux-user-pool). 1. Select **Activated**. 1. For **Provider name**, enter a name for your LDAP provider. 1. For **Hostname or IP address**, enter the hostname or IP address of your LDAP server. 1. For **Port**, enter a port number. 1. For **Base distinguished name (DN)**, enter a distinguished name (DN) for the base. The following attribute types are supported: commonName (CN), localityName (L), stateOrProvinceName (ST), organizationName (O), organizationalUnitName (OU), countryName (C), streetAddress (STREET), domainComponent (DC), and userid (UID). 1. For **Admin group DN**, enter a DN. 1. For **User group DN**, enter a DN. 1. Choose **Save**. Now that you've activated edge capabilities on your SiteWise Edge gateway, you need to configure your asset model for the edge. Your asset model edge configuration specifies where your assets properties are computed. You can compute all properties at the edge, or you can configure your asset model properties separately. Asset model properties include [metrics](concept-overview.md#concept-metric), [transforms](concept-overview.md#concept-transform), and [measurements](concept-overview.md#concept-measurement). For more information about asset properties, see [Define data properties](asset-properties.md). After you create your asset model, you can then configure it for the edge. For more information about configuring your asset model for the edge, see [Create an asset model (console)](create-asset-models.md#create-asset-model-console). **Note** Asset models and dashboards are automatically synced between the AWS Cloud and your SiteWise Edge gateway every 10 minutes. You can also sync manually from the local SiteWise Edge gateway application. # Configure edge data processing for AWS IoT SiteWise models and assets Configure edge data processing **Note** The data processing pack (DPP) feature is no longer availabke to new customers. Existing customers can continue to use the service as normal. For more information, see [Data processing pack availability change](https://docs.aws.amazon.com/iot-sitewise/latest/appguide/iotsitewise-dpp-availability-change.html). You can use AWS IoT SiteWise Edge to collect, store, organize and monitor equipment data locally. You can use SiteWise Edge so that you can model your industrial data and visualize it locally. You can process your data locally and send it to the AWS Cloud, or process it on-premises by using the AWS IoT SiteWise API. With AWS IoT SiteWise Edge, you can process raw data locally and choose to send only aggregated data to the AWS Cloud to optimize your bandwidth usage and cloud storage costs. **Note** AWS IoT SiteWise retains your edge data on your SiteWise Edge gateways up to 30 days. The retention period of your data is dependent on the available disk space of your device. If your SiteWise Edge gateway has been disconnected from the AWS Cloud for 30 days, the [Set up an OPC UA source in SiteWise Edge](configure-opcua-source.md) is automatically disabled. ## Configure an asset model for data processing on SiteWise Edge Process data at the edge You must configure your asset model for the edge before your can process your SiteWise Edge gateway data at the edge. Your asset model edge configuration specifies where your assets properties are computed. You can choose to compute all properties at the edge and send the results to the AWS Cloud, or customize where to compute each asset property separately. For more information, see [Configure edge data processing for AWS IoT SiteWise models and assets](#edge-processing). Asset properties include metrics, transforms, and measurements: + Metrics are the asset's aggregated data over a specified period of time. You can compute new metrics by using existing metric data. AWS IoT SiteWise always sends your metrics to the AWS Cloud for long-term storage. AWS IoT SiteWise computes metrics on the AWS Cloud by default. You can configure your asset model to compute your metrics at the edge. AWS IoT SiteWise sends processed results to the AWS Cloud. + Transforms are mathematical expressions that map an asset property's data points from one form to another. Transforms can use metrics as input data and must be computed and stored at the same location as their inputs. If you configure a metric input to compute at the edge, AWS IoT SiteWise also computes its associated transform at the edge. + Measurements are formatted as raw data that your device collects and sends to the AWS Cloud by default. You can configure your asset model to store this data on your local device. For more information about asset properties, see [Define data properties](asset-properties.md). After you create your asset model, you can then configure it for the edge. For more information about configuring your asset model for the edge, see [Create an asset model (console)](create-asset-models.md#create-asset-model-console). **Note** Asset models and dashboards are automatically synced between the AWS Cloud and your SiteWise Edge gateway every 10 minutes. You can also sync manually from the [Manage SiteWise Edge gateways](manage-gateways-ggv2.md). You can use the AWS IoT SiteWise REST APIs and the AWS Command Line Interface (AWS CLI) to query your SiteWise Edge gateway for data at the edge. Before you query your SiteWise Edge gateway for data at the edge, you must meet the following prerequisites: + Your credentials must be set for the REST APIs. For more information about setting credentials, see [Manage SiteWise Edge gateways](manage-gateways-ggv2.md). + The SDK endpoint must point to the IP address of your SiteWise Edge gateway. You can find more information in the documentation for your SDK. For example, see [Specifying Custom Endpoints](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/specifying-endpoints.html) in the *AWS SDK for Java 2.x Developer Guide*. + Your SiteWise Edge gateway certificate must be registered. You can find more information about registering your SiteWise Edge gateway certificate in the documentation for your SDK. For example, see the [Registering Certificate Bundles in Node.js](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-registering-certs.html) in the *AWS SDK for Java 2.x Developer Guide*. For more information about querying data with AWS IoT SiteWise, see [Query data from AWS IoT SiteWise](query-industrial-data.md).