# Operational metrics
<a name="operational-metrics"></a>

AWS Professional Services can work with your internal teams to establish a delivery governance mechanism for tracking your organization’s progress against the operational metrics defined in the following table. The metrics are based on real-world values and can help define your MLOps engagement. The values in the following table are guidelines only. The target values will depend on your organization’s implementation.


|  |  |  |  | 
| --- |--- |--- |--- |
| **Area** | **Sub area** | **Metric** | **6 month target from platform launch** | 
| 1. Operational efficiency | Faster delivery | Time to value | 3 month average | 
|  | Cost transparency through clear cost allocations to business divisions | AWS management costs | Less than initial | 
| 2. Boundary-less delivery | Freedom of access seamlessly granted in a controlled manner | Time to access | 1 day | 
|  | Support for CI/CD  | Time to value (idea to live) | 3 month average | 
|  | Simplified route-to-live delivery   | Time to deploy (post beta) | 3 month average | 
|  | On-demand lab spin up/down | Time to spin up environment | 24 hours | 
| 3. Controlled | Access aligned to each use case and granted or removed dynamically | Time to access use case | < 1 day | 
|  | Persistent production environment allowing spoke-driven development to live | Time to deploy  | 2 weeks | 
|  | Storage and compute costs managed at point of use and attributed to cost center | Actuals/forecast transparent | Forecast accuracy variance permitted | 
|  | Compliance with internal security standards (automatic assessment and auditing) | Number of non-compliant environments | 0 | 
|  | Data owners accountable for contributing and sharing data (within SLA) | Copies of data in data lake | 0 copies of data in data lake | 
|  | Common processing with support for regulatory separation of data | Compliant access | 0 regulatory findings | 
| 4. Constantly evolving and repeatable patterns | A model development environment with constantly evolving standards | Number of updates per year | 6 | 
|  | Ability to package and deploy environments at will by using spokes | Time to spin up environment | 2 hours | 
|  | Clear layout of services and infrastructure being consumed | Non-tracked infrastructure | 0 | 
| 5. For all of the bank | Reduced number of distinct modelling environments | Number of environments | < 2 | 
|  | Simplified team collaboration across organizational boundaries | Time to access (for example, data) | 1 day | 
|  | Common and singular operating model/framework | Number of domains controlled by model | > 3 |