Almost any piece of industrial equipment today can produce its own rich and steady stream of data. Digital diagnostics embedded in devices can help with on-site troubleshooting when dispatching a trained local technician. But what about extending that to the modern industrial enterprise with thousands of such machines? When each different machine has its own unique data stream and operational logic? There’s no way the ratio of systems to staff can keep up.
The sophistication of these diagnostics creates another problem: technicians can’t all specialize in every type of equipment. On the other hand, remote Operations & Maintenance (O&M) organizations can’t be effective without timely data that’s both broad and deep.
The hype around IoT conjures images of machines that can talk to each other unassisted. But it’s not magic; it’s SaaS-based Enterprise Asset Management (SaaS/EAM).
This SaaS/EAM solution, architected, designed and managed by CloudGeometry, was delivered as a client engagement for a cutting-edge Silicon Valley startup. Developed and deployed using a portfolio of native Amazon Web Services technologies, the SaaS/EAM engagement featured:
- Geographically distributed data integration with real-time streams processing across multiple availability zones
- Cloud-based integration for unified dashboard intelligence that included legacy business process systems
- Model-driven analysis of patterns, within systems and across systems, using machine learning algorithms
- DevOps lifecycle management, so developers can add new functionality and simplify change management with existing systems
- Optimized utilization and maintenance schedules, to avoid downtime and maximize the efficiency of the machine asset portfolio
Industrial infrastructure relies on a wide variety of machines from different manufacturers, often-purpose built to solve problems other than computing. There are few modern standard interfaces or native digital data formats for this equipment (OPC-UA emerging as a potential exception). What’s more, often can’t even get cell data service where this equipment resides, such as at remote locations like solar and wind turbines installations.
Don’t forget, these are expensive machines. Getting a global picture of operations and maintenance can make the difference between a network of profitable assets and a tangle of costly logistics plagued by outages. Solving these problems using SaaS/EAM requires:
- Collecting and analyzing signals from equipment from many different manufacturers, despite inconsistent data connectivity and gaps in continuous feedback
- Large-scale heterogeneous data streams, translating between remote software inputs and consolidated analysis and decision-making
- Constant updates to reflect changes in the equipment environment and operational processes
- New algorithms to make sense of anomalies across the operational landscape, to identify problem patterns without getting bogged down by false alarms
Key to achieving the potential of IoT? Collecting, transforming and analyzing the data fast enough to make the investment in the assets connected to IoT pay off.
The CloudGeometry Solution
To unlock the potential of modern enterprise asset management with IoT, we chose AWS as a cloud platform. The first reason was its global reach, thanks to multiple regions around the world which lets us receive and process data across from the US and East-Asia locations. AWS offers a broad portfolio of technology products and solutions, making it ideal for building scalable SaaS platforms. It combined agility with data processing power to ingest and analyze terabytes of daily input.
Centralizing data took place at multiple levels. The application team developed lightweight agents to collect data. In some cases, data came directly from embedded digital diagnostics on remote equipment in the field.
There was also a ton of data in various proprietary data formats, running on legacy Enterprise Asset Management systems by many existing O&M centers. Such legacy systems included IBM Maximo, Data Historian, OSIsoft PI, and Scada. Smart agents were created to relay data to the nearest AWS region, streamed together into Amazon Kinesis. Amazon Kinesis flowed the data to a SaaS-based decision engine, running a set of rules in real-time, which detected possible equipment performance issues.
Figure 1: Data Flow Diagram
Once processed, data was stored on Amazon RDS (AWS-native SQL relational database) and DynamoDB (AWS-native NoSQL database) to support both trailing historical analytics and archiving. The two native AWS data engines also provide significant benefits that cut the cost of data management, including easier scaling to accommodate growth.
Figure 2: Data Services Deployment
Our applications team also created a centralized dashboard console for remote O&M centers. Designed to simplify both monitoring and decision making, it let users drill down through interactive graphs: to track each asset and location health; schedule remote checks; plan for software maintenance; and the like.
Continuous input for machine learning models
Volume and velocity are not the only challenges in analyzing data from different systems and processes. Patterns within data streams from specific machines are often not obvious from looking at diagnostics. Patterns that play out on separate systems, or as a result of environmental conditions and interactions between different pieces of gear, can also mask misbehavior that’s obvious only in retrospect. While some of these patterns are recognizable to operators, but using Machine Learning (ML) can flag these patterns much faster and more accurately.
What makes ML effective is not only brute-force mining of the data (though that helps), but also selecting the data for training the filtering and sorting algorithms. That tees up new opportunities for improved learning as new data (and the patterns it reveals) adds up.
Sagemaker is an AWS-native service featuring a well-organized environment for developing and running ML algorithms. It removes a lot of the friction between steps, simplifying the development and deployment process, and making it quicker and easier to iterate through new models at every stage of the machine learning lifecycle. (Learn more about continuous dataflow integration here.)
Keeping up with the changes across all connected systems is critical for SaaS/EAM. Constant software improvements are the rule, not the exception. But integrating new features from outside means that the platform itself has to absorb those changes with zero disruption.
To address the need for continuous software upgrades, the CloudGeometry DevOps team implemented a CI/CD pipeline, architected using the based on the Cloud Geometry CI/CD solution. It eliminated delays in coding and testing improvements by the platform development team. With a transparent, predictable release process, developers readily deployed new software to remote data collection agents, interconnected business applications, and internal platform components — with no downtime.
With a design based on microservices, the SaaS platform supports quick deployment of new Docker containers. The modular architecture makes it much easier to add or change (or rollback) of software functionality. As a result, developers have a clear, step-by-step path for quick deployment, testing, and integration, as the platform adds new service capabilities and adopts new technologies.
Figure 3: CloudGeometry CI/CD
24/7 Infrastructure and Security Monitoring
Critical infrastructure requires mission-critical levels of security, monitoring, and compliance assessment. CloudGeometry ensures the integrity of cloud operations by simplifying and streamlining visibility. Instrumenting the IaaS core services as well as the application stack with industry-leading tooling and technology (e.g. Cloudchekr and Amazon Cloudwatch) ensures the SaaS platform has the depth of automated reporting, alerting, analytics, and remediation to meet the strictest of SLAs.
Figure 4: CloudGeometry environment monitoring
SaaS = security, multitenancy, extensibility
Secure multitenancy and distributed processing are keys virtue of the elastic compute approach used by the SaaS Enterprise Asset Management platform. It’s equally well suited to machinery from smaller specialty manufacturers with only a few deployed remote assets, as much as for big international enterprises with locations across continents and 1000s of assets.
SaaS/EAM supports a broad range of data sources and devices from a broad range of equipment. It can support assets as diverse as utility-scale solar power grids, process manufacturing, or transportation logistics.
It also simplifies many supporting business processes which once required their own IT systems. For example, configuring and onboarding new systems can be delivered in a self-service portal that lets manufacturers update and configure their discrete equipment independently. New information about the machines is available to enterprise O&M without having to involve vendors in every single step of the onboarding. Once deployed, new data from a new machine even can be streamed to machine learning, again without needing enterprise technicians to do all of the low-level data configuration efforts.
Integration with Existing Systems and Customization
Another way the SaaS/EAM Platform improves flexibility is through ease of integration between existing equipment and collection systems and other business processes. For example, enterprise businesses process is rarely, if ever, start with a clean slate. business processes change with different Market needs in different requirements of functions, such as changes in regulatory regimes and compliance, or the adoption of new manufacturing software, and so on.
Of course, the sooner new functionality can be added to the Enterprise IT mix, the sooner it can deliver value. CloudGeometry helped design deliver and maintain a broad range of this kind of systems — across new SaaS systems as well as legacy business processes and data operations. For deeper integration, CloudGeometry also offers Integration and Customization services. Data integration can also be extended to establishing development and training for ML models used to optimize anomaly detection.
Cloud-based SaaS/EAM unlocks a wide range of benefits to the process of maintenance and operations of a complex portfolio industrial machinery:
- Timing Maintenance and utilization to maximize duty cycles of the equipment
- Assessing early warning signals for diagnostic problems before critical conditions require dispatching a technician to the field
- Distinguishing false alarms from hidden problems to prevent costly downtime
- Giving remote operations staff real-time information to make maintenance and operations trade-offs
- Cutting the cost of producing and assessing siloed asset inventory reports that are difficult to reconcile with real field conditions
- Accelerating time to value for costly accelerating time to value for costly
- Improving the performance of operations technicians with more complete information and more context for more effective decision-making