How to Implement a Modern Edge Data Stack
A properly implemented Modern Edge Data Stack (MEDS) enables an organization to derive unprecedented operational and business insights on their edge data. In this article, we'll dive into what the modern edge data stack looks like when implemented.
Table of Contents
What is the Modern Edge Data Stack?
The Modern Edge Data Stack helps organizations to bring in unstructured and uncontextualized data from the edge, produce consistent, high-quality data, run analytics and AI to derive operational insights, and transform data into Common Data Format to prepare for BI analysis. The MEDS enables organizations to derive data insights at unprecedented scale.
A robust Modern Edge Data Stack includes the following components:
Edge data pipelines: produce consistent, high-quality data regardless of the type or location of the data. This includes actions such as data acquisition, validation, cleansing, and contextualization.
Common Data Model: unify data description, validation, transformation, and schema across all edge data sources
Real-time analytics: extract real-time, actionable Operational Insights (OI) through physics-based analytics or data-based machine-learning.
Data warehouse: store data persistently to prepare for BI
BI software: generate Business Intelligence through traditional or AI analysis
Who uses the Modern Edge Data Stack?
The Modern Edge Data Stack typically applies to organizations working with unstructured, uncontextualized, and unreliable event-based data. These are often data gathered from the edge, such as sensor data, equipment data, log files, and audio/video data. These types of data require more effort to work with.
Read the key characteristics of edge data in our previous article "What is a Modern Edge Data Stack?"
What is an Example of an Effective Modern Edge Data Stack?
An instrumentation manufacturer builds lab instruments for the oil and gas industry. They have over a dozen different instrument models. These instruments can output instrument status data and measurement data over TCP for new instrument models and RS-485 for old instrument models. Most of their customers do not connect the instruments to the network.
Building a Modern Edge Data Stack solution will help them to:
Gain visibility on instrument status, experiment status, and instrument utilization across their fleet of instruments
Understand how customers use their instruments
Track component reliability
Provide value-added services such as remote management, real-time monitoring, and improved utilization
Provide maintenance alerts to increase service revenue
How They Implemented the Modern Edge Data Stack：
Instrument data collection and processing are performed in edge gateways on site.
Real-time analytics is generated on instrument status, number of products running, alarms, and product utilization.
Sensitive data is masked to protect data privacy.
Non-sensitive data and real-time analytics results are transformed into Common Data Model (CDM). All instrument models use the same CDM.
Transformed data is stored in a data warehouse.
BI software is used to analyze the data in the data warehouse to understand long-term trends:
What are the top features customers use?
How often do they use their instruments?
How often do experiments fail and why?
Which hardware components fail first?
Build a Future-proof Data Stack
Data is the new growth source for many organizations today. As the volume of data and the types of data increase rapidly, it is important for every organization to adopt a robust, scalable, and future-proof data stack. The Modern Edge Data Stack provides the right foundation.