Edge data model management - A growing enterprise challenge
The number and types of edge data models are growing rapidly for any company that has lots of sensors, machines, and edge data operations. How should companies manage this ever growing challenge?
Table of Contents
What is edge data?
Edge data is data that is acquired at the edge. Unlike enterprise data, which is structured and organized, edge data can be unstructured, diverse, and unorganized. Some examples of edge data sources are sensors, videos, machines, lab measurements, log files, to name a few.
Download White Paper: Accelerating Edge Computing on your next IoT project.
Because edge data can be unstructured, decentralized, noisy, and unreliable, edge data is challenging to work with. A great way of working with edge data includes having a hybrid framework, where both the cloud and the edge are used to manage edge data. You can read more about how to build and manage better edge data solutions in our article here.
This time, we focus on another edge data challenge enterprises are facing - the rapid increase of edge data models.
What is an edge data model?
Every edge data source produces data in a predefined format. For example, an NCD long-range environmental sensor produces data in the following format:
{ "nodeId": 0, "firmware": 3, "battery": 3.2844, "battery_percent": "97.840", "counter": 126, "sensor_type": 27, "sensor_data": { "temperature": 7.8, "pressure": 1005.23, "humidity": 48.613, "gas_resistance": 391564, "iaq": 60 }, "sensor_name": "Environmental", "type": "sensor_data", "addr": "00:13:a2:00:41:c6:58:39", "received": 1644620002628, "original": { "mac": "00:13:a2:00:41:c6:58:39", "receive_options": { "ack": 0, "broadcast": 0, "type": "" }, "data": [ 127, 0, 3, 3, 252, 126, 0, 27, 0, 3, 12, 0, 1, 136, 171, 0, 0, 189, 229, 0, 5, 249, 140, 0, 60 ], "type": "receive_packet" }}
This sensor contains sensor values like temperature, pressure, humidity, gas resistance, and indoor air quality (IAQ). It also includes other metadata such as a timestamp for when the data was collected, gateway and sensor information such as firmware version, battery level and sensor type.
A data model is an object that describes the structure and the properties of the data. A data model typically contains a unique model identifier, name, version, and the schema of the receiving data. For the receiving data, relevant properties, types, and data ranges may be included. For example, a data model for the environmental sensor above could look like the following:
{
"model_id": "jy3Q4-j4yt-8nr","model_name": "ncd_lr_env","model_ver": "1.4","data_format": { "nodeId": { "type": "int" }, "battery": { "type": "float", "range": [0, 3.6] }, "battery_percent": { "type": "float", "range": [0, 105] }, "sensor_type": { "type": "int", "range": [27, 27] }, "sensor_data": { "temperature": { "type": "float", "range": [-40, 85] }, "pressure": { "type": "float", "range": [300, 1100] }, "humidity": { "type": "float", "range": [0, 100] }, "gas_resistance": { "type": "float" }, "iaq": { "type": "float" } }, "received": { "type": "timestamp" }}
}
The above data model is called a base model. This means that the model describes the fundamental properties of the data source. Depending on how the data source is used, derived models may need to be created.
For example, if the NCD environmental sensor is used to monitor the temperature and humidity of a server room, then additional properties such as the sampling frequency, required temperature and humidity ranges, location properties, etc. may need to be added.
As an enterprise adds more and more data sources, very quickly the number of data models proliferates. Here are some possible examples:
A paper manufacturing company works with over 5,000 data models inside its manufacturing plants;
A large HVAC company supports over 3,000 data models for its industrial chillers alone.
What is the edge data model management challenge?
The proliferation of edge data models means there is a continuous and rapid growth of edge data models. This is a major enterprise challenge because this growth, when paired with the scale of edge deployments, can make edge data seem unmanageable and data insights feel unattainable.
Due to the large number of data models, creating, modifying, managing, applying and verifying them is a significant pain point for many organizations. In some cases, data models change over time, which can crash existing software or invalidate existing algorithms. Thus, managing data models have become imperative for enterprises.
What can organizations do to manage the ever-increasing number of edge data models ?
While the growing number of edge data models can appear intimidating, there are ways for organizations to keep their edge data models manageable.
Enterprises can manage data models efficiently and effectively with low-code platforms like Prescient Designer, that enables IT/OT teams to manage edge data models with no technical expertise required.
No technical expertise required. Create data models in a visual environment with low-code GUI.
Keep data models consistent. Create data workflows to transform edge data models into homogenized schemas.
Always up-to-date. Automatically detect changes in data models, store and update data models in the cloud.
Flexibility with simplicity. Deploy data models and data workflows to run at the edge or in the cloud.
Organizations can stay on top of their ever-growing and ever-evolving edge data and data models using an enterprise data automation platform like Prescient Designer to make edge data operations manageable and scalable. Find a use case that suits your needs or we'll help you build an application that works for you.