DataOps
How historian compression distorts machine learning
The series in a historian is not the series the sensor saw. Compression settings change what features and models learn.
Prepare OPC UA data for AI
Turn a tag value into a usable observation with units, quality, timestamps and asset context.
In short
- A value like 72 is not an observation until asset identity, signal identity, engineering units and source time are attached to it.
- Bad quality should never become a zero reading, and a carried-forward last value needs an explicit flag and its age.
- A working connection alone does not make data analytically fit, so acceptance should rest on unit, timing and quality checks.
A value is not an observation
The number 72 could mean temperature, speed or load. Keep asset identity, signal identity, engineering units, source time, receipt time and quality status with each reading. Version mappings so that a renamed integration tag does not silently change asset identity.
Preserve quality information
Do not turn bad quality into a zero reading. If carrying a last-known value forward, include an explicit flag and its age. A model must not confuse a three-hour-old reading with a fresh measurement. The process engineer should define acceptable staleness in the data contract.
Test time alignment
Create fixtures for clocks moving backward, duplicate readings and mixed time zones. Monitor the difference between source and receipt time. When choosing a resampling interval, consider whether averaging could hide a brief event. Keep raw measurements separate from derived features.
A proposed minimum contract
Start with asset_id, signal_id, source_timestamp, received_timestamp, value, unit, quality and schema_version. Complete the contract for five critical signals before expanding the pilot. A working connection alone does not establish analytical fitness; use unit, timing and quality checks for acceptance.
Frequently asked
What should an OPC UA data contract contain before AI work?
Each reading should carry asset identity, signal identity, engineering units, source time, receipt time and quality status. A proposed starting set is asset_id, signal_id, source_timestamp, received_timestamp, value, unit, quality and schema_version. Complete the contract for five critical signals before expanding the pilot.
How do I test time alignment in OPC UA data?
Create fixtures for clocks moving backward, duplicate readings and mixed time zones. Monitor the difference between source time and receipt time. When choosing a resampling interval, check whether averaging could hide a brief event, and keep raw measurements separate from derived features.
Sources and further reading
Pilot steps in this guide are proposed engineering workflows. Examples do not constitute measured field performance.
Questions on this topic
Keep reading.
Full library
DataOps
Check data quality before calculating OEE
Availability, performance and quality must use consistent production boundaries.
DataOps
Time-Series Data in Industry: Storage, Trends, and Forecasting
From historians to machine learning, time-series data powers industrial analytics. A practical guide to collection, quality, and the forecasting methods that actually work on plant data.