What is a data historian, and how does it differ from a SQL database?

What a historian stores

A control system reads many tags at short intervals. A historian stores each reading with its timestamp and a quality code, such as good or uncertain. Data is written continuously, kept in time order, and usually queried as one tag over a time range.

The difference from a SQL database lies in the write load and the storage logic. A historian is optimised for constant, wide writes and retains data for long periods, often compressing it. Compression methods such as deadband or swinging-door are lossy: they drop small changes. Interpolation on read rebuilds values between stored points. The trend looks smooth, but it is not the raw signal.

A hypothetical data volume

Hypothetical line: 500 tags, each sampled once per second.

• Rows per day: 500 × 86,400 = 43,200,000
• If each row needs 16 bytes for timestamp and value (an assumption), the raw size is 43,200,000 × 16 = 691,200,000 bytes, about 691 MB (decimal) per day.

This is an upper bound without compression. A deadband setting can reduce it, but by an amount that depends on how each tag behaves. A large deadband can remove small but meaningful changes that a model needs, so check the setting before training.

Misreading historian data

The first mistake is treating the historian as a complete raw record. Compression and interpolation mean a query can return values that were never stored. The second is treating aggregated tags as raw samples; they answer different questions. The third is ignoring quality codes, so values marked uncertain or bad enter a model unnoticed. The fourth is mixing time zones, such as UTC and local time, which can put events in the wrong order.

Follow-up questions