How historian compression distorts machine learning
The series in a historian is not the series the sensor saw. Compression settings change what features and models learn.
Normalization is the discipline of designing relational schemas that stay consistent as data grows. A practical guide to the normal forms that matter for real systems.
A database can hold any data you throw at it — for a while. The problems surface later: duplicate rows that disagree with each other, updates that must be repeated in several places, and reports that return different numbers depending on which table you query. Normalization is the process of structuring tables so that each fact is stored exactly once.
The normal forms are a ladder of design rules. Most production systems aim for third normal form (3NF), which is a strong practical sweet spot.
A table is in first normal form (1NF) when every column holds a single atomic value — no lists, no comma-separated tags, no repeating groups.
| Bad | Good |
|---|---|
order_id, items = "widget, bolt, nut" | order_item(order_id, item) — one row per item |
Storing lists in a column makes every query harder: searching, counting, and joining all require parsing text. 1NF is the baseline, but it does not yet prevent duplication.
A table is in second normal form (2NF) when it is in 1NF and every non-key column depends on the whole primary key, not just part of it. This only matters for tables with composite keys.
Consider order_item(order_id, product_id, product_name, quantity). product_name depends only on product_id, not on the full key — if the product is renamed, every row that mentions it must be updated. Splitting products into their own table fixes the partial dependency.
A table is in third normal form (3NF) when it is in 2NF and no non-key column depends on another non-key column.
Example: invoice(invoice_id, customer_id, customer_address). The address depends on customer_id, which is not the key of the invoice table. If the customer moves, every invoice row must change. Moving customer_address into the customer table removes the transitive dependency.
Normalization is not an absolute. After the schema is in 3NF, engineers sometimes denormalize deliberately — duplicating a few read-mostly columns, or adding summary tables — to make hot queries faster. The rule of thumb:
Analytical workloads (data warehouses) are usually denormalized into star schemas on purpose. Transactional systems (OLTP) should stay normalized. Knowing which workload you have is half the design decision.