When does edge AI make sense, and why run models near the machine?

Reasons to run near the machine

A model can be hosted in the cloud, so the real question is why it should live near the machine. Four reasons are common. Latency: if an alarm or control response must arrive within a few milliseconds, a network round trip may not fit. Connectivity: when the plant link drops, monitoring should keep working. Data volume and privacy: raw high-rate signals, such as vibration, can be summarized locally instead of leaving the plant. Determinism: inference close to the control loop can be given a fixed time budget, while network delay varies and breaks that predictability.

Deciding with numbers (hypothetical)

Suppose one vibration channel is sampled at 10 kHz, with 4-byte values.

• Raw rate: 10 000 × 4 = 40 000 bytes per second.
• Per day: 40 000 × 86 400 = 3 456 000 000 bytes, about 3.46 GB.
• Latency budget: if an alarm must respond within 200 ms, and a cloud round trip takes 300 ms (both figures assumed), the cloud cannot make that decision in time.

This points to a hybrid design: fast decisions at the edge, training and fleet analysis in the central system. Model updates then need a deployment process of their own, and that load grows with the number of devices, so it belongs in the design from the start.

Where it does not fit, and common mistakes

Edge deployment has costs that are easy to underestimate. A device has limited processing power, memory and energy, so the model has to fit. Each deployed model needs versioning, calibration checks and health monitoring on every device, and that work grows with the fleet. The common mistake is choosing the edge because it sounds faster, without measuring whether the delay matters. If latency is not a bottleneck, an edge design mainly adds maintenance work and gains nothing. Measure the latency and the data volume first, then decide.

Follow-up questions