The problem: 50 MB of ML for one intent classifier

Kutup NQ used to load TensorFlow.js plus the Universal Sentence Encoder from a CDN to classify query intent and compute embeddings. The model alone is tens of megabytes, it downloads on every new device, it needs a fast connection, and it makes the app feel heavy on exactly the devices that need it most — field phones on industrial sites.

Most of that machinery was doing a small, well-defined job. So we replaced it with a purpose-built kernel: one 21 KB WebAssembly module written in C, compiled with clang directly to wasm32 (no Emscripten, no runtime, no CDN model download). The same module now powers all three Kutup apps.

What the kernel does

The kernel implements four families of operations, all deterministic and all running on-device:

  • Hashed n-gram embeddings (128 dimensions). Text is tokenized with Turkish/English normalization, then word unigrams and character trigrams are hashed into a signed accumulator (the fastText feature-hashing trick) and L2-normalized. The result is a real vector: "modbus tcp" and "modbus rtu" score 0.76 cosine similarity, "sensor calibration" and Turkish "sensör kalibrasyon" 0.43 — typo- and inflection-tolerant, with zero vocabulary files.
  • Vocabulary naive-Bayes classifiers. Three small models trained on curated bilingual phrase sets — query intent (definition, comparison, technical, procedure, broad search), flowchart sentence roles (start, end, decision, loop, otherwise, parallel, process), and field types for Forge (text, number, textarea, select, checkbox, range). Weights are quantized to 16-bit log-probabilities and embedded in the binary; inference is a prefix match over ~500 vocabulary stems per model plus a weighted sum. Training-set accuracy is 96–97%.
  • Calibrated evidence-quality scoring. A logistic model combines six retrieval signals — coverage, specificity, freshness, source diversity, source count, and intent confidence — into one honest score. NQ shows this as the Evidence Quality panel: it tells the user how much of the question the evidence actually covers, not just how confident the intent classifier was.
  • Graph layout and analysis. Flowchart layout runs barycenter crossing minimization in C (deterministic sweeps over layers), and the analyzer reports nodes, branches, loops, depth, unreachable nodes, and cyclomatic complexity for Flow Maker's analysis panel.

How it is built

The engineering constraints shaped the design:

  • Freestanding C, no libc. A hand-written strlen (guarded against LLVM pattern-matching it back into a libc call), a compact polynomial expf, and a bump arena for marshalling data across the JS boundary. Memory grows on demand via memory.grow; the arena is reset after every call so callers never manage memory.
  • No Emscripten glue. The loader is a small UMD module that works as a classic script, an ES module side-effect, and in Node for tests. One subtle bug this avoids: the arena must never be a large static array — the linker places the biggest static first and can nest other statics (model tables!) inside it.
  • Deterministic by construction. Same input, same output, every time — on every device. That property matters for an engineering tool: results are reproducible and testable.

What it buys

  • Instant start. NQ's first answer is available in seconds even on mid-range phones; there is no model warm-up and no network dependency for the ML layer.
  • Offline-friendly. Intent routing, embeddings, and quality scoring work with the network down. Only the optional LLM synthesis and live Wikipedia/RSS enrichment need connectivity.
  • Privacy. The query's semantic representation never leaves the device; only the plain-text evidence request goes to the backend.
  • One kernel, three apps. Flow Maker and Forge use the same module for layout, analysis, and field-type inference — one binary to cache, one codebase to maintain.

The tradeoffs

Hashed n-gram embeddings are lexical, not deeply semantic. "Modbus" and "RTU" match by surface form, not by concept — which is exactly right for a domain where terminology is precise and identifiers matter. Where meaning must be synthesized across sources, NQ still delegates to an LLM over a constrained evidence packet. The kernel is not a replacement for language models; it is the layer that makes them unnecessary for the small, repeated decisions — and it makes the ones it does make inspectable.

Key point: Not every ML workload needs a neural network framework. When the task is narrow, bilingual, and must run on a field phone, a compiled 21 KB C kernel with trained weights beats a 50 MB runtime — and its decisions are deterministic enough to audit.