Why We Took the AI Box Out of the Customer Site

The first version of Vectiq VMS was hybrid edge plus cloud. Every customer site got a small box: it decoded the camera streams, ran detection, recorded video, and synced events upward. It worked. We removed it anyway.

What the edge box actually cost

The box was never the hard part. What surrounded it was:

• Provisioning. Each box needed an identity before it could be trusted. Claim codes, mutual TLS, a certificate authority, a revocation story.
• Updates. Firmware on hardware you cannot reach needs A/B partitions and a rollback path, because a bad update in a warehouse two provinces away is a truck roll.
• Disk encryption. Video sitting on a box in a customer's back office is video someone can walk out with. That pulls in TPM and LUKS.
• A local UI. When the uplink drops, someone on site still wants to see cameras, so the box needs its own viewer — a second frontend to build and keep in sync.
• Relay. To view a camera from outside the site, traffic has to get out through NAT, which means a relay service in the middle.

None of that serves the customer's actual goal, which is to see their cameras and get told when something happens. All of it is surface area we maintain. Counting it up, the edge deployment was roughly five subsystems whose only reason to exist was that a computer sat in a room we did not control.

What replaced it

Cameras now stream to worker nodes we operate. A node decodes, runs inference, and records. The customer site holds cameras and a network link, nothing else.

Camera (RTSP/ONVIF)
   -> MediaMTX / go2rtc      restream once, many consumers
   -> Worker node            decode, motion gate, batch inference, record
   -> NATS                   events and control, subjects keyed by node_id
   -> Control plane (Go)     API, scheduler, Postgres with RLS
   -> Portal / alerts        browser, mobile PWA, webhooks

The node is not trusted more than the old box was. It authenticates with a token, and it only receives configuration for cameras assigned to it — GET /internal/v1/nodes/{id}/config. A node asking for another node's configuration gets a 404, not a partial answer. We have a test that fails if a node ever picks up a camera on its own.

Tenant isolation lives in the database

A platform that one dealer resells to forty customers has forty blast radii. Putting the tenant_id check in application code means every new endpoint is a chance to forget it.

We put it in PostgreSQL row-level security instead. The application connects as a role that cannot see across tenants; the policy is on the table, not in the handler. A missing WHERE tenant_id = ... is then a query that returns nothing, not a query that returns someone else's cameras.

The scheduler, and what happens when a node dies

Cameras have to land somewhere. The scheduler assigns them with a cost model that respects each node's GPU and bandwidth budget, and prefers to leave a camera where it already is — moving a stream costs a reconnect and a gap in recording, so stickiness is worth more than perfect balance.

The loop runs under a PostgreSQL advisory lock, so several control-plane replicas can be up and exactly one of them schedules at a time. No leader election service, no extra moving part.

Failover is the same loop with a different trigger. A node reports health; when the reports stop, its cameras become unassigned and the next pass places them on healthy nodes. The recording gap is real and visible — the playback timeline shows which spans exist and which do not, rather than presenting an unbroken bar and failing when you click the hole.

The costs we accepted

Central processing is not free, and pretending otherwise would be dishonest:

• Uplink bandwidth. Every camera's stream now leaves the site. A site with twenty 2 MP cameras needs real upstream capacity, and that has to be in the survey before the sale.
• Dependence on the link. When the edge box existed, a dead uplink meant "no remote view". Now it means "no recording". For sites that cannot accept that, the answer is a local recorder, not a resurrection of the AI box.
• Latency. Detection now includes a network round trip. For intrusion and ANPR at a gate this is comfortably inside the useful window; for anything needing sub-100 ms actuation it is not.

We took these because the alternative was maintaining five subsystems forever so that a computer could sit in a room we do not control. For a platform sold through dealers — where the operator is not the customer and a truck roll is a real cost — that trade is not close.

What we verified, not what we assumed

The central topology was tested against real services, not mocks: real PostgreSQL with RLS policies applied, real NATS, real MediaMTX, a real node agent pulling a real RTSP stream. The tests that matter are the negative ones — a node that should not see a camera, a token that should not open another node's config, a tenant that should not appear in another tenant's audit log. Those are the tests that fail loudly when someone refactors the isolation away.