Overview
An architecture study for a highway tolling system where the lane cannot wait for the cloud. OCR cameras and RFID readers feed a local .NET Core process that has to raise or lower a barrier within 200 milliseconds, persist every event to local SQLite, and reconcile with a cloud backend over MQTT whenever connectivity allows — including after hours of being offline.
- Decision budget
- < 200 msvehicle identified to barrier decision
Problem
A vehicle approaching a toll lane at speed gives the system a fixed and very short window to identify it and decide whether the barrier opens. That budget — under 200 milliseconds — is smaller than a single round trip to a cloud service on a good day, and highway sites do not have good days reliably.
The consequence of getting it wrong is physical. A lane that stops working because a link is down is not a degraded service, it is a queue of stationary vehicles on a road.
Constraints
- Identification to barrier decision in under 200 milliseconds
- Full autonomous operation during network loss, for hours rather than seconds
- No event may be lost, including events recorded while offline
- Two identification technologies with different accuracy and failure characteristics
- Constrained roadside hardware with no operator present
Role and contribution
Software Engineer — system architecture
- Defined the failure model and what each layer is allowed to assume about the others
- Placed the decision boundary at the lane rather than in the cloud
- Designed local persistence as the authoritative lane record
- Designed the store-and-forward reconciliation protocol over MQTT
System design
Architecture
Edge and cloud responsibilities
OCR cameras and RFID readers identify a vehicle and pass the result to a .NET Core process running locally in the lane. That process makes the barrier decision within the local latency budget, using locally held data only, and writes the event to a local SQLite database that is authoritative at the lane. A synchronisation component publishes events to the cloud backend over MQTT when connectivity is available and buffers them in SQLite when it is not, so events recorded offline are delivered once the link returns.
- Lane
- OCR cameras
- RFID readers
- .NET Core
- SQLite
- Transport
- MQTT
- Store-and-forward synchronisation
- Cloud
- Central backend
- Account reconciliation
Scope of this project
This is an architecture and academic engineering project. It is not a deployed commercial tolling system, and the figures below are design targets rather than measurements taken from operating roadside hardware.
Where the decision boundary goes
Every design question here follows from one placement: the decision to raise the barrier is made in the lane, using data the lane already holds, and nothing about that decision waits for the network.
That inverts the usual arrangement. The cloud is not the system of record that the edge queries — it is a consumer of events the edge has already committed. Connectivity becomes a property that affects how quickly the central view catches up, never whether a vehicle can pass.
Designing for hours offline, not seconds
The distinction matters because a retry buffer sized for a brief blip is a different component from one sized for an outage. Seconds of disconnection can be held in memory and replayed. Hours cannot: the buffer has to survive a process restart and a power cycle, which makes it a persistence problem rather than a networking one.
Local SQLite therefore does double duty. It is the lane's operating record, and it is the outbound queue. An event is durable the moment the decision is made, and synchronisation is a separate concern that reads from it — so the code path that decides whether a barrier opens contains no network operation at all.
Two identification inputs
OCR and RFID fail differently. A tag read is fast and unambiguous when a tag is present and useless when it is not; plate recognition works for any vehicle but degrades with weather, light, speed and plate condition.
Running both is not redundancy for its own sake — it is coverage of two disjoint failure modes. It does mean the lane may hold two identifications of differing confidence within the same decision window, and resolving that conflict is part of the local logic rather than something deferred to the cloud, because deferring it would break the latency budget.
Key decisions and trade-offs
Decision
Make the lane authoritative and treat the cloud as a consumer
- Context
- A barrier decision must complete in under 200 milliseconds; a cloud round trip cannot be guaranteed within that budget, and the link may be absent entirely.
- Alternatives considered
- Cloud-authoritative with local caching for fallback
- Local decisions requiring cloud confirmation before the barrier moves
- Chosen approach
- The lane decides and commits locally; the cloud reconciles afterwards from published events.
- Reason
- It is the only arrangement where the latency budget and the offline requirement are both met by construction rather than by hoping the network cooperates.
- Trade-off
- The central view is eventually consistent, and reconciliation has to handle events arriving hours late and out of order.
Decision
Use local SQLite as both the operating record and the outbound queue
- Context
- Events created during an outage must survive a restart and be delivered later.
- Alternatives considered
- An in-memory queue with retry
- A dedicated message broker running at the lane
- Chosen approach
- Persist to SQLite at decision time; synchronisation reads from that store.
- Reason
- An in-memory queue loses events on power cycle, which is exactly the condition it needs to survive. A local broker adds a second stateful component to hardware with no operator.
- Trade-off
- Disk write latency sits inside the decision path, and the store needs its own retention policy to avoid unbounded growth during a long outage.
Decision
Publish over MQTT rather than call an HTTP API
- Context
- Roadside connectivity is intermittent and often over constrained links.
- Alternatives considered
- Batched HTTP with retry
- A persistent bidirectional connection to the backend
- Chosen approach
- MQTT publication with local buffering.
- Reason
- MQTT is designed for exactly this shape: small messages, unreliable links, many endpoints, and a broker that decouples publication from delivery. HTTP retry logic would end up reimplementing the same behaviour less well.
- Trade-off
- A broker becomes part of the operational footprint, with its own availability and security surface.
Result
- An architecture where the latency budget and the offline requirement are met by construction rather than by network availability
- No event loss across restarts or extended outages, by making persistence the first step of the decision path
- Two identification technologies covering disjoint failure modes, resolved locally
Lessons learned
- Placing the decision boundary is the architectural choice; almost everything else follows from it.
- 'Offline for seconds' and 'offline for hours' are different requirements that produce different components. Sizing that wrong is a design error, not a tuning error.
- Eventual consistency is easy to accept in a diagram and expensive in reconciliation code, which is where most of the real complexity ends up.
- Physical consequences change the acceptable failure mode: a system that stops is worse than one that is temporarily approximate.
Next improvements
- Measured latency on representative roadside hardware, to confirm the budget holds in practice
- A defined conflict-resolution policy for reconciliation when a late event contradicts central state
- Retention and back-pressure policy for the local store during an extended outage
- Security design for device identity and message authenticity on the MQTT transport