Widget Chat

The problem

A support chatbot that makes things up is worse than no chatbot. The hard part is not calling a model, it is making every answer traceable to something the company actually wrote — and doing it for many companies at once, without one tenant's material ever reaching another's widget.

Grounded answers

Each workspace uploads its own material. Documents are normalised, split into chunks and embedded as vectors stored in PostgreSQL itself through pgvector, so retrieval is a query against the same database as the rest of the application rather than a second service to operate and keep in sync. A question retrieves the passages that match it, and those passages — not the model's general knowledge — are what the reply is built from.

Hosted or local models

The provider is a per-workspace setting, not a global one. Hosted APIs work, and so do local runtimes — Ollama and LM Studio are discovered automatically — which means a client whose data is not allowed to leave their own network runs the same product as everyone else instead of being told no.

Human takeover

A member of staff can step into a live conversation while the assistant is still typing. The in-flight response is cancelled rather than raced to completion, so the visitor does not watch a machine answer and a person answer at the same time. Staff see the reply stream in token by token; the visitor's widget polls, because pushing the same stream to a visitor would also push the fields around it that a visitor has no business seeing.

Boundaries of the embed

The widget is served to arbitrary third-party sites, so it is treated as the exposed surface it is: its own base controller, a workspace widget token plus a signed per-conversation chat token, per-IP rate limits, and an origin allowlist restated as a content-security policy so a stolen snippet does not load inside someone else's page. Every turn also passes a guardrail check on the way out.

Keeping it honest

Around forty RSpec files cover the retrieval path, the guardrails and the embed authentication. Continuous integration runs the suite alongside a static security scan, a style pass and an N+1 query detector that fails the build rather than warning — a query count that grows with the number of messages in a conversation is exactly the bug that does not show up until a customer has a long one.