The scheduling system sets the ceiling
The scheduling system behind an AI receptionist determines whether it can confirm appointments or only take messages. We build and run Vanty, an AI receptionist for service businesses that answers the phone over a Twilio elastic SIP trunk into LiveKit, replies to SMS on the same number and runs a chat widget on the website.
One customer, one thread
Threads are keyed to a contact and an agent, not to a channel. Phone platforms such as SMS, iMessage, WhatsApp and voice use the caller's E.164 number as the external id on a contact. A separate table holds one row per channel per thread with a platform discriminator so the transcript reads in chronological order.
On web chat a visitor starts with an anonymous visitor key stored in local storage. When the widget calls save_contact_details with a name, email or company the tool saves each field immediately. The email becomes an identity on the contact and links subsequent turns across channels. The tool is available only on web replies because on SMS and voice we already know who is calling and asking for email on those channels sounds broken.
Phone calls need different timing than a browser demo
We retired a smaller flash model after it invented business facts on a live receptionist call. Agent instructions now state that the only business facts it may give are those written into its instructions, enforced as a hard constraint.
We use Deepgram Flux for speech-to-text with native end-of-turn detection. Flux defaults to a three second end-of-turn timeout and we measured 4.9 seconds on a hesitant answer from a real phone.
We set eot_timeout_ms to 800 milliseconds and wrap dynamic endpointing bounds of 0.35 to 1.5 seconds around it. This removes dead air without cutting off a caller who pauses mid-sentence.
Interruption handling uses adaptive mode with a minimum duration of 0.35 seconds and a one word minimum. Phone rooms switch to voice activity mode with a minimum of 0.2 seconds and no word minimum. After that change the agent yielded 328 milliseconds after the caller started speaking and returned 1.44 seconds later.
We measured 1.773 seconds median from answer to first audio in production. We measured 1.455 seconds median from turn end to first agent audio. Text-to-speech first byte arrives between 0.162 and 0.271 seconds. We do not measure the end-of-utterance decision itself which usually occurs around 0.35 seconds with outliers close to one second.
Booking APIs define what can be confirmed
With Google Calendar the agent queries free/busy before creating the event. The tool enforces that start and end times carry an explicit UTC offset or a Z by a regex on the input. A language model will otherwise hand back a naive local timestamp and book the job an hour out.
A Housecall Pro job requires a customer id, a description, scheduled start and end times and an arrival window in minutes. The agent must find or create the customer before it can call the job create endpoint.
Jobber has no open-slot availability API. The agent creates a request record and tells callers that the office will confirm the exact time later. Vendors promising direct booking on Jobber are inventing that capability.
Every provider write is mirrored into a local bookings table under an external reference such as gcal:evt_123 and deduplicated on that reference. If the mirror insert fails we log the error and still return success because the appointment already exists in the customer's system and failing the tool would make the agent apologise for a non-existent error.
What broke once real traffic arrived
Two agents in the same account once triggered an endless text loop. We updated the inbound webhook to ignore messages from our own provisioned numbers. The inbound webhook still stores those messages in the transcript but does not trigger an automatic reply.
We enforce a velocity cap of eight agent sends per thread in five minutes. The agent also does not reply when the last message in the thread is already its own.
Early hang ups occurred when the agent invoked the end-call tool before the caller finished. We wrap the end-call tool in an authorization guard that only fires after a clear farewell at the end of the caller's turn or when voicemail greeting phrases match as substrings. An explicit negative such as don't hang up refuses authorization and keeps the call open.
What we do not do
We do not support warm transfer. On a live call the agent cannot hand the caller directly to a human operator. It collects a name, a callback number and a reason for the handoff.
For text and web chat handoff we use a status flag on the thread. When that flag is set the agent stops replying. Your team can then answer from the inbox and the widget will pick up replies on its poll.
There is no call recording. Turn-by-turn text transcripts and metadata are stored and posted into the thread when the call ends.
How the running cost behaves
Call time is metered in whole minutes rounded up with a floor of one minute. New conversations are counted once per thread rather than per message.
When usage passes the plan cap inbound messages continue to store in the inbox. Only the agent's automatic replies are skipped until the cap resets.
Common questions
Do I need a new phone number?
We provision voice and SMS on the same Twilio number in a single purchase. The trunk SID routes voice into the SIP trunk and the SMS URL routes inbound texts to the app webhook. You can point an existing number at the same trunk. A workspace holds one number by default.
Will it work with the software I already use?
If your scheduling system exposes availability through an API the agent can book real slots in it. If it does not the agent will take a request and a person must confirm the time afterwards. Ask any vendor which of these two you are actually getting.
Why does it sound worse on the phone than in the demo?
The ceiling on a PSTN call is narrowband audio not the voice model. In our own listening tests line noise rather than synthesis quality separates a call from the same agent running in a browser.
What should it do when it does not know something?
The agent must say that it does not know and take a message. We enforce a hard constraint that the agent may only state business facts written into its instructions.
Your first decision is which system holds your calendar and what its API will promise because that determines whether an AI receptionist will book work or only take messages. We offer a paid AI audit at $3,000 credited toward the build. Book a 30 minute intro, or get in touch.