Skip to content
All work
AIWeb2025 · ~1 min read

AI answers during a live call

The operator hears a question and sees the answer at once: speech from the shared tab is transcribed live, and an AI answers from the company knowledge base.

# The problem

An operator on a call had to answer whatever a client asked. Some answers they simply did not know, others they forgot in the moment, and searching the documentation mid-conversation is not an option: a pause in the dialogue costs more than an imprecise answer.

# The solution

I built an interface that captures the system audio of a chosen tab. The microphone is ignored on purpose — what gets transcribed is the other party, not the operator. Speech is recognised on the fly and collected into a dialogue feed. Next to any line there is a Query button: it sends the question to an AI connected to the company knowledge base through RAG, and the answer appears in the neighbouring column without interrupting the conversation. It all works across languages. The recording shows a video instead of a real call, so the demonstration can be shown without exposing either clients or the content of their calls.

# The outcome

Delivered in a week: the interface, audio capture, the transcription feed and the AI queries. I have no usage metrics — I had no access to the client data, and the AI backend was built by another party. I have assembled an equivalent backend on another project: noise cleanup through Silero, recognition through Whisper and a model call, all on FastAPI. So I can build a system like this end to end, not only its interface.

# How it works

  1. Browser tabThe operator picks the tab the conversation is happening in.
  2. Audio captureThe system audio of that tab is taken. The microphone is ignored on purpose — the transcript carries the other party, not the operator.
  3. TranscriptionSpeech becomes text as it is spoken, in any of several languages.
  4. Conversation feedLines stack up in a feed, each one with a Query button beside it.
  5. AI and knowledge baseAnswers from the company knowledge base through RAG. The server side was another party’s work — I built the interface and the query logic.

Dashed: not my work — platforms and the client’s own systems

Next project

A WhatsApp assistant that books the client itself

Need something similar?

Discuss the task