README

AI voice agents for Laravel, built on top of LaravelEasyAI.
Speech-to-text, text-to-speech, voice sessions, and voice-enabled tool-calling agents.

Built by Easy IT

v0.3.0 is published on Packagist. This README documents what's actually implemented today. See CHANGELOG.md for the phase-by-phase build history and the reasoning behind each design decision — main accumulates changes across several phases before each version tag, rather than tagging every phase.

Why LaravelEasyVoice?

easybdit/laraveleasyai already gives Laravel a unified LLM interface, tool-calling, RAG, and conversation memory. It has no speech capability worth building on — its STT/TTS is a single blocking, OpenAI-only HTTP call, outside its own stable provider contract, with no session model, no events, and no authorization layer for tool calls. LaravelEasyVoice adds the voice-specific layer on top of it — STT/TTS provider abstraction, voice sessions and turns, an event system, and mandatory tool authorization — without reimplementing anything LaravelEasyAI already does well.

// Register an agent once (e.g. in a service provider's boot()) — never in
// config/voice.php, since tool handlers are closures.
Voice::registerAgent('receptionist', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')
        ->systemPrompt('You are a friendly front-desk receptionist.')
        ->tools([
            AuthorizedTool::make(
                name: 'check_appointment',
                description: "Look up today's appointments",
                parameters: ['type' => 'object', 'properties' => []],
                ability: 'view-appointments',
                handler: fn (array $args) => Appointment::today()->get(),
            ),
        ])
        ->limits(maxTurns: 50, maxSessionSeconds: 1800);
});

// Then, per caller:
$agent = Voice::agent('receptionist');
$session = $agent->startSession(['user_id' => auth()->id()]);

$assistantTurn = $agent->handleTurn($session, $uploadedAudioPath);
// $assistantTurn->transcript  -> the AI's text reply
// $assistantTurn->audio_path  -> synthesized speech, on your private disk

Requirements

• PHP ^8.1
• Laravel 9–13 (illuminate/support, illuminate/http)
• easybdit/laraveleasyai ^2.21 — required, not optional. This package has no LLM logic of its own. ^2.21 specifically (not just any ^2.x) is needed for accurate token/cost accounting on multi-step tool-calling turns — see the Analytics\VoiceUsage note below.

Installation

composer require easybdit/laraveleasyvoice
php artisan voice:install

voice:install publishes config/voice.php, runs migrations (with your confirmation), configures your OpenAI key (reusing OPENAI_API_KEY automatically if LaravelEasyAI already set one — no need to enter it twice), optionally asks for a Deepgram key too (skip it if you only plan to use OpenAI for STT), and asks whether to turn on the HTTP API. Prefer to do it by hand instead:

composer require easybdit/laraveleasyvoice
php artisan vendor:publish --tag=voice-config
php artisan migrate

easybdit/laraveleasyai is a required dependency and is pulled in automatically from Packagist.

"repositories": [
    { "type": "path", "url": "../path/to/laraveleasyai", "options": { "symlink": true } }
]

What's implemented today

• Voice::stt($driver) / Voice::tts($driver) — provider-agnostic speech contracts (SpeechToTextProvider, TextToSpeechProvider), with OpenAI (STT+TTS), Deepgram (STT+TTS), and ElevenLabs (TTS) implementations.
• Voice::registerAgent() / Voice::agent() — named, reusable voice agents that wire STT → LaravelEasyAI's AI::provider()->tools()->run() agent loop → TTS into one call.
• Sessions & turns (voice_sessions, voice_turns) — persisted history per caller, with token/latency accounting on the session.
• Events — SessionStarted, SessionEnded, SpeechTranscribed, ToolCallStarted, ToolCallCompleted, ResponseSynthesized, VoiceError, all fired through Laravel's own Event facade.
• AuthorizedTool — a tool wrapper that structurally requires a Gate ability check before its handler ever runs, failing closed on denial or an undecidable guest check.
• HTTP routes (opt-in) — POST /voice/sessions, POST /voice/sessions/{id}/end, POST /voice/sessions/{id}/turns (audio in, transcript + audio URL out), GET /voice/sessions/{id}/turns/{turn}/audio. See below.
• php artisan voice:install — guided setup: publishes config/migrations, configures the OpenAI key, asks about the HTTP API.
• Streaming text responses (opt-in) — VoiceAgent::streamResponses() fires a ResponseChunkReceived event per delta as the LLM's reply streams in, for showing text progressively before the full turn (including TTS) finishes.
• Multi-tenancy wiring — voice.routes.tenant_resolver resolves the caller's tenant per request; every session-scoped HTTP route enforces it via VoiceSession::isOwnedBy().
• Tool tiers — AuthorizedTool::make(..., tier: 'destructive') (read/write/destructive/privileged) surfaces on ToolCallStarted/ToolCallCompleted for auditing, alongside the Gate check that actually authorizes the call.
• Chat-history mirroring (opt-in) — link a session to an existing ai_chat_sessions row (chat_session_id) and every turn is also written to ai_chat_messages, so a voice call and a text chat can share one transcript.
• Analytics\VoiceUsage — a query service over voice_sessions/voice_turns for aggregate or per-session usage (STT/TTS duration, tokens, estimated cost, latency), for building your own admin view or a future billing layer. Token/cost totals are summed across every real LLM call a turn makes, including the intermediate step(s) of a tool-calling turn, not just the final answer — this relies on easybdit/laraveleasyai v2.21.0's own per-step $onStep callback on AIProviderInterface::run() (this package consumes that callback; it isn't this package's own API). estimated_cost stays null, never a fabricated 0, whenever no step in a turn has a configured ai.pricing rate.
• Human handoff — VoiceAgent::requestHandoff()/completeHandoff() fire HandoffRequested/HandoffCompleted events for notifying a human agent through whatever channel you already use.
• Cross-session memory (opt-in tools) — RememberFactTool/RecallFactTool give an agent a small per-caller key-value fact store that survives across separate sessions, gated by the same Gate-based authorization as any other tool.
• Browser widget (voice-widget.js, opt-in, zero dependencies) — a small vanilla-JS client for the HTTP API: start a session, record with MediaRecorder, upload the turn, play the reply, live mic/playback level meters, upload progress. See "Browser voice agent" in the Cookbook below.
• Realtime voice, browser-direct, provider-selectable (opt-in) — voice.realtime.default/voice.realtime.providers.<name>, same pattern as voice.stt/voice.tts. /voice/realtime/token mints a short-lived token server-side, live-verified against both OpenAI and Deepgram real accounts. Each provider has its own browser client, since the two protocols are genuinely different transports: voice-realtime.js (VoiceRealtimeSession) speaks OpenAI's WebRTC + its own event schema; voice-realtime-deepgram.js (VoiceRealtimeDeepgramSession) speaks Deepgram's raw WebSocket + linear16 PCM protocol, hand-rolled and grounded in Deepgram's own official SDK source rather than prose docs alone. Both open the connection directly from the browser (Laravel is never in the audio path) and expose an interrupt() control. Requesting an unsupported provider fails clearly (422) rather than silently misconnecting. Neither browser connection can be exercised by this package's test suite — see "Realtime voice" in the Cookbook for the honest testing caveat, and its "why a second provider isn't just a config entry" for what a third provider would actually need.
• RealtimeVoiceProvider/RealtimeConnection — design-stage contracts only, not implemented by anything yet (the browser-direct clients above don't need them for OpenAI/Deepgram specifically). See "Not implemented yet" below.

With this, every item in the original v0.1 scope is implemented — see CHANGELOG.md for the full build history.

HTTP API (opt-in, off by default)

VOICE_ROUTES_ENABLED=true

With that set, the routes above run under ['web', 'auth'] by default — every request triggers billed STT/LLM/TTS calls, so there is no guest-open default. A request:

POST /voice/sessions          {"agent": "receptionist"}          -> {"id": 1, "agent": "receptionist", "status": "active"}
POST /voice/sessions/1/turns  multipart: audio=<file>             -> {"id": 5, "transcript": "...", "audio_url": "...", "tool_calls": [...]}
GET  /voice/sessions/1/turns/5/audio                              -> streamed audio, ownership-checked
POST /voice/sessions/1/end    {}                                  -> {"id": 1, "status": "ended"}

Every session-scoped route 403s for anyone who isn't the session's owner (VoiceSession::isOwnedBy()). To allow anonymous callers, set both VOICE_ROUTES_ALLOW_GUEST=true and VOICE_ROUTES_REQUIRE_AUTH=false in .env — a long-lived signed cookie (separate from LaravelEasyAI's own guest cookie) identifies a returning guest, same pattern LaravelEasyAI uses for its chat widget, deliberately kept as an independent config surface so the two packages' access policies can never silently affect each other. VOICE_ROUTES_REQUIRE_AUTH is deliberately an env var, not something you hand-edit in the middleware array of the published config file — a local test toggle that lives in a config file gets silently lost the next time that file is republished (vendor:publish --force), which is exactly the mistake that happened while dogfooding this package and is now fixed at the source.

Retrying a turn safely — Idempotency-Key (optional): a network timeout doesn't tell you whether the server actually finished processing your last request. Send the same client-generated key on a retry and POST /voice/sessions/{id}/turns becomes safe to repeat — STT/LLM/tools/TTS never run twice for it, and you get back the exact same response instead of a second billed turn:

POST /voice/sessions/1/turns
Idempotency-Key: <a client-generated id, unique per attempt>
multipart: audio=<file>

A key already tied to a turn still being processed returns 409; a key tied to one that already finished returns the original result again, unchanged. A key already tied to one that failed returns that same failure — retrying under that exact key is not re-attempted, so use a new key if you genuinely want to try again. Omit the header entirely for the exact behavior this package has always had.

Recovering from a crashed attempt: if the process handling a turn dies mid-flight (a killed worker, a server restart) before it ever reaches a final completed/failed state, its row is left stuck at pending. A retry sent with that same Idempotency-Key, once the stuck row has been pending for longer than voice.turn_recovery.stale_after_seconds (VOICE_TURN_STALE_AFTER_SECONDS, default 300 seconds), safely resumes it instead of returning 409 forever:

• If STT itself never finished, it runs again from your retry's audio.
• If STT already finished and persisted a transcript, it is not re-run — only the LLM/tool-calling/TTS portion resumes, reusing that same transcript.
• A 409 still applies to a pending turn that hasn't been stuck for longer than the threshold — a genuinely slow but still-alive attempt is never mistaken for an abandoned one.

stale_after_seconds is a timestamp-based heuristic, not a computed safe value — it has no way to distinguish a crashed process from one that's simply still working, and the shipped default of 300 seconds is not guaranteed to be enough for every legitimate turn. Concretely, this package's own shipped defaults alone already sum to more than 300 seconds in the worst case, before your agent's own tools are even counted:

That ~540s is the sum of the shipped provider-timeout budgets, not a guaranteed absolute ceiling — a real turn usually finishes far faster, but nothing stops it from using the full budget against a slow provider or a loaded model. On top of it, every tool call your agent makes runs your own closure (Tool::execute()) with no timeout enforced anywhere in this package or in LaravelEasyAI, so real tool-handler latency (a database query, an external API call) can push the true worst case higher still.

Set VOICE_TURN_STALE_AFTER_SECONDS above the real maximum time one of your own turns can legitimately take — computed from your configured STT/TTS timeouts and retries, your agent's maxSteps and LLM provider timeout, and your slowest tool handler's expected duration, plus a safety margin — before relying on this in production. Leaving every default untouched is not, by itself, a safe configuration for this setting; this package has no reliable way to know your configured LLM provider's own timeout or your tools' real latency, so it does not guess a safe value for you.

Recovery is at-least-once, not exactly-once: if a stale attempt is recovered after its LLM step already called a tool, that tool may run again — there is no way for this package to know whether an arbitrary tool handler's own external side effect (an email sent, a record charged) already happened right before the original process died. Any tool with a real side effect should be made idempotent by the application itself (e.g. accept and de-duplicate on its own idempotency argument) — this package does not and cannot solve that for you.

This same at-least-once boundary applies to provider usage/cost accounting, not just tool side effects: a provider call can succeed — and be billed by the provider — moments before the process handling it crashes, before this package ever gets to record that call's usage. Recovery may then run that same provider work again. Because usage/cost is persisted incrementally as each step completes (not batched until the whole turn finishes), an abandoned attempt's already-recorded partial usage is never rolled back or deduplicated against the attempt that eventually succeeds — voice_sessions.total_prompt_tokens/total_completion_tokens/estimated_cost can therefore end up reflecting more than one execution's worth of work for what is, from the caller's side, a single logical turn. This package does not claim exactly-once billing across a crash/recovery boundary — it is an inherent limitation of recovering from a crash without a distributed lease/coordination system, not a bug to be silently papered over.

Every write this package makes to a turn or its session counters after a recovery attempt begins is itself ownership-fenced: if another execution has already reclaimed a turn, an execution that was merely slow (not actually dead) is prevented from overwriting the recovered result or crediting usage for work whose outcome was already discarded — it stops and raises a conflict instead. What is not prevented, and cannot be, is a genuine provider-side charge that already happened before that detection.

Streaming text responses (opt-in)

Voice::registerAgent('receptionist', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')->streamResponses();
});

Event::listen(ResponseChunkReceived::class, function ($event) {
    // $event->chunk  -> the partial text delta
    // $event->type   -> 'content' or 'thinking' (reasoning-capable models)
    broadcast(new YourOwnBroadcastEvent($event->session->id, $event->chunk));
});

Text only — TTS still synthesizes the complete reply once at the end of the turn, since none of the shipped TTS providers support streaming audio output. Full duplex audio streaming is a realtime-transport feature, tracked separately below.

Cookbook

Real, runnable patterns for common voice-agent shapes. Everything below is composition of what's already documented above — none of it needed new package code, which is itself the point: LaravelEasyVoice orchestrates, LaravelEasyAI thinks.

Want a complete, drop-in page instead of a snippet? See examples/ — three full working pages (the turn-based widget with a real polished UI, OpenAI realtime, Deepgram realtime), each live-tested against a real provider account, with setup instructions.

Basic voice assistant

Voice::registerAgent('assistant', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')
        ->systemPrompt('You are a helpful voice assistant. Keep replies short - they will be spoken aloud.');
});

Tool-calling (customer support: "Where is my order?")

Voice::registerAgent('support', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')->tools([
        AuthorizedTool::make(
            name: 'get_order_status',
            description: 'Look up the shipping status of an order by its order number',
            parameters: ['type' => 'object', 'properties' => [
                'order_number' => ['type' => 'string'],
            ], 'required' => ['order_number']],
            ability: 'view-own-orders',
            handler: fn (array $args) => Order::where('number', $args['order_number'])
                ->where('user_id', auth()->id()) // never trust the model to scope this itself
                ->firstOrFail()
                ->only(['status', 'estimated_delivery']),
        ),
    ]);
});

If a handler throws (e.g. firstOrFail() above on a nonexistent order), the turn doesn't fail — LaravelEasyAI's own Tool::execute() catches the exception and hands the model {"error": "..."} instead, so the agent can react conversationally ("I couldn't find that order") rather than the whole turn erroring out. Verified end-to-end through both VoiceAgent::handleTurn() and the real HTTP turn endpoint, not just assumed.

RAG voice agent ("What is the admission policy?") — no new API, just call AI::rag() from inside a tool or bake it into the system prompt per turn:

Voice::registerAgent('school-info', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')->tools([
        AuthorizedTool::make(
            name: 'search_school_policies',
            description: 'Search the school knowledge base (admissions, fees, routine, policies)',
            parameters: ['type' => 'object', 'properties' => ['query' => ['type' => 'string']], 'required' => ['query']],
            ability: 'view-public-info', // define this Gate to always allow - it's public information
            handler: fn (array $args) => AI::rag()->source('school-kb')->search($args['query']),
        ),
    ]);
});

Appointment/booking, with read vs. destructive tools distinguished

Voice::registerAgent('booking', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')
        ->systemPrompt('Before booking, always call check_availability first. Only call create_appointment after the user has explicitly confirmed the specific time.')
        ->tools([
            AuthorizedTool::make(
                name: 'check_availability',
                description: 'Check open appointment slots for a given date',
                parameters: ['type' => 'object', 'properties' => ['date' => ['type' => 'string']], 'required' => ['date']],
                ability: 'view-availability',
                handler: fn (array $args) => Appointment::availableSlots($args['date']),
                tier: AuthorizedTool::TIER_READ,
            ),
            AuthorizedTool::make(
                name: 'create_appointment',
                description: 'Book a confirmed appointment slot',
                parameters: ['type' => 'object', 'properties' => [
                    'date' => ['type' => 'string'], 'time' => ['type' => 'string'],
                ], 'required' => ['date', 'time']],
                ability: 'create-appointment',
                handler: fn (array $args) => Appointment::book(auth()->id(), $args['date'], $args['time']),
                tier: AuthorizedTool::TIER_WRITE,
            ),
        ]);
});

School AI receptionist — a single agent combining the RAG-info and booking patterns above, plus a couple more read-only tools (check_attendance, get_class_routine, get_fee_status), each gated by its own ability. No new capability needed; it's the same composition at a larger scale.

Multilingual — pass the caller's language into both STT and TTS per call, and let the model reply in kind:

$agent->handleTurn($session, $audioPath, [
    'stt' => ['language' => 'bn'],       // ISO-639-1: bn, en, hi, ur, ar, ...
    'tts' => ['voice' => 'alloy'],
]);

Voice::registerAgent('multilingual', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')
        ->systemPrompt('Always reply in the same language the user spoke in.');
});

Automatic language detection (rather than the caller specifying it) isn't built - Whisper can auto-detect if language is omitted, but nothing here inspects the transcript to switch the reply language or TTS voice automatically yet.

Speech-to-text with Deepgram — a second STT provider alongside OpenAI, useful if you already have a Deepgram account or want its pricing/latency profile instead:

VOICE_STT_PROVIDER=deepgram        # or leave the default (openai) and pass 'deepgram' per call instead
VOICE_DEEPGRAM_API_KEY=...
# optional overrides - all have working defaults:
# VOICE_DEEPGRAM_BASE_URL=https://api.deepgram.com/v1
# VOICE_DEEPGRAM_STT_MODEL=nova-2
# VOICE_DEEPGRAM_STT_TIMEOUT=60
# VOICE_STT_MAX_FILE_SIZE=26214400   # 25MB, shared with the openai STT provider
# VOICE_STT_RETRIES=2                # connection-level failures only, never retries a non-2xx response
# VOICE_STT_RETRY_SLEEP_MS=250

use EasyAI\LaravelVoice\Facades\Voice;

$result = Voice::stt('deepgram')->transcribe($audioPath, [
    'model' => 'nova-2',   // optional - falls back to voice.stt.providers.deepgram.model
    'language' => 'bn',    // optional ISO-639-1 code - omit it to let Deepgram auto-detect where the model supports it
]);

$result->text;              // transcript string, '' if Deepgram heard no speech
$result->durationSeconds;   // audio duration Deepgram reported, or null
$result->raw;                // full decoded Deepgram response, for anything not already normalized

Or wire it into an agent the same way as OpenAI: $agent->stt('deepgram')->tts('openai')->llm('openai').

DeepgramSttProvider::transcribe() validates everything it can before making a network call - a missing/unreadable file, an oversized file (max_file_size), an empty (0-byte) file, and a missing/blank API key all throw \InvalidArgumentException immediately, no request sent. Once the request is made: a non-2xx response, an unparseable (non-JSON) response body, or a response missing the expected results field all throw EasyAI\LaravelVoice\Exceptions\ProviderException; a timeout, DNS failure, or other connection-level problem throws EasyAI\LaravelVoice\Exceptions\ConnectionException. Neither exception ever includes your API key - only Deepgram's own returned status/body. A genuinely empty transcript (Deepgram understood the audio but heard no speech) is not an error - it comes back as a normal result with text === ''.

Text-to-speech with Deepgram — a third TTS provider alongside OpenAI and ElevenLabs, using Deepgram's Aura voices. Shares the same VOICE_DEEPGRAM_API_KEY as Deepgram STT above (one Deepgram account, one key) — no separate VOICE_DEEPGRAM_TTS_API_KEY exists or is needed:

VOICE_TTS_PROVIDER=deepgram        # or leave the default (openai) and pass 'deepgram' per call instead
VOICE_DEEPGRAM_API_KEY=...         # reused from the STT config above - set it once
# optional overrides - all have working defaults:
# VOICE_DEEPGRAM_BASE_URL=https://api.deepgram.com/v1   # shared with Deepgram STT
# VOICE_DEEPGRAM_TTS_MODEL=aura-2-thalia-en
# VOICE_DEEPGRAM_TTS_FORMAT=mp3
# VOICE_DEEPGRAM_TTS_TIMEOUT=60
# VOICE_DEEPGRAM_TTS_MAX_INPUT_LENGTH=2000   # Deepgram's own documented per-request character cap

use EasyAI\LaravelVoice\Facades\Voice;

$result = Voice::tts('deepgram')->synthesize('Hello there', [
    'model' => 'aura-2-thalia-en',   // optional - falls back to voice.tts.providers.deepgram.model
    'format' => 'mp3',               // optional - Deepgram's own "encoding" value (mp3, linear16, mulaw, alaw, flac, aac, opus)
]);

$result->binary;             // raw audio bytes
$result->mimeType;           // e.g. audio/mpeg for mp3
$result->durationSeconds;    // always null - Deepgram's /v1/speak response doesn't include one, so this package doesn't invent it
$result->raw;                 // always [] - the response body is audio, not JSON, so there's no structured payload to keep

Or wire it into an agent the same way as the others: $agent->stt('openai')->tts('deepgram')->llm('openai').

Unlike OpenAI/ElevenLabs, Deepgram has no separate "voice" parameter — the model value itself selects the voice (e.g. aura-2-thalia-en is the "Thalia" voice), confirmed against Deepgram's own current API reference, so this provider has no voice option to set. DeepgramTtsProvider::synthesize() validates empty text, max_input_length (Deepgram's own documented 2000-character cap — a longer request would otherwise get a 413 from Deepgram itself), and a missing/blank API key before making any network call, all as \InvalidArgumentException. Once the request is made: a non-2xx response or an empty audio body both throw ProviderException; a timeout, DNS failure, or other connection-level problem throws ConnectionException. Neither exception ever includes your API key. This is a buffered implementation only — the full audio is generated and returned in one response, the same way the OpenAI and ElevenLabs providers already work; streaming isn't implemented for any TTS provider yet.

A full voice agent on Deepgram end-to-end — VoiceAgent::handleTurn() (audio → STT → LaravelEasyAI → TTS → stored AudioResult) is provider-agnostic by construction, so pointing both ends at Deepgram is just driver selection, no different from any other combination:

Voice::registerAgent('deepgram-assistant', function ($agent) {
    $agent->stt('deepgram')->tts('deepgram')->llm('openai')
        ->systemPrompt('You are a helpful voice assistant. Keep replies short - they will be spoken aloud.');
});

Providers can also be mixed freely per agent, e.g. ->stt('deepgram')->tts('openai') — VoiceAgent never special-cases which driver is behind either interface.

Tool-calling on Deepgram + Together — the tool-calling pattern shown earlier isn't OpenAI-only either; the same AuthorizedTool handler, Gate check, and ToolCallStarted/ToolCallCompleted events work unchanged with a non-OpenAI STT/TTS pair and a non-OpenAI LLM:

Voice::registerAgent('school-attendance', function ($agent) {
    $agent->stt('deepgram')->tts('deepgram')->llm('together')->tools([
        AuthorizedTool::make(
            name: 'check_attendance',
            description: "Check today's attendance",
            parameters: ['type' => 'object', 'properties' => []],
            ability: 'view-attendance',
            handler: fn (array $args) => ['present' => 42],
            tier: AuthorizedTool::TIER_READ,
        ),
    ]);
});

Verified live end-to-end, not just against Http::fake(): a real speech WAV transcribed by Deepgram, a genuine tool call made by Together's LLM, the handler executing and its result being fed back, Together's final response being persisted, and Deepgram TTS synthesizing it into real audio — from a separate Laravel R&D app installing this package from its published GitHub commit. VoiceAgent doesn't special-case which LLM sits behind llm() any more than it special-cases which STT/TTS driver is behind stt()/tts().

Memory — within one session, prior turns are already included as context (contextTurns(), default 10) — "My name is Murad" followed by "What's my name?" works today as long as both are in the same call. For the same fact to survive a separate call (the caller phoning back next week), opt an agent into the two memory tools:

Voice::registerAgent('receptionist', function ($agent) {
    $agent->stt('openai')->tts('openai')->llm('openai')
        ->systemPrompt('If the caller shares their name or a preference, call remember_fact. If they ask you to recall something, call recall_fact first.')
        ->tools([
            RememberFactTool::make(),
            RecallFactTool::make(),
        ]);
});

This is a small, flat per-caller key-value store (voice_memories), not a general memory/retrieval system — no embeddings, no staleness handling, nothing is captured automatically. A session with no identity at all (no tenant, no user, no guest token) can't use it, since there'd be nothing to scope the fact to.

Human handoff

// Inside a tool's handler, once the agent recognizes it can't help:
$agent->requestHandoff($session, 'Caller wants a refund - outside this agent\'s scope.');

Event::listen(HandoffRequested::class, function ($event) {
    // Notify a human however your app already does - Slack, a support queue, email.
    Notification::route('slack', config('services.slack.support_channel'))
        ->notify(new VoiceHandoffRequested($event->session, $event->reason));
});

Once the human has actually taken over, call $agent->completeHandoff($session, $reason) to end the session with the reason recorded in its metadata.

Tenant-scoped multi-agent SaaS

// config/voice.php (or .env-driven, same pattern as identity_resolver)
'tenant_resolver' => fn ($request) => $request->user()?->current_tenant_id,

Every session created through the HTTP API now carries tenant_id, and VoiceSession::isOwnedBy() enforces it on every subsequent request to that session - a 403 for a mismatched tenant, not just a mismatched user.

Browser voice agent

VOICE_ROUTES_ENABLED=true

<script src="/vendor/laraveleasyvoice/voice-widget.js"></script>
<meta name="csrf-token" content="{{ csrf_token() }}">

<button id="talk">Hold to talk</button>
<script>
  const widget = new VoiceWidget({
      agent: 'receptionist',
      onStateChange: (state) => console.log('state:', state), // idle, starting, ready, recording, uploading, speaking, ended
      onResponse: (turn) => console.log('assistant said:', turn.transcript),
      onError: (error) => console.error(error),

      // Optional - live feedback for a nicer UI, all off by default:
      onLevel: (level) => {}, // 0-1 mic input amplitude, ~60fps, while recording
      onPlaybackLevel: (level) => {}, // 0-1 reply-audio amplitude, ~60fps, while speaking
      onUploadProgress: (percent) => {}, // 0-100 while the recorded turn uploads
  });

  const button = document.getElementById('talk');

  (async () => {
      await widget.start();

      button.addEventListener('mousedown', () => widget.startRecording());
      button.addEventListener('mouseup', () => widget.stopRecording());
  })();
</script>

Publish the widget file first: php artisan vendor:publish --tag=voice-assets. The widget only talks to this package's own HTTP API (already covered by the test suite above) — it ships no third-party code. widget.stopSpeaking() stops local playback immediately (a "stop talking" control), but note it is not server-side barge-in — see the note in the widget's own file, and "Realtime" below.

The uploading state covers the entire server round-trip (upload + STT + the LLM turn + TTS), not just the HTTP upload itself - there's no separate "server is thinking" state, since how long each provider call takes varies per turn. To show that distinction in your own UI, watch for onUploadProgress reaching 100 while onResponse/onTranscript haven't fired yet - that gap is "still processing," without the widget guessing at a label for it.

Realtime voice (browser-direct, OpenAI)

VOICE_REALTIME_ENABLED=true
VOICE_REALTIME_OPENAI_API_KEY=sk-...   # a real OpenAI key - deliberately separate from
                                         # VOICE_OPENAI_API_KEY (voice.stt/tts), which may be
                                         # pointed at a different, OpenAI-compatible provider
VOICE_REALTIME_OPENAI_VOICE=alloy

<script src="/vendor/laraveleasyvoice/voice-realtime.js"></script>
<meta name="csrf-token" content="{{ csrf_token() }}">
<button id="start">Start realtime call</button>
<button id="stop">Interrupt</button>
<script>
  const session = new VoiceRealtimeSession({
      onConnected: () => console.log('live'),
      onEvent: (event) => console.log('event:', event.type, event), // transcripts, turn boundaries, etc. - see OpenAI's Realtime event reference
      onRemoteLevel: (level) => {}, // 0-1 amplitude of the model's voice, for a live waveform
      onError: (error) => console.error(error),
  });

  document.getElementById('start').addEventListener('click', () => session.connect());
  document.getElementById('stop').addEventListener('click', () => session.interrupt());
</script>

Publish it with the same voice-assets tag as the turn-based widget. VoiceRealtimeSession mints a token from /voice/realtime/token, then opens a RTCPeerConnection directly from the browser to OpenAI — Laravel is never in the audio path. interrupt() sends OpenAI's response.cancel event as an explicit "stop" control; note that OpenAI's own server-side voice-activity detection already auto-interrupts a response when it detects you speaking in the default session config, so interrupt() is a backstop, not the only thing making this feel like a real conversation.

Realtime is provider-selectable, like voice.stt/voice.tts — voice.realtime.default and voice.realtime.providers.<name>, same shape, same reason (so you can choose per your own cost/latency/data-residency needs rather than being locked to one vendor). The token endpoint accepts a provider parameter ('openai' or 'deepgram', both live-verified), and each provider has its own browser client class — VoiceRealtimeSession (OpenAI, WebRTC) and VoiceRealtimeDeepgramSession (Deepgram, WebSocket) — since the two protocols are not close enough to share one class's connection logic. Requesting an unsupported provider from the token endpoint returns a clear 422, rather than silently misconnecting.

VOICE_REALTIME_ENABLED=true
VOICE_REALTIME_DEEPGRAM_API_KEY=...   # needs Owner/Admin role (or explicit keys:write scope) -
                                        # a default/member key gets a live 403 INSUFFICIENT_PERMISSIONS,
                                        # confirmed against a real account, since minting a scoped
                                        # key is itself a privileged key-management operation

<script src="/vendor/laraveleasyvoice/voice-realtime-deepgram.js"></script>
<meta name="csrf-token" content="{{ csrf_token() }}">
<button id="start">Start realtime call</button>
<button id="stop">Interrupt</button>
<script>
  const session = new VoiceRealtimeDeepgramSession({
      greeting: 'Hello! How can I help you today?',
      onConnected: () => console.log('live'),
      onTranscript: (text, role) => console.log(role, text), // 'user' or 'assistant'
      onEvent: (event) => console.log('event:', event.type, event),
      onLocalLevel: (level) => {}, // 0-1 mic input amplitude
      onRemoteLevel: (level) => {}, // 0-1 agent speech amplitude
      onError: (error) => console.error(error),
  });

  document.getElementById('start').addEventListener('click', () => session.connect());
  document.getElementById('stop').addEventListener('click', () => session.interrupt());
</script>

What's actually been verified live vs. what hasn't. OpenAiRealtimeTokenBroker and DeepgramRealtimeTokenBroker (the server-side token-minting halves) have both been tested against real accounts with real credentials — and that testing genuinely mattered: two documentation-review passes on OpenAI's request body both missed fields a real API call rejected outright (session.type was required and missing; session.voice doesn't exist, it's nested under session.audio.output.voice), and Deepgram's scopes field turned out to be required despite being documented as optional. All fixed and re-verified live. Deepgram's full browser connection has been verified live, end-to-end: a real browser session against a real Deepgram account, mic audio in, ConversationText transcripts for both sides of the conversation, synthesized speech played back, clean disconnect. OpenAI's WebRTC connection has not — token minting works, but the connection step itself is still blocked on that particular account's billing (see the 429 troubleshooting note below), separate from anything this package controls. Neither browser connection can be exercised by vendor/bin/phpunit at all — there is no way to open a real WebRTC/WebSocket connection or use real microphone/speaker hardware from a PHP test process — so this live-testing distinction matters more than usual here. Expect the occasional event-reliability rough edge on OpenAI's side specifically (response.cancel/conversation.item.truncate community reports) once its connection is unblocked — that's OpenAI's API surface, not something this package can smooth over.

Not implemented yet

Barge-in reliability guarantees. Both realtime clients above genuinely open a live, interruptible connection — but whether an interruption always lands cleanly depends on each provider's own server-side behavior, which this package doesn't control; Deepgram's client in particular has no confirmed explicit "cancel" message the way OpenAI's response.cancel is (see that client's own docblock). A PHP-mediated realtime connection (Contracts\RealtimeVoiceProvider/RealtimeConnection) remains design-stage only — no class implements either — since the browser-direct approach above makes that unnecessary for OpenAI/Deepgram specifically; it would still matter for a provider without a public browser-direct realtime API. Telephony/SIP is deliberately untouched — it also carries call-recording-consent obligations that belong to your application, not this package. These are sequenced, not overlooked. See CHANGELOG.md for the full reasoning.

Security & Trust

This package assumes it will run in front of real, paid, third-party APIs and handle personal (often biometric) voice data, in applications it doesn't control the tenancy model of. Concretely, as of this phase:

• Fail-closed authorization. AuthorizedTool never invokes its handler without a passing Gate::allows() check; an undecidable check (e.g. a guest caller against an ability closure that doesn't declare a nullable $user parameter) is denied, not allowed.
• Fail-closed ownership. VoiceSession::isOwnedBy() denies access to any session without a verifiable matching identity — there's no "no identity recorded, so allow" fallback.
• Cost and abuse limits are on by default. VoiceAgent::limits() bounds both turns-per-session and session duration, and actually ends the session once exceeded — a voice turn triggers billed STT, LLM, and TTS calls, so unbounded usage was never treated as a safe default.
• No SSRF surface. STT only ever reads a local file path you already control; there is no remote-URL ingestion anywhere in this package.
• Size limits before network calls. Both STT (file size) and TTS (text length) reject oversized input before making any outbound request.
• Private storage by default. Synthesized audio is written to voice.storage.disk (default local), never a public disk.
• No invented tenancy, but real enforcement once you plug in a resolver. tenant_id/user_id/guest_token are nullable, FK-less columns — the same posture LaravelEasyAI itself takes, because this package cannot assume your app's tenant model — but once voice.routes.tenant_resolver is set, it's actually checked on every request, not just stored.
• Routes are opt-in and authenticated by default. voice.routes.enabled defaults to false; once enabled, auth stays in the middleware list unless you explicitly set VOICE_ROUTES_REQUIRE_AUTH=false, and guest access needs a second, explicit flag (voice.routes.allow_guest) on top of that.
• Errors are redacted before they reach the client. A provider-side failure over HTTP returns a generic "temporarily unavailable" message and a 502 — the real exception is still logged server-side via report(), never echoed back.
• Double-checked upload limits. The HTTP layer validates file size/type before touching disk; the STT provider re-checks independently, so a bypassed or custom-built controller still can't push an oversized file through to a billed API call.
• Concurrent turns for one session can't corrupt data or lose usage. handleTurn() locks the session row (lockForUpdate(), released before any STT/LLM/TTS call — never held across network latency) while claiming the next turn sequence, backed by a voice_turns(voice_session_id, sequence) unique index; estimated_cost accumulates via an atomic database-level addition rather than a read-then-write, so two turns updating it around the same time can never silently overwrite each other's contribution.

If you find a security issue, please report it privately rather than as a public GitHub issue.

Testing

composer install
vendor/bin/phpunit

Tests use Http::fake() against the real provider URL patterns (mirroring LaravelEasyAI's own test conventions) — no live API calls are made.

Optional: live Deepgram STT integration test. tests/Feature/DeepgramSttIntegrationTest.php calls the real Deepgram API with a small bundled audio fixture (tests/Fixtures/sample-tone.wav) to confirm the full audio → Deepgram → transcript round trip. It's skipped automatically (no failure, no key required) unless a real key is present:

DEEPGRAM_API_KEY=your-real-key vendor/bin/phpunit --filter DeepgramSttIntegrationTest

Optional: live Deepgram TTS integration test. tests/Feature/DeepgramTtsIntegrationTest.php calls the real Deepgram API with a short fixed string to confirm the full text → Deepgram → audio round trip (checks for non-empty binary audio and the expected content type, not exact bytes). Same opt-in gate, same key:

DEEPGRAM_API_KEY=your-real-key vendor/bin/phpunit --filter DeepgramTtsIntegrationTest

Never commit a real key — pass it as an inline env var, as above, or in an uncommitted .env/phpunit.xml.local used only for local runs.

License

MIT.