Full-Duplex Speech-to-Speech AI Explained: Audio Tokens and Labels

A full-duplex speech-to-speech model takes audio in and produces audio out, listening and talking at the same time, without converting speech to text in the middle. A neural audio codec turns sound into discrete tokens, and a language model predicts the next tokens for its own voice while it keeps hearing yours.

That is why these models laugh mid sentence, sigh, hesitate and cut in at the right moment: no transcript strips out timing and emotion. It is also why they raise the stakes for voice impersonation. This guide covers what Block announced about its Dessa model on 23 October 2026, how codecs and audio tokens work with worked numbers, how cascaded and speech-to-speech stacks differ, what the FCC and the EU AI Act say about synthetic voices, and how watermarks, content credentials and classifiers label synthetic speech, including where each one breaks.

Key takeaways

• Block describes Dessa as a full-duplex speech-to-speech model trained from scratch on human conversation only, with no text and no pretrained models, that runs locally on a MacBook Pro M4 Max or better. There is no public access; Block runs a waitlist and is giving a small group of AI safety researchers early access.
• A cascaded stack chains speech recognition, a text model and text to speech. A speech-to-speech model generates audio tokens directly. Full duplex means it models your audio stream and its own at once, so overlap, interruptions and backchannels are possible.
• Neural audio codecs use residual vector quantization (RVQ). EnCodec at 24 kHz produces 75 frames per second with 10 bit codebooks, so 2 codebooks give 1.5 kbps. Mimi, the codec inside Moshi, runs at 12.5 frames per second with 8 codebooks of 11 bits, which is 1.1 kbps.
• The FCC's ruling FCC 24-17 (adopted 2 February 2024) treats AI generated voices as "artificial or prerecorded voice" under the TCPA. Article 50 of the EU AI Act requires machine readable marking of synthetic audio and disclosure of AI interaction and deep fakes, applying from August 2026 according to the European Commission.
• Labels come in three families: watermarks, signed provenance metadata such as C2PA Content Credentials, and passive classifiers. Each fails in a known way. The Moshi paper found that a round trip through its own low bitrate codec made an AudioSeal watermark indistinguishable from unmarked audio.
• The dependable defence is procedural: no approvals by voice alone, callback to a number of record, code words for high risk requests, disclosure whenever an AI voice speaks to a customer, and an inventory entry for every AI voice feature.

What did Block actually announce about Dessa?

Block published the announcement on its own page at block.xyz/dessa, and TechCrunch reported it on 24 October 2026. Facts below come from Block's page unless marked as press coverage.

• What it is. "Audio in, audio out. Dessa is a true speech-to-speech model." Block says it is full duplex: "it listens and talks at the same time, like you do."
• Training. Block says it trained Dessa "from scratch on nothing but human conversation", with "no text, no pretrained models", on one NVIDIA GB200 NVL72.
• Behaviour. Block says it learned when to cut in and when to backchannel ("yeah!", "got it!", "ooh"), and that it "natively hears and speaks emotion, sarcasm, laughter, a frustrated sigh". Block describes each published clip as a live recording of a person talking to Dessa, with subtitles added afterwards.
• Where it runs. "Inference runs faster than real time", and it "runs locally on a MacBook Pro M4 Max or better."
• Data claim. Block says Dessa learned language from "hundreds of times fewer words" than large language models. The page gives no figures, so treat it as a claim.
• Who built it. A small research lab at Block, founded by former members of the Toronto deep learning startup also called Dessa. The lab reports to Jack Dorsey and says it has built generative speech models for over eight years.
• Safety and access. Block lists safety as one of three motivations and is giving "a small group of AI safety researchers early access to find the risks we haven't anticipated". One demo pairs Dessa with a third party AI voice, labelled "Both voices are AI-generated."

TechCrunch reported "no public access to Dessa at the moment" and described the training hardware as a single NVIDIA GPU rack, which is consistent with the rack scale GB200 NVL72 that Block names.

What Block has not published as of 25 October 2026: a technical report, a parameter count, the audio tokenizer it uses, training data sources and hours, benchmark results, or anything about watermarking its output. The rest of this guide therefore explains how published full-duplex systems work, from the research papers. It does not describe Dessa's internals.

For voice agent pipelines, realtime APIs, turn taking and securing agent tools, see our guide on voice AI agents. This post stays on model internals and synthetic speech safety.

Cascaded voice stacks versus speech-to-speech models

The cascaded pipeline

Most voice assistants built before 2024 chained components: voice activity detection (VAD) to find the end of your turn, automatic speech recognition (ASR) to transcribe it, a text language model to write a reply, and text to speech (TTS) to read it aloud.

The Moshi paper from Kyutai (September 2024) names three problems with this design. The chain induces "a latency of several seconds between interactions". Because text is the intermediate modality, non-linguistic information that changes meaning, "such as emotion or non-speech sounds", is lost. And segmenting speech into speaker turns cannot represent overlap, interruptions or interjections.

The cascade has one big advantage: every hop produces an inspectable artifact. A transcript can be logged, a text reply can be filtered before it is spoken, and a TTS request can be rate limited and watermarked at the server. Safety teams like the text in the middle because it is a choke point.

The speech-to-speech model

A speech-to-speech model removes the transcript. A neural codec encodes audio into tokens, a sequence model predicts the next audio tokens, and the codec decoder turns them back into a waveform.

OpenAI's Realtime API documentation describes the hosted version of this category: a speech-to-speech voice agent where "the model works directly with audio, maintains conversation state, and can call tools", connected over WebRTC, WebSocket or SIP for telephony. The same docs point to a separate product, GPT-Live, "for full-duplex conversations with a separate delegated backend". Block's claim is different: Dessa runs on a laptop.

What "full duplex" means

In telecoms, half duplex is one direction at a time (a walkie-talkie) and full duplex is both at once (a phone call). Applied to voice AI:

• Half duplex turn taking. The system waits for an end of turn signal, then speaks. A separate "barge in" detector cuts its audio if you talk over it. It cannot follow what you say while it is speaking.
• Full duplex. At every time step the model sees what it hears and what it is saying, and predicts its own next audio from both. It can keep talking through your "mm hm", stop for a real interruption, or laugh at your joke before you finish it.

Moshi implements this as what the paper calls the first "multi-stream audio language model", processing input and output audio "jointly into two autoregressive token streams" so there is no concept of a speaker turn. When the user talks and Moshi stays silent, Moshi's stream decodes into "natural silence".

Training data has to look like that too. Moshi gained full-duplex behaviour by fine tuning on the Fisher corpus, about 2,000 hours of telephone conversations with each side on a separate channel. Block says Dessa learned from real conversation; it has not said whether that data was multi-channel.

How neural audio codecs turn speech into tokens

Raw audio at 24 kHz is 24,000 numbers per second, each typically a 16 bit integer. A language model needs a short sequence of discrete symbols. A neural audio codec bridges the two:

1. Encoder. A neural network reads the waveform and outputs one vector per frame, each summarising a short slice of audio.
2. Quantizer. Each vector is replaced by the index of its nearest entry in a learned codebook. That index is the token.
3. Residual stacking. One codebook is too coarse, so the trick is repeated on the leftover error.
4. Decoder. A mirror network turns the codebook entries back into audio. Codecs are trained with reconstruction and adversarial losses so the output sounds right to a listener.

Residual vector quantization

SoundStream (July 2021) introduced the residual vector quantizer for neural codecs, trained end to end with the encoder and decoder. EnCodec (October 2022) uses the same idea and puts it simply: "RVQ refines this process by computing the residual after quantization, and further quantizing it using a second codebook, and so forth."

• Codebook 1 picks the entry e1 closest to the frame vector. The residual r1 is the vector minus e1.
• Codebook 2 picks the entry e2 closest to r1. The new residual is r1 minus e2.
• Repeat for as many codebooks as the bitrate allows. The decoder sums e1, e2 and the rest.

Early codebooks capture the broad shape of the sound and later ones add detail, so a model can predict coarse tokens first and fine ones after. SoundStream trained with "structured dropout applied to quantizer layers" so one model runs at several bitrates, 3 kbps to 18 kbps in that paper.

Worked example 1: EnCodec bitrates

The EnCodec paper gives every number needed:

• The 24 kHz encoder outputs 75 latent steps per second.
• Each codebook has 1,024 entries, which is 10 bits per index (2 to the power 10 is 1,024).
• It uses at most 32 codebooks at 24 kHz.

Bitrate is frames per second times codebooks times bits per index:

These match the five bandwidths the paper says the 24 kHz model supports. Uncompressed 24 kHz mono at 16 bits is 24,000 x 16 = 384,000 bit/s, so at 1.5 kbps EnCodec carries the same second in 1/256 of the bits. At 6 kbps one second is 75 x 8 = 600 tokens, or 36,000 tokens per minute of one speaker.

Worked example 2: Mimi, the codec inside Moshi

Kyutai designed Mimi to make audio tokens cheap enough for real-time dialogue. The Moshi paper states that Mimi runs at 12.5 Hz with codebooks of 2,048 entries, 8 quantizers, and that "at 12.5Hz, this represents a bitrate of 1.1kbps". Check it:

• 2,048 entries is 11 bits per index.
• 8 codebooks x 11 bits = 88 bits per frame.
• 88 bits x 12.5 frames per second = 1,100 bit/s = 1.1 kbps, about 1/349 of raw 24 kHz PCM.

The token counts are what make full duplex affordable:

• 12.5 x 8 = 100 audio tokens per second per speaker, or 6,000 per minute.
• A full-duplex model tracks two speakers, so 200 audio tokens per second of conversation, plus any text tokens.

Each Mimi frame covers 80 ms, and Mimi is causal: from a first 80 ms of audio it outputs a first latent step that decodes to 80 ms of audio. The complete Moshi system reports 160 ms theoretical latency and 200 ms in practice, against the "several seconds" the same authors attribute to cascades.

Semantic tokens and acoustic tokens

Tokens trained only for reconstruction carry sound well and meaning poorly. AudioLM (September 2022) combined "semantic" tokens from a self-supervised speech model with "acoustic" tokens from a codec. Trained on speech without transcripts, it produced "syntactically and semantically plausible speech continuations" while keeping speaker identity and prosody. Mimi folds both into one codec with what the paper calls a split RVQ: semantic information from a self-supervised speech model is distilled into the first quantizer level, and the remaining levels carry acoustic detail.

How a model generates speech tokens directly

Predicting a grid, not a line

A text model predicts one token per step. An audio model faces a grid: several codebook levels at every frame. Flattening it multiplies sequence length by the number of codebooks, so published systems split the work:

• Hierarchical prediction. Moshi uses an RQ-Transformer: a large Temporal Transformer steps once per 80 ms frame, and a small Depth Transformer predicts the codebook levels inside each frame. The big model runs 12.5 times per second, not 100.
• Acoustic delay. Moshi offsets acoustic tokens by one or two frames relative to the semantic token, so the coarse decision for a frame comes before its detail.

Two streams for two speakers

Each time step carries the user's tokens and the model's tokens. During training Moshi predicts both. At inference, the paper says, its prediction for the user stream "is actually ignored, as the actual user audio is used instead", and only the model's own stream is sampled and played. The model learns from data when to speak, stay silent or overlap.

With or without an inner text stream

Moshi starts from Helium, a 7B parameter text language model, and adds an "Inner Monologue": time aligned text tokens predicted as a prefix to each frame's audio tokens. The authors report this "significantly improves the linguistic quality of generated speech".

Block describes Dessa as trained with no text and no pretrained models. If so, there is no text backbone and no inner text stream, and everything it knows came from sound, closer to AudioLM's textless approach than to Moshi's. Until a technical report exists, the architecture is unknown.

This matters for safety engineering. A model with an inner text stream produces text you can log and filter. A purely textless model produces only audio tokens, so any record of what it said has to come from a separate speech recognizer.

Why speech-to-speech models sound human

What a transcript throws away

Block's page offers a test: say "I didn't say she stole my CUDA code" aloud, stressing a different word each time. The words never change; the meaning does. A transcript records the same string every time. What changed the meaning was prosody, and it lives only in the audio.

A cascade loses prosody twice: ASR drops how you said it, and TTS must invent how the reply should sound from text. Codec tokens keep the sound itself:

• Prosody: stress, pitch, rhythm and rate.
• Non-speech sounds: laughter, sighs, breaths, gasps, a voice cracking.
• Timing: pauses, hesitations, overlap and reply speed.
• Backchannels: "yeah", "uh huh" and "ooh" without taking the turn.

Block's demos lean on these. It says the model "laughs the moment something's funny, mid-sentence" and gets louder when asked to speak up ("Nobody taught it that").

Timing alone is persuasive. A two second gap before every reply was the old giveaway of a voice bot. Moshi's authors note that their 160 ms theoretical latency is below the average gap measured in natural conversations across 10 languages. A model that replies inside normal human gaps and backchannels removes cues people have used to spot a machine.

The new risks

Voice cloning and robocalls predate full-duplex models. What changes is how convincing a live synthetic voice can be and how cheaply it can run.

1. Live impersonation that survives questions. A pre-recorded deepfake collapses when the listener asks something unexpected. A full-duplex model can answer, interrupt and laugh off a challenge. The "executive who needs a payment today" and the "caller who needs a password reset" become more convincing when the voice can converse.
2. Emotional pressure as a feature. A model that hears and speaks emotion can mirror distress or warmth. The capability Block frames as safety (noticing a user in pain) can be pointed at a target to build rapport.
3. Voice drift toward the caller. These models condition on the other speaker's audio. Moshi's authors measured whether their model drifted into the user's voice: over 100 hours of generated conversations, 98.7 percent of segments (10,249) stayed closer to the model's own voice and 1.3 percent (133) were closer to the other speaker. A good result, and proof the failure mode is real enough to test.
4. Local inference removes provider controls. A hosted API can enforce usage policies, require consent for custom voices and watermark at the server. A model on a laptop has none of that unless the operator adds it.
5. Less text to audit. In a textless model there is no transcript in the loop; the audit trail is audio unless you add a recognizer.

For team procedures against cloned voice calls, see our verification policy guide for deepfake calls.

Cascaded pipeline versus speech-to-speech model

Speech-to-speech wins on naturalness and latency; cascades win on inspectability. If you deploy speech-to-speech for customer contact, rebuild the inspection points the cascade gave you for free.

What do the rules say about synthetic voices?

United States: FCC Declaratory Ruling FCC 24-17

The FCC adopted this Declaratory Ruling on 2 February 2024 and released it on 8 February 2024 (CG Docket No. 23-362). The Commission confirmed "that the TCPA's restrictions on the use of 'artificial or prerecorded voice' encompass current AI technologies that generate human voices." Consequences spelled out in the ruling:

• Such calls need the called party's prior express consent, absent an emergency purpose or exemption.
• All artificial or prerecorded voice messages must provide identification and disclosure information for the entity responsible for the call.
• Advertising or telemarketing messages must also offer the specified opt-out methods.

An outbound call placed by any AI voice, cascaded or speech-to-speech, is an artificial voice call for TCPA purposes. This is not legal advice; take specific campaigns to counsel.

European Union: AI Act Article 50

Regulation (EU) 2024/1689 sets transparency duties in Article 50:

• Article 50(1): providers must design AI systems that interact directly with people so those people are informed they are dealing with AI, unless that is obvious from the context.
• Article 50(2): providers of systems, including general purpose AI systems, that generate synthetic audio, image, video or text must ensure outputs are "marked in a machine-readable format and detectable as artificially generated or manipulated". The technical solutions must be effective, interoperable and reliable as far as technically feasible, considering each content type, cost and the state of the art.
• Article 50(4): deployers of a system that generates audio constituting a deep fake must disclose that it is artificial, with lighter disclosure for evidently artistic, satirical or fictional work.

The European Commission's AI Act page says the Act became applicable on 2 August 2026 with listed exceptions and that "the transparency rules of the AI Act will come into effect in August 2026". It also describes the AI Omnibus, in force since 27 July 2026, which moved several high risk deadlines, and a voluntary Code of Practice on Marking and Labelling of AI-generated Content that includes disclosure icons.

The qualifier "as far as technically feasible" is doing real work for audio, as the next section shows.

Provider terms

OpenAI's text to speech documentation says its usage policies "require you to provide a clear disclosure to end users that the TTS voice they are hearing is AI-generated and not a human voice", and that custom voices are created from "a speaker's consent recording and matching audio sample". Check the current terms of any speech service you use. A local model has no such terms.

How can synthetic speech be labelled or detected?

NIST AI 100-4, "Reducing Risks Posed by Synthetic Content" (November 2024), splits the approaches into provenance data tracking (watermarks and metadata, signed or unsigned) and synthetic content detection (classifying content as synthetic, including by finding a watermark). NIST notes they can be combined: a covert watermark that is hard to strip in transit, plus signed metadata for whoever receives the file intact.

1. Audio watermarking: AudioSeal

AudioSeal (January 2024) was built to detect AI generated speech, including inside longer recordings:

• Generator and detector trained together. The generator adds a watermark waveform; the detector outputs, per sample, the probability the mark is present.
• Localised detection at "sample level resolution (1/16k seconds)", so synthetic words spliced into a real recording can be found.
• Optional message. Attribution experiments embed 16 bit messages to tell model versions apart.
• Imperceptibility from a loudness loss based on auditory masking, and a single pass detector the authors report as up to two orders of magnitude faster than earlier methods.

In the paper's results table, detection accuracy was 1.00 after MP3 and AAC compression and resampling and 0.98 after EnCodec compression, but 0.61 after a highpass filter. Against a passive classifier baseline on re-synthesized audio, AudioSeal stayed at 1.0 where the classifier fell to 0.704. The authors tested removal attacks with different levels of attacker knowledge and concluded that the training code and the fact that audio is watermarked can be public, but "the detector's weights should be kept confidential".

Where watermarks break: the Moshi codec test

The Moshi team tested AudioSeal on their own model's output, averaged over 1,000 generations:

Their conclusion: Mimi "removes the mark to a level that makes a watermarked audio indistinguishable from a non-watermarked audio", because low bitrate codecs "discard anything not related to the signal reconstruction". This sits alongside AudioSeal's own 0.98 result for EnCodec; different codecs and bitrates behave differently, which is why you test the channel you care about.

They also tried watermarking the token sampling itself, as text watermarks do. It ran into the fact that codecs are not idempotent: decode tokens to audio and re-encode, and you often get different tokens back even with no added noise, with the semantic level most stable and deeper levels less so.

NIST reaches a compatible view: early research suggests audio watermarks hold up against the perturbations they were trained on, are much weaker against other distortions and are "not secure against adversarial perturbation". It also flags forgery, where a mark is added to genuine content.

None of these sources measures the telephone network, which band limits audio (often to 8 kHz sampling on narrowband paths) and passes it through speech codecs and loss concealment. Do not assume a watermark survives a phone call unless the vendor shows results for that exact path.

2. Provenance metadata: C2PA Content Credentials

The C2PA Technical Specification (version 2.2 here) defines a signed manifest that travels with a file and records claims about how it was made:

• Hard binding: cryptographic hashes of the asset. Change a hashed byte and validation fails.
• Soft binding: "a content identifier that is either (a) not statistically unique, such as a fingerprint, or (b) embedded as an invisible watermark", which lets a stripped file be matched back to its manifest.
• AI origin: a c2pa.actions assertion starting with c2pa.created whose digitalSourceType is the IPTC value trainedAlgorithmicMedia marks generative AI output.
• Audio containers: MP3 and FLAC carry the manifest in an ID3v2 General Encapsulated Object, WAV uses RIFF chunk handling, and MP4 based audio follows the ISO base media file format rules. The open source C2PA SDK lists wav, mp3, flac and m4a as supported.

The limit is structural. A manifest rides on a file or file segments. A live call has nowhere to carry it, and re-recording the output through a microphone produces a new file with no credentials. A missing credential never proves content is human; see our post on what a missing content credential means.

3. Passive detection classifiers

A passive detector estimates whether audio is synthetic from artifacts alone. It is the only option for a generator that did not mark its output, which includes any local model whose operator chose not to. Classifiers learn the artifacts of the generators they were trained on, generalise poorly to new ones and degrade with compression and re-recording, as the AudioSeal comparison above shows. As models improve, those artifacts shrink.

Comparing the methods

Our explainer on watermarking and provenance techniques for AI models covers text and image methods in the same frame.

Step by step: a voice channel policy and an AI voice inventory

These steps suit a team that receives voice requests, ships an AI voice feature, or both. None depends on detecting synthetic audio.

Step 1: Write the voice channel rule

No approval, payment, credential reset, MFA change or data release is granted on the strength of a voice alone, however familiar it sounds.

Then define what replaces the voice:

1. Callback to a number of record from your directory or vendor master file, never one given on the call.
2. A second channel: confirm in ticketing or chat from an authenticated account.
3. Code words for executives and finance staff, agreed in person and rotated.
4. Dual control: payments and bank detail changes need a second approver who did not take the call.
5. A no blame stop: anyone may pause a voice request to verify.

Step 2: Disclose your own AI voice

An AI voice says what it is at the start of every interaction:

Hi, this is an automated assistant for Example Co. I'm an AI voice, not a person.
This call may be recorded. Say "agent" at any time to reach a human.

For US outbound calls, involve counsel on consent, identification and opt-out. For people in the EU, map the disclosure to Article 50(1), and published generated audio to Articles 50(2) and 50(4).

Step 3: Add every AI voice feature to your AI inventory

# ai-inventory/voice-support-agent.yaml
id: voice-support-agent
owner: contact-centre-platform
architecture: speech-to-speech            # or: cascaded (asr + llm + tts)
duplex: full                              # or: half
deployment: hosted-api                    # or: local-weights
provider: example-speech-provider
model: example-realtime-model
model_version_pinned: "2026-09-30"
weights:
  source: null                            # set for local-weights deployments
  sha256: null
voices:
  - id: support-voice-1
    type: stock                           # or: custom
    consent_record: null                  # required when type is custom
disclosure:
  spoken_at_start: true
  text: "I'm an AI voice, not a person."
provenance:
  watermark: "unknown, asked vendor 2026-10-25"
  c2pa_for_recordings: false
logging:
  audio_retention_days: 30
  transcript_source: separate-asr         # inner-text-stream | separate-asr | none
  pii_redaction: true
outbound_calls: false
eu_users: true

A small CI check stops entries landing without the fields that matter:

#!/usr/bin/env bash
# fail if an AI voice entry lacks spoken disclosure or has a custom voice without consent
set -euo pipefail
status=0
for f in ai-inventory/*.yaml; do
  grep -q 'spoken_at_start: true' "$f" || { echo "missing spoken disclosure: $f"; status=1; }
  if grep -q 'type: custom' "$f" && grep -q 'consent_record: null' "$f"; then
    echo "custom voice without consent record: $f"; status=1
  fi
done
exit $status

Treat a failure like any other policy gate: Block the merge, Warn the owner or Notify the platform team, and route genuine one-off cases through a documented exception.

Step 4: Verify local voice weights like code

A local speech model is a large binary you execute. Download from the publisher's official location, pin a revision and compare the checksum the publisher lists:

shasum -a 256 model.safetensors   # macOS
sha256sum model.safetensors       # Linux

Prefer tensor only formats such as safetensors over formats that can run code when loaded, record source, revision and hash in the inventory, and keep the inference runtime's packages in your normal dependency scanning.

Step 5: Ask speech vendors five questions

1. Is generated audio watermarked by default, with what method, and can you get a detector or detection API?
2. Has the watermark been tested over your call path (narrowband telephony, conferencing codecs, your SIP provider)? Ask for results.
3. Can generated files be exported with C2PA credentials declaring trainedAlgorithmicMedia?
4. How are custom voices created, and what consent evidence is kept?
5. What is logged (audio, transcripts, tool calls), for how long, and can you get it for an investigation?

Step 6: Check published audio for Content Credentials

If you publish AI generated audio, sign it with C2PA, then verify what shipped with the open source c2patool:

c2patool generated-ad.wav            # summary report of C2PA manifests
c2patool generated-ad.wav --detailed # low level manifest report

Look for a c2pa.created action with a digitalSourceType ending in trainedAlgorithmicMedia. If there is no manifest, find where your publishing chain stripped it.

Step 7: Test labels over the real channel, and log sessions

If you rely on a vendor watermark, send a generated sample through the exact channel (a test call, your conferencing platform, voice notes), capture what arrives and run the detector on it. If detection fails, your policy cannot depend on it. For speech-to-speech sessions, keep both audio channels as separate tracks, a transcript from a separate recognizer if the model has no inner text stream, and every tool call, under your retention and redaction rules.

Common mistakes

• Trusting a familiar voice. Familiarity was never authentication, and a full-duplex model makes a cloned voice conversational.
• Assuming watermarks survive phone compression. A low bitrate codec round trip erased a strong watermark in the Moshi test. Test, or assume it does not survive.
• Treating a missing label as proof of a human. No watermark and no manifest means nothing was detected.
• Running local voice models from unverified weights. Pin a revision, check the hash, prefer tensor only formats and scan the runtime's dependencies.
• No disclosure when an AI voice speaks to customers. It breaks provider terms, creates TCPA exposure for US outbound calls and conflicts with Article 50 in the EU.
• Switching a cascaded feature to speech-to-speech without rebuilding inspection. Text filters and transcript logs disappear with the transcript.
• Letting the AI voice inventory drift. Voice features ship as SDK imports and config changes; if nobody owns the inventory, nobody knows which products speak with a synthetic voice.

FAQ

What is a full-duplex speech-to-speech AI model?

A model that takes audio in and generates audio out while listening and speaking at the same time. It models the user's audio and its own as parallel streams, so it can overlap, interrupt and backchannel instead of waiting for a turn to end.

How is speech-to-speech different from speech recognition plus text to speech?

A cascaded assistant transcribes you, sends text to a language model and reads the reply aloud. A speech-to-speech model skips the transcript and generates audio tokens directly, keeping tone, timing and non-speech sounds and cutting latency.

What are audio tokens?

Integer indices produced by a neural audio codec. An encoder turns each short frame into a vector, and residual vector quantization replaces it with entries from several codebooks. A model predicts those indices the way a text model predicts word pieces.

Why do AI voices like Dessa sound so human?

Nothing converts the conversation to text in the middle. Codec tokens carry prosody, laughter, sighs and timing, and a full-duplex model learns when to pause, cut in and backchannel from real conversations.

Is Block's Dessa available to the public?

Not as of 25 October 2026. Block offers a waitlist and early access for a small group of AI safety researchers, and TechCrunch reported no public access.

Can you detect AI generated speech reliably?

Not from audio alone in every case. Watermarks work when the generator cooperated and the audio was not heavily re-encoded; C2PA works for files that kept their metadata; classifiers give a probability that weakens with new generators. Use procedural verification for anything important.

Do audio watermarks survive phone calls?

There is no general guarantee. Research shows watermarks surviving common file compression but erased by low bitrate neural codec round trips, and the sources reviewed here do not measure phone networks. Test the specific path.

What do US and EU rules require for AI voices?

In the US, the FCC treats AI generated voices as "artificial or prerecorded voice" under the TCPA, so calls need prior express consent unless exempt, plus identification and opt-out rules. In the EU, Article 50 of the AI Act requires disclosure of AI interaction and audio deep fakes and machine readable marking of synthetic audio, from August 2026.

Why software supply chain hygiene still matters for voice AI

A speech-to-speech feature is still software: an inference runtime, codec and audio libraries, WebRTC and SIP stacks, Python and native packages and container images, all shipping open source components that receive security fixes like anything else. Safeguard cannot tell you whether a voice is synthetic, but the stack that runs your voice feature is where unpatched dependencies quietly accumulate.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation categories include dependencies, licenses, security posture and findings, and each one is set separately. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds.

3. Gold Registry: start from components that are already fixed

Gold Registry provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs and a completed license check, with signed build provenance (signed with Safeguard's key and verifiable with cosign, SLSA Build Level 3 as the design target) and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. Ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

To be plain about the fit: Safeguard does not detect synthetic speech, embed or read audio watermarks, verify C2PA credentials or verify callers, and Safeguard's own models are single purpose security models that do not process audio. It helps with the software around a voice feature. It scans repositories, container images and SBOM files for dependencies, matches them against vulnerability data and shows which projects contain a given component and version, including the audio, codec, WebRTC and inference libraries a voice agent pulls in. AI-BOM discovery inventories AI SDK imports in code and produces an ML-BOM, a useful start for the inventory in step 3. Policies and gates can Block, Warn or Notify when a vulnerable version would ship, with exceptions for cases you accept. Gold Open Source at gold.safeguard.sh gives free CVE intelligence with affected and fixed versions.

The first demo is a real CLI session. Asked to fix a lodash CVE, the assistant reads the finding and advisory, asks before any command that changes something, edits package.json to lodash 4.17.21, runs npm install and npm test with your approval, prints a summary and pushes nothing. The same flow applies when the Node.js signalling server behind a voice agent pins a vulnerable WebSocket or media package.

Click through it at your own pace, or open it full screen.

The second demo sets up the local runner on your own machine, approves it with its device code, then scans from the terminal. Teams trying local speech models often keep the inference code and its dependencies on a workstation or internal GPU box, and the runner scans them where they live.

Click through it at your own pace, or open it full screen.

The third demo connects Claude to the Safeguard MCP server with OAuth and lets you choose which tools Claude can use. It is a quick way to ask which of your projects carry a given audio or WebRTC library version on the day an advisory lands, with tool permissions you control.

Click through it at your own pace, or open it full screen.

To try the CLI, download Safeguard and follow the CLI documentation.

Quick check

Sources

• Block: Dessa, the conversation model trained only on speech
• TechCrunch: Block's new voice model sounds eerily human (24 October 2026)
• Mashable: Block's latest AI model Dessa was trained entirely on human speech (24 October 2026)
• Défossez et al., High Fidelity Neural Audio Compression (EnCodec), arXiv:2210.13438
• Zeghidour et al., SoundStream: An End-to-End Neural Audio Codec, arXiv:2107.03312
• Borsos et al., AudioLM: a Language Modeling Approach to Audio Generation, arXiv:2209.03143
• Défossez et al., Moshi: a speech-text foundation model for real-time dialogue, arXiv:2410.00037
• San Roman et al., Proactive Detection of Voice Cloning with Localized Watermarking (AudioSeal), arXiv:2401.17264
• FCC Declaratory Ruling FCC 24-17, CG Docket No. 23-362
• NIST AI 100-4, Reducing Risks Posed by Synthetic Content (November 2024)
• C2PA Technical Specification 2.2
• C2PA SDK supported formats
• C2PA Tool (c2patool)
• EU AI Act Article 50, AI Act Service Desk, European Commission
• European Commission: AI Act regulatory framework and timeline
• OpenAI: Getting started with the Realtime API
• OpenAI: Text to speech guide

Last reviewed: 25 October 2026. Block has not published a technical report for Dessa; check Block's page for updates and the European Commission's AI Act pages for changes to transparency timelines.