version: "1.0.1" name: flutter-edge-ai-speech description: Use when adding speech to a flutter_edge_ai app — speech-to-text (transcribe a voice note, dictation, Whisper, moonshine, Parakeet), text-to-speech (Matcha, Qwen3-TTS, Inflect), or a push-to-talk voice assistant with VoiceSession. Also use when transcripts come back in English for non-English audio, a WAV file has to become 16 kHz PCM, synthesized audio plays at the wrong pitch, or getActiveTts throws a StateError about the language. For audio sent straight to Gemma in a chat, use flutter-edge-ai-inference.

Speech with flutter_edge_ai_speech

Rules

1. Depend on flutter_edge_ai and flutter_edge_ai_speech, and import both. The speech package does not re-export core.
2. transcribe takes raw PCM — 16 kHz, mono, 16-bit little-endian, as a Uint8List — and returns the text. Not a WAV file, not 44.1 or 48 kHz: nothing resamples or converts it.
3. Play synthesized audio at synth.sampleRate. It differs per model.
4. Only Whisper has a selectable output language. moonshine-tiny and Parakeet are English-only, and passing a language to them throws ArgumentError.
5. STT language: set a default with getActiveStt(language:) or override one call with transcribe(pcm, language:). Nothing reloads.
6. TTS language: close() the synthesizer first. Asking a live synthesizer for another language throws StateError.
7. Android needs minSdk 30. There is no web support — the web backends throw UnsupportedError.
8. Close recognizers and synthesizers.

Setup

flutter pub add flutter_edge_ai flutter_edge_ai_speech

import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_speech/flutter_edge_ai_speech.dart';
 
await FlutterEdgeAi.initialize(
  sttBackends: [LiteRtSttBackend()],
  ttsBackends: [LiteRtTtsBackend()],
);

Speech runs on the same native libraries as the .litertlm engine. Its build setup — Android minSdk 30, the Apple entries — is in the flutter-edge-ai-inference skill's platform setup, installed alongside this one.

Speech-to-text

An STT model is two files — the model and its tokenizer — usually from different repos. install() skips files already on disk.

await FlutterEdgeAi.installStt()
    .modelFromNetwork('https://huggingface.co/litert-community/whisper-tiny/resolve/main/whisper_tiny_30s_f32.tflite')
    .tokenizerFromNetwork('https://huggingface.co/openai/whisper-tiny/resolve/main/tokenizer.json')
    .ofType(SttModelType.whisper)
    .install();
 
final SpeechRecognizer recognizer = await FlutterEdgeAi.getActiveStt(language: 'de');
try {
  final String german = await recognizer.transcribe(germanPcm);
  final String french = await recognizer.transcribe(frenchPcm, language: 'fr');
} finally {
  await recognizer.close();
}

Audio longer than the window is silently truncated, not rejected: it is zero-padded when shorter and cut when longer, so a 40-second clip on Whisper returns the first 30 seconds with no error. Split long recordings yourself.

Whisper tiny is weak outside English. Whisper base int8 is the next size up in the catalog; install it the same way from https://huggingface.co/litert-community/whisper-base/resolve/main/whisper_base_30s_i8.tflite with the tokenizer https://huggingface.co/openai/whisper-base/resolve/main/tokenizer.json.

Getting 16 kHz mono PCM

The package has no resampler and no WAV reader. Record in the right format from the start — with the record package:

import 'package:record/record.dart';
 
const config = RecordConfig(
  encoder: AudioEncoder.wav,
  sampleRate: 16000,
  numChannels: 1,
);

Recording needs microphone access. On Android record declares android.permission.RECORD_AUDIO in its own manifest and the manifest merger adds it to the app; iOS and macOS need entries of your own:

<!-- ios/Runner/Info.plist and macos/Runner/Info.plist -->
<key>NSMicrophoneUsageDescription</key>
<string>Speech is transcribed on this device.</string>
 
<!-- macos/Runner/DebugProfile.entitlements and macos/Runner/Release.entitlements -->
<key>com.apple.security.device.audio-input</key>
<true/>

The macOS sandbox withholds the microphone without the entitlement, and it has to be in both entitlements files. On Android, iOS and macOS the user must also allow it at run time; hasPermission() asks, so call it before recording:

final recorder = AudioRecorder();
if (!await recorder.hasPermission()) return;

That produces a WAV file. Its header is not always 44 bytes — take the samples from the data chunk:

import 'dart:typed_data';
 
/// The samples of a 16 kHz mono 16-bit WAV file, without its header.
Uint8List pcmFromWav(Uint8List wav) {
  final view = ByteData.sublistView(wav);
  var offset = 12; // after 'RIFF', the size and 'WAVE'
  while (offset + 8 <= wav.length) {
    final id = String.fromCharCodes(wav, offset, offset + 4);
    final size = view.getUint32(offset + 4, Endian.little);
    final start = offset + 8;
    if (id == 'data') {
      final end = start + size > wav.length ? wav.length : start + size;
      return Uint8List.sublistView(wav, start, end);
    }
    offset = start + size + (size & 1); // chunks are padded to an even size
  }
  throw const FormatException('WAV file has no data chunk');
}

A file recorded at another rate or channel count — 44.1 kHz stereo, say — has to be converted first: average the channels to mono, then resample with a low-pass filter. Dropping samples instead aliases and costs accuracy.

Traps

Transcript comes back in English

• Symptom: German audio, fluent English text, no error.
• Cause: Whisper's language token decides the output language, not what it understands — with 'en' it translates. moonshine only ever produces English.
• Fix: use Whisper and pass language:.

A language is rejected

• Whisper codes are bare and lowercase: 'de', not 'de-DE', 'DE' or 'german'. Malformed codes throw ArgumentError from getActiveStt.
• A well-formed code the installed checkpoint lacks (e.g. 'zz') throws ArgumentError from transcribe.

Text-to-speech

await FlutterEdgeAi.installTts()
    .fromNetwork('https://huggingface.co/litert-community/Matcha-TTS/resolve/main/')
    .ofType(TtsModelType.matcha)
    .install();
 
final SpeechSynthesizer synth = await FlutterEdgeAi.getActiveTts();
try {
  final audio = await synth.synthesize('Hello world.'); // 16-bit PCM
  final rate = synth.sampleRate;                        // 22050 for Matcha
} finally {
  await synth.close();
}

TtsModelType.supertonic and TtsModelType.kokoro are in the enum but throw UnimplementedError — do not use them.

To ship the voice with the app instead of downloading it, swap fromNetwork for one of:

Inflect's four Matcha G2P files go next to its own two. A missing file fails install() with the full list before the current voice is replaced; a switch that fails partway (lost connection) leaves no voice active, so getActiveTts throws until install() is run again. install() throws UnsupportedError on web. To ship a changed voice in an app update, use a new asset directory (a bundled voice refreshes itself); with fromFile on iOS, call install() at launch, because the data directory moves on update.

Switching the Qwen3 language — full lowercase names, not ISO codes, and only with the Qwen3 bundle installed (Matcha still throws the same StateError but the language changes nothing):

final english = await FlutterEdgeAi.getActiveTts(language: 'english');
await english.close();
final german = await FlutterEdgeAi.getActiveTts(language: 'german');

Without the close(), the second call throws StateError: Active TTS synthesizer was created for language 'english'; call close() before requesting 'german'.

Voice assistant

VoiceSession runs one push-to-talk turn: transcribe, generate, speak, with barge-in. It uses the recognizer's current language.

Wrap the loop in try/catch: a failed stage — transcribe, generate or synthesize — arrives as a stream error, not as an event. VoiceErrorEvent is reserved in this release and never emitted; the case is only there because the switch must be exhaustive. To barge in, call await voice.interrupt() — cancelling the subscription is not a portable stop.

final reply = StringBuffer();
final voice = VoiceSession.fromChat(
  recognizer: await FlutterEdgeAi.getActiveStt(language: 'de'),
  chat: chat,
  synthesizer: await FlutterEdgeAi.getActiveTts(),
);
 
await for (final event in voice.runTurn(pcm16kMono)) {
  switch (event) {
    case VoiceTranscriptEvent(:final text):
      print('heard: $text');
    case VoiceReplyTextEvent(:final chunk):
      reply.write(chunk);
    case VoiceReplyAudioEvent(:final sampleRate):
      print('audio at $sampleRate Hz');
    case VoiceTurnInterruptedEvent():
      print('interrupted — stop the player');
    case VoiceTurnCompleteEvent():
      print('done');
    case VoiceErrorEvent(:final error):
      print('failed: $error');
  }
}

chat is an InferenceChat from the flutter-edge-ai-inference skill. A chat created with tools also needs onToolCall: — without it fromChat throws.

Audio turns: no speech-to-text

A model that hears audio (Gemma 4 E2B or E4B as .litertlm, native platforms only) can take the recording itself: runAudioTurn skips the transcription and sends the PCM to the chat. Set supportAudio: true on getActiveModel, which loads the audio encoder; createChat takes the flag from the model unless you pass it, and on a chat whose supportAudio is false runAudioTurn throws StateError. It takes the same 16 kHz mono 16-bit PCM as runTurn, and wraps it in a WAV header, because a .litertlm model decodes an audio file; a custom VoiceResponder.respondToAudio gets that WAV and can pass it to AgentSession.ask(audioBytes:) as is. There is no VoiceTranscriptEvent, the complete and interrupted events carry an empty transcript, and a recording under 2 bytes completes without asking the model. After a barge-in the chat keeps earlier audio turns with flutter_edge_ai_litertlm 1.15.0 and later; with 1.14.0 and earlier, and on the NPU, it is rebuilt from text and forgets them. Requires flutter_edge_ai_speech 0.5.7.

final model = await FlutterEdgeAi.getActiveModel(
  maxTokens: 4096,
  supportAudio: true,
);
final chat = await model.createChat(supportAudio: true, maxOutputTokens: 128);
final voice = VoiceSession.fromChat(
  recognizer: recognizer,
  chat: chat,
  synthesizer: synth,
);
 
await for (final event in voice.runAudioTurn(pcm16kMono)) {
  if (event is VoiceReplyTextEvent) print(event.chunk);
}