denisovav/flutter_edge_ai/flutter-edge-ai-mediapipe

The MediaPipe engine

Rules

1. Depend on flutter_edge_ai and flutter_edge_ai_mediapipe, and import both. The engine package does not re-export core.
2. Declare fileType: ModelFileType.task for .task files and ModelFileType.binary for .bin files.
3. An app that includes this package needs iOS 16.0.
4. There is no desktop support.
5. maxTokens is the real context limit — small values are not raised as they are on .litertlm. maxOutputTokens is ignored; stop generation with session.stopGeneration().
6. On Android and iOS, createChat inherits the model's audio support but not its image support — pass supportImage: true to the chat as well, or the image is dropped.
7. Upstream MediaPipe LLM Inference is maintenance-only; prefer a .litertlm model (flutter-edge-ai-inference) for new work.
8. On iOS, do not load a .task model with PreferredBackend.gpu: Gemma 3 1B reaches the app's memory limit while loading and iOS kills the app with no crash report. Use .litertlm, or PreferredBackend.cpu.

Setup

flutter pub add flutter_edge_ai flutter_edge_ai_mediapipe

import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_mediapipe/flutter_edge_ai_mediapipe.dart';
 
await FlutterEdgeAi.initialize(inferenceEngines: [MediaPipeEngine()]);
 
await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.task,
).fromNetwork(url).install();
 
final InferenceModel model = await FlutterEdgeAi.getActiveModel(maxTokens: 1024);

Sessions, chats, streaming and the common traps work as in the flutter-edge-ai-inference skill — except that openSession / openChat (its concurrent-conversation pattern) work on Android and iOS only; on web they throw UnsupportedError.

iOS

ios/Podfile, declared once:

platform :ios, '16.0'
use_frameworks! :linkage => :static

With Swift Package Manager, also set iOS Deployment Target to 16.0 on the Runner target in Xcode. This package ships no Swift package manifest, so the app gets an ios/Podfile either way.

Core and the other engines build from iOS 15.0, except flutter_edge_ai_llama, which needs 16.4. If the app does not use .task models, leave this package out and stay on 15.

Large models also need Extended Virtual Addressing, Increased Memory Limit and Increased Debugging Memory Limit, added in Xcode under Signing & Capabilities, or the app is killed for memory.

Run .task models on PreferredBackend.cpu on iOS: on the GPU even Gemma 3 1B is killed for memory while loading, with the entitlements present.

Images and audio

final model = await FlutterEdgeAi.getActiveModel(maxTokens: 4096, supportImage: true);
final chat = await model.createChat(supportImage: true);
await chat.addQueryChunk(
  Message(text: 'Describe this image.', isUser: true, imageBytes: bytes),
);

On Android and iOS, a chat without supportImage: true drops the image and the model answers the text alone. On web the chat follows the model: an image sent to a model loaded without supportImage: true throws ArgumentError.

Audio input works on Android and iOS only with a .task that carries an audio encoder. The Gemma 3n .task files do not, so for audio use a .litertlm model (flutter-edge-ai-inference).

Bounding output

final session = await model.createSession();
await session.addQueryChunk(Message(text: prompt, isUser: true));
final reply = StringBuffer();
var produced = 0;
await for (final token in session.getResponseAsync()) {
  reply.write(token);
  if (++produced >= 200) {
    await session.stopGeneration();
    break;
  }
}
await session.close();

Web

Load a web build of the model — the -web.task file, e.g. gemma3-1b-it-int4-web.task rather than gemma3-1b-it-int4.task. Mobile .task files generally do not run on web, so install a separate URL there.

Add to web/index.html <head>, before Flutter boots:

<script type="module">
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.29';
window.FilesetResolver = FilesetResolver;
window.LlmInference = LlmInference;
</script>
<script src="cache_api.js"></script>
<script src="opfs_helper.js"></script>

Pin the version — an unpinned import takes whatever was published last. Copy cache_api.js and opfs_helper.js from the flutter_edge_ai package's web/ directory into the app's web/; find it with grep -A1 '"name": "flutter_edge_ai"' .dart_tool/package_config.json.

Web is GPU-only. Models over about 2 GB need OPFS streaming storage:

await FlutterEdgeAi.initialize(
  webStorageMode: WebStorageMode.streaming,
  inferenceEngines: [MediaPipeEngine()],
);

Android

Text inference runs on arm64-v8a, x86_64 and armeabi-v7a. The release build needs <uses-permission android:name="android.permission.INTERNET"/> in android/app/src/main/AndroidManifest.xml to download a model — Flutter's template declares it only for debug and profile builds.