denisovav/flutter_edge_ai/flutter-edge-ai-builtin-ai

The built-in OS model

Rules

1. The OS owns the weights, but the model is still installed — as an identity: fileType: ModelFileType.builtIn with .fromBundled(...). The app downloads nothing.
2. Call BuiltInAi.ensureReady() before getActiveModel(), from a user action: the first call downloads the model and can take minutes, and on web the browser refuses to start that download without a user gesture. Call it straight from the tap handler, with no slow await in front of it. It throws TimeoutException after timeout — 10 minutes by default.
3. On web, ensureReady called outside a user gesture throws a BuiltInAiUnavailableException whose status is BuiltInAiAvailability.downloadable — in the log, LocalAiUserActivationRequiredException: Chrome only starts the Gemini Nano download inside a user gesture…, a subclass. The browser can run the model; it is waiting for a tap. Check e.status == BuiltInAiAvailability.downloadable first, ask the user to tap, and call ensureReady again from that tap handler. Do not fall back to a download here — the fallback below would fetch gigabytes the browser was about to provide.
4. Catch BuiltInAiUnavailableException with any other status and fall back to a downloadable model.
5. Android apps need minSdk 26, or the manifest merge fails, and Kotlin 2.3.21. The native layer, flutter_local_ai, no longer applies the Kotlin Gradle Plugin itself (since 0.2.0), so it does not block built-in Kotlin on AGP 9; other plugins in the app still may.
6. Linux has no built-in model: BuiltInAiModels.forCurrentPlatform is null there and BuiltInAi.availability() reports BuiltInAiAvailability.unavailableDeviceUnsupported. Windows runs Phi Silica through Windows AI Foundry on Copilot+ hardware, in a packaged app; a Windows build that could not resolve the Windows App SDK reports the same unavailable state, so the fallback below covers it.

Setup with a fallback

flutter pub add flutter_edge_ai flutter_edge_ai_builtin_ai flutter_edge_ai_litertlm

import 'dart:async';
 
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_builtin_ai/flutter_edge_ai_builtin_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
 
await FlutterEdgeAi.initialize(
  inferenceEngines: [BuiltInAiEngine(), LiteRtLmEngine()],
);
 
final spec = BuiltInAiModels.forCurrentPlatform; // null on Linux
 
Future<InferenceModel> downloadGemma() async {
  await FlutterEdgeAi.installModel(
    modelType: ModelType.gemma4,
    fileType: ModelFileType.litertlm,
  ).fromNetwork(
    // 2.6 GB — ask first. On web: gemma-4-E2B-it-web.litertlm
    'https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm',
  ).install();
  return FlutterEdgeAi.getActiveModel(maxTokens: 1024);
}
 
InferenceModel model;
if (spec == null) {
  model = await downloadGemma();
} else {
  try {
    // The first await in the tap handler: on web the browser starts the
    // download only inside a user gesture, so nothing slow may run before it.
    // onProgress reports real percentages on Android and the web; Apple
    // downloads nothing and reports none — ensureReady just waits.
    await BuiltInAi.ensureReady(onProgress: (int percent) => print('$percent%'));
    await FlutterEdgeAi.installModel(
      modelType: ModelType.general,
      fileType: ModelFileType.builtIn,
    ).fromBundled(spec.name).install();
    model = await FlutterEdgeAi.getActiveModel(maxTokens: 4096);
  } on BuiltInAiUnavailableException catch (e) {
    if (e.status == BuiltInAiAvailability.downloadable) {
      // Web, called outside a user gesture. The browser can run the model:
      // show an "Enable AI" button and call ensureReady from its handler.
      return;
    }
    model = await downloadGemma();
  } on TimeoutException {
    model = await downloadGemma();
  }
}

The latest install is the one getActiveModel loads, so the fallback replaces the built-in model. Sessions and chats then work as in the flutter-edge-ai-inference skill.

Checking before offering the feature

final availability = await BuiltInAi.availability();
final usable = availability == BuiltInAiAvailability.available ||
    availability == BuiltInAiAvailability.downloadable ||
    availability == BuiltInAiAvailability.downloading;

downloadable and downloading mean "not yet" — ensureReady finishes the job. The unavailable* states describe the device now, not forever:

Platforms

Images work on Android, one per message, and only when asked for: pass supportImage: true to getActiveModel (getActiveModel(maxTokens: 4096, supportImage: true)); chats from createChat and sessions inherit it. An image sent to a model loaded without it throws UnsupportedError. Windows and web are text-only, and Apple is text-only on OS 26: supportImage: true there fails when getActiveModel creates the model, not mid-conversation — so do not build an image path there.

Web

There is no script to add: the Prompt API is part of the browser.

• Chrome — since Chrome 148 the Prompt API is available to ordinary web pages: no origin-trial token and no chrome://flags entry.
• Microsoft Edge — enable edge://flags → "Prompt API for on-device language model" and restart. Both browsers use BuiltInAiModels.chromePromptApi, which is what forCurrentPlatform returns on web: the spec names the API, and the browser picks the model.
• Sampling — on web, topP and maxOutputTokens are ignored (warned once); on web pages Chrome ignores temperature and topK too (they work in Chrome Extensions only) and logs a deprecation warning, so a topK: 1 there is not greedy.

Trade-offs

No choice of weights, no LoRA, and capabilities that vary by OS version. The right pick when zero download and zero disk matter more than choosing the model.

Tool calls go through the prompt on every platform. Apple's native tool calling and schema-constrained output are reachable only through BuiltInAiModel.localAiModel / BuiltInAiSession.localAiSession — @experimental, and they change with flutter_local_ai's next major release.