google-ai-edge/litert-samples/litert-compiled-model-migration

Skill: LiteRT Compiled Model Migration SKILL

Description

This skill guides an AI agent to rapidly migrate an Android application from legacy TensorFlow Lite (TFLite) to the modern LiteRT CompiledModel API (2.x) in Open Source GitHub repositories. It prioritizes a high-speed, 1st-pass "Like for Like" baseline migration with automated self-testing, and encourages advanced performance upgrades including off-main-thread execution with buffer reuse and NPU JIT compilation.

Read first: what the Kotlin CompiledModel API (com.google.ai.edge.litert, 2.3.0; 2.2.0 has the same API) has

• CompiledModel.create(context.assets, assetName, CompiledModel.Options(Accelerator.CPU), env) and create(filePath, options, env); Options takes one or more of Accelerator.CPU / GPU / NPU (e.g. Options(Accelerator.GPU, Accelerator.CPU)); env is Environment.create(context) (optional; enables the compiler cache) or Environment.create(context, BuiltinNpuAcceleratorProvider(context)) for the NPU. Nothing takes a ByteBuffer or a mapped file.
• Buffers come from the model (createInputBuffers() / createOutputBuffers(), List<TensorBuffer>; writeFloat(FloatArray) / writeInt8(ByteArray) / writeInt / writeLong, readFloat() and friends; create them once). run(inputs, outputs) is synchronous on the CPU; on the GPU it can return before the GPU finishes and read*() waits; there is no callback API. CompiledModel, TensorBuffer and Environment are AutoCloseable: close buffers, then the model, then an Environment the app created.
• CompiledModel.Options() with no accelerator compiles, but create then throws LiteRtException. Interpreter, GpuDelegate, NnApiDelegate and the Support library's Model (com.google.ai.edge.litert.support.model.Model, which runs on InterpreterApi) do not appear in migrated code; the rest of LiteRT Support does, under com.google.ai.edge.litert.support (Step 1), with org.tensorflow.lite.DataType as the type its classes take. The litert AAR also contains org.tensorflow.lite.Interpreter, so leftover Interpreter code compiles (all but setUseNNAPI, which 2.x does not have) and the Stage 1 import check is what catches it. templates/LiteRtModel.kt compiles against the 2.3.0 and the 2.2.0 artifacts; start from it.

0. Automatic Discovery & Upfront Planning

Before writing code, the agent MUST inspect the project workspace and present the upfront planning interview to align migration parameters:

A. Automatic Workspace & Toolchain Discovery

The agent must automatically inspect the repository structure:

1. Ecosystem & Build Engine:

• Gradle Build System: Detected by build.gradle, build.gradle.kts, or settings.gradle. -> Enable Gradle & GitHub PR Workflow.

1. Language & JNI Toolchain:

• Native C++ / NDK: Detected if CMakeLists.txt, Android.mk, or *.cpp files exist. -> Enable C++ / JNI Migration Rules.
• Pure Kotlin / Java: Default to JVM / Android SDK Migration Rules.

B. Upfront User Interview (Questions Asked Prior to Migration)

The agent must present the following review options to the user:

Before initiating the LiteRT Compiled Model Migration, please confirm your project preferences:
 
1. Model Workload & Domain:
   What type of data does this application process?
   - [A] Vision (Images / Video / Camera Feeds) -> Enables the camera-frame recipe: preprocess into a reused input buffer via writeFloat / writeInt8 (AHardwareBuffer interop is C++-only).
   - [B] Audio (Speech / Sound Classification) -> Enables the streaming FloatArray recipe (writeFloat per window).
   - [C] Text / NLP / GenAI -> Enables the token-id recipe (writeInt / writeLong into the model's input buffers).
 
2. Hardware Acceleration & Conditional INT8 Quantization:
   Do you want to enable NPU hardware acceleration via JIT on-device compilation?
   - [A] Yes (Recommended - replaces deprecated NNAPI; needs the vendor libraries of Upgrade 2.C on a supported SoC) [Default]
         * If the app uses a Float32 model: Would you like to generate an INT8 integer-quantized model via AI Edge Quantizer for peak NPU speed, or run the original Float32 model?
           -> Option A.1: Convert to INT8 (Generates model_int8.tflite with int8 weights and activations; needs a few representative inputs for calibration) [Default]
           -> Option A.2: Keep Float32 (Runs baseline float model directly on NPU)
   - [B] No (GPU and CPU acceleration only)
 
3. Encouraged Performance Upgrades:
   Should the agent upgrade the calling code to use LiteRT's advanced features?
   - [A] Yes (Move inference off the main thread and reuse buffers) [Default]
   - [B] No (Keep strict 1-to-1 synchronous baseline execution)
 
4. Automated Pull Request Provisioning:
   Should the agent automatically stage, commit, and create a GitHub PR when self-testing passes?
   - [A] Yes [Default] (Attaches verification test logs and before/after summary diff)
   - [B] No (Keep changes local in current working branch)

With no user to answer (CI, batch runs): take Q1 from the code, Q2 [A] only when the vendor libraries of Upgrade 2.C are already in the repo (otherwise [B]), Float32 unless calibration inputs exist, and the defaults of Q3 and Q4 with the PR opened as a draft; record the choices in the PR description. Today the runtime is the standalone com.google.ai.edge.litert:litert AAR: no LiteRT artifact for Google Play services is on Google Maven as of October 2026 (only the play-services-tflite-* Interpreter artifacts exist).

[!IMPORTANT]

LiteRT Support (`litert-support` 2.3.0): an app on the TensorFlow Lite Support Library keeps it. The library now ships as LiteRT Support, with the same classes, methods and behavior; the migration is two steps of Step 1, swap the Gradle coordinates and rename the package org.tensorflow.lite.support to com.google.ai.edge.litert.support (its guide: https://github.com/google-ai-edge/LiteRT/blob/main/litert/support_lib/MIGRATION_GUIDE.md). ImageProcessor, ResizeOp, NormalizeOp, TensorImage, TensorLabel, FileUtil and MetadataExtractor stay as they are. The one class replaced is its Model, which runs on the Interpreter API: it becomes CompiledModel like Interpreter (Step 4), with templates/SupportBridge.kt between the Support tensors and the model's buffers. An app that never used the Support library does not add it; Bitmap.toModelInput in templates/LiteRtModel.kt is its preprocessing.

1. Phase 1: "Like for Like" Baseline Migration (1st Pass Success)

Phase 1 prioritizes functional equivalence, fast compilation, and immediate 1st pass self-test success.

Step 0: Capture the Golden Output

Before Step 1 changes any dependency, with the legacy runtime and preprocessing still in the project (no device attached: skip Step 0, stop after Stage 1, and open the PR as a draft that says so):

• Define one fixed input: for a vision app a deterministic bitmap (fixedBitmap in templates/GoldenCaptureTest.kt, or a small image file in app/src/androidTest/assets/), otherwise a generated array; never a camera frame. A bitmap of the model's input size with rotation 0 leaves the resize and the rotation out of the gate: use another size (and aspect ratio), and the rotation the camera path passes, in the capture and in Stage 2 alike.
• Declare the androidx.test dependencies and testInstrumentationRunner (Stage 2 needs them too), copy templates/GoldenCaptureTest.kt into app/src/androidTest/java/<package>/, make it call the app's existing inference entry point (the class the UI calls, with its own preprocessing) on the fixed input on the CPU (no GPU or NNAPI delegate, XNNPACK at its default); when that entry point returns only a thresholded or sorted list, first give the class a function returning the raw output and one returning the preprocessed input (preprocess(bitmap): Stage 2 calls it too), still on the legacy runtime. Then run the four commands in its header: they install both APKs, run the test on the device, and pull golden.txt into app/src/androidTest/assets/golden/output.txt. A raw tensor fed straight to the model is not a golden output: it leaves the preprocessing out of the gate.
• Delete the capture test. Stage 2 compares against the file, so the test APK never needs org.tensorflow:tensorflow-lite.

Step 1: Clean & Modernize Dependencies