Discover TensorFlow Lite for Flutter: A Comprehensive Guide to Image and Object Detection

Published on by Flutter News Hub

Discover TensorFlow Lite for Flutter: A Comprehensive Guide to Image and Object Detection

TensorFlow Lite, an optimized mobile-first version of TensorFlow, seamlessly integrates with Flutter, empowering you to harness the power of machine learning in your mobile applications. This guide will walk you through the setup, usage, and implementation of TensorFlow Lite with Flutter for image and object detection tasks.

Installation

  1. Add tflite as a dependency in your pubspec.yaml file:
dependencies:
  tflite: ^latest_version

Setup

Android

In android/app/build.gradle, add the following setting in the android block:

aaptOptions {
  noCompress 'tflite'
  noCompress 'lite'
}

iOS

  • If you encounter the error 'vector' file not found, change the Compile Sources As setting to Objective-C++ in Xcode.
  • If the header path error 'tensorflow/lite/kernels/register.h' file not found occurs, set CONTRIB_PATH to true in TflitePlugin.mm.

Usage

Load the model and labels:

String res = await Tflite.loadModel(
    model: "assets/mobilenet_v1_1.0_224.tflite",
    labels: "assets/labels.txt",
    numThreads: 1,
    isAsset: true,
    useGpuDelegate: false);

Image Classification

Output format:

{
  index: 0,
  label: "person",
  confidence: 0.629
}

Run on image:

var recognitions = await Tflite.runModelOnImage(path: filepath);

Run on binary:

var recognitions = await Tflite.runModelOnBinary(binary: imageToByteListFloat32(image, 224, 127.5, 127.5));

Object Detection

SSD MobileNet

Output format:

{
  detectedClass: "hot dog",
  confidenceInClass: 0.123,
  rect: {
    x: 0.15,
    y: 0.33,
    w: 0.80,
    h: 0.27
  }
}

Run on image:

var recognitions = await Tflite.detectObjectOnImage(path: filepath, model: "SSDMobileNet");

Run on binary:

var recognitions = await Tflite.detectObjectOnBinary(
    binary: imageToByteListUint8(resizedImage, 300),
    model: "SSDMobileNet");

Tiny YOLOv2
Run on image:

var recognitions = await Tflite.detectObjectOnImage(
    path: filepath,
    model: "YOLO",
    anchors: anchors,
    blockSize: 32,
    numBoxesPerBlock: 5);

Run on binary:

var recognitions = await Tflite.detectObjectOnBinary(
    binary: imageToByteListFloat32(resizedImage, 416, 0.0, 255.0),
    model: "YOLO",
    anchors: anchors,
    blockSize: 32,
    numBoxesPerBlock: 5);

Example: Real-Time Detection

For an in-depth example of real-time object detection, refer to flutter_realtime_detection.

Testing

To run test cases:

flutter test test/tflite_test.dart

Additional Resources

Flutter News Hub