tf — Tensor Flow

The tf module is capable of executing Quantized TensorFlow Lite Models on the OpenMV Cam (not supported on the OpenMV Cam M4).

You can read more about how to create your own models that can run on the OpenMV Cam here. In particular:

  • Supported operations are listed here.
  • Convert your model to a FlatBuffer by following the instructions here.
  • Finally, quantize your model by following the instructions here.

Alternatively, just follow Google’s in-depth guide here.

The final output .tflite model can be directly loaded and run by your OpenMV Cam. That said, the model and the model’s required sratch RAM must fit within the available frame buffer stack RAM on your OpenMV Cam.

  • The OpenMV Cam M7 has about 384KB of frame buffer RAM. Please try to keep your model and it’s required scratch buffer under 320 KB.
  • The OpenMV Cam H7 has about 496KB of frame buffer RAM. Please try to keep your model and it’s required scratch buffer under 400 KB.
  • The OpenMV Cam H7 Plus has about 31MB of frame buffer RAM. That said, running a model anywhere near the that size will be extremely slow.

Alternatively, you can also load a model onto the MicroPython Heap. However, this significantly limits the model size on all OpenMV Cams.

Functions

tf.classify(path, img[, roi[, min_scale=1.0[, scale_mul=0.5[, x_overlap=0[, y_overlap=0]]]]])

Executes the TensorFlow Lite image classification model on the img object and returns a list of tf_classification objects. This method executes the network multiple times on the image in a controllable sliding window type manner (by default the algorithm only executes the network once on the whole image frame).

path a path to a .tflite model to execute on your OpenMV Cam’s disk. The model is loaded into memory, executed, and released all in one function call to save from having to load the model in the MicroPython heap. Pass "person_detection" to load the built-in person detection model from your OpenMV Cam’s internal flash.

roi is the region-of-interest rectangle tuple (x, y, w, h). If not specified, it is equal to the image rectangle. Only pixels within the roi are operated on.

min_scale controls how much scaling is applied to the network. At the default value the network is not scaled. However, a value of 0.5 would allow for detecting objects 50% in size of the image roi size…

scale_mul controls how many different scales are tested out. The sliding window method works by multiplying a default scale of 1 by scale_mul while the result is over min_scale. The default value of scale_mul, 0.5, tests out a 50% size reduction per scale change. However, a value of 0.95 would only be a 5% size reductioin.

x_overlap controls the percentage of overlap with the next detector area of the sliding window. A value of zero means no overlap. A value of 0.95 would mean 95% overlap.

y_overlap controls the percentage of overlap with the next detector area of the sliding window. A value of zero means no overlap. A value of 0.95 would mean 95% overlap.

tf.segment(path, img[, roi])

Executes the TensorFlow Lite image segmentation model on the img object and returns a list of grayscale image objects for each segmentation class output channel.

path a path to a .tflite model to execute on your OpenMV Cam’s disk. The model is loaded into memory, executed, and released all in one function call to save from having to load the model in the MicroPython heap.

roi is the region-of-interest rectangle tuple (x, y, w, h). If not specified, it is equal to the image rectangle. Only pixels within the roi are operated on.

class tf_classification – tf classification dection result

The tf_classification object is returned by tf.classify() or tf_model.classify().

Constructors

class tf.tf_classification

Please call tf.classify() or tf_model.classify() to create this object.

Methods

tf_classification.rect()

Returns a rectangle tuple (x, y, w, h) for use with image methods like image.draw_rectangle() of the tf_classification’s bounding box.

tf_classification.x()

Returns the tf_classification’s bounding box x coordinate (int).

You may also get this value doing [0] on the object.

tf_classification.y()

Returns the tf_classification’s bounding box y coordinate (int).

You may also get this value doing [1] on the object.

tf_classification.w()

Returns the tf_classification’s bounding box w coordinate (int).

You may also get this value doing [2] on the object.

tf_classification.h()

Returns the tf_classification’s bounding box h coordinate (int).

You may also get this value doing [3] on the object.

tf_classification.classification_output()

Returns a list of the classification label scores. The size of this list is determined by your model output channel size. For example, mobilenet outputs a list of 1000 classification scores for all 1000 classes understood by mobilenet. Use zip in python to combine the classification score results with classification labels.

You may also get this value doing [4] on the object.

class tf_model – TensorFlow Model

If your model size is small enough and you have enough heap space you may wish to directly load the model into memory to save from having to load it from disk each time you wish to execute it.

Constructors

class tf.tf_model

Please call tf.load(path) to create this object. path a path to a .tflite model to load into memory. NOTE! This method stores the model on your micropython heap which is only ~50 KB on the OpenMV Cam M7 and ~256 KB on the OpenMV Cam H7. Pass "person_detection" to load the built-in person detection model from your OpenMV Cam’s internal flash. This built-in model does not use any Micropython Heap as all the weights are stored in flash which is accessible in the same way as RAM.

Returns a tf_model object which can operate on an image.

Methods

tf_model.len()

Returns the size in bytes of the tf_model.

tf_model.height()

Returns the input height of the model. You can use this to size your input image height appropriately.

tf_model.width()

Returns the input width of the model. You can use this to size your input image width appropriately.

tf_model.channels()

Returns the number of color channels in the model. 1 for grayscale and 3 for RGB.

tf_model.classify(img[, roi[, min_scale=1.0[, scale_mul=0.5[, x_overlap=0[, y_overlap=0]]]]])

Executes the TensorFlow Lite image classification model on the img object and returns a list of tf_classification objects. This method executes the network multiple times on the image in a controllable sliding window type manner (by default the algorithm only executes the network once on the whole image frame).

roi is the region-of-interest rectangle tuple (x, y, w, h). If not specified, it is equal to the image rectangle. Only pixels within the roi are operated on.

min_scale controls how much scaling is applied to the network. At the default value the network is not scaled. However, a value of 0.5 would allow for detecting objects 50% in size of the image roi size…

scale_mul controls how many different scales are tested out. The sliding window method works by multiplying a default scale of 1 by scale_mul while the result is over min_scale. The default value of scale_mul, 0.5, tests out a 50% size reduction per scale change. However, a value of 0.95 would only be a 5% size reductioin.

x_overlap controls the percentage of overlap with the next detector area of the sliding window. A value of zero means no overlap. A value of 0.95 would mean 95% overlap.

y_overlap controls the percentage of overlap with the next detector area of the sliding window. A value of zero means no overlap. A value of 0.95 would mean 95% overlap.

tf_model.segment(img[, roi])

Executes the TensorFlow Lite image segmentation model on the img object and returns a list of grayscale image objects for each segmentation class output channel.

roi is the region-of-interest rectangle tuple (x, y, w, h). If not specified, it is equal to the image rectangle. Only pixels within the roi are operated on.