3. OpenMV IDE Overview

Let’s talk about OpenMV IDE now. OpenMV IDE is the tool you’ll be using to program your OpenMV Cam with. It features a powerful text editor powered by QtCreator, a frame buffer viewer, histogram display, and an integrated serial terminal for debug output from your OpenMV Cam.

Anyway, the IDE is more or less straight forward to use. A lot of effort was put in to stream-line things in the IDE. However, there are a few things you should know about in the IDE.

3.1. File Menu and Examples

Under the file menu in OpenMV IDE you have all your standard text editor options. New, Open, Save, Save As, Print, etc. Anyway, when you create a new file or open one it will appear in the text editor pane. Please note that you can have multiple files open which you can select between by clicking on the combo box that displays the name of the current file. OpenMV IDE uses QtCreator as the text editor backend and actually supports opening multiple files in multiple windows with horizontal and vertical splitting capabilities. However, all of that was hidden to make it obvious which script will get run when you click the run button like in the Arduino IDE. If you want to edit scripts in a different editor or you need to open more than one file at a time just open the files in the editor of your choice and work on them. OpenMV IDE will automatically detect the files have changed and ask you if you want to load the changes when you click on the OpenMV IDE window again.

Note

Only one instance of OpenMV IDE is allowed one your computer at a time. This is done so that if you try to open a file via your file explorer the one instance of OpenMV IDE will open the file. This also allows you to open files using drag-and-drop.

Moving on, under the file menu you’ll find the examples menu which has tons of scripts showing off how to use different features of the OpenMV Cam. As we implement more features more examples will show up. Make sure to familiarize yourself with the example scripts so you can use your OpenMV Cam effectively.

Lastly, whenever you create a new file in OpenMV IDE or open an example script OpenMV IDE creates a memory only copy of the file that is not stored on disk like in the Arduino IDE. This let’s you quickly test out ideas without having to create a file. If you want to save the file you need to use Save As to save the file to disk. Then the regular file Save option will work. The QtCreator text editor backend only allows use of the Save command for files that exist on disk. Note that if you forget to Save As a new file or an edited example file OpenMV IDE will ask if you want to save the file(s) on close.

3.2. Text Editing

OpenMV IDE has a professional text editor powered by the QtCreator backend. You’ve got infinite undo and redo across all open files, white-space visualization (important for MicroPython), font-size controls, and best-of-all QtCreator’s find and replace. The find and replace features regular expression matching with the ability to capture text during the find and use that captured text for the replace. Additionally, it also can preserve case while replacing. Finally, the find and replace features works not only on the current file, but, it can work on all files in a folder or all open files in OpenMV IDE.

3.3. Connecting to your OpenMV Cam

Connecting to your OpenMV Cam was covered in detail in the previous Hardware Setup tutorial page. Once you’ve gotten past all the startup issues that involve programming firmware and etc. you just have to click the connect button in the bottom left-hand corner of OpenMV IDE to connect to your OpenMV Cam.

OpenMV IDE is smart about connecting and automatically filters our all serial ports that aren’t an OpenMV Cam. If there’s only one OpenMV Cam attached to your computer it will find it and connect immediately. If you have two OpenMV Cams it will ask you which serial port to connect to then. Note that OpenMV IDE will remember your pick so that next time if you connect the serial port for the OpenMV Cam you want to connect to will already be selected.

After connecting to your OpenMV Cam’s serial port OpenMV IDE will then try to determine the USB flash drive on your computer associated with your OpenMV Cam. OpenMV IDE will do some smart filtering on USB flash drives to automatically connect to the right one if possible. However, it may not be able to determine the correct one automatically and will ask you to help it if it can’t. Like with serial ports above OpenMV IDE will remember your selection so the next time you connect it will automatically highlight your previous choice.

Note

OpenMV IDE needs to associate the USB flash drive on your OpenMV Cam with your OpenMV Cam’s virtual serial port so that it can parse compile error messages and open the file with the error easily. It also needs the drive information for the Save open script to OpenMV Cam command. That all said, OpenMV IDE can still function without knowing your OpenMV Cam’s USB flash drive and will just disable features that depend on this knowledge.

Finally, after connecting to your OpenMV Cam the connect button will be replaced with the disconnect button. Click the disconnect button to disconnect from your OpenMV Cam. Note that disconnecting will stop whatever script is currently executing on your OpenMV Cam. You can also just unplug your OpenMV Cam from your computer without disconnecting and OpenMV IDE will detect that and automatically disconnect from your OpenMV Cam. If your OpenMV Cam crashes OpenMV IDE will detect this too and disconnect from your OpenMV Cam.

Note

It is possible on Windows/Mac/Linux for the serial port driver to crash in a terrible way if your OpenMV Cam becomes unresponsive. On Windows you’ll notice this as an unkillable OpenMV IDE process without any window in the Task Manager. If this happens the only recourse is to restart your computer as the unkillable process will prevent any new instance of OpenMV IDE from connecting to any OpenMV Cam. This issues exists as a won’t fix bug in Windows that’s been around since Windows XP. On Mac/Linux this same issue can occur but is much harder to run into. Basically what happens on Windows at least is that when the USB virtual serial port driver crashes from an OpenMV Cam becoming unresponsive it never let’s OpenMV IDE’s serial thread return from Kernel function call which makes OpenMV IDE’s serial thread unkillable. Anyway, we’ve done a lot of work to make sure this doesn’t happen but please note that it still can.

3.4. Running scripts

Once you’ve finished editing your code and are ready to run a script just click the green run button in the bottom left-hand corner of OpenMV IDE. The script will then be sent to your OpenMV Cam to be compiled into Python byte-code and executed by your OpenMV Cam.

If there were any errors in your script your OpenMV Cam will send back the compile error in the Serial Terminal which OpenMV IDE automatically parses looking for errors. When OpenMV IDE detects and error it will automatically open the file in error and highlight the line in error along with displaying a nice error message box. This feature saves you tons of time fixing errors.

Anyway, if you want to stop the script just click on the stop button which will replace the run button while a script is running. Note that scripts can automatically stop by themselves either due to finishing or a compile error. In either case the run button will re-appear again.

3.5. Frame Buffer Viewer

What makes OpenMV IDE special is the integrated frame buffer viewer. This let’s you easily see what your OpenMV Cam is looking at while working on your code.

The frame buffer viewer displays whatever was in your OpenMV Cam’s frame buffer previously when sensor.snapshot() is called. Anyway, we’ll talk more about that later. For now, here’s what you need to know about the frame buffer viewer:

  • The Zoom button on the top right-hand corner of OpenMV IDE controls the zoom-to-fit feature for the frame buffer viewer. Enable or disable the feature as your please.
  • The Disable button on the top right-hand corner of OpenMV IDE controls whether or not your OpenMV Cam will send images to OpenMV IDE. Basically, your OpenMV Cam has to JPEG compress images constantly to stream them to OpenMV IDE. This reduces performance. So, if you want to see how fast your script will run after in the wild without your OpenMV Cam being connected to your computer just click the Disable button and then restart your script. While the frame buffer is disabled you won’t be able to see what your OpenMV Cam is looking at anymore but you’ll still see debug output from your OpenMV Cam in the Serial Terminal.

Finally, you can right click on whatever image you see in the frame buffer viewer to save that image to disk. Additionally, if you select an area in the frame buffer by clicking and dragging you can save just that area to disk instead. Note that you should stop the script before trying to save the frame buffer to disk. Otherwise, you may not get the exact frame you want.

To de-select an area in the frame buffer just click anywhere without dragging to remove the selection. However, it’s possible to create one pixel selections when de-selecting so try to click on the blank area in the frame buffer.

3.6. Histogram Display

The integrated histogram display in OpenMV IDE exists primarily to fill the empty space under the frame buffer viewer and provide some eye candy for you. However, it also actually useful for getting feedback about the lighting quality in the room, determining color tracking settings, and in general just giving you an idea about the quality of the image your OpenMV Cam is looking at.

You can select between four different color spaces in the histogram. Either RGB, Grayscale, LAB, and YUV. Only Grayscale and LAB are useful for use with programmatically controlling your OpenMV Cam. RGB is nice for looking at as eye candy and YUV is there because we use it for JPEG compression and thought we might as well add it.

Anyway, by default the histogram shows information about the whole image. But, if you select an area of the frame buffer by clicking and dragging on it then the histogram will only show the distribution of colors in that area. This features makes the histogram display super useful for determining the correct Grayscale and LAB color channel settings you need to use in your scripts for image.find_blobs and image.binary.

3.7. Serial Terminal

To show the Serial Terminal click on the Serial Terminal button located on the bottom of OpenMV IDE. The Serial Terminal is built-into the main window to be easier to use. It just splits with your text editing window.

Anyway, all debug text from your OpenMV Cam created by print will be displayed in the Serial Terminal. Other than that there’s not much else to say.

Note that the serial terminal more or less will infinitely buffer text. It will keep the last one million lines of text in RAM. So, you can use it to buffer a large amount of debug output. Additionally, if you press ctrl+f in Windows/Linux or the equivalent shortcut on Mac you’ll be able to search through the debug output. Finally, the Serial Terminal is smart enough not to auto-scroll on you if you want to look at previous debug output making it really nice to use. Auto-scrolling will turn on again if you scroll to the bottom of the text output.

3.8. Status Bar

On the status bar OpenMV IDE will display your OpenMV Cam’s Firmware Version, Serial Port, Drive, and FPS. The Firmware Version label is actually a button you can click to update your OpenMV Cam if your OpenMV Cam’s firmware is out of date. The Serial Port label just displays your OpenMV Cam’s serial port and nothing else. The Drive label is another button which you can click on to change the drive linked to your OpenMV Cam. Finally, the FPS label displays the FPS OpenMV IDE is getting from your OpenMV Cam.

Note

The FPS displayed by OpenMV IDE may not match the FPS from your OpenMV Cam. The FPS label on OpenMV IDE is the FPS OpenMV IDE is getting from your OpenMV Cam. But, your OpenMV Cam can actually run faster than OpenMV IDE sometimes and OpenMV IDE is only sampling some of the frames from your OpenMV Cam and not all. Anyway, OpenMV IDE’s FPS will never be faster than your OpenMV Cam’s FPS, but, if may be slower.

3.9. Tools

You’ll find useful tools for your OpenMV Cam under the Tools Menu in OpenMV IDE. In particular, the Save open script to your OpenMV Cam and Reset OpenMV Cam tools are useful for using your OpenMV Cam when developing an application.

  • The Save open script to your OpenMV Cam command saves whatever script you’re currently looking at to your OpenMV Cam. Additionally, it can also automatically strip white-space and comments from your script so it takes up less space. Use this command once you think your program is ready to deploy without OpenMV IDE. Note that this command will save your script as main.py on your OpenMV Cam’s USB flash drive. main.py is the script your OpenMV Cam will run once it finishes booting up.
  • The Reset OpenMV Cam` command resets and then disconnects from your OpenMV Cam. You’ll want to use this option if you run a script that creates files on your OpenMV Cam as Windows/Mac/Linux won’t show you any files on your OpenMV Cam created programmatically in a Python Script after your OpenMV Cam’s USB flash drive is mounted.

Next, under the Tools Menu you can invoke the boot-loader to re-program your OpenMV Cam. The boot-loader can only be invoked while OpenMV IDE is disconnected for your OpenMV Cam. You can either give it a binary .bin file to re-program your OpenMV Cam with or a .dfu file. The boot-loader feature is only for advanced users who plan on changing the default OpenMV Cam firmware.

3.10. Options

Under the Tools Menu (or About Menu for Mac) you can access OpenMV IDE’s option dialog.

Note

At the time of writing this OpenMV IDE is not complete yet. In particular, the options menu has not been cleaned up.

You can configure the editor font, size, zoom, tab behavior, automatic cleaning of white space, column margins for folks who like keeping lines to 80 characters, and more.