4. OpenMV IDE Overview¶
Let’s talk about OpenMV IDE now. OpenMV IDE is the tool you’ll be using to program your OpenMV Cam with. It features a powerful text editor powered by QtCreator, a frame buffer viewer, histogram display, and an integrated serial terminal for debug output from your OpenMV Cam.
Anyway, the IDE is more or less straight forward to use. A lot of effort was put in to stream-line things in the IDE. However, there are a few things you should know about in the IDE.
4.2. Text Editing¶
OpenMV IDE has a professional text editor powered by the QtCreator backend. You’ve got infinite undo and redo across all open files, white-space visualization (important for MicroPython), font-size controls, and best-of-all QtCreator’s find and replace. The find and replace features regular expression matching with the ability to capture text during the find and use that captured text for the replace. Additionally, it also can preserve case while replacing. Finally, the find and replace features works not only on the current file, but, it can work on all files in a folder or all open files in OpenMV IDE.
In addition to the nice text editing enviornment OpenMV IDE also provides
auto-completion support hover tool-tips on keywords. So, after typing
example in Python OpenMV IDE will detect that you’re about to write a function
or method name and it will show you an auto-completion text box. Once you’ve
written the function/method name it will also walk you through the arguments.
Finally, if you hover your mouse cursor over any keyword OpenMV IDE will
display the documentation for that keyword in a tooltip. These two features
make editing scripts a joy!
4.3. Connecting to your OpenMV Cam¶
Connecting to your OpenMV Cam was covered in detail in the previous
Hardware Setup tutorial page. Once you’ve gotten past all the startup
issues that involve programming firmware and etc. you just have to click
the connect button in the bottom left-hand corner of OpenMV IDE to connect
to your OpenMV Cam.
OpenMV IDE is smart about connecting and automatically filters our all serial ports that aren’t an OpenMV Cam. If there’s only one OpenMV Cam attached to your computer it will find it and connect immediately. If you have two OpenMV Cams it will ask you which serial port to connect to then. Note that OpenMV IDE will remember your pick so that next time if you connect the serial port for the OpenMV Cam you want to connect to will already be selected.
After connecting to your OpenMV Cam’s serial port OpenMV IDE will then try to determine the USB flash drive on your computer associated with your OpenMV Cam. OpenMV IDE will do some smart filtering on USB flash drives to automatically connect to the right one if possible. However, it may not be able to determine the correct one automatically and will ask you to help it if it can’t. Like with serial ports above OpenMV IDE will remember your selection so the next time you connect it will automatically highlight your previous choice.
OpenMV IDE needs to associate the USB flash drive on your OpenMV Cam with
your OpenMV Cam’s virtual serial port so that it can parse compile error
messages and open the file with the error easily. It also needs the drive
information for the
Save open script to OpenMV Cam command. That all
said, OpenMV IDE can still function without knowing your OpenMV Cam’s USB
flash drive and will just disable features that depend on this knowledge.
Finally, after connecting to your OpenMV Cam the connect button will be replaced with the disconnect button. Click the disconnect button to disconnect from your OpenMV Cam. Note that disconnecting will stop whatever script is currently executing on your OpenMV Cam. You can also just unplug your OpenMV Cam from your computer without disconnecting and OpenMV IDE will detect that and automatically disconnect from your OpenMV Cam. If your OpenMV Cam crashes OpenMV IDE will detect this too and disconnect from your OpenMV Cam.
It is possible on Windows/Mac/Linux for the serial port driver to crash in a terrible way if your OpenMV Cam becomes unresponsive. On Windows you’ll notice this as an unkillable OpenMV IDE process without any window in the Task Manager. If this happens the only recourse is to restart your computer as the unkillable process will prevent any new instance of OpenMV IDE from connecting to any OpenMV Cam. This issues exists as a won’t fix bug in Windows that’s been around since Windows XP. On Mac/Linux this same issue can occur but is much harder to run into. Basically what happens on Windows at-least is that when the USB virtual serial port driver crashes from an OpenMV Cam becoming unresponsive it never let’s OpenMV IDE’s serial thread return from a Kernel function call which makes OpenMV IDE’s serial thread unkillable. Anyway, we’ve done a lot of work to make sure this doesn’t happen but please note that it still can.
4.4. Running scripts¶
Once you’ve finished editing your code and are ready to run a script just click the green run button in the bottom left-hand corner of OpenMV IDE. The script will then be sent to your OpenMV Cam to be compiled into Python byte-code and executed by your OpenMV Cam.
If there were any errors in your script your OpenMV Cam will send back the compile error in the Serial Terminal which OpenMV IDE automatically parses looking for errors. When OpenMV IDE detects and error it will automatically open the file in error and highlight the line in error along with displaying a nice error message box. This feature saves you tons of time fixing errors.
Anyway, if you want to stop the script just click on the stop button which will replace the run button while a script is running. Note that scripts can automatically stop by themselves either due to finishing or a compile error. In either case the run button will re-appear again.
OpenMV IDE automatically scans imports in your script when you click the run button and copies any externally required scripts that are missing to your OpenMV Cam. OpenMV IDE will also automatically update any out-of-date external modules on your OpenMV Cam when you click the run button too. OpenMV IDE looks for external modules in your personal “OpenMV” Documents Folder first and then the example folder. OpenMV IDE is able to parse both single file modules and directory modules.
4.5. Frame Buffer Viewer¶
What makes OpenMV IDE special is the integrated frame buffer viewer. This let’s you easily see what your OpenMV Cam is looking at while working on your code.
The frame buffer viewer displays whatever was in your OpenMV Cam’s frame buffer
sensor.snapshot() is called. Anyway, we’ll talk more about
that later. For now, here’s what you need to know about the frame buffer
Reocordbutton on the top right-hand corner of OpenMV IDE records whatever is in the frame buffer. Use it to quickly make videos of what your OpenMV Cam sees. Note that recording works by recording whatever is in OpenMV IDE’s frame buffer at 30 FPS. However, the frame buffer may be updating faster or slower than this depending on the application. Anyway, after the recording is complete OpenMV IDE will use FFMPEG to transcode the recording to whatever file format you want for sharing.
Zoombutton on the top right-hand corner of OpenMV IDE controls the zoom-to-fit feature for the frame buffer viewer. Enable or disable the feature as your please.
Disablebutton on the top right-hand corner of OpenMV IDE controls whether or not your OpenMV Cam will send images to OpenMV IDE. Basically, your OpenMV Cam has to JPEG compress images constantly to stream them to OpenMV IDE. This reduces performance. So, if you want to see how fast your script will run after in the wild without your OpenMV Cam being connected to your computer just click the
Disablebutton. While the frame buffer is disabled you won’t be able to see what your OpenMV Cam is looking at anymore but you’ll still see debug output from your OpenMV Cam in the Serial Terminal.
Finally, you can right click on whatever image you see in the frame buffer viewer to save that image to disk. Additionally, if you select an area in the frame buffer by clicking and dragging you can save just that area to disk instead. Note that you should stop the script before trying to save the frame buffer to disk. Otherwise, you may not get the exact frame you want.
To de-select an area in the frame buffer just click anywhere without dragging to remove the selection. However, it’s possible to create one pixel selections when de-selecting so try to click on the blank area in the frame buffer.
4.6. Histogram Display¶
The integrated histogram display in OpenMV IDE exists primarily to fill the empty space under the frame buffer viewer and provide some eye candy for you. However, it also actually useful for getting feedback about the lighting quality in the room, determining color tracking settings, and in general just giving you an idea about the quality of the image your OpenMV Cam is looking at.
You can select between four different color spaces in the histogram. Either RGB, Grayscale, LAB, and YUV. Only Grayscale and LAB are useful for use with programmatically controlling your OpenMV Cam. RGB is nice for looking at as eye candy and YUV is there because we use it for JPEG compression and thought we might as well add it.
Anyway, by default the histogram shows information about the whole image. But,
if you select an area of the frame buffer by clicking and dragging on it then
the histogram will only show the distribution of colors in that area. This
features makes the histogram display super useful for determining the correct
Grayscale and LAB color channel settings you need to use in your scripts for
Finally, the image resolution and ROI (x, y, w, h) of the bounding box you select on the image are displayed above the histogram graphs.
4.7. Serial Terminal¶
To show the Serial Terminal click on the Serial Terminal button located on the bottom of OpenMV IDE. The Serial Terminal is built-into the main window to be easier to use. It just splits with your text editing window.
Anyway, all debug text from your OpenMV Cam created by
Note that the serial terminal more or less will infinitely buffer text. It will
keep the last one million lines of text in RAM. So, you can use it to buffer
a large amount of debug output. Additionally, if you press
Windows/Linux or the equivalent shortcut on Mac you’ll be able to search
through the debug output. Finally, the Serial Terminal is smart enough not
to auto-scroll on you if you want to look at previous debug output making it
really nice to use. Auto-scrolling will turn on again if you scroll to the
bottom of the text output.
4.8. Status Bar¶
On the status bar OpenMV IDE will display your OpenMV Cam’s Firmware Version, Serial Port, Drive, and FPS. The Firmware Version label is actually a button you can click to update your OpenMV Cam if your OpenMV Cam’s firmware is out of date. The Serial Port label just displays your OpenMV Cam’s serial port and nothing else. The Drive label is another button which you can click on to change the drive linked to your OpenMV Cam. Finally, the FPS label displays the FPS OpenMV IDE is getting from your OpenMV Cam.
The FPS displayed by OpenMV IDE may not match the FPS from your OpenMV Cam. The FPS label on OpenMV IDE is the FPS OpenMV IDE is getting from your OpenMV Cam. But, your OpenMV Cam can actually run faster than OpenMV IDE sometimes and OpenMV IDE is only sampling some of the frames from your OpenMV Cam and not all. Anyway, OpenMV IDE’s FPS will never be faster than your OpenMV Cam’s FPS, but, it may be slower.
You’ll find useful tools for your OpenMV Cam under the Tools Menu in OpenMV
IDE. In particular, the
Save open script to your OpenMV Cam and
OpenMV Cam tools are useful for using your OpenMV Cam when developing an
Configure OpenMV Cam Settings fileallows you to modify an
.inifile stored on your OpenMV Cam using OpenMV IDE which your OpenMV Cam will read on bootup for particular hardware configurations.
Save open script to your OpenMV Camcommand saves whatever script you’re currently looking at to your OpenMV Cam. Additionally, it can also automatically strip white-space and comments from your script so it takes up less space. Use this command once you think your program is ready to deploy without OpenMV IDE. Note that this command will save your script as
main.pyon your OpenMV Cam’s USB flash drive.
main.pyis the script your OpenMV Cam will run once it finishes booting up.
Reset OpenMV Camcommand resets and then disconnects from your OpenMV Cam. You’ll want to use this option if you run a script that creates files on your OpenMV Cam as Windows/Mac/Linux won’t show you any files on your OpenMV Cam created programmatically in a Python Script after your OpenMV Cam’s USB flash drive is mounted.
Next, under the Tools Menu you can invoke the boot-loader to re-program your
OpenMV Cam. The boot-loader can only be invoked while OpenMV IDE is
disconnected for your OpenMV Cam. You can either give it a binary
to re-program your OpenMV Cam with or a
.dfu file. The boot-loader
feature is only for advanced users who plan on changing the default OpenMV Cam
4.10. WiFi Debug¶
If you’d like to debug you camera over WiFi and have a WINC1500 shield you can use the
Tools->Configure OpenMV Cam Settings file dialog to setup your OpenMV Cam to connect
to the IDE over WiFi (please ensure that your OpenMV Cam can run WiFi examples first and
that your firmware and WiFi module firmware is up to date before enabling WiFi debug).
Once WiFi Debug is enabled your OpenMV Cam will connect to your WiFi network (or create one) on turn on. During this time the status LED onboard will be solid white. After connecting OpenMV IDE will display a WiFi icon on the connect button. When you click connect OpenMV IDE will give you the option to connect to your OpenMV Cam over WiFi.
When connected over WiFi you can do everything but work with the file system on your OpenMV Cam (unless your OpenMV Cam is still connected to your computer over USB).
4.11. Open Terminal¶
The Open Terminal feature allows you to create new serial terminals using OpenMV IDE to remotely debug your OpenMV Cam while it is not connected to your computer. The Open Terminal feature can also be used to program any MicroPython development board.
You can use the Open Terminal feature to open terminals connected to serial ports, tcp ports, or udp ports. Note that serial ports can be bluetooth ports for wireless connectivity.
4.12. Machine Vision¶
The Machine Vision submenu includes a lot of Machine Vision tools for your
OpenMV Cam. For example, the you can use the color threshold editor to get
the best color tracking thresholds for
image.find_blobs(). We’ll make
new machine vision tools available regularly to make life easier for you.
4.13. Video Tools¶
If you need to compress a
.gif file produced by your OpenMV Cam or convert
.mjpeg or ImageIO
.bin video file to
.mp4 you can use the
convert video file action to do this. Alternatively, if you’d just like to play
these videos instead you can do that too using the play video file action.
Note that you should copy video files from your OpenMV Cam’s flash drive to your computer first before playing them since disk I/O over USB on your OpenMV Cam is slow.
Finally, FFMPEG is used to provide conversion and video player support and may be used for any non-OpenMV Cam activites you like. FFMPEG can convert/play a large number of file formats.
- To convert a video file into a set of pictures select the video file as the source and make the target a “%07d.bmp”/”%07d.jpg”/etc. file name. FFMPEG understands
printf()like format statements with an image file format extension to mean it should break the video file up into still images of the target format.
- To convert a series of still images into a video set the source file name to “%7d.bmp”/”%7d.jpg”/etc. where all the images in a directory have a name like
2.bmp, etc. FFMPEG understands
printf()like format statements with an image file format extension to mean it should join those images files together into a video.
- To convert an
ImageIOfile into any other video format select the file as the source and target to be whatever file format you want.
- To convert a video file of any format into an
ImageIOfile select the video file you want to convert as the source and set the target to be a
.binfile. FFMPEG will then break the video into JPGs and OpenMV IDE will turn these JPGs into RAW Grayscale or RGB565 frames saves to the
.binfile using the
- To optimize a
.giffile saved by your OpenMV Cam for the web set the source file to be that
.gifand the target file to be another
- To convert an MJPEG file saved by your OpenMV Cam to another format set the MJPEG file as the source and another file using another format (like
.mp4) as the target.
Under the Tools Menu (or About Menu for Mac) you can access OpenMV IDE’s option dialog.
You can configure the editor font, size, zoom, tab behavior, automatic cleaning of white space, column margins for folks who like keeping lines to 80 characters, and more.