Using ML Models with OptimEyes Developer

ML Image Processing Made Easi(er)

Using ML (Machine Learning) models to process images can be an arcane art. Search the internet for guides, and you'll generally find pages of Python code accompanied by explanations that may as well be written in Klingon for all their lack of clarity.

The OptimEyes Developer ML Tool aims to make ML image processing accessible to humans. Without writing a single line of code, you can use the Tool's single dialog box to:

• Indicate the ML model you wish to use
• Select images to pass to it
• Indicate which portions of the image (a.k.a. ROIs or Regions of Interest) you want to process -- or just process the entire image
• Indicate which inputs (a.k.a. Input Operations) the images correspond to (the Tool can help you figure out what those need to be)
• Indicate the format of the image to pass to the ML model (eg. grayscale or color, using floating point or integer values, etc.)
• Indicate the outputs (a.k.a. Output Operations) you wish to process
• Indicate what to do with the output data (eg. display it, store it in a variable that can be used by other OptimEyes Tools, use it to draw a box around an object in the image, etc.)
• Indicate how many output values to process and how to select them (eg. based on the highest or lowest values in one of the outputs, etc.)
• and more.

Once you've got an ML model working the way you want it, just click the "Record Script" button to add it to a VisualEyes Script, where it will work with our other Tools to automate your computer vision workflows.

Help With Specific ML Models

ML models can be difficult to work with. What are the names of the inputs and outputs? What image format does the model require? Are coordinates output by the image expressed in pixels? percentages of the image size?

The documentation for many ML models can be hard to find, and the answers to the questions you need to know can be difficult to understand even when you've found the documentation. Therefore, we will build a list of ML models with human-readable explanations of some of the more important details and the settings you'll need to use them with the OptimEyes ML Tool.

Here are the ML models we've documented so far* (this Tool is brand new -- check back soon for a larger list!):

• Faster R-CNN with Resnet V2: an object detection model that can locate and identify the types of objects in images.

* Inclusion in this list is not meant to imply endorsement or recommendation of any particular ML model. Ideal Eyes Systems is not affiliated with their creators. This list is provided for informational purposes to assist our customers who wish to use these models.