One of the outstanding visionaries of our industry is without doubt Cognex co-founder and Senior Vice President Bill Silver. By getting him away for a short time from both, programming (which he still does daily) as well as ultimate Frisbee (which he still does daily also), INSPECT was able to obtain some of his insights on the future developments and challenges in machine vision.

INSPECT: What would you name the most important development in machine vision software during the last decade?
B. Silver: The machine vision industry seems to be well past the stage where any one software development could be called the most important of the decade. In the 1980's one could point to Cognex's Search (normalized correlation) or Itran's GUI, and in the 1990's PatMax, but for the last decade it's much less clear. This is a sign of both the maturity of industrial vision, and the broad diversity of industrial applications.

Well then, what will be the most important development in machine vision software in the next decade?
B. Silver: Hopefully, what I'm working on right now... I hope I'm proven wrong about industrial vision being past the stage where a single new development stands out as "most important", and I hope some ambitious young kid figures it out, although he or she is going to have to compete with some pretty ambitious middle-aged guys still hard at work. Look for developments in one of the following areas:
Image analysis: We've done a great job at squeezing information out of a single image, and I've long believed that to do better we need more information from the scene, such as could be produced using 3D or motion. I have a strong personal interest in motion, because it produces lots of information about objects, requires no fancy hardware, and few others in industry seem to be seriously working on it.
ID: The next decade will see image-based ID largely replace laser scanners in almost all applications areas, not just industrial ID. This will be driven to some extent by increasing use of 2D symbology and demand for capabilities such as saving images of codes that can't be read, but to really replace lasers the image-based systems will have to read 1D barcodes as well as or better than lasers, and this means yield, speed, field of view, and cost. We can already do this in some industrial applications, but for wider use look for some groundbreaking software.
User Interface: User interface revolutions are rare but powerful. I still think that the most influential in the 30-year history of industrial vision is Itran's 1983 GUI. There has been a lot of great work since, but none as broadly influential. I don't know what we might see in the coming decade, but the potential for importance is always there.
Computational Optics: The laws of physics impose limits on depth of field and object speed as a function of illumination brightness, and exceeding these limits causes loss of the higher spatial frequency information that is crucial for pattern recognition, ID, and the like. Computational optics allows one to recover the higher spatial frequency information that we need by giving up some information in lower frequencies in which we are less interested, thereby extending the physical limits. This requires some fancy hardware as well as software, and it may well be of great importance in the coming decade.

What do you see as the major challenges for machine vision yet to solve?
B. Silver: Newton could have been describing machine vision in 2010 when he said, "I was like a boy playing on the sea-shore, and diverting myself now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me." Unlike Newton we pretty much know what that "great ocean" looks like, and that is human vision, but we are hardly closer to it than we were in 1980 when the vision industry started, or in the 1960's when they connected an image dissector camera to a PDP-6 at the Artificial Intelligence Lab at MIT. I'm one of those who believe that in principal a machine can do anything human vision can do, but that in practice it may take centuries. Indeed I don't think it's possible to separate competence in machine vision from the general problem of machine intelligence.

We observe a trend that machine vision software more and more migrates into hardware (like vision sensors, 3D sensors, vision processors) and is sold as integrated part of such hardware as opposed to be sold separately as a library. Do you at Cognex see this trend as well?
B. Silver: We do see that trend, but I wouldn't say that it is in opposition to vision being sold as a library. The trend represents an expansion of the market, not a zero-sum shift from one to the other.

Right now a lot of 3D functionality in machine vision software enters the market with suppliers putting the „3D" label on very different tools and approaches. What can the vision user do to make a right decision here among the different products?
B. Silver: Vision suppliers may be causing trouble for each other in the area of 3D. Most 3D functionality seems to be at prototype or demo level rather than production quality. The technology looks pretty good at a trade show but is either difficult or impossible to set up, calibrate, train or deploy without custom engineering from the vision supplier. When potential customers give it a try, they end up with a bad experience and tend to avoid 3D in the future.
There are many 3D vision methods, starting with different approaches for image acquisition and illumination (single camera, multiple camera, time-of-flight camera, simple illumination, laser light, structured illumination, coded illumination, etc.) and continuing through a myriad of techniques for estimating/inferring 3D structure from the image information (triangulation, stereo, photometric stereo, structure from motion, shape from shading, etc) and then multiple techniques for alignment and inspection that take the 3D information as input. It is clearly difficult for a user to choose the right approach and right vendor.
So the advice to users is: 1) Consider a 2D system first (instead of a 3D system) that they already understand, especially if the vision task may not merit the additional work of 3D; 2) Consider a reputable vision vendor with experience and technical depth; 3) Be skeptical of technical claims; 4) Ask lots of questions and listen for answers that make sense; and 5) Ask the vendor/integrator to walk you through their proposed solution to your problem.

Which are the topics on Cognex' software road map into the future?
B. Silver: We intend to maintain and expand our technical leadership in pattern recognition, ID (1D and 2D symbology), 3D, and other areas of interest to our customers. This is not just meaningless spin; our best people are actively at work in these areas right now.

Thanks a lot for your insights, Bill. It was - as always - enlightening.