Over the past 20 years the Munich software company MVTec has gained a leading position in the market for machine vision software. The demands on the products are accordingly great. What can software now achieve? Dr. Wolfgang Eckstein, CEO from MVTec Software, spoke with inspect about the potential performance of modern vision software.

inspect: Dr. Eckstein, how long would our modern, technical world continue to function if all image processing software suddenly disappeared?

Dr. Eckstein: Problems would arise more quickly and be far more devastating than one could imagine. Commerce and logistics would be directly affected. The automatic scanning of food, packages and documents would no longer function. Work at the major online retailers would thus grind to a halt. In the same way, production throughout the electronics industry would collapse. Semiconductor components and electronic components would no longer be possible, not to mention the manufacture of technical products. In next to no time, there would be no more new cell phones, computers, televisions, computer games, solar panels or cars. Other areas would also be affected, however, such as medicine, food, safety systems, robotics, photography, telephones, Internet and road traffic controls. In other words, a world without image processing would permanently be like a world without electric power: nothing would work any more. It is a good thing that we have image processing!

inspect: In the field of image processing, the combination of eye and brain is used as an analogy for high performance vision and recognition. What models does modern software utilise?

Dr. Eckstein: The eye and the brain are a very appropriate analogy for image processing. The eye consists of the lens and the retina; these correspond to the optics and the camera. Moreover, the resolution, use of color, sensitivity and repetition rate of the eye and the camera are comparable these days. The eye is also capable of fast preprocessing, such as filtering and color space transformation; this approach is commonly adopted in modern cameras or directly connected special hardware. Data are processed and interpreted in the brain. This corresponds to the software and hardware of an image processing system. There are clear differences in implementation, however. Processing in the brain is massively parallel, to a much greater extent than we can expect from computers in the future. Another feature of the computer, in contrast to the brain, is the separation of software and hardware. This separation provides us with a very useful degree of flexibility for the simple and cost-effective adaptation of applications to specific requirements. The performance of the brain nonetheless far exceeds that of current image processing systems in many respects. We thus still have plenty of work ahead of us for the future.

inspect: The development of powerful algorithms is far ahead of software development. There have been mathematical approaches for solutions to image processing problems, but no adequately powerful computers for their commercial implementation. What is the present situation?

Dr. Eckstein: Research and development in image processing software is being carried out on two fronts. The first is the one you mentioned: there are good processes that cannot be used due to excessive runtime or memory requirements. The objective is to optimize these methods so that it becomes possible to use them. On the other hand, there are also problems for which basic solutions have yet to be found. The performance of our brains is a good indication of what we can expect in the future. MVTec is known for its innovations in both of these areas. Processes such as sample-based identification are impressive examples of the capabilities of new technologies and open up new fields of application for image processing.

inspect: 3D vision and pattern detection are complex and sophisticated fields of image processing. What can software achieve in combination with modern vision hardware?

Dr. Eckstein: It is amazing what is already possible with current 3D software technologies. Most industrial applications are currently being used in the fields of robotics, mechanical engineering, semiconductors and the food industry. In the coming years, we will see 3D processes becoming as important as conventional 2D processes are now. This is the development that MVTec is actively helping to push forward.

inspect: Where is the emphasis for the development of algorithms and software solutions in your field?

Dr. Eckstein: In addition to the introduction of new software technologies referred to earlier, our focus is on usability, i.e. the simple and efficient creation of solutions. 3D image processing still requires extensive technical knowledge on the part of the user. We are currently able to offer the full range of 3D processes. Our goal now - alongside normal further development - is to open them up to less experienced users.
We must also focus on continuous improvement in terms of robustness and processing speed. These are two points that are of intrinsic importance for customers, as the demands of their customers become ever greater. That is why MVTec is continually working on existing processes in search of alternatives that are even more robust or even faster. This effort is not to be underestimated when one considers that HALCON has more than 1800 operators and consists of almost 5 million lines of code. That is one reason why it is now virtually impossible for a user to develop such a library of his own.

inspect: The sky is the limit! - How far would you say that this applied to the potential performance of image processing algorithms?

Dr. Eckstein: First of all, the sky is not a limit at all. Our product HALCON, for example, has been used successfully with the humanoid robot R2 on board the ISS space station. This is one good example of how putative limits can be overcome. Our brains impressively demonstrate the capabilities of image processing. We are rising to this challenge and are highly confident that we can continue to provide the worlds of industry and business with many useful innovations - innovations that we can currently not even imagine. Image processing is thus an area in which we can still expect decades of further development accompanied by continuous growth.

inspect: Even among engineers, image processing used to have the aura of an occult science, which was closer to the esoteric than to mathematics. Its handling was the preserve of absolute experts. How much expert knowledge is now needed to operate a machine vision system?

Dr. Eckstein: Simple applications can already be implemented with a minimum of specialist knowledge. At MVTec, however, we envisage the use of image processing in practically all fields of application. To enable this, we are working on making it vastly simpler to use - even for previously complex processes. It is interesting to take a look at the definition of innovation. It includes the requirement that an idea is not only implemented, but also accepted by the market. In the case of image processing, this means that we need a good balance between new processes and techniques that make these processes easy to use.

inspect: Software is no longer installed on central computers, but rather provides individual image processing components with intelligence. How does this work and what does this mean for the complexity and performance of the overall system?

Dr. Eckstein: We offer our customers the entire range of processes for practically all kinds of hardware and every operating system. It is possible to make a camera intelligent, for example, or enable a PLC to be used for processing data in addition to controlling the machine. This has major advantages for the user, such as simple installation or low prices - combined with use of the latest software technologies.

inspect: The capability of mobile systems to perceive and interpret their environment is progressing. What challenges does this present to the software developers in your company?

Dr. Eckstein: MVTec is pursuing two parallel strategies. First of all, our HALCON software can be used directly for the development of apps on smart phones. This makes it possible for software companies to develop attractive products in a short space of time. Alternatively, the recorded images can be transferred from the smart phone to the cloud for evaluation on the server - using HALCON once again. The customer can thus choose the model that corresponds to his technical requirements and product strategy.

inspect: Image processing has already left the realm of industrial and scientific applications. It is increasingly becoming a part of everyday life. Where do you expect the most far-reaching and sustainable changes to everyday life due to the use of machine vision?

Dr. Eckstein: One area in which we are all familiar with it already is that of computer games. In this field, 3D technology is used with extremely low costs and astonishingly high performance combined with astonishingly intuitive usability. That is just the beginning, however. The performance of today's smart phones is already comparable to that of a normal computer just a few years ago. This allows the implementation of extremely sophisticated applications with minimal size and unbeatable portability. Image processing will also lead to advances in the use of Internet services in fields such as image searches, recognition or navigation. Many household appliances will also benefit from image processing in the future: examples might include a washing machine that can detect the color and degree of soiling of the garments, as well as automatically reading the labels, in order to offer assistance in the choice of program. A camera in the oven could ensure that no cake ever burnt again. One thing that is certain is that the future holds many interesting applications in store, as a source of both assistance and delight for the user.