Machine vision systems have fundamental limitations with respect to resolution, Field-Of-View, and Depth-Of-Focus. Angular resolution and FOV are conflicting specifications, limited by the size and resolution of the image sensor.
Expanding the FOV while maintaining high resolution is crucial for a variety of applications. For example, in the field of public surveillance at train stations or airports, a large scene needs to be imaged with a resolution that is high enough to manage detailed object detection or face recognition. Another example is traffic sign detection in autonomous driving, which benefits from FOV expansion by enabling to read signs at a larger distance. Other applications where good resolution and a large FOV are beneficial are barcode scanning and iris reco­gnition. To maintain high resolution, the barcode or the human eye to be identified need to be accurately positioned to be within the camera’s FOV. This limits the ease-of-use of such imaging systems or may make it necessary to use multiple devices to cover the necessary FOV.
Although high-end devices are available on the market that are capable of producing wide-angle gigapixel pictures and even videos, these devices are not practical for most applications. They rely on an expensive brute-force approach to use an array of image sensors and to provide the extreme computational and signal processing power to cope with the high data rate.

Commercially available single-device multi-camera systems provide a larger FOV by combining a few standard image sensors. However, these cameras are still relatively wide angle and have small aperture, hence the benefit in resolution is limited.

2D Mirror and Tunable Lens Combined

The core competence of the Swiss company Optotune lies in the optoelectromechanical design of dynamic light controlling components, in the understanding of the materials in use and in the control of efficient and clean manufacturing processes. Due to its specialization in this area, the company has used its competence to develop an option to extend previous FOV limits and thus enable efficient image processing systems that can cover much larger areas based on an innovative idea.

The company’s approach is to equip an industrial camera and objective with a 2D fast steering mirror and a liquid lens. The mirror is used to increase the FOV of the imaging system while the liquid lens allows to acquire all images in perfect focus.

As bright illumination is the key to high quality images, a large mirror size is a precondition for the system. Common 2D-MEMS mirrors have a very limited angular travel range and are generally too small. On the other hand, a simple serial combination of two single axis scan mirrors is also not an ideal solution, because the second mirror needs to be considerably bigger than the first one, which leads to pretty bulky systems.

Optotune’s solution consist of the company’s 2D mirror product MR-15-30 with a 15 mm aperture and large ±25° scan angles, in combination with a tunable lens called EL-16-40, also developed by manufacturer, with its unrivalled 16 mm aperture. Used in a combined way, both products are ideally suited for FOV expansion, allowing for a compact and flexible system.

Stitched Image with Corrected Distortion

In order to investigate the functionality of the setup, a test system that combines the two Optotune products, the tunable lens EL-16-40 and the 2D beam steering mirror MR-15-30, with a 3 Megapixel monochrome camera and a 75 mm CCTV lens was realized. The EL-16-40 is used in a front-lens configuration, and the MR-15-30 is mounted at a 45° angle. This demo setup was placed centered in front of a 2.4 m x 2.4 m square exhibition wall at a distance of only 1.7 m.
While the camera’s regular FOV only covers a small area of 5.2° by 3.9°, the 2D-mirror’s ±25° tilt angles extend this FOV by factor 500. This means that despite the moderate 3 Megapixel sensor, high resolution images with more than a gigapixel can be obtained by stitching individual images captured by the system.

The second image in the image  galary shows the resulting Field-Of-View and the distortion introduced by the 45° mirror arrangement. As a consequence, the resulting FOV of 70° in the horizontal direction is noticeably smaller than in the vertical axis that reaches 100°, despite the equal 50° travel range of the two mirror axes. For reference, the silhouette of the wall is shown in green.

From the Individual Images Acquired in this Way, a Combined Overall Image is Created in the Next Steps

First of all, a set of pictures is collected by scanning the mirror over one of the images on the wall. Optotune’s recommendation is that those images should have about 30 percent overlap to facilitate the following stitching process. The Swiss engineers used the demo version of the fully automated AutoStitch tool, available to license from the University of British Columbia that uses a spherical projection to render the stitched image. For the final distortion removing step, the open-source image mani­pulation program GIMP 5 was used.

Impressive Results

The resulting stitched and distortion corrected overall image calculated as described impresses with an exceptional quality. In order to demonstrate the high level of detail that can be achieved with that setup, a 10-mil barcode was attached to the wall. The smallest feature size of a 10-mil barcode is 0.254 mm which corresponds to 8.6 mdeg when looking from a distance of 1.7 m. This is smaller than the resolution limit of the human eye of about 15 mdeg. However, the setup described is able to resolve the barcode with an impressive quality, thanks to 2048 pixels along the small 5.2° horizontal FOV, which yields 2.5 mdeg/pixel.

Conclusion and Outlook

Combining Optotune’s 2D mirror MR-15-30 and the EL-16-40 tunable lens with a standard camera and lens plus suitable stitching and image manipulation software tools enables the development of compact and flexible image acquisition systems that can enormously extend the Field-Of-View of those systems without the need to move the camera. Areas of application for that kind of system can be found in logistics, ­autonomous driving and many other fields where scanning of large areas with high resolution is required.

To help users get started with the system’s capabilities, the company recently introduced a FOV development kit that features two imaging systems with a wide-angle objective for overview and a narrow-angle objective (50 mm or 75 mm) for high resolution in the area of interest. This development kit is accompanied by a powerful software to demonstrate its suitability for surveillance and inspection applications, incorporating options for face recognition, autofocus, and an area of interest selection.

Author
Andreas Amrein
Application Engineer