The vision industry is on the verge of a technological revolution with consumer-based technologies driving industrial vision applications. This revolution is being driven in part by the increasing use of portable consumer electronics such as smart phones, tablets, and embedded systems, which now have more on-board processing capabilities, are widely available to consumers, and are low cost making it easier than before to build an intelligent vision system that is both economical and efficient.

Embedded systems will play a large part of this revolution: packaging computing hardware, software, and interfaces into one simplified consumer-friendly solution.

A large portion of vision systems using embedded computing will benefit from ARM-based processors. They use a Reduced Instruction Set Computing (RISC) which means the code is compiled into small atomic instructions and then executed very efficiently. This requires more work up front by the compiler, but the benefits are passed along to the processor. The reduced instruction set is optimized to be handled by the processor, requiring fewer resources during execution. This translates to the processor requiring less power.

ARM processors are also designed using “big.LITTLE” technology, where the number and size of the cores can vary to further optimize the processor. An example of this would be a processor with four A7 cores and four A15 cores. The A7 cores would take care of smaller tasks with high efficiency whereas the A15 cores will perform more process-intensive tasks with more power. This architecture allows tasks to be balanced across the processor depending on their complexity.

Mobile computing, a driving force

There are a number of processors and Systems on a Chip (SoC) using ARM currently on the market, which is being driven by the demand for mobile computing. ARM processors are a perfect fit to meet this demand, as they are ideal for low power, high performance devices. Typical SoCs will include combinations of a CPU, GPU, RAM, and various interfacing technologies such as WiFi, USB, and Bluetooth, among others.

Embedded systems can benefit from the advancements that SoCs have seen thanks to the consumer mobile industry. Popular devices known as Single Board Computers (SBC) are a great example of systems using ARM-enabled SoCs. These devices will contain the SoC along with storage slots, IO ports such as USB and Gig-E ports, general purpose I/O ports, and a DC power input. Popular devices include the Raspberry Pi, the ODroid XU4, and the NVidia Jetson TX1. While these devices are popular, accessible, and typically inexpensive, they are not ideal for large scale industrial deployment – rather they are best suited for development and prototyping (fig.1). Since they are a consumer level product, they tend to have new models available every year and are quick to discontinue older variants.

A strategy for the industry

For industrial vision applications, a more suitable approach would be to use a Computer on Module (CoM). Similar to Single Board Computers, CoMs also use ARM-enabled System on Chip technology along with memory and storage; however they do not come with any physical I/O ports. Instead the CoM is inserted into a custom carrier board containing only the required ports for the specific application. Since CoMs are intended for industrial use, the module’s product lifecycle is lengthier, making them suitable for OEM production.

ARM certainly has its place in select vision applications considering it has a long history of supporting Linux. There are a number of ARM-based vision libraries for Linux such as OpenCV – initially developed over 15 years ago, SimpleCV, and OpenMV, to name a few. There are also more advanced and proprietary libraries available for purchase from companies like MVTech Halcon Embedded, MATLAB, and FastCV. In addition to these libraries, many camera manufacturers, such as Lumenera, have purpose-built Software Development Kits (SDK) that are designed to work specifically with ARM-based systems.

ITS an UAVs

Two industries that can benefit from ARM-based embedded vision systems are the ITS (Intelligent Transportation Systems) and Unmanned Systems markets. In the case of ITS, applications like red light, speed, and parking enforcement and high speed tolling – systems are often detached from the power grid and will rely on solar panels or batteries to supply power. Processing requirements for ITS systems tends to be less stringent as they usually do not need to capture more than 5 frames per second and image processing tasks such as Automated Number Plate Recognition (ANPR), demosaicing, and compression can be done in background threads when the system sits idle. The processed images can then be sent over cellular data networks to a central processing station for analysis like enforcement or toll collection.

Specific ITS applications can come with additional requirements such as multiple cameras for red-light enforcement and radar triggered systems for speed enforcement. ARM-based embedded systems are well equipped to handle all of these design requirements and offer cost-effective alternatives to be placed in ITS solutions. They also offer the opportunity to expand on the ITS system’s capabilities, allowing for solutions such as traffic flow management and the identification and reporting of vehicles of interest, both of which performed in real time.

Turning to the Unmanned Systems market, in particular Unmanned Aerial Vehicles (UAV’s), where size, weight, and power are scrutinized extensively, embedded vision systems are an ideal fit for the UAV’s payload. In such applications, the imaging payload will typically weigh one to two kilograms and require minimal processing from the computing system. UAVs used for imaging tend to capture images at roughly one to four frames per second and most image processing such as demosaicing and image stitching is done post-flight by systems with more intensive computing power. Suitable applications for UAV-based and ARM-enabled embedded vision systems are photogrammetry – precision mapping that comes at a lower cost than manned aircraft, precision agriculture – monitoring the health of crops using the Normalized Differential Vegetative Index (NDVI) (fig. 2) or other multi spectral methods, and inspection – visually assessing large structures such as turbines, dams, and high voltage power lines.

With the advent of more powerful processors, embedded vision equipped UAVs can enhance their abilities by making use of vision data for navigation. Adaptive flight paths can be adopted based on the data recorded from a scene below, avoiding low interest areas – as may be the case when plotting and pre-programming a flight path over an unknown area. Real-time collision avoidance can also be accomplished as well as over the horizon (out of sight) operation.

Conclusion

The high performance and cost effectiveness of ARM-based embedded systems make them very attractive for implementation within various vision systems. The incremental improvement of ARM-based processors and SoC technology is allowing for the development and introduction of new and smarter applications without breaking the bank or demanding too much power. Linux and ARM-based embedded systems will continue to grow in the market and find themselves in vision applications around the world for years to come.