30.04.2021 • News

Networked vision system assists drivers

System automatically prevents car accidents and reduces casualties

Researchers at the the University of Tokyo, Japan, have developed an evasive maneuver assist (EMA) system using an array of Mikrotron high-speed cameras capable of communicating with each other via a network. The system has a wide field-of-view (FOV) and high responsiveness, enabling vehicles to receive information about potential dangers immediately after it appears around them, thus allowing them to perform evasive maneuvering as quickly as possible to prevent accidents and reduce casualties.

One of the most challenging situations for drivers is avoiding collision with an obstacle that abruptly appears in front of the vehicle. This occurs because humans require a longer time to perform an evasive maneuver after sensing danger. The University of Toyko researchers were investigating methods of assisting drivers to perform evasive maneuvers at the exact moment when instantaneous reaction is required, known as evasive maneuver assist (EMA). One technique, having cameras onboard the vehicles, suffers from a limited field-of-view (FOV) and cannot observe pedestrians or motorcycles in a dead spot of the onboard camera. Another technique, vehicle-to-vehicle communications, notify drivers of each other's position. However, this type of system requires almost all vehicles to be equipped with an alert and notification unit. In addition, the accuracy of GPS for vehicle navigation has room for improvement.

Broad field of view can tracken target even under extreme conditions

A single camera has limited object-tracking capability in large working areas, or whenever a target enters and leaves its FOV rapidly. In contrast, using multiple precisely synchronized high-speed cameras in the system provides a broader FOV that can successfully track a target even under extreme conditions, such as when the target moves rapidly across the FOV of each camera. Owing to the high acquisition rate, the system also recognizes events immediately after they occur around a tracking target, which is critical where even a slight latency can have adverse effects.

The University of Tokyo researchers developed a 12V networked high-speed vision system composed of Mikrotron EoSens CMOS monochrome cameras aligned to cover all areas of interest. These CameraLink cameras are connected via a network to enable them to communicate with each other, with each camera forming a node with its processing unit. All nodes are synchronized to sub-millisecond order using a software-defined precision time protocol. The cameras can robustly track objects moving around the entire space of interest and recognize events in every frame. Applications of the proposed system include various kinds of ITS tasks, such as motion planning and platooning vehicles.

For testing a 1/10-scale robotic car has been used (Source: U. Tokyo /...

Using multiple precisely synchronized high-speed cameras provides a broader FOV...

To test the system, researchers built a 1/10-scale robotic car platform featuring a real-time operating system to control steering and Wi-Fi communication modules. Two high-speed Mikrotron cameras were set to 600 fps and connected to a workstation for image processing. One of the two workstations was set to transmit an obstacle map containing fifty obstacles.

Obstacles were placed by researchers at a fixed point who drew a line indicating the threshold distance of 800 mm. Once the robotic vehicle reached the line, it performed evasive maneuvering. The cameras recognized the position of the vehicle during its entire path, including the area in which the FOVs overlapped, while it was traveling at 72 km/h. Networked cameras detected the moment the vehicle crossed the line. These results clearly show that it is critical to detect dangers as soon as possible to perform evasive maneuvering safely.

Researchers are now working on ways to reduce communication latency by introducing low-latency real-time wireless communication techniques such as RT-WiFi and developing new applications beyond EMA based on the proposed system. (Source: Mikrotron)