Status Quo

Deep Learning (DL) based Computer ­Vision (CV) market is undergoing a relentless transformation with continual technological advancements and consequently a surge of new solutions is becoming available. Likewise, many companies are either already implementing or at least considering the idea of CV on their manufacturing sites to establish “smart factory” habitats. From the viewpoint of an economist, this is an ideal example of “Supply and Demand”. One just had to sit back and enjoy the show. Alas, the show has been rather bumpy! The predominant reason for this circumstance is the complexity of requirements by the manufacturing industry, emerging from the bridging of existing operational tech infrastructure and integrating it with the rest of IT. On the other hand, the lack of flexibility and scalability by the “one size fits all” solutions in the market does not help with mitigating the status quo. Robotron’s Real-time Computer Vision (RCV) platform aims to close this gap.
 

Off-the-Rack, Yet Unique

With the introduction of CV technologies into automated manufacturing, new requirements emerge. A decisive factor for success is the integration of the solution and the CV models into existing standardized company processes and IT systems. The RCV platform meets these requirements through a modular and scalable design based on container virtualization and expandable interfaces. This enables operational interaction with almost any physical or virtualized hardware. In addition, data from control systems such as MES and ERP can be integrated. Standardized interfaces (such as Modbus, OPC-UA, etc.) can be used to connect the platform to manufacturing/­assembling units, for example to inspect faulty components. The administration of the RCV toolkit is carried out via a central GUI and thus offers a comprehensive and consistent overview of the end-to-end process.

Flexible Data Acquisition and Processing

Different inspection scenarios and circumstances at times require disparate or specialized camera technology. In addition to the free choice of hardware, diverse infrastructure scenarios are also supported to assist with the requirements of the shop floor. This means that the toolkit is operable entirely in the cloud or on-premises and as well as in a hybrid environment with limited or no internet connectivity. This flexibility is particularly helpful with projects that address a heterogeneous workflow and can include everything from robot commissioning to standardized shopfloor procedures.

Collaborative Image Annotation

Annotating images and generating training dataset is key to creating a precise and robust DL model. To optimally identify and mark the defect classes in the images, domain experts such as quality engineers, process engineers and machine operators are needed. They often work from different and geographically distributed locations. The RCV toolkit enables multiple users access to the same image dataset and facilitates the annotation process from spatially separated locations. This collaborative process increases the quality of the training dataset and consequently the resulting model and facilitates its ongoing maintenance. In addition, a supervisor with administrator rights can coordinate the workflow, for example, by partitioning the image dataset to ensure fast and orderly annotation of the images by several users.

Artificial Augmentation of the Dataset

Due to sustained quality control measures, proportionally fewer images of defective parts are available, which can have a negative impact on the quality and accuracy of the model. To compensate for this, the toolkit offers the functionality to artificially expand the data set through prevalent augmentation techniques (e. g. rotation, mirroring, etc.) and as well as the possibility to generate synthetic images. Besides the selection of conventional methods, it is also possible to integrate custom scripts. With these functionalities, the size of the data set and quantity of particular defect classes can be individually controlled or even fully automatically compensated to optimize the accuracy of the DL models.

Creation of CV Models

The usage of DL in the recent years has led to a strong research progress in the field of CV. Various frameworks (TensorFlow, PyTorch, Keras, etc.) have been developed to make it easier for users to create neural networks. Additionally, different architectures of convolutional networks have been designed, which are utilized depending on the use case and the prescribed requirements (accuracy, time, resource demands, etc.). The RCV platform follows a no-code AI approach to facilitate the creation of models without any prior programming knowledge. The user can easily create several models and compare their metrics. Different frameworks and architectures are supported to determine the ideal solution for the use case. The provisioning of the trained CV models is similarly performed by means of container virtualization.

Supporting Model Lifecycle

When going live with a productive rollout, deployment of AI models across several manufacturing lines and plants that involve a release management procedure can become a challenge. It is important to comply with both IT and organizational guidelines and procedures. The RCV toolchain offers an end-to-end management and release of models via an easy-to-use dashboard. The modular structure of the platform facilitates the integration of additional tasks such as monitoring for data drift or triggering a pipeline for retraining.

Scaling and Coverage through Partnerships

As a modular platform, RCV enables a professionally scalable creation, distribution and maintenance of DL-based quality and conformity checks throughout the entire company. The information contained in the image data can be analyzed and processed in near real time to address a variety of complex issues, making inspections more comprehensive, stable, and efficient, and thus providing the basis for future process optimization. Thus, the use of the platform makes a significant contribution to increasing the success of the company. 

Strong partnerships in the software and hardware sectors will result in further innovative and interesting areas of application in the future. As a long-standing partner of Microsoft in the cloud sector, Robotron is very familiar with the extensive services of Azure, so that a seamless integration of the RCV toolchain is guaranteed. Robotron works together with industry leaders such as Intel to integrate various hardware components with the RCV toolkit. The use of the platform in industries operating under extreme conditions such as railway, mining or oil & gas is easily possible, thanks to the partnership with Moxa Europe GmbH, the manufacturer of robust edge devices. 

Authors
Tom Marvin Ihme, Systems Analyst, Realtime Computer Vision
Dr. Deepa Kasinathan, Group Leader & Product Owner, Realtime Computer Vision