AI aided high-speed robot vision of 3D- shaped product surfaces

Motivation and Goal

In most countries of industry and suppliers, sales in the range of several billion Euros per year depend on the visual quality impression that products leave on potential customers. Almost no sector is exempt from this, but it applies particularly to the automobile and airplane industry, electronic products, homecare products, as well as for all industry branches connected with them. These industry branches form the backbone of every modern economy. Austria, in particular, is a country with many suppliers producing for large OEMs and who depend on the quality of their products. This quality not least concerns the flawless appearance of all visible components.

Figure 1: 3D-shaped products and components represent a special challenge for the quality management in the industry.

Sophisticated 3D-shaped visual product surfaces characterize the daily business of many companies (see examples in Figure 1). Among them are automobile interior components, parts that form enclosures of various devices or high-quality packaging. Manual inspection of these 3D-surfaces during production is only possible for very few products. The reasons are time constraints and the “zero defect” requirement. In order to totally avoid the sale of defective parts, a too high reject rate of the produced parts is daily business. However, the demand for zero-defect production and the associated recall of faulty components result in enormous expenditure. This is an economically and ecologically unacceptable situation for the industry.

Therefore, the present project aimed at the development of a new AI aided 3D-component machine vision technology for the industry.


Automatic high-speed full-surface inspection of 3D-shaped products is an ambitious objective. Component geometries and surface structures need to be completely covered and generalized (understood), by the machine vision system within seconds. Thereby, the inspection system must recognize a multitude of different surface faults (out of specification documents), such as local point defects, streaks, surface waviness, different gloss phenomena, defects in surface structures and many more. Weak anomalies do not necessarily lead to the rejection of a part and must be distinguished from clear defects.

In this project the first surface inspection system for real-time full inspection of complex 3D-shaped product and component surfaces was developed, which can perform automatic inspection and perceptual defect evaluation in 5 to 10 seconds. This allows to perform a 3D-surface evaluation in real-time during production.

Figure 2: Example for the PCCL robot vision of a complex 3D-shaped automotive part using a multi-axis robot.

New solutions could be found, both in the high-speed detection of 3D-surfaces and in the area of artificial neural networks for the recognition and classification of surface structures and defects. For the detection of surface anomalies an artificial neural network was developed that can scan and classify the entire component in a parallelized process. The results of the defect classification are available within a few milliseconds after the surface scan was completed. The training of the artificial neural network is performed by a data set that includes good parts, defect parts and augmented data. The required training effort for new components could be reduced to a minimum in this inspection system. A special property is the precise differentiation between “good surface anomalies” and “defect structures and anomalies”. This is an important point, as correctly detected non-critical “good anomalies” lead to a significant yield of good components.

Figure 3: PCCL inspection robot in cooperation with Flextronics International kft at the site in Sarvar (Hungary). The 3D-inspection system tests the entire surface of a component in less than 10 seconds.

Benefit for the Industry

The presented inspection system for 3D-surfaces will considerably reduce the amount of parts wrongly classified as “defective”. At the same time the “slip” of delivered defective parts can be reduced to almost zero by a 100%-in-line inspection. The presented cyber-physical system is equipped with artificial vision in combination with a fundamentally new artificial intelligence and it enables a complete and reproducible inspection of 3D-components for currently more than 60 different types of defects at plastic, aluminium and steel surfaces.

Since a full-inspection takes less than 10 seconds, it is for the first time possible to carry out a fully controlled production of high-quality 3D-components.
Currently, the PCCL Division “Machine Vision and Artificial Intelligence” cooperates with about 20 partner companies and research institutions. The 3D-surface inspection system has already been successfully tested in the facilities of our industrial project partners and it results in saving of material, energy and time.


Priv.-Doz. DI Dr. Dieter P. Gruber is Division Manager for “Machine Vision and Artificial Intelligence” at the Polymer Competence Center Leoben GmbH. He studied “Technical Physics” at the Graz University of Technology. He focused on “Computational Physics and Information Technology” and “Material Physics”, which enabled him to build up knowledge in the fields of material physics, machine vision, data analysis and artificial intelligence. Winning two Excellence Fellowships in 2001 and 2002 led him to research stays at the Albert Ludwig University, Freiburg, and at the Fraunhofer Institute for Solar Energy Systems (ISE), Freiburg.

Dieter P. Gruber moved to the Polymer Competence Center Leoben (PCCL) in 2003 and habilitated to Associate Professor in 2015 at the Montanuniversiät Leoben. He is the author of more than 80 publications and 12 patents and he awarded numerous research prizes such as the Magna ACS Innovation Award, the SFG Fast Forward Award and the Houska Prize. He was Austrian of the Year 2014 in the category “Scientific Research”.


The research work of this article was performed at the Polymer Competence Center Leoben GmbH (PCCL, Austria) within the framework of the COMET-program of the Federal Ministry for Climate Action, Environment, Engergy, Mobility, Innovation and Technology and the Federal Ministry of Digital and Economic Affairs with contributions by Flextronics International kft. The PCCL is funded by the Austrian Government and the State Governments of Styria, Lower Austria and Upper Austria.

Article written by: Dieter P. Gruber