Real time control of curved laser welding processes by cellular neural networks ( CNN ) : first results

In the last decades the laser beam welding (LBW) has outclassed older welding techniques in the industrial scenario. Despite the improvement in welding technology, sophisticated methods of fault detection are not commonly used in commercially available equipments yet. A recent analysis of process images have revealed the possibility to build up a real time closed loop control system. By the use of image based quality features, a feedback signal can be provided to maintain the process in the desired state. The development of the presented visual control system has been focused on the adjustment of the laser power according to the detection of the so called full penetration hole. Due to the high dynamics of the laser welding, a fast real time image processing with controlling rates in the multi kilo Hertz range is necessary to have a robust feedback control. In this paper an algorithm for the real time control of welding processes is described. It has been implemented on the Eye-RIS v1.2, a visual system which mounts a cellular structure. By applying this algorithm in real time applications, controlling rates of about 7 kHz can be reached. In the following some real time control results are also described.


Introduction
Nowadays image processing is used to monitor the quality of laser material processes such as laser welding, cut or ablation.In this paper laser beam welding processes are treated.The most important characteristic of the laser in welding processes is the huge power density that the beam can convey on the joining partners (Fortunato and Ascari, 2008).Such energy allows the local vaporization of the metal and the formation of a capillary, conventionally called "keyhole" (Fabbro et al., 2004).If an opportune laser power is used, due Correspondence to: L. Nicolosi (leonardo.nicolosi@tu-dresden.de) to the hydrostatic pressure of the metal vapor, all the material sheets are penetrated through the keyhole, creating a full penetration.Meanwhile the laser beam advances, the fluid material solidifies behind the keyhole creating a deep and slender weld seam.By the use of this strategy, moreover, high feeding rates up to 50 m/min can be reached.Figure 1 shows a longitudinal section of the welding setup and the resulting image of a coaxial process control camera.The latter represents the thermal radiation of the melt in the spectral range of 820-980 nm.The state of full penetration is visible in the coaxial camera image as a dark area directly behind the laser interaction zone, the so called "full penetration hole".It ensures that the two materials are properly connected over the whole cross section after re-solidification and, therefore, it represents an important quality feature which guarantees the strength of the connection.In the past, several authors have proposed closed-loop control systems by using, e.g. the laser power, the focal-point position, or other parameters as actuator (Bardin et al., 2004).This paper proposes a Cellular Neural Network (CNN) based visual closed loop control system for the real time adjustment of the laser power according to the detection of the full penetration hole in process images.It is important to underline the importance of controlling laser welding processes.The conventional welding strategy consists in regulating the laser power or the feeding rate until the full penetration is reached and add 10% of the laser power as a safety factor.Such uncontrolled full penetration weld is characterized by significant defects like smoke residue, spatters and craters.Furthermore, this strategy does not allow changing welding conditions during the process, such as feeding rate and material thickness.Due to the high dynamics of LBW, visual algorithms with controlling rates in the multi kilo Hertz range are needed to provide a robust real time feedback control.In order to reach such controlling rates, only a few CNN based operations for the full penetration hole detection in process images can be used to speed the algorithm up.
Published by Copernicus Publications on behalf of the URSI Landesausschuss in der Bundesrepublik Deutschland e.V.

A visual system based on CNN operations
This paper is focused on the description of a new algorithm for the full penetration hole detection and for the real time control of laser welding processes.It has been implemented on the Eye-RIS v1.2 (Anafocus), a visual system essentially composed by a cellular structure, called "Q-Eye", for image sensing and early processing, an "FPGA board" (Altera NIOS II processor) for controlling the execution flow and for decision-making operations and "I/O modules" to interface external devices.In particular, the Q-Eye chip hosts the whole image sensing and evaluation.It has a quarter CIF resolution fully programmable smart image sensor array.It consists of 176×144 cells, each containing an optical sensor merged with a processor, memory circuitry and interconnections to the 8 neighboring cells.This means that the Q-Eye can both sense the corresponding spatial sample of the image and process this data in close interaction and cooperation with other neighboring pixels.This concept finds its basis in the CNN theory (Chua and Yang, 1988).
CNNs are analog dynamic processors which have some of the key features of neural networks and important potential applications in such areas as image processing.The first model of a CNN was introduced by Chua and Yang which consists of a grid of M × N interconnected cells.Each cell C(i,j ) can directly interact with its neighboring cells and indirectly with the other cells because of the propagation effects of the continuous-time network dynamics.The fundamental property that distinguishes a CNN from a general neural network is the local connectivity of the coupling structure.The hardware realization of large-complexity CNN chips has required to modify the original Chua-Yang model into an other one, called Full Range model (Corinto et al., 2002).It is described by: where where m is supposed to be large enough for approximating the nonlinear characteristic shown in Linan et al. (1999).
In the previous equations x ij , u ij and y ij represent the state, the input and the output of the cell (i,j ), r denotes the neighborhood of interaction of each cell and the indexes i,j are such that 1 ≤ i ≤ M and 1 ≤ j ≤ N for a CNN arranged in M rows and N columns.Finally, the matrices Âkl and Bkl are linear weight functions defined, respectively as "feedback" and "feedforward" templates and allow programming the CNN in order to obtain the desired output.Defining an image as a grid of pixels, the pixel of the input image in the position (i,j ) can be mapped on the CNN cell (i,j ), so that each pixel is associated with the input or initial state of a particular cell.The input image dynamically evolves to a corresponding output image due to the CNN evolution, according to the state equation.
Unfortunately, the Q-Eye chip is not governed by traditional state equations like in Eq. ( 1) and its internal status is not accessible.Nevertheless it can be programmed by the specification of templates, thereby it is possible to implement algorithms based on traditional local neighborhood operations which can be efficiently run on any CNN architecture.

Algorithm for the full penetration hole detection
As described in Nicolosi et al. (2009a), algorithms based on dilating operations are allowing the detection of the full penetration hole in laser welding processes.By these strategies an evident improvement in the quality of the welding result has been obtained, especially in situations characterized by speed or material thickness changes.They have been implemented supposing the welding orientation constant during the whole process.This means that they can be applied only for welding of straight lines.Nevertheless, some real life welding processes are executed with orientation changes, such as curved weld lines.With regards to this, a new orientationindependent strategy called "omnidirectional algorithm" has been implemented (Nicolosi et al., 2009b).In the following,  besides the omnidirectional algorithm, a tool for creating automatically a mask at the beginning of the process will be described.The mask is used to reduce the defects resulting from the image elaboration.

Omnidirectional algorithm
The first step consists in enhancing the source image contrast by the application of sharpening filters and binarizing the resulting image by the use of a global threshold value within 1 and 255.The second step consists in detecting the full penetration hole by the use of closing and opening operations (Chua and Roska, 2002).The former is a dilation followed by an erosion and allows filling small black objects in a white background.The latter, instead, is an erosion followed by a dilation and it is used to delete small isolated white objects in a black background.Thereby, at first iterative morphological closings are executed in order to fill the full penetration hole.
The application of a logical XOR and a logical AND comes after to keep white pixels in the full penetration hole area only.Since the closing operations generate defects due to the elaboration of the external edges of the interaction zone, the last step consists in reducing such defects by the use of opening operations and by the subsequent application of a mask.As the typical size of the full penetration hole is pretty known, its presence can be decided simply referring to the total number of white pixels in the last resulting image.Figure 2 shows the algorithm flow chart.
The omnidirectional algorithm has been simulated with several image sequences acquired by the Eye-RIS v1.2 during laser welding processes.As shown in Table 1, hit percentages greater than 82% have been obtained by analysing the simulation results through visual inspections.False detections can be explained making reference to the following: False positives are a consequence of the opening and closing operations.The defects can reach the typical size of the full penetration hole and the application of the opening operation could be insufficient to delete it.False negatives are due to the fact that the opening operation, used to reduce the defects, deletes small white objects despite they represent the full penetration hole.As explained in Nicolosi et al. (2009b), only a few false detections of the same type occur consecutively (from 1 to 4).In order to analyse the effect of such false detections, the reacting time of the physical process has been estimated in the range of 4-20 ms, by applying the Fourier transform to the laser response of different welding processes.Assuming that the controlling rate was slowed down to about 2 kHz by the image storage (not needed in real time control), the worst case of four consecutive false detections of the same type would take a total "false detection" time of about 2 ms.It is considerably less than the estimated reacting time of the physical process and made even worse by the image storage.This means that such false detections should not significantly influence the real time control of welding processes.

Mask builder
As shown in the previous section, the last elaboration step consists in applying a mask to the resulting image.The mask builder allows to fit automatically the mask to the interaction www.adv-radio-sci.net/8/117/2010/Adv.Radio Sci., 8, 117-122, 2010 zone position, making the algorithm independent of the specific optical setup.A default mask centered in the image is adopted.It has a roundish shape whose diameter must be chosen in order to limit the extension of the interaction zone tail which could be source of false detections.At the beginning of the process the center coordinates of the interaction zone in the first available source image are extracted.Through this information it is possible to shift the default mask from the image center to the interaction zone center.Figure 3 clarifies in details the mask builder.

Available hardware
In order to obtain a closed loop control of the process, the Eye-RIS system must be connected to the welding machine.
The available hardware essentially consists of the Eye-RIS system v1.2, a numerically controlled (NC) machine, the laser and the interface board.The NC machine is responsible for safety interlocks, laser ON/OFF signals and movement.The laser is controlled by an analog voltage between 0 and 10 V which corresponds to 0-100% of laser power.The NC machine sends the starting 24 V signal in order to activate the laser and the algorithm flow.Consequently, the machine movement is set in action and the Eye-RIS system starts executing the control algorithm.At first the mask builder draws the mask by the use of the first available acquired image.
As soon as the mask is created, while a new image is being sensed, the image acquired at the previous time step is evaluated by the omnidirectional algorithm.If the full penetration hole is detected the Eye-RIS system decreases the laser power, or increases it otherwise.Consecutively sensing, evaluation and control operations are performed until the stopping signal is received from the NC machine.
The interface board was built to interface the laser and the NC machine with the Eye-RIS system.More details can be found in Abt et al. (2008).The welding experiments discussed in this paper have been carried out with a 2-D-laser scanner setup.The laser source is a 6 kW, 1030 nm Trumpf TruDisk 6002 Yb:YAG thin disk with a 200 µm transport fibre.The laser scanner used -a Trumpf PFO-33 -was equipped with a 450 mm focusing optic which resulted in a focal diameter of 600 µm.The Eye-RIS system is connected to the scanner optic through a 90 • beam splitter.Thus the camera perspective is coaxial to the laser beam, allowing an invariant field of view regardless of the scanner position.Furthermore, it is necessary to use an aligned lens system in combination with an optical band-pass filter in order to acquire high quality camera images.

Experimental results
As explained in previous sections, the omnidirectional algorithm can be used to detect the full penetration hole.In addiction the mask builder represents an important tool to create the mask, which can be used to cut away the defects resulting from the image elaboration.It is important to consider for the real time control sensing and controlling operations which are two additional phases.As already mentioned, sensing and image evaluation are simultaneously performed: while the image i is being acquired, the image i-1 is elaborated.Therefore, the sensing exposure time directly depends on the elaborating time.Controlling operations, instead, allow changing the laser power accordingly to the image evaluation.The application of these operations takes an additional time leading to total controlling rates up to about 7 kHz.
The experiments shown in the following have been executed to weld an overlap joint of two 0.9 mm thick steel sheets with a gap of 0.2 mm and a constant feeding rate.Despite the omnidirectional algorithm has been conceived for welding of curved seams, in a first attempt it has been used to control the weld of straight lines, to be compared with the results obtained by the dilation algorithm in (Nicolosi et al., 2009a).Furthermore, some of the following experiments have been executed with image saving in order to analyse the process images offline.As already mentioned, image storing slows down the controlling rate from 7 to about 2 kHz.Nevertheless, in the considered speed range (3-9 m/min) the image storing has not negatively influenced the process control.Therefore, the omnidirectional algorithm is expected to work even more so for higher controlling rates.
The experimental results pointed out a dependence of the threshold value on the process speed.In low speed processes (1-4 m/min) the full penetration hole is placed inside the interaction zone (Fig. 4a).Because of the vapor plume, there is not a significant intensity difference between full penetration hole and interaction zone.Thereby, a high threshold value in this case is needed to distinguish the two areas.On the contrary, in higher speed processes the full penetration hole is situated directly behind the interaction zone which presents an elongated shape (Fig. 4b).In this case full penetration hole and interaction zone can be easily distinguished by the use of a low threshold value, since they show a big intensity difference.By the application of a high threshold value several areas inside the interaction zone would be discriminated because of the intensity fluctuation due to the vapor plume.Thus the false positives would increase and the laser power level would decrease, creating full penetration Fig. 6.Weld of an overlap joint of two sheets 0.9 mm thick, with 0.2 mm gap at 7 m/min, controlled by the omnidirectional algorithm setting the threshold value at 180.The welding result at the top presents a full penetration loss due to the low level power showed at the bottom.The control was executed without image storing (Controlling rates around 7 kHz).losses.Nevertheless, it is possible to overcome to this problem choosing the opportune threshold value according to the feeding rate.In fact, as shown in Figs. 5 and 6, setting the threshold value to 180, the full penetration has been reached at the speed of 5 m/min, but not at the speed of 7 m/min.In order to obtain the full penetration also in 7 m/min speed processes a threshold value of 130 had to be used, as shown in Fig. 7.This behavior is confirmed by the experimental result in Fig. 8 obtained by using 130 as threshold value for a 8 m/min process.

Conclusions
A novel closed loop visual control system for the laser power adjustment in omnidirectional LBW processes has been presented.It is based on the detection of the full penetration hole in process images.The proposed algorithm is "omnidirectional" because no a priori knowledge of the welding direction is needed, allowing also the control of the weld of curved seams.Controlling rates up to 7 kHz can be reached.First experiments have pointed out a dependence between the process speed and the threshold value used to binarized the source images.In particular, increasing the process speed the threshold value must be decreased to reach the full penetration state.Finally, experimental results have shown that variable orientation welding processes can be also controlled by CNN based strategies.Further studies will be focused on the algorithm improvement in order to apply the same threshold value for a wide range of welding processes and feeding rates.

Fig. 1 .
Fig. 1.Schematics of a welding process in an overlap-joint and nomenclature.The picture shows at the top the longitudinal section of the materials during the welding process and at the bottom the resulting image of a coaxial process control camera.

Fig. 2 .
Fig. 2. Omnidirectional algorithm flow chart.The source image (a) is filtered (b) and binarized (c).Afterwards, closings are executed (d) and logical operations (XOR and AND) are applied in order to keep white pixels in the closed area only (e).Openings are performed in order to reduce the defects (f) and a mask (g) is applied to obtain the result shown in (h).The mask used in this example was built offline to fit the specific source image.The images have been zoomed to improve the visibility.

Fig. 3 .
Fig. 3. Mask builder for the omnidirectional algorithm.The source image (a) is binarized (b) and the coordinates of the interaction zone center are extracted (c).By the use of this information, the default mask, initially placed in the image center IC, is shifted to the interaction zone center (IZC) position, as in (d), in order to obtain the final mask (e).

Fig. 5 .
Fig. 5. Weld of an overlap joint of two sheets 0.9 mm thick, with 0.2 mm gap at 5 m/min, controlled by the omnidirectional algorithm setting the threshold value at 180.The welding result and the laser power are shown at the top and at the bottom respectively.The full penetration state was reached and kept during the process.The control was executed without image storing (Controlling rates around 7 kHz).

Fig. 7 .
Fig. 7. Weld of an overlap joint of two sheets 0.9 mm thick, with 0.2 mm gap at 7 m/min, controlled by the omnidirectional algorithm setting the threshold value at 130. The welding result and the laser power are shown at the top and at the bottom respectively.The full penetration state was reached and kept during the process.The control was executed with image storing (Controlling rates around 2 kHz).

Fig. 8 .
Fig.8.Weld of an overlap joint of two sheets 0.9 mm thick, with 0.2 mm gap at 8 m/min, controlled by the omnidirectional algorithm setting the threshold value at 130. The welding result at the top presents a full penetration loss due to the low level power showed at the bottom.The control was executed with image storing (Controlling rates around 2 kHz).