Send Orders for Reprints to Reprints@benthamscience.ae 3d Reconstruction Algorithm of Weld Pool Surface Based on Computer Vision Technology

As is well known that sensing and measuring the weld pool surface is very important to design intelligent welding machines which is able to imitate a skilled human welder who can choose suitable welding parameters. Therefore, in this paper, we concentrate the problem of weld pool surface 3D reconstruction, which is a key issue in intelligent welding machines development. Firstly, the framework of the weld pool surface 3D reconstruction system is described. The weld pool surface 3D reconstruction system uses a single camera stereo vision system to extract original data from weld pool, and then the left and right image are collected. Afterwards, we utilize Pixel difference square and matching algorithm and Stereo matching algorithm to process images. Next, the 3D reconstruction of weld pool surface is constructed using the point cloud data. Secondly, stereo matching based weld pool surface 3D reconstruction algorithm is illustrated. In this algorithm , the matching cost function is computed through the Markov random field, and then the weighted matching cost is calculated via the guided filter. Thirdly, to test the performance of our proposed algorithm, we develop an experimental platform to measure weld pool width, length, convexity and the previous inputs based on a linear model predictive controller. Experimental results demonstrate that the proposed 3D reconstruction algorithm of weld pool surface can achieve high quality under both current disturbance and speed disturbance.


INTRODUCTION
Currently, manufacturing industry of welding product needs the mode of automation and intelligent, and it also requires precise control of weld penetration with the widely utilized of welding robot [1].In recent years, some skilled welders observe the surface of weld pool using eyes and choose weld parameters based on the penetration state information, however this mode is not accurate [2].Different from the human operations, we can introduce the vision sensors to extract 3D topography information instead of human operations to optimize the weld parameters [3].
Sensing and measuring the weld pool surface is of great importance to develop intelligent welding machines which is able to imitate a skilled human welder who can choose suitable welding parameters (for example welding current, speed, arc length, and so on) [4].As is well known that Weld pool is made up of weld status, e.g.weld defects and penetration.Therefore, 3D reconstruction of weld pool surface is very complex problem welding research and construction [5,6].It is very important for developing the next generation intelligent welding machines.Additionally, estimating the weld joint penetration in automatic welding is very interesting [7,8].Therefore, in this paper, the images of front-side free surface and back-side width of the weld pool are synchronously sensed by two different CCD cameras are welding process using the laser vision, and then the 3D weld pool surface can be reconstructed utilized our proposed algorithm [9].
Particularly, in recent years, there are four main types of 3D weld pool surface reconstruction methods [10], in the following parts, we will discuss them.Type 1: Model based reconstruction: The 3D weld pool surface is partially reconstructed with 2D weld pool images utilizing a simple method, and it only requires estimating the height of the weld bead at the rear of the weld pool.
Type 2: Stereovision measurement: using this method, 3D weld pool surface is reconstructed utilizing the two weld pool images.Type 3: Shape from shading reconstruction 3D weld pool reconstruction algorithms have also been proposed using shape from shading approach.The rest of the paper is organized as follows.Section 2 illustrates related works about Stereo Matching.In section 3, framework of the weld pool surface 3D reconstruction system is discussed.In section 4, Stereo matching based weld pool surface 3D reconstruction is proposed.Section 5 gives experimental results and related analysis.Finally, the conclusions are drawn in section 6.

RELATED WORKS
The main innovations of this paper lie in that stereo matching is utilized to solve the weld pool surface 3D reconstruction problem.Vision is an important mode to discover and obtain our world for persons, almost 75 percent of human's information are collected from eyes.Stereo vision refers to a subject on how to understand and perceive the objective world by machine rather than human beings.That is to say, visual stereo matching is belonged to one of the fundamental and significant problems in the computer vision.Afterwards, we will show the related works about the applications of Stereo Matching Kim et al. present a new DCT-based local binary descriptor for the dense matching of multiple views stereo.To solve these problems effectively, in the proposed dense descriptor, 2D DCT-based local features are exploited to obtain high discriminative power even for the non-salient regions [11].
Raghavendra et al. proposes an anchor-diagonal-based shape adaptive support region construction for the problem of stereo matching.The proposed algorithm dynamically makes up local support region, and the aggregated matching cost is utilized for Normalized Cross-Correlation-based similarity estimating [12].
Yang et al. proposed a non-local solution for the cost aggregation issue.The matching cost values are aggregated adaptively using pixel similarity on a tree structure obtained from the stereo image pair to hold depth edges.The nodes of this tree are all the image pixels, and the edges are all the edges between the nearest neighboring pixels.The similarity between any two pixels is represented as the shortest distance on the tree [13].Shi et al. proposes a new high-accuracy stereo matching policy using adaptive ground control points.Apart from the existing fixed GCP-based approaches, we use color dissimilarity, spatial relation, and the pixel-matching reliability to choose GCP adaptively in each local support window.In order to reduce the global energy, the authors developed a practical solution, which is denoted as alternating updating method of disparity and confidence map, that is able to effectively eliminate the redundant and interfering information of unreliable pixels [14].Mozerov et al. propose to execute stereo matching as a two-phase energy-minimization method.The authors introduce 2 Markov random field models.The first on is a fully connected model defined on the complete set of pixels in an image, and the second one is a conventional locally connected model.The authors tackle the energy-minimization problem for the fully connected model [15].
Xu et al. propose a radiometrically invariant stereo matching algorithm which is named Optimal Local Adaptive Radiometric Compensation.In this algorithm, this paper approximates the spatially varying Pixel Value Correspondence Function between a corresponding pixel pair as a locally consistent polynomial in an optimal local adaptive window [16].Zhu et al. proposed a new local stereo matching approach which is radiometric invariant.The main innovation of this paper lies in that it uses a combined matching cost of intensity and gradient based similarity measure.Furthermore, they developed an adaptive cost aggregation scheme via constructing an adaptive support window for each pixel, which can solve the boundary and low texture problems [17].
Inspired by the above works, in this paper, we propose a novel 3D reconstruction algorithm of weld pool surface using the Stereo matching approach.

FRAMEWORK OF THE WELD POOL SURFACE 3D RECONSTRUCTION SYSTEM
In this section, the framework of the weld pool surface 3D reconstruction system is given (shown in Fig. 2).
As is shown in Fig. (2), the weld pool surface 3D reconstruction system utilizes the single camera stereo vision system to obtain original data from weld pool, and then the left and right images are constructed.Next, using the filtering process, left and right images is processed by two algorithm, that is, 1) Pixel difference square and matching algorithm and 2) Stereo matching algorithm.Finally, the 3D reconstruction of weld pool surface is constructed by point cloud data which is made up of time difference map using camera parameters.
Next, we will discuss how to convert a same point from pixel to millimeter unit, and the method is defined as follows.
Then, the above representation can be defined as follows.
where f u is equal to 1 dx , and f v is equal to 1 dy , and f u , f v denote the corresponding length for a given pixel on the u and u axis respectively.
Supposing that P x c , y c , z c ( ) is a point in the space, and then the coordinate of this point in the camera coordinate system is represented as x c , y c , z c ,1 ( ) T , and its coordinate under the world coordinate system is represented as x w , y w , z w ,1 ( ) T as follows.
where R is a 3 3 orthogonal matrix and t denotes a translation vector

STEREO MATCHING BASED WELD POOL SUR-FACE 3D RECONSTRUCTION
To implement the Stereo matching based weld pool surface 3D reconstruction, the guided filter is illustrated in advanced.Afterwards, the matching cost function is calculated using the Markov random field, and then the weighted matching cost is computed using the guided filter.For a given gray image, the kernel weight of a pixel is defined as follows.
where i and j denote the indexes of pixels, parameters u k and k 2 are the mean and variance of the image I in w k , and parameter is a regularization value.
Then, for the color images, the weight in Eq. 5 can be redefined as follows.
Using this rule, the guided filter refers to an edge preserving smoothing filter.Therefore, an optimal disparity map can obtain higher quality.
The aim of the stereo matching process is to compute the disparity for each pixel.Supposing that there are two images, that is, the left image I l and the right image I r , particularly, I l is used as the reference image.Based on the Markov ran- dom field, the disparity set is represented as Furthermore, stereo matching denotes the process of calculating the correct disparity value l L d for the pixel using the coordinate x, y ( ).Hence, the absolute luminance difference is represented as the following equation.
Next, the difference of absolute gradient is defined as follows.

C GRAD i,l
( ) = x I r j+l ( ) x I l j ( ) where I refers to the luminance of the pixel, and the symbol N i ( ) means the set of pixels in a specific window which is located at the pixel x, y ( ), P Q means the size of a window.In addition, x represents the gradient operation of the horizontal direction.Based on the above analysis, the matching cost function is defined as follows.
where is ranged in 0,1 ( ) means a harmonic coeffi- cient, which is utilized to balance the luminance and gradient difference.
Afterwards, the final weighted matching cost function is defined as follows.
CF i,l ( ) where the symbol W i, j refers to a weight factor.
Then, using the cost function which is defined in Eq. 10, the disparity related to the min or max cost can be chosen at the pixel level.
Hence, disparity of each pixel is computed by Eq. 11, and the original disparity map can be calculated as well.

EXPERIMENT
Before presenting the experimental results, experimental platform is provided in Fig. (3).This experimental platform is able to measure weld pool width, length, convexity and the previous inputs are utilized by the linear model predictive controller.Afterwards, the calculated welding cur-rent/welding speed is exploited to the welding process to obtain a specific3D weld pool state.Particularly, the arc length is set to 5mm in the following experiments.

Experiment 1: Current Disturbance
In this scheme, the robustness of the control algorithm against current disturbances is provided.The set points in this scheme we used are 5mm, 0.4mm, 0.2mm for the width of weld pool, length, and convexity respectively.Afterwards, the artificial error between the computed and utilized values of the welding current is exploited.In particular, the initial current is 54A and the initial welding speed is set to 1.1mm/s.Additionally, in the first forty seconds, in the openloop period, there are no errors between the computed current and utilized current.Therefore, the controller can let the weld pool width to 5mm, the length to 4mm and the convexity to 0.2mm.Then, input parameters with time varying in experiment 1 is provide in Fig. (4).
Afterwards, we set the weld pool width, length and convexity to 3.5mm, 4mm and 0.19mm respectively, and then weld pool parameters with time varying in experiment 1 is shown in Fig. (5), where, the current disturbance is also outside the input range chosen for the model can represent the intended normal.

Experiment 2: Speed Disturbance
In scheme 2, an artificial error between the actual and calculated values of the welding speed is exploited to evaluate the robustness of the proposed control algorithm against welding pool speed disturbances.In particular, the initial current 54A and the initial welding speed is set to 1.1mm/s, and then the width, length, and convexity of weld pool characteristic parameter are set to 4mm, 3.5mm, and 0.2mm re-    On the other hand, weld pool parameters with time varying in experiment 2 is given in Fig. (7).The weld pool width, length, and convexity are set to 5.2mm, 5.4mm, and 0.28mm, respectively.Then, we can see that the controller can solve the problem of the disturbance through decreasing the welding current to nearly 54A.Combining all the above experimental results together, the conclusions can be drawn that the proposed 3D reconstruction algorithm of weld pool surface can achieve high quality when 3D weld pool surface geometry when different disturbances occur.

CONCLUSION
This paper focuses on the problem of weld pool surface 3D reconstruction, which is a key issue in intelligent welding machines development.The weld pool surface 3D reconstruction system uses a single camera stereo vision system to extract original data from weld pool, and then the left and right image are collected.In addition, the 3D reconstruction of weld pool surface is constructed using the point cloud data based on stereo matching.To implement the stereo matching process, the matching cost function is computed through the Markov random field, and then the weighted matching cost is calculated based on the guided filter.Experimental results verify the effectiveness of our proposed algorithm.

Type 4 :
Structured-light based sensing: In this framework, an iterative approach is developed to reconstruct the 3D weld pool surface.As is shown Fig. (1), two examples of weld pools are given.