Thermal field prediction in DED manufacturing process using Artificial Neural Network

Seifallah Fetni. University of Liège, UEE Research Unit, MSM division, allée de la Découverte, 9 B52/3, B 4000 Liège, Belgium Quy Duc Thinh Pham. Thu Dau Mot University, Vietnam University of Liège, UEE Research Unit, MSM division, allée de la Découverte, 9 B52/3, B 4000 Liège, Belgium Van Xuan Tran. Thu Dau Mot University, Vietnam Corresponding author: xuantv@tdmu.edu.vn Laurent Duchêne. University of Liège, UEE Research Unit, MSM division, allée de la Découverte, 9 B52/3, B 4000 Liège, Belgium Hoang Son Tran. University of Liège, UEE Research Unit, MSM division, allée de la Découverte, 9 B52/3, B 4000 Liège, Belgium Anne Marie Habraken. University of Liège, UEE Research Unit, MSM division, allée de la Découverte, 9 B52/3, B 4000 Liège, Belgium Fund of Scientific Research F.R.S-FNRS


Intr Introduction oduction
Additive Manufacturing (AM) processes are today widely spread and prove their ability to form complex shapes and are applied on a wide variety of alloys [1,2]. Meanwhile, industrial applications face various kinds of defects during and after fabrication: porosity, formation of cracks, keyhole defects [3], difficulties to produce net shapes etc. This is due to the complex phenomena occurring during the manufacturing process.
Experimentations and trials constitute a way to adjust process parameters (laser power, scanning speed, spot size, building direction, hatch spacing, feed rate, distance nozzle-substrate etc.) but are time and cost consuming. Therefore, numerical techniques, in particular the Finite Element Method (FEM), have been used as promising tools to predict various outputs such as microstructure, mechanical properties, melt pool shape, and dimensions.
Within literature, several approaches to model AM processes have been developed [4][5][6][7][8]. These numerical models help to better optimize process parameters and offer the possibility for industrials to reasonably predict distortions, shapes, residual stresses, thermal histories etc. However, numerical modeling applied on AM still faces various challenges.
Indeed, multi-physical simulations are time and CPU consuming and require simplifying assumptions, which result in accuracy issues consisting in discrepancy between computed and experimental results. To improve the accuracy, ESAFORM 2021. MS13 (Additive Manufacturing), 10.25518/esaform21.2812 2812/1 additional high cost must be paid; taking in account real thermo-physical properties of the cladded materials, multitechnique validations : thermocouple records, high speed cameras, microstructural analysis, nano-indentations…Thus, simulation should be validated by at least two techniques [8,9]. Boundary conditions are still a serious challenge in AM modeling due to the difficulty of experimental measurements. These FEM drawbacks trigger scientists to look for alternative approaches.
Machine Learning (ML) is a promising technique which has been enhanced by the increasing computing capacity, mostly in the last decade [10]. It trends to prove its ability to be complementary and even substitute for traditional techniques in computational material science. It has been considered as a technology development accelerator in various fields like chemistry, molecule design [11], finance [12], Web … but is still a new approach in materials science and especially in the field of metals. The limitation of ML in this research area is due to the lack of large dataset of materials, while other domains take advantage of large available datasets: Kaggle, AWS, Socrata, etc. Most recently, Integrated Computational Materials Engineering (ICME) approach has been developed to enhance emergence of new materials and better selection, especially in advanced applications. The aim is to develop a strong network relating actors in material science field. ML framework provides a good opportunity to couple ICME tools and experimentations [10].
Researchers in AM field are giving increasing attention to harness the benefits of Artificial Intelligence (AI) in order to optimize materials design and control process parameters. It should be noted that ML is the set of soft techniques applied to realize AI. AI approaches can be classified into two main groups: supervised and unsupervised learning.
Supervised learning is used in case of known responses and englobes regression algorithms on the one hand and classification ones on the other hand, while unsupervised learning is used in case of unknown response (prediction, recognition, etc.). Regression techniques in AM field have the aim to predict continuous responses (thermal histories [13], melt pool depth [14], deformations [15], etc.). For that, linear or nonlinear regressions, Gaussian Process (GP), Support Vector Machine (SVM), Logistic and Artificial Neural Networks (ANN) are good candidates. Classification techniques are attracting attention of material's researchers to predict various defects occurring during manufacturing such as abnormal grain growth. For that, K-nearest neighbors, Logistic regression and SVM are strongly recommended according to [16].
The selection of the appropriate algorithm strongly depends on a variety of criteria such as the size of the training data, memory usage, target accuracy, interpretability of results and number of features. Statistical methods and pure-datadriven models are developed in AM field to quantify the influence of process parameters on mechanical properties, surface characteristics, build shape, etc.
Within literature, some interesting ML models applied on AM of metals can be highlighted. Tapia et al. [14] developed a framework based on a GP surrogate model to predict melt pool depth in single track Laser Powder-Bed Fusion (L-PBF) deposition of 316L Stainless Steel (SS). The model was used to identify appropriate process parameters to get the desirable heat conduction and reduce keyhole mode effects. Waqas et al. [17] proposed a model of the deposition of AlSi10Mg alloy by Selective Laser Melting (SLM) in order to predict plastic anisotropy, local strain distribution and failure in tensile tests.
As the application of ML in AM field is still not widely applied in particular in metal alloys and is often considered as a black box for several researchers of this field, this work proposes an exploratory and comprehensive study that highlights the methodology of implementation of an AI framework composed of a physically based model (FEM) and an ANN algorithm. An initial architecture was chosen through trial and error as a first step. Then, different configurations were studied with varying numbers of hidden layers, numbers of nodes per hidden layers and activation functions.
Thermal field prediction in DED manufacturing process using Artificial Neural Network 2812/2 2 FEM model and ML fr 2 FEM model and ML framew amewor ork k 2.1 FEM model A FEM model for 2D thermal simulation of the Directed Energy Deposition (DED) of the composite coating (316L SS and tungsten carbides) was developed to predict the thermal history during the building stage and to get insights about possible microstructures of the solidified material [18]. The 2D model is shown in Fig. 1. The clad consisted in 19 superimposed layers. The validation of the model was based on two validation criteria: measured melt pool depths and a substrate thermocouple records. In addition to the parameters gathered in Fig. 1b, thermo-physical properties and boundaries conditions (radiation and convection) were considered [18]. set of pr set of process and simulation input data [18]. ocess and simulation input data [18].

ANN model ANN model
From ML regression techniques, which predict continuous responses such as temperature field history, ANN was here chosen. This choice is recommended for highly non-linear problems and consists in fully connected perceptrons to mimic the human brain. The thermal history predicted by the FEM model was used as the input data of ANN. Input dataset, organized as a matrix X, is indeed composed of following input features: =( ) ; x, y are the coordinates, t is the FE time step, xL and yL define the laser position at each Finite Element (FE) time step, dx and dy are the distance between laser head and each point P(x,y) at each time step. It was reported that the distance between laser head and substrate points has direct impact on their thermal history and melt pool shape [19]. Overall, the input matrix X X has 7 as columns number (7 input features) and the output layer contains nodal temperature for each point at each FE time step.
It should be noted that the case where the matrix only consists in x, y and t as input was also investigated in order to highlight the impact of adding more features on the model accuracy. In a related work, input laser power (input energy) was varied in the FEM model to get a bigger dataset and was considered in the input matrix. Moreover, the layer number was taken into account [20].
The actual work took into account only one FEM simulation due to its exploratory purpose. The ANN architecture was first chosen using trials and error which led to 4 hidden layers; each one contains 100 nodes, as shown in Fig. 2 and here a is the output value of a given perceptron, f is the activation function (often non-linear), wi the weight of an input connexion i whose value is xi and b the bias. The activation functions are various such as the Rectified Linear Unit (Relu), Logistic (sigmoid), the hyperbolic tangent (tanh), and the softmax (also called softargmax) [17,11].  It should be noted that this architecture corresponds to deep learning, as the number of hidden layers exceeds one.
The data preprocessing was ensured by feature transformation integrated into the single matrix (X) then normalized.
Indeed, normalization is a key step in data preparation to allow the ML algorithm a better modeling of the data. In particular, the data was normalized to the range of 0 and 1. A Python script code was used to implement the framework using the free scientific libraries: Numpy (high dimensional arrays and matrix), Scipy (scientific computing), Matplotlib (graphics), Pandas (data analysis), Tensorflow (differential programming and data flow), Keras (interface for ANN), Sklearn (data regression) and Shap (feature importance). Using the Keras sequential model, the input data were divided into a training data set and a validation one through the choice of 0.6 as ratio (ts). The ANN model was trained using only 60% of input data, while the remaining 40% were used to validate the model. Data normalization was processes using the sklearn object "MinMaxScaler".
Thermal field prediction in DED manufacturing process using Artificial Neural Network 2812/4

R Results and discussion esults and discussion
After the training and validation phases, predicted ANN solutions were compared to those obtained by FEM simulations.
Thermal histories for selected important material points (gathered in Fig. 3a) were carried out. These points were selected from different heights of the clad, at the beginning of the deposition and from the substrate. For comparison purposes, various metrics evaluate the ML model performances called loss fucntions. The mean squared error (MSE) is given by the formula: The mean absolute error (MAE) is expressed by: These errors were used to evaluate the accuracy of regression problems. Here yi, ŷ and n are respectively the reference value (FEM result), the predicted one (ANN) and the number of samples or rows in the validation dataset (ts*nb_node * nb_time_steps). MSE has the advantage of providing a quadratic loss function and letting data scientist punish big errors, but that may enhance difficulty of interpretability by varying from 0 to infinity. MAE simply gives the absolute difference between actual (from FEM model) and predicted values (by ANN), but is an absolute value like MSE.
The Mean Absolute Percentage Error (MAPE) is suitable for regression problems and has the advantage to be more interpretable as it is a relative error. It is given by the formula: Likewise, the R 2 score is interpretable as it varies between 0 and 1. It links the variance from the model to the total variance through the relationship: where ̅ is the mean of actual values. The more data are correlated, the more the tendency of R 2 is to be close to 1.
It was here chosen to begin by highlighting results of the ANN model when only x, y and t are considered. As inferred from Fig.3   When considering all features (7 inputs), there is a good agreement between FEM and ANN predicted curves, as shown in Fig. 4. The model loss is a number indicating how well the model is doing. It was computed for each example in training and validation sets. The lower is the loss, the better is the model. Fig. 4a   The test accuracy is higher than 99%, which suggests a high fidelity model. Output data were postprocessed for a better visualization. As shown in Fig.5, the ANN model predicted a thermal field distribution similar to that predicted from the FEM model. The big data set (training and validation) explains the high fidelity prediction. Indeed, it consists of 4079768 samples (1628 nodes * 2506 time steps). Moreover, the most influential parameters, such as laser power, idle time, distance from substrate to nozzle as well as the geometry were not varied. That allows ANN a good modeling Thermal field prediction in DED manufacturing process using Artificial Neural Network 2812/6 of data without too much training time. given in Fig.6a, confirm the initial choice of Relu which was based on the fact that Relu functions are commonly used in regression problems with neural networks [21]. The huge error given by the softmax function was excepted because it is often applied in only the last hidden layer as a classifier, used e.g., in image analysis problems with deep neural networks [18]. Tanh also gives acceptable results for this highly non-linear problem because the data has been normalized to the range of 0 and 1. For sigmoid, deeper analysis should be applied to interpret the obtained huge error. Indeed, Waqas et al. [17] (see Introduction) had obtained similar losses evolution by applying sigmoid and tanh, when they investigated the effect of the choice of the activation function. In their case, a neural network with 8 hidden layers was selected while 2, 3 and 4 hidden layers (using Relu) were here investigated as shown in Fig. 6b. The performed trials demonstrate that increasing the number of hidden layers leads to progressively reduce the mean percentage error to reach an acceptable value of 2.28%. For comparison purposes, the architecture (8 HL + "sigmoid" + 100 nodes/HL) was also tried. It results in a decrease of the model error from 58% to 12.7%, which is a significant decrease but not sufficient. Another interesting analysis consists in testing the effect of the number of the HL nodes on the mean percentage error and the training time. For simplifying purposes, all hidden layers of the model are affected by the same number of nodes (100). As inferred from Fig.7, increasing the number of nodes in the hidden layers leads to progressively decrease the model error to reach a steady value near 2%. It is observed that increasing the number of nodes to greater than 300 does not significantly improve the model error while it can consume significantly more CPU training time. Based on these trials, it could be deduced that an architecture of 3 or 4 hidden layers associated with about 100-200 nodes leads to obtain a good accuracy (more than 99% in this study) and in a reasonable computing time. It should be emphasized that these parameters appeared to be goal for this specific data set and problem.

Conclusions Conclusions
In this work, a ML framework was proposed consisting in a physically-based FEM model and a deep learning algorithm.
The choice of such an architecture was carefully justified. That gave high fidelity prediction for FEM results through more than 99% of accuracy and a model error (mean of all MAPE errors) of about 2% for an ANN model containing 4 HL and 100 nodes/HL. The effect of varying the number of HL and the number of nodes was investigated. The obtained high fidelity is not only explained by the choice of the architecture but also the quantity of input data. Indeed, the deposition of 19 superimposed layers was modeled by FEM, which provided a dataset containing more than 4 million samples.
ANN regression algorithms are well adapted to AM problems because output data are generally continuous such as temperature and strain evolutions. Even if ANN algorithms can substitute FEM through a framework composed of physical analysis and ML model, FEM has the particularity to give big data set alloying ANN to better model input parameters. Thus, combining FEM and ANN in one framework is a suitable choice to get good accuracy. When in-situ control is possible in AM field, real-time computing and adjustment of processing parameters can be possible using ML.
Thus, it would be interesting to focus on recurrent neural network algorithms to get better insights of the application