Deep learning-based CNC milling tool wear stage estimation with multi-signal analysis

▪ The convolutional neural network (CNN) was performed to detect the wear stages of the milling tool.


Introduction
Industrial condition monitoring systems are developing in parallel with advancing technology and artificial intelligence applications. The effective use of condition monitoring and machine learning systems is increasing in machines or production facilities where continuous, uninterrupted, and error-free operation is desired [6]. Monitoring milling tool wear has become day-by-day important in smart manufacturing systems to enhance product reliability and efficiency. During the machining process, damaged or worn tools can result in a poor surface finish on the workpiece and even product wastage. Therefore, tool wear is monitored in real-time in smart manufacturing. To prevent the negative impact of tool damage on the machining process, online monitoring systems can be used [46].
The tool condition changes depending on factors such as the speed of cutting, cutting depth, the material of the workpiece, and the geometrical properties of the tool. Taking these parameters into account, the tool wear stage can be diagnosed in real-time using measurements such as cutting force, vibration signal, acoustic emission signal, and spindle motor current signal [45]. Vision-based techniques such as the contactless optical method, laser scattering pattern, scanning electron microscopy, and optoelectronic imaging are also used in the investigation and analysis of more complex and unpredictable tool faults [44]. Sensors used for measurements such as accelerometers, ultrasonic sensors, and current sensors are critical components of data acquisition and tool condition monitoring systems. Time, frequency, and time-frequency domain features are obtained from the signals acquired from these sensors, and these features are utilized in the analysis, classification, and estimation of the tool wear [41]. With different approaches in time or frequency domain analysis of signals, features can be extracted manually or automatically from experimental data and used in different decision-making algorithms to estimate tool conditions [16]. Determining the decision-making algorithm is one of the essential phases in the condition monitoring of the tool. Algorithms such as Artificial Neural Networks (ANNs) [7], Long Short-Term Memory Networks (LSTMs) [63], Support Vector Machines (SVMs) [37,65], Gaussian Process Regression (GPR) [34], Decision Trees [33], Fuzzy Logic [3,13], Genetic Algorithm (GA) [53], Hidden Markov Model (HMM) [36], and Adaptive Network Based Fuzzy Inference Systems (ANFIS) [56] have been used to estimate tool wear.
There are also deep learning-based methods in which features are automatically extracted from signals or images, such as Recurrent Neural Networks (RNNs), Deep Multi-layer Perceptron, Deep Reinforcement Learning, and Convolutional Neural Networks (CNNs) [48]. In recent years, the amount of studies on the application of CNN in tool wear estimation has increased [55]. Liu et al. [39] conducted a new CNNtransformer neural network model to obtain a more suitable tool wear estimation. In this model, they used the transformer model and convolutional neural networks together to obtain condition monitoring data such as shear force in parallel. Yin et al. [60] performed multi-sensor-based tool wear detection using onedimensional CNN and deep generalized canonical correlation analysis. Experiments showed that their performed approach can acquire sufficient accuracy and real-time implementation.
Terrazas et al. [51] presented an online tool wear estimation method by using condition monitoring for dry milling of steel.
In their study, they preferred the CNN approach to determine flank wear by using cutting force measurements. Cao et al. [12] conducted a new intelligent system for tool wear condition monitoring by utilizing spindle vibration signals. In this system, they combined wavelet frames with CNN. They utilized CNN to apply a deep learning approach to 2D vibration images and observed that with the integration of wavelet frames and CNN, tool wear stages can be effectively diagnosed. Wu et al. [57] conducted a CNN tool wear estimation model by using an image dataset and utilized a convolutional autoencoder to pre-train the network model. They also used the backpropagation algorithm First, image representations of vibration signals were obtained by using STFT. These images were then utilized for feature extraction to estimate tool wear automatically by using CNN.
Dai et al. [18] performed a CNN-based monitoring method to recognize intermediate anomalies and estimate tool wear in aerospace-related multi-stage manufacturing processes. Using their proposed approach, they expanded the criteria for evaluating anomalous conditions for practical applications and increased recognition stability through features. Cao et al. [11] introduced a powerful milling tool wear estimation system by using two-dimensional CNN and derived wavelet frames. With their proposed methodology, they achieved sufficient recognition accuracy and confirmed that the derived wavelet frames were effective. Duan et al. [19] [38] enhanced the deep learning regression approach to estimate tool wear through features obtained from 2D visual data of the workpiece's surface. They compared the models developed based on CNN and deep neural networks for predictive accuracy. Ambadekar and Choudhari [5] proposed an estimation method to estimate the flank wear of the cutting tool with CNN.
They used carbide inserts as cutting tools in their work and carried out the experiments under dry conditions. To detect the development of flank wear, visual data of the cutting tool and workpiece were acquired at constant intervals by utilizing a microscope. Images collected in one of three wear classes, low, medium, and high, were used as inputs to the CNN tool monitoring model. Bergs et al. [9] investigated a deep learning method for 2D visual data processing to determine tool wear conditions. Accordingly, they trained a CNN to classify the cutting tool wear stages. As a result of the evaluation they made with the test dataset, they reached a satisfactory result. Huang et al. [28] conducted a new tool wear estimation approach by using multi-domain feature fusion with CNN. They showed that with this method, the low prediction accuracy seen in manual feature fusion can be avoided and the tool state can be predicted effectively. Huang et al. [29] used reshaped time series signals in another study and presented a multisensory tool wear estimation approach by using CNN. Xu et al. [58] improved a tool wear estimation system by using a deep learning method for industrial applications. They performed multiscale feature fusion with CNN and improved the prediction results. Cooper et al. [17] conducted a CNN-based tool wear estimation model for vertical machining workings by utilizing acoustic emission signals. Ma et al. [40] conducted the mechanism of tool wear in the milling operations of a titanium alloy. They developed two new tool wear estimation models by using deep learning utilizing a convolutional bidirectional long short-term memory network and a convolutional bidirectional gated recurrent unit.
Huang and Lee [26] presented a study in which they estimated the tool wear formation and roughness of the surface by using vibration signals and sound signals with deep learning and sensor fusion approaches. The realized design was used for online condition monitoring of the tool via an alarm. Zhou et al.  [4]. Deep learning mostly needs big data and training time is longer than conventional machine learning algorithms. However, it is thought that these disadvantages will be overcome with the development of highperformance computers in the future [59,62]. In this study, different CNN models were trained for tool wear stage estimation using vibration data, acoustic emission data, and motor current data in The Nasa Ames Milling Dataset [21]. In the proposed method, STFT was applied to all signal data and 2D spectrograms were obtained. The tool wear stages are designated as the initial wear stage (IW), steady state wear stage (SSW), and accelerated wear stage (AW). Unlike the literature, CNN models with different architectures (GoogleNet, ResNet-50, AlexNet, and EfficientNet-B0) were trained with spectrograms, used for tool wear stage estimation, and comparatively examined in terms of complexity, training time, testing, and verification performance. Different CNN models were also trained for tool wear stage estimation using cutting force data, vibration data, and acoustic emission data from the 2010 PHM Data Challenge Dataset.

Tool Wear Mechanism
During milling, the forces and temperature caused by the surface deformation and friction between the tool and the workpiece directly affect the milling tool's life [24,43]. The milling tool is also affected by chemical reactions occurring on the contact surfaces. As a result, tool wear gradually develops due to mechanical (adhesion, abrasion, fatigue, plastic deformation, etc.) thermal (thermo-mechanical), and chemical (diffusion, oxidation, etc.) factors [47,50].
In machining operations with a worn tool, cutting forces increase, the surface quality of the workpiece decreases and it becomes difficult to manufacture within tolerances.
Consequently, it is required to detect cutting tool wear on time and replace it. Wear types such as flank wear, crater wear, groove wear, notch wear, and nose wear can take place on the cutting tool. The stage of flank wear (VB) is an important indicator for monitoring tool condition and is the best parameter for replacement decisions [52].
The change of tool wear over time can be considered a continuous function or it can be evaluated as stages. For this reason, in many searches in the literature, tool flank wear is evaluated in stages as initial wear (IW), steady state wear (SSW), and accelerated wear (AW) [14,42]. The development of these stages of the milling tool is not an accidental process. Accordingly, the first region is the IW stage where wear appears. The second region is the SSW stage, in which wear progresses at a uniform rate. The third region, in which wear takes place at a gradually growing rate, is the AW stage [49]. for each experiment [21]. Details on feed, depth of cut, workpiece materials, and measurement sensors in the experimental studies are presented in Table 1. The details of the data acquisition system are given in Fig. 2. Fig. 2. Details of data acquisition system [21].
The signals from vibration sensors are amplified, filtered, and preprocessed before entering the computer. The signals from acoustic emission sensors are amplified and preprocessed before entering the computer for data acquisition. The signals from the spindle motor current sensors are sent to the computer without being processed [21]. More information about the other constituents of the experimental setup and the data set can be found in the cited reference.

The 2010 PHM Data Challenge Dataset and Experimental System
The 2010 PHM Data Challenge is another dataset used to validate the methods proposed in this study [1]. The 2010 PHM Data Challenge Dataset is derived from experimental studies performed under consistent operating conditions. The details of the experimental system can be seen in Fig. 3. A three-flute ball nose tungsten carbide milling cutter was used in the experiments. The working parameters were set as given in Table 2.   Signals were acquired using 7 channels, and flank wear was assigned as the target value of these signals [1,54].

Time-Frequency Signal Analysis
In this study, time-frequency domain analysis of different source signals acquired from the milling experiment system for tool wear stage estimation was performed. So, Short Time Fourier Transform (STFT) was applied to vibration, acoustic emission, and motor current signals, and 2D spectrograms were obtained for different operating conditions. After the spectrograms of each signal type were recorded as 2D data, they were used as a dataset for training the CNN models. STFT is performed by dividing the signal into short segments and applying the Fourier transform for each segment [30]. Fig. 4 shows the STFT of a time domain signal. In the case of discrete time, STFT can be calculated as in Eq.
(2). Here, the discrete signal is represented by [ ], and the window function is represented by ( ). The discrete-time intervals are represented by , and . Accordingly, the 2D spectrogram of the STFT function can be obtained by using Eq.
(3) [32].  shares its weights, the optimization time is shortened and the complexity of CNN is decreased [59].
If the input to the convolution layer is considered to be ∈ , the layer output is calculated as in Eq. (4). Here, and demonstrate the input data dimensions, * represents the convolution operator, represents the th feature map of the convolution layer, represents the input data matrix, represents the weight matrix of the th filter of the actual layer, represents the th bias, and demonstrates the nonlinear activation function implemented to the result [31].
The pooling layers follow the convolution layers and decrease the dimensions of the network features and network parameters by subsampling. The maximum pooling is the most preferred function in CNN for calculating activation statistics.
The maximum pooling activation function can be given as in Eq. (5). Here, demonstrates the pooling block dimension, and ∈ , represents the pooling layer output [31].

Tool Wear Stage Estimation with Modified Pre-trained CNNs
Pre-trained CNN models can be modified and used for new regression and classification problems. Thus, the time and effort required to develop and train an entirely new network become less [25]. Therefore, in this study, pre-trained CNNs were used for tool wear stage classification. The specifications for these CNNs are given in Fig. 6 and Table 3.

Loss function Cross Entropy Cross Entropy Cross Entropy Cross Entropy
The amount of data used in training CNNs is usually quite large. However, it is also possible to train CNNs with a relatively limited number of 2D data. In the case of using a limited number of 2D data, CNN training performance can be increased by fine-tuning the hyperparameters of the network and increasing the resolution of the 2D data [30,59]. To enhance the performance of the pre-trained CNNs utilized in this study, firstly, the network hyperparameters were updated with finetuning, and data augmentation was applied to the 2D spectrograms. Details on the training parameters of pre-trained CNNs can be seen in Table 4. In addition, 50% of the spectrograms of the signals were used for training the networks.
30% of the remaining spectrograms were used for validation, and 20% for testing.  Finally, the trained networks were tested.  The difference in parameters such as feed rate and depth of cut given in Table 5 and Table 6 is due to the different test conditions in which two separate datasets were obtained [1, 21,54]. In the test in Table 5, the feed rate of the device was set at different speeds, while in the test in Table 6, the feed rate of the device was configured at a constant speed.

Models for The NASA Ames Milling Dataset
In this study, milling tool wear estimation was performed using pre-trained CNN models. In the training of the models, spectrogram images obtained from vibration, acoustic emission, and motor current signal data were utilized. In the training processes of CNNs, validation accuracies and loss values were calculated, weights of the networks were updated and the results of the models were also analyzed comparatively. Finally, the tool wear estimation models were tested with the reserved data and the performances of the models were compared with each other. In addition, CNN models were trained with different numbers of spectrograms and the results were analyzed.      Each of the CNN models, whose validation and test results are given in Table 7, Fig. 8, Fig. 9, and Fig. 10, was developed based on a single signal source. In this study, CNN models in which spectrograms of vibration signals, motor current signals, and acoustic emission signals are used together in the data set in order to determine the milling wear stage have also been developed. Table 9 indicates the validation and test performances of the CNN models acquired by using the spectrograms of all signals. In addition, Fig. 11 shows the training process and loss variation during the training of these models. When Table 9 and Fig. 11 are investigated together, it can be seen that CNN models with more complex architecture have higher validation accuracy and test success. Additively, the training times of these models are longer than the CNN models with simpler architecture. Considering the results obtained from CNN models using a single signal source, the use of spectrograms obtained from different signal sources together in the same data set adversely affected the model performance.

Models for The 2010 PHM Data Challenge Dataset
In this section, milling tool wear was estimated using pretrained CNN models using The 2010 PHM Data Challenge Dataset. In the training of the models, spectrogram images obtained from vibration, acoustic emission, and shear force data were used. In the training periods of CNNs, validation accuracies and loss values were calculated, the weights of the networks were updated and the results of the models were analyzed comparatively. Finally, the tool wear prediction models were tested with the separated data, and the performances of the models were compared with each other.  Additionally, if different signal ranges are to be examined in detail for further analysis, a cutoff should be applied.
Vibration, acoustic, AC, and DC signals from the spindle motor were captured using specialized sensors. However, frequency calculations based on spindle speed were not performed in our study. Rather, we obtained spectrograms of the vibration, acoustic, and motor current signals and used them to detect tool wear stage. By analyzing the frequency content of the signals over time, we were able to identify changes in spectral characteristics that corresponded to changes in cutting conditions and tool wear. While frequency calculations based on spindle speed could have provided additional information, our approach of using spectrograms allowed us to capture more comprehensive information about the tool wear process and enabled us to develop a robust tool wear stage detection method that can be applied to various machining conditions.
In our study, we did not perform frequency calculations from the radial runout of individual cutter blades. However, we acknowledge that radial runout has a significant impact on the wear of individual tool blades. Our study focused on analyzing the signals from the sensors and using spectrograms to detect tool wear stage. Future works can explore the relationship between the radial runout of the cutting blades and the corresponding frequency content of the signals, which can provide useful information regarding the tool wear process.

Conclusions
In this study, a method based on deep learning and different source signals, which takes into account the wear stages holistically, is proposed for the timely and accurate detection of