Data mining process

Data Mining is the core of the more comprehensive process of knowledge dis-coveiy in data bases (KDD). However, the term data mining" is often used synonymously with KDD. KDD describes the process of extracting and storing data and also includes methods for data preparation such as data cleaning, data selection, and data transformation as well as evaluation, presentation, and visualization of the results after the data mining process. 

Higher quality of the resulting patterns The natural capabiUty of human beings to visually recognize patterns and relations can be used and leads to a more effective data mining process. 

Increased trust in pattern recognition The active user involvement in the data mining process can lead to a deeper understanding of the data and increases the trust in the resulting patterns. In contrast, "black box" systems often lead to a higher uncertainty, because the user usually does not know, in detail, what happened during the data analysis process. This may lead to a more difficult data interpretation and/or model prediction. 

G. H. John. Enhancements to the Data Mining Process, Ph.D thesis, Stanford University, 1997. 

Lastly, because the results obtained from the data mining process can be difficult to interpret, it is extremely useful for the results to be presented in a graphical form that allows the user to interact with both the data and the results. This allows the end user to further explore and better understand the results obtained. By being able to go from a... 

Selection of the data in the target database. The data that are stored in the primary source databases are often collected by different users using different automated methods and business rules. As a consequence, the quality of the data is not the same for all the records in the database and data will be contaminated. Depending on the goal of the data mining process and method, the data in the target database should be cleaned first and obvious inconsistencies between data points should be resolved. 

Evaluation and interpretation. A very important step in the data mining process is to evaluate and interpret the obtained patterns and models. This process is also known as the translation of the information from the data into knowledge. This knowledge can eventually guide the search for better and new patterns and/or models. 

Table 1 The Data Mining Process Employed by Lipinski et al. [9]...

In the data mining process, one uses the combination of methods and tools from three areas Statistics, machine... 

Data mining process takes place in four main stages Data Pre-processing, Exploratory Data Analysis, Data Selection, and Knowledge Discovery. 

The Data Mining process may be regarded as taking place in four main stages (Fig. 1) ... 

Knowledge Discovery is the main objective in Data Mining and many different technologies have been employed in this context. In the Data Mining Process we frequently need to iterate round the EDA, Data Selection, Knowledge Discovery part of the process, as once we discover some new knowledge, we often then want to go back to the data and look for new or more detailed patterns. 

When we carry out Data Mining, we are often working with large, possibly heterogeneous data. It therefore frequently happens that some of the data values are missing because data were not recorded in that case or perhaps was represented in a way that is not compatible with the remainder of the data. Nonetheless, we need to be able to carry out the Data Mining process as best we can. A number of techniques have been developed which can be used in such circumstances, as follows ... 

---