Abstract

Research of data mining technology based on hadoop platform

Author(s): Dezhi An, Guangli Wu, Jun Lu

With the constant development and improvement of information technology, the Internet technology has become the essential ingredients of people's daily life in current days. Due to the traditional computer architectures can not meet the needs of the majority of people in front of deal with big data, the cloud computing has been come up with to provide an effective solution of handling and processing a huge data. Hadoop platform is a major project developed by the Apache Foundation, and is used in a cluster of general business computers commonly, And its most significant characteristics are of a super calculating power, flexible storage capacity and various scheduling capacity. On this basis, the data mining technology supported by Hadoop platform, after being deep processed on its model, has entered the development phase of the information age. The present paper, based on the research of Hadoop platform and in the study and application of the platform, makes an in-depth research on the algorithm. And finally a corresponding operation platform is set up, and a Hadoop version which can run well is offered, the purpose of which is to provide an effective basis for data mining personnel in the application of this platform.