Novel network host vulnerability scanning method based on Hadoop platform

The invention provides a new network host vulnerability scanning method based on a Hadoop platform, which is characterized by comprising a data cleaner based on the Hadoop platform, which can quickly clean repeated, invalid and misreported data in a large number of scanning messages and finally store the data in a cluster in a distributed manner according to a format defined by an Hbase; a task scheduler based on a MapReduce engine can dynamically allocate computing resources, distribute tasks to idle computing resources and improve the reliability of the tasks, and wrong tasks can be re-issued to the idle resources through the task scheduler. According to the method, a large amount of repeated invalid data accumulated in one task detection result can be greatly reduced through cleaning processing of the cleaner, hardware resources such as a CPU and a memory of each node can be fully utilized, the advantages of a distributed cluster are fully played, and the scanning speed and the fault-tolerant rate are impr