@lama

Web Robot Detection - Preprocessing Web Logfiles for Robot Detection

, , and . New Developments in Classification and Data Analysis, (2005)

Abstract

Web usage mining has to face the problem that parts of the underlying logfiles are created by robots. While cooperative robots identify themselves and obey to the instructions of server owners not to access parts or all of the pages on the server, malignant robots may camouflage themselves and have to be detected by web robot scanning devices. We describe the methodology of robot detection and show that highly accurate tools can be applied to decide whether session data was generated by a robot or a human user. ER -

Links and resources

Tags