Kyoung Soo, P., Pai, V.S., Lee, K.-W. and Calo, S.B. (2006) Securing Web Service by Automatic Robot Detection. USENIX Annual Technical Conference, General Track, 255-260.

has been cited by the following article:

• TITLE: Agglomerative Approach for Identification and Elimination of Web Robots from Web Server Logs to Extract Knowledge about Actual Visitors

  AUTHORS: Dilip Singh Sisodia, Shrish Verma, Om Prakash Vyas

  KEYWORDS: Web Robots, Web Server Log Repositories, Ensemble Learning, Bagging, Boosting, and Voting, Actionable Knowledge, Usable Knowledge, Browsing Behavior, Genuine Visitors

  JOURNAL NAME: Journal of Data Analysis and Information Processing, Vol.3 No.1, April 27, 2015

  ABSTRACT: In this paper we investigate the effectiveness of ensemble-based learners for web robot session identification from web server logs. We also perform multi fold robot session labeling to improve the performance of learner. We conduct a comparative study for various ensemble methods (Bagging, Boosting, and Voting) with simple classifiers in perspective of classification. We also evaluate the effectiveness of these classifiers (both ensemble and simple) on five different data sets of varying session length. Presently the results of web server log analyzers are not very much reliable because the input log files are highly inflated by sessions of automated web traverse software’s, known as web robots. Presence of web robots access traffic entries in web server log repositories imposes a great challenge to extract any actionable and usable knowledge about browsing behavior of actual visitors. So web robots sessions need accurate and fast detection from web server log repositories to extract knowledge about genuine visitors and to produce correct results of log analyzers.