Characterizing web spam using content and HTTP session analysis Conference Paper uri icon

abstract

  • Web spam research has been hampered by a lack of statistically significant collections. In this paper, we perform the first large-scale characterization of web spam using content and HTTP session analysis techniques on the Webb Spam Corpus-a collection of about 350,000 web spam pages. Our content analysis results are consistent with the hypothesis that web spam pages are different from normal web pages, showing far more duplication of physical content and URL redirections. An analysis of session information collected during the crawling of the Webb Spam Corpus shows significant concentration of hosting IP addresses in two narrow ranges as well as significant overlaps among session header values. These findings suggest that content and HTTP session analysis may contribute a great deal towards future efforts to automatically distinguish web spam pages from normal web pages.

name of conference

  • CEAS 2007 - The Fourth Conference on Email and Anti-Spam, 2-3 August 2007, Mountain View, California, USA

published proceedings

  • 4th Conference on Email and Anti-Spam, CEAS 2007

author list (cited authors)

  • Webb, S., Caverlee, J., & Pu, C.

complete list of authors

  • Webb, S||Caverlee, J||Pu, C

publication date

  • January 2007