Sampling for Big Data: A Tutorial Conference Paper uri icon

abstract

  • One response to the proliferation of large datasets has been to develop ingenious ways to throw resources at the problem, using massive fault tolerant storage architectures, parallel and graphical computation models such as MapReduce, Pregel and Giraph. However, not all environments can support this scale of resources, and not all queries need an exact response. This motivates the use of sampling to generate summary datasets that support rapid queries, and prolong the useful life of the data in storage. To be effective, sampling must mediate the tensions between resource constraints, data characteristics, and the required query accuracy. The state-of-the-art in sampling goes far beyond simple uniform selection of elements, to maximize the usefulness of the resulting sample. This tutorial reviews progress in sample design for large datasets, including streaming and graph-structured data. Applications are discussed to sampling network traffic and social networks. 2014 Authors.

name of conference

  • Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining

published proceedings

  • PROCEEDINGS OF THE 20TH ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING (KDD'14)

author list (cited authors)

  • Cormode, G., & Duffield, N.

citation count

  • 36

complete list of authors

  • Cormode, Graham||Duffield, Nick

publication date

  • August 2014