Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure Conference Paper uri icon


  • This paper describes a measurement infrastructure used to collect detailed IP traffic measurements from an IP backbone. Usage, i.e, bytes transmitted, is determined from raw NetFlow records generated by the backbone routers. The amount of raw data is immense. Two types of data sampling in order to manage data volumes: (i) (packet) sampled NetFlow in the routers; (ii) size-dependent sampling of NetFlow records. Furthermore, dropping of NetFlow records in transmission can be regarded as an uncontrolled form of sampling. We show how to manage the trade-off between estimation accuracy and data volume. Firstly, we describe the sampling error that arises from all three types of sampling when estimating usage per traffic class: how it can be predicted from models and raw data, and how it can be estimated directly from the sampled data itself. Secondly, we show how to determined the usage of resources - bandwidth, computational cycle, storage - within the components of the infrastructure. These two sets of methods allow dimensioning of the measurement infrastructure in order to meet accuracy goals for usage estimation. Copyright 2003 ACM.

name of conference

  • Proceedings of the 2003 ACM SIGCOMM conference on Internet measurement - IMC '03

published proceedings

  • Proceedings of the 2003 ACM SIGCOMM conference on Internet measurement - IMC '03

author list (cited authors)

  • Duffield, N., & Lund, C.

citation count

  • 57

complete list of authors

  • Duffield, Nick||Lund, Carsten

publication date

  • January 2003