Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure
Conference Paper
Overview
Identity
Additional Document Info
Other
View All
Overview
abstract
This paper describes a measurement infrastructure used to collect detailed IP traffic measurements from an IP backbone. Usage, i.e, bytes transmitted, is determined from raw NetFlow records generated by the backbone routers. The amount of raw data is immense. Two types of data sampling in order to manage data volumes: (i) (packet) sampled NetFlow in the routers; (ii) size-dependent sampling of NetFlow records. Furthermore, dropping of NetFlow records in transmission can be regarded as an uncontrolled form of sampling. We show how to manage the trade-off between estimation accuracy and data volume. Firstly, we describe the sampling error that arises from all three types of sampling when estimating usage per traffic class: how it can be predicted from models and raw data, and how it can be estimated directly from the sampled data itself. Secondly, we show how to determined the usage of resources - bandwidth, computational cycle, storage - within the components of the infrastructure. These two sets of methods allow dimensioning of the measurement infrastructure in order to meet accuracy goals for usage estimation. Copyright 2003 ACM.
name of conference
Proceedings of the 2003 ACM SIGCOMM conference on Internet measurement - IMC '03
