CompactFlow: A Hybrid Binary Format for Network Flow Data - Information Security Theory and Practice
Conference Papers Year : 2020

CompactFlow: A Hybrid Binary Format for Network Flow Data

Michal Piskozub
  • Function : Author
  • PersonId : 1093945
Riccardo Spolaor
  • Function : Author
  • PersonId : 1093946
Ivan Martinovic
  • Function : Author
  • PersonId : 1042907

Abstract

Network traffic monitoring has become fundamental to obtaining insights about a network and its activities. This knowledge allows network administrators to detect anomalies, identify faulty hardware, and make informed decisions. The increase of the number of connected devices and the consequent volume of traffic poses a serious challenge to carrying out the task of network monitoring. Such a task requires techniques that process traffic in an efficient and timely manner. Moreover, it is crucial to be able to store network traffic for forensic purposes for as long a period of time as possible.In this paper, we propose CompactFlow, a hybrid binary format for efficient storage and processing of network flow data. Our solution offers a trade-off between the space required and query performance via an optimized binary representation of flow records and optional indexing. We experimentally assess the efficiency of CompactFlow by comparing it to a wide range of binary flow storage formats. We show that CompactFlow format improves the state of the art by reducing the size required to store network flows by more than 24%.
Fichier principal
Vignette du fichier
492809_1_En_12_Chapter.pdf (275.97 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03173900 , version 1 (18-03-2021)

Licence

Identifiers

Cite

Michal Piskozub, Riccardo Spolaor, Ivan Martinovic. CompactFlow: A Hybrid Binary Format for Network Flow Data. 13th IFIP International Conference on Information Security Theory and Practice (WISTP), Dec 2019, Paris, France. pp.185-201, ⟨10.1007/978-3-030-41702-4_12⟩. ⟨hal-03173900⟩
50 View
64 Download

Altmetric

Share

More