An Evaluation of Combinations of Lossy Compression and Change-Detection Approaches for Time-Series Data

  • Author:

    Gregor Hollmig, Matthias Horne, Simon Leimkühler, Frederik Schöll, Carsten Strunk, Adrian Englhardt, Pavel Efros, Erik Buchmann, Klemens Böhm
  • Source:

    Information Systems

  • Download pdf

    Abstract

    Today, time series of numerical data are ubiquitous, for instance in the Internet of Things. In such scenarios, it is often necessary to compress the data and to detect changes on it. More specifically, both methods are used in combination, i.e., data is lossily compressed and later decompressed, and then change detection takes place. There exists a broad variety of compression as well as of change-detection techniques. All this calls for a systematic comparison of different combinations of compression and change-detection techniques, for different data sets, together with recommendations on how the values of the various (typically non-linear) parameters should be chosen. This article is such an evaluation. Its design is not trivial, necessitating a number of decisions. We work out the details and the rationale behind our design choices. Next to other results, our study shows that the choice of combinations of change detection and compression algorithm and their parameterization does affect result quality significantly. Our evaluation also indicates that results are highly contingent on the nature of the data.

    Framework

    We implemented a framework for our evaluation, that can flexibly be adapted to various change detection techniques.
    You will easily be able to extend it by implementing your own compression or change-detection algorithms with the provided Interfaces IChangeDetectionAlgorithm and ICompressionAlgorithm.
    This framework also offers a user friendly experience with dynamic parameter input and all operations are intuitionally accessible.

    Next to the control interface there is a monitor interface which shows the most important results, such as the MILTON distance, the compression ratio, the detected changes before and after lossy transformation and their exact position.

    Download framework code

    Due to license restrictions we cannot provide all sources, therefore some algorithm implementations are missing.

    Algorithm Implementations

    In the context of our evaluation we implemented several change-detection and compression algorithms. Our implementations are written in MATLAB code. We provide the code for Wavelet, Online Kernel Change Detection, Change Finder and Event Detection

    Download algorithms

    Artificial Datasets

    In our evaluation we used five datasets of which two are artificial.

    We provide the code to generate them.

    Download datasets