An online algorithm for separating sparse and low-dimensional signal sequences from their sum, and its applications in video processing

Guo, Han
Journal Title
Journal ISSN
Volume Title
Source URI
Research Projects
Organizational Units
Journal Issue

In signal processing, ``low-rank + sparse'' is an important assumption when separating two signals from their sum. Many applications, e.g., video foreground/background separation are well-formulated by this assumption. In this work, with the ``low-rank + sparse'' assumption, we design and evaluate an online algorithm, called practical recursive projected compressive sensing (prac-ReProCS) for recovering a time sequence of sparse vectors St and a time sequence of dense vectors Lt from their sum, Mt = St + Lt, when the Lt's lie in a slowly changing low-dimensional subspace of the full space.

In the first part of this work (Chapter 1-5), we study and discuss the prac-ReProCS algorithm, the practical version of the original ReProCS algorithm. We apply prac-ReProCS to a key application -- video layering, where the goal is to separate a video sequence into a slowly changing background sequence and a sparse foreground sequence that consists of one or more moving regions/objects on-the-fly. Via experiments we show that prac-ReProCS has significantly better performance compared with other state-of-the-art robust-pca methods when applied to video foreground-background separation.

In the second part of this work (Chapter 6), we study the problem of video denoising. We apply prac-ReProCS to video denoising as a preprocessing step. We develop a novel approach to video denoising that is based on the idea that many noisy or corrupted videos can be split into three parts -- the ``low-rank laye'', the ``sparse layer'' and a small residual which is small and bounded. We show using extensive experiments, layering-then-denoising is effective, especially for long videos with small-sized images that those corrupted by general large variance noise or by large sparse noise, e.g., salt-and-pepper noise.

In the last part of this work (Chapter 7), we discuss an independent problem called logo detection and propose a future research direction where prac-ReProCS can be combined with deep learning solutions.

compressive sensing, low-rank, robust PCA, sparse recovery, video denoising, video layering