TV-L¹ optical flow for vector valued images

The variational TV-L1 framework has become one of the most popular and successful approaches for calculating optical flow. One reason for the popularity is the very appealing properties of the two terms in the energy formulation of the problem, the robust L1-norm of the data fidelity term combined with the total variation (TV) regular- ization that smoothes the flow, but preserve strong discontinuities such as edges. Specifically the approach of Zach et al. [1] has provided a very clean and efficient algorithm for calculating TV-L1 optical flows between grayscale images. In this paper we propose a generalized algorithm that works on vector valued images, by means of a generalized projection step. We give examples of calculations of flows for a number of multi- dimensional constancy assumptions, e.g. gradient and RGB, and show how the developed methodology expands to any kind of vector valued images. The resulting algorithms have the same degree of parallelism as the case of one-dimensional images, and we have produced an efficient GPU implementation, that can take vector valued images with vectors of any dimension. Finally we demonstrate how these algorithms generally produce better flows than the original algorithm.