Lab for Mathematical Methods in Computer Vision and Machine Learning

Head: Prof. Dr. Martin Storath

The lab develops mathematical methods and fast algorithms for image and data processing. The methods include non-smooth variational approaches, non-convex optimization in linear and non-linear spaces, complex wavelet transforms, and machine learning. Fields of application include segmentation, regularization of inverse problems in imaging, and robust signal estimation.

Optimal partitioning of images. Image partitioning is often modeled as an approximation problem by piecewise constant functions, whose discontinuities describe the segmentation curves; this is known as Potts model or piecewise constant Mumford-Shah model. Finding solutions of these problems in reasonable time is challenging as they are typically NP hard problems. The lab develops new fast algorithms for such problems.

A color image - Caps hanging on a wall. — A color image.

The same picture - partitioned by the computer — Optimal partitioning using the piecewise constant Mumford-Shah/Potts model

Discontinuity-preserving image reconstruction. It is often desirable to reconstruct an image from indirect measurements from as few measurements as possible. In computed tomography, for example, restrictions on the acquisition geometry, on the acquisition time, or on the radiation dose may limit the available number of measurements. If the measurement process is highly undersampled, image details typically get lost. However, the macrostructures, i.e. the segments, can possibly be recovered. The lab develops methods for such situations.

(b) Regularized filtered backprojection (FBP).

(c) Segmentation of FBP result using graph cuts.

(d) Proposed joint reconstruction and segmentation.

Reconstruction and segmentation from highly undersampled computed tomography data (7 projection angles). The proposed joint reconstruction and segmentation algorithm (d) outperforms the two-stage process (b,c).

Mathematical methods in machine learning. Machine learning methods and, in particular, convolutional neural networks, are important tools for all types of computer vision problems. The lab focuses on two main setups: learning of optimal features for texture segmentation and developing convolutional neural networks for unsupervised image segmentation.

A texture mosaic of the Prague benchmark

Segmentation using the proposed unsupervised learning approach.

Harmonic analysis of complex wavelets transforms
Wavelet transform are well-established tools for signal and image processing:
They allow to decompose a signal or image into components of different scales, from small details to large structures. Complex-valued wavelet transforms have several advantages over the real-valued one, for example better higher robustness to noise and better invariance properties. The lab investigates such complex-valued wavelet transforms and it develops new methods that utilize the information gained by using those transforms.

Left: Real and imaginary part of a complex wavelet. Right: The proposed approach detects singularities of the test signal f (top) based on the phase angle of the wavelet coefficients, and it delineates step from cusp singularities by the resulting phase angle (bottom).

Open Source Software

Pottslab - Multilabel image segmentation (color/gray/multichannel) based on the Potts model (aka piecewise constant Mumford-Shah model)
DCEBE - Estimation of bolus arrival times for DCE-MRI signals
CircleMedianFilter - Fast median filter for circle-valued data, for example signals or images describing phase or orientation
L1TV - Denoising/reconstruction of piecewise constant signals using the L1TV model
MumfordShah2D - Algorithms for edge preserving smoothing based on the Mumford-Shah model
PALMS Image Partitioning - A New Parallel Algorithm for the Piecewise Affine-Linear Mumford-Shah Model