Home > Software

Removing Atmospheric Turbulence via Space-Invariant Deconvolution

To correct geometric distortion and reduce space and time-varying blur, a new approach is proposed in this paper capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach reduces the space and time-varying deblurring problem to a shift invariant one. It first registers each frame to suppress geometric deformation through B-spline based non-rigid registration. Next, a temporal regression process is carried out to produce an image from the registered frames, which can be viewed as being convolved with a space invariant near-diffraction-limited blur. Finally, a blind deconvolution algorithm is implemented to deblur the fused image, generating a final output. Experiments using real data illustrate that this approach can effectively alleviate blur and distortions, recover details of the scene and significantly improve visual quality. You can download the code here .

Robust Multichannel Blind Deconvolution via Fast Alternating Minimization

Blind deconvolution, which comprises simultaneous blur and image estimation, is a strongly ill-posed problem. It is by now well-known that if multiple images of the same scene are acquired, this multichannel blind deconvolution problem is better posed and allows of blur estimation directly from the degrade images. We improve the multichannel idea by adding robustness to noise and stability in the case of large blurs or if the blur size is vastly overestimated. We formulate blind deconvolution as a L1-regularized optimization problem and seek a solution by alternately optimizing with respect to the image and with respect to blurs. Each optimization step is converted to a constrained problem by variable splitting and then addressed with an augmented Lagrangian method, which permits simple and fast implementation in the Fourier domain. Our current implementation can also perform superresolution. The algorithm is implemented in MATLAB (requires Image Processing Toolbox). Instructions on how to set parameters are in parameters.m. You can download the code here .

Restoration for Weakly Blurred and Strongly Noisy Images

We present an adaptive sharpening algorithm for restoration of an image which has been corrupted by mild blur, and strong noise. Most existing adaptive sharpening algorithms can not handle strong noise well due to the intrinsic contradiction between sharpening and denoising. To solve this problem we propose an algorithm that is capable of capturing local image structure and sharpness, and adjusting sharpening accordingly so that it effectively combines denoising and sharpening together without either noise magnification or over-sharpening artifacts. It also uses structure information from the luminance channel to remove artifacts in the chrominance channels. The file Example.m provides a demonstration of the code .

Q: A Powerful No-reference Image Content Metric with Application to Parameter Optimization

Implementation of our no-reference image content metric Q based on singular value decomposition of local image gradients. It correlates well with subjective quality evaluations for both blur and random noise distortions. The metric can be used for parameter optimization of any image denoising algorithm in an unsupervised fashion. The file Example.m provides a demonstration of the code.

Training-free, Generic Object Detection using Locally Adaptive Regression Kernels (LARK)

We present a generic detection/localization algorithm capable of searching for a visual object of interest without training. The proposed method operates using a single example of an object of interest to find similar matches; does not require prior knowledge (learning) about objects being sought; and does not require any pre-processing step or segmentation of a target image. Our method is based on the computation of locally adaptive regression kernels as descriptors from a query, which measure the likeness of a pixel to its surroundings. Salient features are extracted from said descriptors and compared against analogous features from the target image. This comparison is done using a matrix generalization of the cosine similarity measure. The algorithm yields a scalar resemblance map, indicating the likelihood of similarity between the query and all patches in the target image.

Action Recognition From One Example

We present a novel human action recognition method based on space-time locally adaptive regression kernels and the matrix cosine similarity measure. The proposed method operates using a single example (e.g., short video clip) of an action of interest to find similar matches. It does not require prior knowledge (learning) about actions being sought; and does not require foreground/background segmentation, or any motion estimation or tracking. Our method is based on the computation of the so-called local steering kernels as space-time descriptors from a query video, which measure the likeness of a voxel to its surroundings. Salient features are extracted from said descriptors and compared against analogous features from the target video. This comparison is done using a matrix generalization of the cosine similarity measure. The algorithm yields a scalar resemblance volume with each voxel here, indicating the likelihood of similarity between the query video and all cubes in the target video. By employing nonparametric significance tests and non-maxima suppression, we detect the presence and location of actions similar to the given query video. High performance is demonstrated on the challenging set of action data indicating successful detection of actions in the presence of fast motion, different contexts and even when multiple complex actions occur simultaneously within the field of view of the camera.

Fundamental Limits of Image Denoising

Image denoising has been well-studied problem where considerable performance improvement has been achieved in the last decade. Yet, surprisingly, the state-of-the-art has not been considerably improved over the last few years, even though many methods of widely varying approaches have been proposed. In this work we investigate the fundamental limits to denoising a given image. The Matlab toolbox provided implements our method of estimating the bounds assuming the noise to be additive white Gaussian. Example scripts provided illustrate how the software can be used to calculate the MSE bounds of denoising any arbitrary image.

Clustering-based Denoising using Locally Learned Dictionaries (K-LLD)

K-LLD is a patch-based framework for denoising which generalizes kernel regression-based denoising methods through the use of locally learned dictionaries that best represent a group of patches sharing a common geometric structure. The denoising framework consists of three steps - clustering, dictionary selection and coefficient calculation stages. The software provided implements the method in Matlab. Example scripts are provided that explain how the software can be used to perform denoising.

Visual Saliency by Self-Resemblance

We present a novel unified framework for both static and space-time saliency detection. Our method is a bottom-up approach and computes so-called locally adaptive regression kernels (LARK) from the given image which measure the likeness of a pixel to its surroundings. Visual saliency is then computed using the said ``self-resemblance" measure. The framework results in a saliency map where each pixel (or voxel) indicates the statistical likelihood of saliency of a feature matrix given its surrounding feature matrices. State of the art performance is demonstrated on commonly used human eye fixation data. The included demonstration files (demo*.m) provide the easiest way to learn how to use the code.

Superresolution and Demosaicing

The main function of this software tool is the implementation of several superresolution techniques in our published work, and the comparison of these to other work. The techniques implemented cover robust methods, dynamic color superresolution methods, and simultaneous demosaicing and resolution enhancement.
Some of the test superresolution datasets used in our papers.

Kernel Regression for Image Processing and Reconstruction

Kernel regression (KR) is a universally applicable tool. For example, it is capable of denoising, interpolation (both regularly and irregularly-sampled data), fusion, super-resolution, and more. This software package implements our data-adaptive version of KR for two-dimensional data, and the example scripts show the effectiveness of our approach in image processing and reconstruction.

3-D Kernel Regression for Super-resolution without Explicit Subpixel Motion estimation

The need for precise (subpixel accuracy) motion estimates in conventional super-resolution has limited its applicability to only video sequences with relatively simple motions such as global translational or affine displacements. In this work, we introduce a novel framework for adaptive enhancement and spatiotemporal upscaling of videos containing complex transitions without explicit need for subpixel motion estimation.

Local Orientation Estimation

Based on the work reported in our paper, the objective of this Matlab-based software is to produce a dense orientation field (unit vector field orthogonal to the gradient direction) that defines the local direction of edges and features in an image. This method can be used to produced vector fields from noisy fingerprint images, which can then be used to extract features for matching and retrieval. The method can also be used as a preprocessor for locally adaptive image filtering methods such as bandelets, and steering kernel regression (see above).

Regularized Kernel Regression-Based Deblurring (AKTV)

In this work, we extended the use of kernel regression for deblurring applications. In some earlier examples in the literature, such non-parametric deblurring was sub-optimally performed in two sequential steps, namely, denoising followed by deblurring. In contrast, our optimal solution jointly denoises and deblurs images. The proposed algorithm takes advantage of an effective and novel image prior (Adaptive Kernel Total Variation -- AKTV) that generalizes some of the most popular regularization techniques in the literature.