(Click the screenshot below for a larger version)
This is a Matlab-based software package for resolution enhancement from video, developed at the Multi-Dimensional Signal Processing (MDSP) research lab at the University of California at Santa Cruz, led by Peyman Milanfar.
The main objective of this software tool is the implementation of several superresolution techniques. In particular, the techniques described in [1], [2], [3], and several references therein are included. The techniques implemented cover robust methods, dynamic color superresolution methods, and simultaneous demosaicing and resolution enhancement.
Some specific features of the software package are:
- As part of this software package, motion estimation is done automatically by the program, or independently estimated motion vectors may be provided by the user.
- The user is able to specify the region of interest to be processed.
- A basic tracking algorithm is incorporated in the program so that if only a certain part of the input images are important for the user (a car moving in a crowded street), this region can be tracked and another data sequence containing only that particular object is produced.
- The parameters of the imaging system (such as the point-spread function) may be specified by the user.
- The input image files may be given as .mat (Matlab data file) or .avi format.
- The output generated by the program can be .mat (Matlab data file) or .avi format.
- Producing color or grayscale output images are optional, given color input frames.
- For purposes of experimentation, the software package is capable of producing simulated video data for different imaging scenarios from a single high resolution input image, with user-controlled parameters.
Examples
Sample results illustrating the performance of our techniques can be viewed here:
User’s Manual
MDSP Resolution Enhancement Software User’s Manual, May 2004
Sample datasets
A webpage containing many of the test superresolution data-sets used in our papers is online.
References
- S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Advances and Challenges in Super-Resolution”, Invited Paper, International Journal of Imaging Systems and Technology, Special Issue on High Resolution Image Reconstruction, vol. 14, no. 2, pp. 47-57, 2004.
- S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and Robust Multi-frame Super-resolution”, IEEE Transactions on Image Processing, vol. 13, no. 10, pp. 1327-1344, October 2004.
- S. Farsiu, M. Elad, and P. Milanfar, “Multi-Frame Demosaicing and Super-Resolution of Color Images”, IEEE Trans. on Image Processing, vol. 15, no. 1, pp. 141-159, January 2006.
- S. Farsiu, M. Elad, and P. Milanfar, “Video-to-Video Dynamic Superresolution for Grayscale and Color Sequences”, EURASIP Journal of Applied Signal Processing, Special Issue on Superresolution Imaging, Volume 2006, Article ID 61859, Pages 1-15.
Our other relevant publications on this topic.
Download, Licensing and IP
The software package can be downloaded from here. Please note: The software runs on Matlab 7.1 or earlier. It is not supported on any later version of Matlab. This is experimental software. It is provided for noncommercial research purposes only. Use at your own risk. No warranty is implied by this distribution. Copyright © 2009 by University of California
If you are interested in commercial applications and development, please contact MotionDSP, Inc.
The following patents held by the University of California cover the operation of the software and related technology.
- US Patent 7,379,612, P. Milanfar, S. Farsiu, M. Elad, "Dynamic Reconstruction of High-Resolution Video from Color-Filtered Low-Resolution Video-to-Video Super-Resolution," issued 2008
- US Patent 7,412,107 P. Milanfar, S. Farsiu, M. Elad, "System and Method for Robust Multi-Frame Demosaicing and Color Super-Resolution," issued 2008
- US Patent 7,477,802, P. Milanfar, S. Farsiu, D. Robinson, M. Elad , "Robust reconstruction of high resolution grayscale images from a sequence of low resolution frames," issued Jan 2009.
Acknowledgements
The work implemented in this software was carried out in collaboration with Prof. Michael Elad of the Technion CS department. Sina Farsiu is the principal architect of the code, with significant contributions from Dirk Robinson, particularly on the motion estimation algorithms.
This work was supported in part by the National Science Foundation Grant CCR-9984246, US Air Force Grant F49620-03-01-0387, and by the National Science Foundation Science and Technology Center for Adaptive Optics, managed by the University of California at Santa Cruz under Cooperative Agreement No. AST-9876783.