21.July Honolulu, Hawaii

NTIRE 2017

New Trends in Image Restoration and Enhancement workshop

and challenge on image super-resolution

in conjunction with CVPR 2017

Sponsors




Call for papers

Image restoration and image enhancement are key computer vision tasks, aiming at the restoration of degraded image content or the filling in of missing information. Recent years have witnessed an increased interest from the vision and graphics communities in these fundamental topics of research. Not only has there been a constantly growing flow of related papers, but also substantial progress has been achieved.

Each step forward eases the use of images by people or computers for the fulfillment of further tasks, with image restoration or enhancement serving as an important frontend. Not surprisingly then, there is an ever growing range of applications in fields such as surveillance, the automotive industry, electronics, remote sensing, or medical image analysis. The emergence and ubiquitous use of mobile and wearable devices offer another fertile ground for additional applications and faster methods.

This workshop aims to provide an overview of the new trends and advances in those areas. Moreover, it will offer an opportunity for academic and industrial attendees to interact and explore collaborations.

Papers addressing topics related to image restoration and enhancement are invited. The topics include, but are not limited to:

  • Image inpainting
  • Image deblurring
  • Image denoising
  • Image upsampling and super-resolution
  • Image filtering
  • Image dehazing
  • Demosaicing
  • Image enhancement: brightening, color adjustment, sharpening, etc.
  • Style transfer
  • Image generation and image hallucination
  • Image-quality assessment
  • Video restoration and enhancement
  • Hyperspectral imaging
  • Methods robust to changing weather conditions
  • Studies and applications of the above.

IMPORTANT! The competition announced its final results!

Jointly with NTIRE2017 there is the example-based single image super-resolution challenge. The authors of the top methods in each category will be invited to submit papers to NTIRE2017 workshop.

The authors of the top methods co-author the NTIRE 2017 challenge paper report:

bibtex:

@InProceedings{Timofte_2017_CVPR_Workshops,
author = {Timofte, Radu and Agustsson, Eirikur and Van Gool, Luc and Yang, Ming-Hsuan and Zhang, Lei and others},
title = {NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {July},
}

Contact:

Radu Timofte, radu.timofte@vision.ee.ethz.ch

Computer Vision Laboratory

ETH Zurich, Switzerland

NTIRE challenge on example-based single image super-resolution

In order to gauge the current state-of-the-art in example-based single-image super-resolution, to compare and to promote different solutions we are organizing an NTIRE challenge in conjunction with the CVPR 2017 conference. We propose a large DIV2K dataset with DIVerse 2K resolution images.

The challenge has 2 tracks:

  1. Track 1: bicubic uses the bicubic downscaling (Matlab imresize), one of the most common settings from the recent single-image super-resolution literature.
  2. Track 2: unknown assumes that the explicit forms for the degradation operators are unknown, only the training pairs of low and high images are available.

To learn more about the challenge, to participate in the challenge, and to access the newly collected DIV2K dataset with DIVerse 2K resolution images everybody is invited to register at the following links, accordingly:

The training data is made available to the registered participants.

The top ranked participants co-author the challenge paper report.

NTIRE 2017 Challenge report:

@InProceedings{Timofte_2017_CVPR_Workshops,
author = {Timofte, Radu and Agustsson, Eirikur and Yang, Ming-Hsuan and Zhang, Lei and Van Gool, Luc and others},
title = {NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {July},
year = {2017}
}

DIV2K dataset and study:

@InProceedings{Agustsson_2017_CVPR_Workshops,
author = {Agustsson, Eirikur and Timofte, Radu},
title = {NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {July},
year = {2017}
}

Important dates



Challenge Event Date (always 5PM Pacific Time)
Site online January 21, 2017
Release of train data (low-res and high-res images) and validation data (only low-res) February 14, 2017
Validation server online March 1, 2017
Final test data release (only low-res), validation data (high-res) released, validation server closed April 10, 2017
Test high-res results submission deadline April 17, 2017
Fact sheets submission deadline April 17, 2017
Code/executable submission deadline April 17, 2017
Final test results release to the participants April 24, 2017
Paper submission deadline for entries from the challenge May 4, 2017
Workshop Event Date (always 5PM Pacific Time)
Paper submission server online March 1, 2017
Paper submission deadline April 24, 2017 (extended!)
Paper submission deadline (only for methods from the challenge!) May 4, 2017
Decision notification May 8, 2017
Camera ready deadline May 19, 2017
Workshop day July 21, 2017

Submit



Instructions and Policies
Format and paper length

A paper submission has to be in English, in pdf format, and at most 8 pages (excluding references) in double column. The paper format must follow the same guidelines as for all CVPR 2017 submissions.
http://cvpr2017.thecvf.com/submission/main_conference/author_guidelines

Double-blind review policy

The review process is double blind. Authors do not know the names of the chair/reviewers of their papers. Reviewers do not know the names of the authors.

Dual submission policy

Dual submission is allowed with CVPR2017 main conference only. If a paper is submitted also to CVPR and accepted, the paper cannot be published both at the CVPR and the workshop.

Submission site

https://cmt3.research.microsoft.com/NTIRE2017

Proceedings

Accepted and presented papers will be published after the conference in CVPR Workshops proceedings together with the CVPR2017 main conference papers.

Author Kit

http://cvpr2017.thecvf.com/files/cvpr2017AuthorKit.zip
The author kit provides a LaTeX2e template for paper submissions. Please refer to the example egpaper_for_review.pdf for detailed formatting instructions.

People



Organizers

Radu Timofte

Radu Timofte obtained a PhD degree in Electrical Engineering at the KU Leuven, Belgium in 2013, the MSc at the Univ. of Eastern Finland in 2007, and the Dipl. Eng. at the Technical Univ. of Iasi, Romania in 2006. Currently, he is research group leader in the Computer Vision Lab, from the ETH Zurich, Switzerland. He serves as a reviewer for top journals (such as TPAMI, TIP, IJCV, TNNLS, TCSVT, CVIU, PRL) and conferences (ICCV, CVPR, ECCV, NIPS). His work received a best scientific paper award at ICPR 2012, the best paper award at CVVT workshop (ECCV 2012), the best paper award at ChaLearn LAP workshop (ICCV 2015) and his team won a number of challenges including traffic sign detection (IJCNN 2013) and apparent age estimation (ICCV 2015). He is co-founder of Merantix. His current research interests include sparse and collaborative representations, classification, deep learning, optical flow, image restoration and enhancement.

Eirikur Agustsson

Thanks Wenzhe!

Eirikur Agustsson received a MSc degree in Electrical Engineering and Information Technology from ETH Zurich and a double BSc degree in Mathematics and Electrical Engineering from the University of Iceland. Currently he is a Research Assistant and PhD student at ETH Zurich, under the supervision of Prof. Luc Van Gool. His main research interests include deep learning for regression & classification and super-resolution.

Ming-Hsuan Yang

Ming-Hsuan Yang received the PhD degree in Computer Science from University of Illinois at Urbana-Champaign. He is an associate professor in Electrical Engineering and Computer Science at University of California at Merced. He has published more than 120 papers in the field of computer vision. Yang serves as a program co-chair of ACCV 2014, general co-chair of ACCV 2016, and program co-chair of ICCV 2019. He serves as an editor for PAMI, IJCV, CVIU, IVC and JAIR. His research interests include object detection, tracking, recognition, image deblurring, super resolution, saliency detection, and image/video segmentation.

Lei Zhang

Lei Zhang (M’04, SM’14) received his B.Sc. degree in 1995 from Shenyang Institute of Aeronautical Engineering, Shenyang, P.R. China, and M.Sc. and Ph.D degrees in Control Theory and Engineering from Northwestern Polytechnical University, Xi’an, P.R. China, respectively in 1998 and 2001, respectively. From 2001 to 2002, he was a research associate in the Department of Computing, The Hong Kong Polytechnic University. From January 2003 to January 2006 he worked as a Postdoctoral Fellow in the Department of Electrical and Computer Engineering, McMaster University, Canada. In 2006, he joined the Department of Computing, The Hong Kong Polytechnic University, as an Assistant Professor. Since July 2015, he has been a Full Professor in the same department. His research interests include Computer Vision, Pattern Recognition, Image and Video Processing, and Biometrics, etc. Prof. Zhang has published more than 200 papers in those areas. As of 2016, his publications have been cited more than 20,000 times in the literature. Prof. Zhang is an Associate Editor of IEEE Trans. on Image Processing, SIAM Journal of Imaging Sciences and Image and Vision Computing, etc. He is a "Highly Cited Researcher" selected by Thomson Reuters.

Luc Van Gool

Luc Van Gool received a degree in electro-mechanical engineering at the Katholieke Universiteit Leuven in 1981. Currently, he is a full professor for Computer Vision at the ETH in Zurich and the Katholieke Universiteit Leuven in Belgium. He leads research and teaches at both places. He has authored over 200 papers in his field. Luc Van Gool has been a program committee member of several, major computer vision conferences (e.g. Program Chair ICCV'05, Beijing, and General Chair of ICCV'11, Barcelona, and of ECCV'14, Zurich). His main interests include 3D reconstruction and modeling, object recognition, and tracking and gesture analysis. He received several Best Paper awards (eg. David Marr Prize '98, Best Paper CVPR'07, Tsuji Outstanding Paper Award ACCV'09, Best Vision Paper ICRA'09). He is a co-founder of 10 spin-off companies. In 2015 he received the 5-yearly Excellence Award in Applied Sciences by the Flemish Fund for Scientific Research. He is the holder of an ERC Advanced Grant (VarCity).

Program committee

Invited Talks



Peyman Milanfar

Title: Regularization by Denoising - "The little engine that could"

Abstract: Image denoising is the most fundamental problem in image enhancement, and it is largely solved: It has reached impressive heights in performance and quality -- almost as good as it can ever get. But interestingly, it turns out that we can solve many other problems using the image denoising "engine". I will describe the Regularization by Denoising (RED) framework: using the denoising engine in defining the regularization of any inverse problem. The idea is to define an explicit image-adaptive regularization functional directly using a high performance denoiser. Surprisingly, the resulting regularizer is guaranteed to be convex, and the overall objective functional is explicit, clear and well-defined. With complete flexibility to choose the iterative optimization procedure for minimizing this functional, RED is capable of incorporating any image denoising algorithm as a regularizer, treat general inverse problems very effectively, and is guaranteed to converge to the globally optimal result. I will show examples of its utility, including state-of-the-art results in image deblurring and super-resolution problems.

Bio: Peyman leads the Computational Imaging/Image Processing team in Google Research. Prior to this, he was a Professor of Electrical Engineering at UC Santa Cruz from 1999-2014, where he is now a visiting faculty. He was Associate Dean for Research at the School of Engineering from 2010-12. From 2012-2014 he was on leave at Google-x, where he helped develop the imaging pipeline for Google Glass. Peyman received his undergraduate education in electrical engineering and mathematics from the University of California, Berkeley, and the MS and PhD degrees in electrical engineering from the Massachusetts Institute of Technology. He holds 11 US patents, several of which are commercially licensed. He founded MotionDSP in 2005. He has been keynote speaker at numerous technical conferences including Picture Coding Symposium (PCS), SIAM Imaging Sciences, SPIE, and the International Conference on Multimedia (ICME). Along with his students, has won several best paper awards from the IEEE Signal Processing Society. He is a Fellow of the IEEE "for contributions to inverse problems and super-resolution in imaging.

Wenzhe Shi & Christian Ledig

Title: Neural networks for image and video super resolution

Abstract: The most important considerations when applying neural networks for super resolution (SR) are the training data, the network architecture and the objective function(s). In this talk we will present our recent works on both network architecture and objective functions for SR. In 2016 we developed an innovative sub-pixel convolution layer which greatly increases the speed of using neural networks for super resolution. By leveraging the speed up, it is also now possible to efficiently train deep residual network for the task. Deep residual networks current provides the most accurate of reconstructions in terms of peak signal-to-noise ratios. However, they are limited by the pixel-wise objective functions (e.g MSE, L2 distance, etc) used in training and struggle to resolve all the high-frequency details, so results are still perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. By redefining the objective functions we achieved a step-change in the perceived quality of super resolved images in our more recent work with GANs. Finally, we will briefly comment on the link between image, video super-resolution and compression and discuss what will be the challenges moving forward.

Bio: Wenzhe Shi works at Magic Pony of Twitter as a computer vision research lead. He received his Ph.D. training under Prof. Daniel Rueckert in the Biomedical Image Analysis group within Imperial College London from 2009 to 2012 where he stayed as a research associate from 2012 to 2014. His research interests includes image/video super resolution, compression, frame synthesis, motion estimation and segmentation.
Christian Ledig (@LedigChr) is a Computer Vision Researcher at Magic Pony, Twitter. He received a PhD from Imperial College London in 2015, where he was working on medical image analysis under the supervision of Prof. Daniel Rueckert. His current research focuses on deep learning approaches and generative models, in particular generative adversarial networks, for image and video super-resolution.

Liang Lin

Title: Attention-aware Face Hallucination via Deep Reinforcement Learning

Abstract: Face hallucination is a domain-specific super-resolution problem with the goal to generate high-resolution (HR) faces from low-resolution (LR) input images. In contrast to existing methods that often learn a single patch-to-patch mapping from LR to HR images and are regardless of the contextual interdependency between patches, we study a novel Attention-aware Face Hallucination (Attention-FH) framework which resorts to deep reinforcement learning for sequentially discovering attended patches and then performing the facial part enhancement by fully exploiting the global interdependency of the image. The Attention-FH approach jointly learns the recurrent policy network and local enhancement network through maximizing the long-term reward that reflects the hallucination performance over the whole image. Therefore, our proposed Attention-FH is capable of adaptively personalizing an optimal searching path for each face image according to its own characteristic.

Bio: Liang Lin is the Executive R&D Director of SenseTime Group Limited and a full Professor of Sun Yat-sen University. He is the Excellent Young Scientist of the National Natural Science Foundation of China. He received his B.S. and Ph.D. degrees from the Beijing Institute of Technology (BIT), Beijing, China, in 2003 and 2008, respectively, and he was a joint Ph.D. student with the Department of Statistics, University of California, Los Angeles (UCLA). From 2008 to 2010, he was a Post-Doctoral Fellow at UCLA. From 2014 to 2015, as a senior visiting scholar he was with The Hong Kong Polytechnic University and The Chinese University of Hong Kong. He currently leads the SenseTime R&D teams to develop cutting-edges and deliverable solutions on computer vision, data analysis and mining, and intelligent robotic systems. He has authorized and co-authorized on more than 100 papers in top-tier academic journals and conferences (e.g., 10 papers in TPAMI/IJCV and 40+ papers in CVPR/ICCV/NIPS/IJCAI). He has been serving as an associate editor of IEEE Trans. Human-Machine Systems, The Visual Computer and Neurocomputing. He served as Area/Session Chairs for numerous conferences such as ICME, ACCV, ICMR. He was the recipient of Best Paper Runners-Up Award in ACM NPAR 2010, Google Faculty Award in 2012, Best Student Paper Award in IEEE ICME 2014, and Hong Kong Scholars Award in 2014.

Jan Kautz

Title: Unsupervised Image-to-Image Translation Networks

Abstract: Most of the existing image-to-image translation frameworks---mapping an image in one domain to a corresponding image in another---are based on supervised learning, i.e., pairs of corresponding images in two domains are required for learning the translation function. This largely limits their applications, because capturing corresponding images in two different domains is often a difficult task. To address the issue, we propose the UNsupervised Image-to-image Translation (UNIT) framework, which is based on variational autoencoders and generative adversarial networks. The proposed framework can learn the translation function without any corresponding images in two domains. We enable this learning capability by combining a weight-sharing constraint and an adversarial training objective. Through visualization results from various unsupervised image translation tasks, we verify the effectiveness of the proposed framework. An ablation study further reveals the critical design choices. Moreover, we apply the UNIT framework to the unsupervised domain adaptation task and achieve better results than competing algorithms do in benchmark datasets.

Bio: Jan leads the Visual Computing Research team at NVIDIA, working predominantly on computer vision problems (from low-level vision through geometric vision to high-level vision), as well as machine learning problems (including deep reinforcement learning, generative models, and efficient deep learning). Before joining NVIDIA in 2013, Jan was a tenured faculty member at University College London. He holds a BSc in Computer Science from University of Erlangen-Nürnberg (1999), an MMath from the University of Waterloo (1999), received his PhD from the Max-Planck-Institut für Informatik (2003), and worked as a post-doc at the Massachusetts Institute of Technology (2003-2006). Jan has chaired numerous conferences (Eurographics Symposium on Rendering 2007, IEEE Symposium on Interactive Ray-Tracing 2008, Pacific Graphics 2011, CVMP 2012, Eurographics 2014) and has been on several editorial boards (IEEE Transactions on Visualization & Computer Graphics, The Visual Computer, Computer Graphics Forum, International Journal of Image and Graphics).

Eli Shechtman

Title: Image Stylization – from Patches to Neural Networks and Back

Abstract: Neural stylization methods became highly popular in the last couple years following the work by Gatys et al. These methods showed impressive results of transforming real photos into paintings given just a single style example. However, neural stylization methods have some limitations - they do not work well for some combinations of photos and styles and do not provide any control to the user to fix or change the result. Furthermore, they are effective for painterly styles and often result in non-photorealistic outputs. A more established family of stylization methods, based on patch-based synthesis and goes back to 2001 with the seminal Image Analogies work by Herzmann et al. These techniques require guiding channels in addition to the style example, but allow a finer-level of control over the output.
I will discuss about recent progress on both fronts. I will first show how simple changes to the Image Analogies framework lead to significant improvements in challenging problems like fluid animation, lighting-aware 3D model stylization and face stylization in video. I will then describe how control over spatial location, color information and control across spatial scale, can be introduced to neural stylization. Finally, I will show how photo realistic results can be obtained with neural stylization.

Bio: Eli Shechtman is a Principal Scientist at the Creative Intelligence Lab at Adobe Research. He received the B.Sc. degree in Electrical Engineering (magna cum laude) from Tel-Aviv University in 1996. Between 2001 and 2007 he attended the Weizmann Institute of Science where he received with honors his M.Sc. and Ph.D. degrees in Applied Mathematics and Computer Science. In 2007 he joined Adobe and started sharing his time as a post-doc with the University of Washington in Seattle. He published over 60 academic publications and holds over 20 issued patents. He served as a Technical Paper Committee member at SIGGRAPH 2013 and 2014, as an Area Chair at CVPR’15, ICCV’15 and CVPR’17 and serves an Associate Editor at TPAMI. He received several honors and awards, including the Best Paper prize at ECCV 2002, a Best Poster Award at CVPR 2004, a Best Reviewer Award at ECCV 2014 and published two Research Highlights papers in the Communication of the ACM journal.

Sabine Süsstrunk

Title: TBA

Alexei Efros

Title: TBA

Schedule (preliminary)