Image Co-Saliency Detection: Novel Approaches with Convex Optimization and Deep Neural Networks

Abstract

The advance of digital technologies has endowed people with easier access to massive collections of image or video data than ever before. For its capability to borrow signal strengths across images or video frames, image co-saliency detection has become an active research topic to help address many advanced computer vision applications, such as image retrieval and object tracking. Co-saliency detection is to distill the essential content of an image group by locating the eye-catching objects commonly present in multiple images. In this dissertation, we present several novel approaches based on convex optimization and deep neural networks for accurate image co-saliency detection. First, we introduce a region-wise saliency map fusion approach to amend the inherent flaw of traditional map-wise fusion methods. The effectiveness of region-wise fusion motivates us to the improved model by integrating segmentation revealed objectness in the following work. Second, we explore the complementary relationship between image co-saliency detection and co-segmentation for higher quality performance on both tasks with scalability to multiple input images. Third, we improved our region-wise fusion scheme by exploring the power of unsupervised deep learning methods by stacked auto-encoder to relax the underlying foreground consistency assumption in most state-of-the-art fusion models. To achieve the desired practical significance, we further combine our stacked autoencoder-enabled fusion with the convolutional neural networks (CNNs). Our proposed two-stage co-saliency detection model can retain the highly discriminative power from the CNNs without the requirement of massive human labeling. Comprehensive experimental results demonstrate its state-of-the-art performance on the publicly available benchmark data sets. We expect solving these issues in image co-saliency detection can lead to significant contributions to the computer vision community for better image understanding and consequent decision making based on that.