Due to improvements in hardware and software performance, deep learning algorithms have been used in many areas and have shown good results. In this paper, we propose a noise reduction framework based on a convolutional neural network (CNN) with deconvolution and a modified residual network (ResNet) to remove image noise. Simulation results show that the proposed algorithm is superior to the conventional noise eliminator in subjective and objective performance analyses.

This letter presents an objective quality index for benchmarking image inpainting algorithms. Under the guidance of the masks of damaged areas, the boundary region and the inpainting region are first located. Then, the statistical features are extracted from the boundary and inpainting regions respectively. For the boundary region, we utilize Weibull distribution to fit the gradient magnitude histograms of the exterior and interior regions around the boundary, and the Kullback-Leibler Divergence (KLD) is calculated to measure the boundary distortions caused by imperfect inpainting. Meanwhile, the quality of the inpainting region is measured by comparing the naturalness factors between the inpainted image and the reference image. Experimental results demonstrate that the proposed metric outperforms the relevant state-of-the-art quality metrics.

In this research, we propose a novel method to determine fingerprint liveness to improve the discriminative behavior and classification accuracy of the combined features. This approach detects if a fingerprint is from a live or fake source. In this approach, fingerprint images are analyzed in the differential excitation (DE) component and the centralized binary pattern (CBP) component, which yield the DE image and CBP image, respectively. The images obtained are used to generate a two-dimensional histogram that is subsequently used as a feature vector. To decide if a fingerprint image is from a live or fake source, the feature vector is processed using support vector machine (SVM) classifiers. To evaluate the performance of the proposed method and compare it to existing approaches, we conducted experiments using the datasets from the 2011 and 2015 Liveness Detection Competition (LivDet), collected from four sensors. The results show that the proposed method gave comparable or even better results and further prove that methods derived from combination of features provide a better performance than existing methods.

In recent years, much research interest has developed in image enhancement and haze removal techniques. With increasing demand for real time enhancement and haze removal, the need for efficient architecture incorporating both haze removal and enhancement is necessary. In this paper, we propose an architecture supporting both real-time Retinex-based image enhancement and haze removal, using a single module. Efficiently leveraging the similarity between Retinex-based image enhancement and haze removal algorithms, we have successfully proposed an architecture supporting both using a single module. The implementation results reveal that just 1% logic circuits overhead is required to support Retinex-based image enhancement in single mode and haze removal based on Retinex model. This reduction in computation complexity by using a single module reduces the processing and memory implications especially in mobile consumer electronics, as opposed to implementing them individually using different modules. Furthermore, we utilize image enhancement for transmission map estimation instead of soft matting, thereby avoiding further computation complexity which would affect our goal of realizing high frame-rate real time processing. Our FPGA implementation, operating at an optimum frequency of 125MHz with 5.67M total block memory bit size, supports WUXGA (1,920×1,200) 60fps as well as 1080p60 color input. Our proposed design is competitive with existing state-of-the-art designs. Our proposal is tailored to enhance consumer electronic such as on-board cameras, active surveillance intrusion detection systems, autonomous cars, mobile streaming systems and robotics with low processing and memory requirements.

To expand the use of systems using a camera on portable devices such as tablets and smartphones, we have developed and propose a memory saving feature descriptor, the use of which is one of the essential techniques in computer vision. The proposed descriptor compares pixel values of pre-fixed positions in the small patch around the feature point and stores binary values. Like the conventional descriptors, it extracts the patch on the basis of the scale and orientation of the feature point. For memories of the same size, it achieves higher accuracy than ORB and BRISK in all cases and AKAZE for the images with textured regions.

This paper proposes an enhanced affinity graph (EA-graph) for image segmentation. Firstly, the original image is over-segmented to obtain several sets of superpixels with different scales, and the color and texture features of the superpixels are extracted. Then, the similarity relationship between neighborhood superpixels is used to construct the local affinity graph. Meanwhile, the global affinity graph is obtained by sparse reconstruction among all superpixels. The local affinity graph and global affinity graph are superimposed to obtain an enhanced affinity graph for eliminating the influences of noise and isolated regions in the image. Finally, a bipartite graph is introduced to express the affiliation between pixels and superpixels, and segmentation is performed using a spectral clustering algorithm. Experimental results on the Berkeley segmentation database demonstrate that our method achieves significantly better performance compared to state-of-the-art algorithms.

Saliency quality assessment aims at estimating the objective quality of a saliency map without access to the ground-truth. Existing works typically evaluate saliency quality by utilizing information from saliency maps to assess its compactness and closedness while ignoring the information from image content which can be used to assess the consistence and completeness of foreground. In this letter, we propose a novel multi-information fusion network to capture the information from both the saliency map and image content. The key idea is to introduce a siamese module to collect information from foreground and background, aiming to assess the consistence and completeness of foreground and the difference between foreground and background. Experiments demonstrate that by incorporating image content information, the performance of the proposed method is significantly boosted. Furthermore, we validate our method on two applications: saliency detection and segmentation. Our method is utilized to choose optimal saliency map from a set of candidate saliency maps, and the selected saliency map is feeded into an segmentation algorithm to generate a segmentation map. Experimental results verify the effectiveness of our method.

A new fuzzy level set method (FLSM) based on the global search capability of quantum particle swarm optimization (QPSO) is proposed to improve the stability and precision of image segmentation, and reduce the sensitivity of initialization. The new combination of QPSO-FLSM algorithm iteratively optimizes initial contours using the QPSO method and fuzzy c-means clustering, and then utilizes level set method (LSM) to segment images. The new algorithm exploits the global search capability of QPSO to obtain a stable cluster center and a pre-segmentation contour closer to the region of interest during the iteration. In the implementation of the new method in segmenting liver tumors, brain tissues, and lightning images, the fitness function of the objective function of QPSO-FLSM algorithm is optimized by 10% in comparison to the original FLSM algorithm. The achieved initial contours from the QPSO-FLSM algorithm are also more stable than that from the FLSM. The QPSO-FLSM resulted in improved final image segmentation.

We propose a method for extracting multi-view images from a light field (plenoptic) camera that accurately handles the physical pixel arrangement of this camera. We use a Lytro Illum camera to obtain 4D light field data (a set of multi-viewpoint images) through a micro-lens array. The light field data are multiplexed on a single image sensor, and thus, the data is first demultiplexed into a set of multi-viewpoint (sub-aperture) images. However, the demultiplexing process usually includes interpolation of the original data such as demosaicing for a color filter array and pixel resampling for the hexagonal pixel arrangement of the original sub-aperture images. If this interpolation is performed, some information is added or lost to/from the original data. In contrast, we preserve the original data as faithfully as possible, and use them directly for the super resolution reconstruction, where the super-resolved image and the corresponding depth map are alternatively refined. We experimentally demonstrate the effectiveness of our method in resolution enhancement through comparisons with Light Field Toolbox and Lytro Desktop Application. Moreover, we also mention another type of light field cameras, a Raytrix camera, and describe how it can be handled to extract high-quality multi-view images.

This letter proposes a representation learning framework of convolutional neural networks (Convnets) that aims to rectify and improve the feature representations learned by existing transformation-invariant methods. The existing methods usually encode feature representations invariant to a wide range of spatial transformations by augmenting input images or transforming intermediate layers. Unfortunately, simply transforming the intermediate feature maps may lead to unpredictable representations that are ineffective in describing the transformed features of the inputs. The reason is that the operations of convolution and geometric transformation are not exchangeable in most cases and so exchanging the two operations will yield the transformation error. The error may potentially harm the performance of the classification networks. Motivated by the fractal statistics of natural images, this letter proposes a rectifying transformation operator to minimize the error. The proposed method is differentiable and can be inserted into the convolutional architecture without making any modification to the optimization algorithm. We show that the rectified feature representations result in better classification performance on two benchmarks.

This paper presents a 6th-order quadrature bandpass delta sigma AD modulator (QBPDSM) with 2nd-order image rejection using dynamic amplifier and noise coupling (NC) SAR quantizer embedded by passive adder for the application of wireless communication system. A novel complex integrator using dynamic amplifier is proposed to improve the energy efficiency of the QBPDSM. The NC SAR quantizer can realize an additional 2nd-order noise shaping and 2nd-order image rejection by the digital domain noise coupling technique. As a result, the 6th-order QBPDSM with 2nd-order image rejection is realized by two complex integrators using dynamic amplifier and the NC SAR quantizer. The SPICE simulation results demonstrate the feasibility of the proposed QBPDSM in 90nm CMOS technology. Simulated SNDR of 76.30dB is realized while a sinusoid -3.25dBFS input is sampled at 33.3MS/s and the bandwidth of 2.083MHz (OSR=8) is achieved. The total power consumption in the modulator is 6.74mW while the supply voltage is 1.2V.

In this paper, we propose a method for enhancing the visibility of omnidirectional spherical images by enlarging the foreground and compressing the background without provoking a sense of visual incompatibility by using a simplified spring model.

We investigate image recognition of multiple food items in a single photo, focusing on a buffet restaurant application, where menu changes at every meal, and only a few images per class are available. After detecting food areas, we perform hierarchical recognition. We evaluate our results, comparing to two baseline methods.

Encryption-then-Compression (EtC) systems have been proposed to securely transmit images through an untrusted channel provider. In this study, EtC systems were applied to social media like Twitter that carry out image manipulations. The block scrambling-based encryption schemes used in EtC systems were evaluated in terms of their robustness against image manipulation on social media. The aim was to investigate how five social networking service (SNS) providers, Facebook, Twitter, Google+, Tumblr and Flickr, manipulate images and to determine whether the encrypted images uploaded to SNS providers can avoid being distorted by such manipulations. In an experiment, encrypted and non-encrypted JPEG images were uploaded to various SNS providers. The results show that EtC systems are applicable to the five SNS providers.

MDC-4 is the enhanced version of MDC-2, which is a well-known hash mode of block ciphers. However, it does not guarantee sufficient securities required for a cryptographic hash function. In the ideal cipher model, the MDC-4 compression function has the collision security bound close to 25n/8 and the preimage security bound close to 25n/4, where the underlying block cipher has the block size of n bits. We have studied how to improve MDC-4 with simple modification to strengthen its security. It is meaningful work because users often want to improve their familiar systems with low cost. In this paper, we achieve it by proposing MDC-4+, which is a light variation of MDC-4. We prove that MDC-4+ is much more secure than MDC-4 by showing that it has the collision security bound close to optimal 2n and the preimage security bound close to 24n/3. We also discuss its efficiency by comparing existing hash modes.

In the case that images are shared via social networking services (SNS) and cloud photo storage services (CPSS), it is known that the JPEG images uploaded to the services are mostly re-compressed by the providers. Because of such a situation, a new image identification scheme for double-compressed JPEG images is proposed in this paper. The aim is to detect a single-compressed image that has the same original image as the double-compressed ones. In the proposed scheme, a feature extracted from only DC coefficients in DCT coefficients is used for the identification. The use of the feature allows us not only to robustly avoid errors caused by double-compression but also to perform the identification for different size images. The simulation results demonstrate the effectiveness of the proposed one in terms of the querying performance.

Depth image based rendering (DIBR), which is utilized to render virtual views with a color image and the corresponding depth map, is one of the key procedures in the 2D to 3D conversion process. However, some troubling problems, such as depth edge misalignment, disocclusion occurrences and cracks at resampling, still exist in current DIBR systems. To solve these problems, in this paper, we present a robust depth image based rendering scheme for stereoscopic view synthesis. The cores of the proposed scheme are two depth map filters which share a common domain transform based filtering framework. As a first step, a filter of this framework is carried out to realize texture-depth boundary alignments and directional disocclusion reduction smoothing simultaneously. Then after depth map 3D warping, another adaptive filter is used on the warped depth maps with delivered scene gradient structures to further diminish the remaining cracks and noises. Finally, with the optimized depth map of the virtual view, backward texture warping is adopted to retrieve the final texture virtual view. The proposed scheme enables to yield visually satisfactory results for high quality 2D to 3D conversion. Experimental results demonstrate the excellent performances of the proposed approach.

Dai-Yuan (DY) conjugate gradient method is an effective method for solving large-scale unconstrained optimization problems. In this paper, a new DY method, possessing a spectral conjugate parameter βk, is presented. An attractive property of the proposed method is that the search direction generated at each iteration is descent, which is independent of the line search. Global convergence of the proposed method is also established when strong Wolfe conditions are employed. Finally, comparison experiments on impulse noise removal are reported to demonstrate the effectiveness of the proposed method.

This paper proposes a block-permutation-based encryption (BPBE) scheme for the encryption-then-compression (ETC) system that enhances the color scrambling. A BPBE image can be obtained through four processes, positional scrambling, block rotation/flip, negative-positive transformation, and color component shuffling, after dividing the original image into multiple blocks. The proposed scheme scrambles the R, G, and B components independently in positional scrambling, block rotation/flip, and negative-positive transformation, by assigning different keys to each color component. The conventional scheme considers the compression efficiency using JPEG and JPEG 2000, which need a color conversion before the compression process by default. Therefore, the conventional scheme scrambles the color components identically in each process. In contrast, the proposed scheme takes into account the RGB-based compression, such as JPEG-LS, and thus can increase the extent of the scrambling. The resilience against jigsaw puzzle solver (JPS) can consequently be increased owing to the wider color distribution of the BPBE image. Additionally, the key space for resilience against brute-force attacks has also been expanded exponentially. Furthermore, the proposed scheme can maintain the JPEG-LS compression efficiency compared to the conventional scheme. We confirm the effectiveness of the proposed scheme by experiments and analyses.

Recently, image forensics community has paid attention to the research on the design of effective algorithms based on deep learning technique. And facts proved that combining the domain knowledge of image forensics and deep learning would achieve more robust and better performance than the traditional schemes. Instead of improving algorithm performance, in this paper, the safety of deep learning based methods in the field of image forensics is taken into account. To the best of our knowledge, this is the first work focusing on this topic. Specifically, we experimentally find that the method using deep learning would fail when adding the slight noise into the images (adversarial images). Furthermore, two kinds of strategies are proposed to enforce security of deep learning-based methods. Firstly, a penalty term to the loss function is added, which is the 2-norm of the gradient of the loss with respect to the input images, and then an novel training method is adopt to train the model by fusing the normal and adversarial images. Experimental results show that the proposed algorithm can achieve good performance even in the case of adversarial images and provide a security consideration for deep learning-based image forensics.