A side channel attack is a means of security attacks that tries to restore secret information by analyzing side-information such as electromagnetic wave, heat, electric energy and running time that are unintentionally emitted from a computer system. The side channel attack that focuses on the running time of a cryptosystem is specifically named a “timing attack”. Timing attacks are relatively easy to carry out, and particularly threatening for tiny systems that are used in smart cards and IoT devices because the system is so simple that the processing time would be clearly observed from the outside of the card/device. The threat of timing attacks is especially serious when an attacker actively controls the input to a target program. Countermeasures are studied to deter such active attacks, but the attacker still has the chance to learn something about the concealed information by passively watching the running time of the target program. The risk of passive timing attacks can be measured by the mutual information between the concealed information and the running time. However, the computation of the mutual information is hardly possible except for toy examples. This study focuses on three algorithms for RSA decryption, derives formulas of the mutual information under several assumptions and approximations, and calculates the mutual information numerically for practical security parameters.

Recently, image compression systems based on convolutional neural networks that use flexible nonlinear analysis and synthesis transformations have been developed to improve the restoration accuracy of decoded images. Although these methods that use objective metric such as peak signal-to-noise ratio and multi-scale structural similarity for optimization attain high objective results, such metric may not reflect human visual characteristics and thus degrade subjective image quality. A method using a framework called a generative adversarial network (GAN) has been reported as one of the methods aiming to improve the subjective image quality. It optimizes the distribution of restored images to be close to that of natural images; thus it suppresses visual artifacts such as blurring, ringing, and blocking. However, since methods of this type are optimized to focus on whether the restored image is subjectively natural or not, components that are not correlated with the original image are mixed into the restored image during the decoding process. Thus, even though the appearance looks natural, subjective similarity may be degraded. In this paper, we investigated why the conventional GAN-based compression techniques degrade subjective similarity, then tackled this problem by rethinking how to handle image generation in the GAN framework between image sources with different probability distributions. The paper describes a method to maximize mutual information between the coding features and the restored images. Experimental results show that the proposed mutual information amount is clearly correlated with subjective similarity and the method makes it possible to develop image compression systems with high subjective similarity.

Mining software artifacts is a useful way to understand the source code of software projects. Topic modeling in particular has been widely used to discover meaningful information from software artifacts. However, software artifacts are unstructured and contain a mix of textual types within the natural text. These software artifact characteristics worsen the performance of topic modeling. Among several natural language pre-processing tasks, removing stop words to reduce meaningless and uninteresting terms is an efficient way to improve the quality of topic models. Although many approaches are used to generate effective stop words, the lists are outdated or too general to apply to mining software artifacts. In addition, the performance of the topic model is sensitive to the datasets used in the training for each approach. To resolve these problems, we propose an automatic stop word generation approach for topic models of software artifacts. By measuring topic coherence among words in the topic using Pointwise Mutual Information (PMI), we added words with a low PMI score to our stop words list for every topic modeling loop. Through our experiment, we proved that our stop words list results in a higher performance of the topic model than lists from other approaches.

Receiver implementations with very low quantization resolution will play an important role in 5G, as high precision quantization and signal processing are costly in terms of computational resources and chip area. Therefore, low resolution receivers with quasi optimum performance will be required to meet complexity and latency constraints. The Information Bottleneck method allows for a novel, information centric approach to design such receivers. The method was originally introduced by Naftali Tishby et al. and mostly used in the machine learning field so far. Interestingly, it can also be applied to build surprisingly good digital communication receivers which work fundamentally different than state-of-the-art receivers. Instead of minimizing the quantization error, receiver components with maximum preservation of relevant information for a given bit width can be designed. All signal processing in the resulting receivers is performed using only simple lookup operations. In this paper, we first provide a brief introduction to the design of receiver components with the Information Bottleneck method. We keep referring to decoding of low-density parity-check codes as a practical example. The focus of the paper lies on practical decoder implementations on a digital signal processor which illustrate the potential of the proposed technique. An Information Bottleneck decoder with 4bit message passing decoding is found to outperform 8bit implementations of the well-known min-sum decoder in terms of bit error rate and to perform extremely close to an 8bit belief propagation decoder, while offering considerably higher net decoding throughput than both conventional decoders.

Let X, Y be two correlated discrete random variables. We consider an estimation of X from encoded data φ(Y) of Y by some encoder function φ(Y). We derive an inequality describing a relation of the correct probability of estimation and the mutual information between X and φ(Y). This inequality may be useful for the secure analysis of crypto system when we use the success probability of estimating secret data as a security criterion. It also provides an intuitive meaning of the secrecy exponent in the strong secrecy criterion.

Ensemble learning is widely used in the field of sensor network monitoring and target identification. To improve the generalization ability and classification precision of ensemble learning, we first propose an approximate attribute reduction algorithm based on rough sets in this paper. The reduction algorithm uses mutual information to measure attribute importance and introduces a correction coefficient and an approximation parameter. Based on a random sampling strategy, we use the approximate attribute reduction algorithm to implement the multi-modal sample space perturbation. To further reduce the ensemble size and realize a dynamic subset of base classifiers that best matches the test sample, we define a similarity parameter between the test samples and training sample sets that takes the similarity and number of the training samples into consideration. We then propose a k-means clustering-based dynamic ensemble selection algorithm. Simulations show that the multi-modal perturbation method effectively selects important attributes and reduces the influence of noise on the classification results. The classification precision and runtime of experiments demonstrate the effectiveness of the proposed dynamic ensemble selection algorithm.

In this paper, we treat (k, L, n) ramp secret sharing schemes (SSSs) that can detect impersonation attacks and/or substitution attacks. First, we derive lower bounds on the sizes of the shares and random number used in encoding for given correlation levels, which are measured by the mutual information of shares. We also derive lower bounds on the success probabilities of attacks for given correlation levels and given sizes of shares. Next we propose a strong (k, L, n) ramp SSS against substitution attacks. As far as we know, the proposed scheme is the first strong (k, L, n) ramp SSSs that can detect substitution attacks of at most k-1 shares. Our scheme can be applied to a secret SL uniformly distributed over GF(pm)L, where p is a prime number with p≥L+2. We show that for a certain type of correlation levels, the proposed scheme can achieve the lower bounds on the sizes of the shares and random number, and can reduce the success probability of substitution attacks within nearly L times the lower bound when the number of forged shares is less than k. We also evaluate the success probability of impersonation attack for our schemes. In addition, we give some examples of insecure ramp SSSs to clarify why each component of our scheme is essential to realize the required security.

Bayesian networks are a powerful approach for representation and reasoning under conditions of uncertainty. Of the many good algorithms for learning Bayesian networks from data, the bio-inspired search algorithm is one of the most effective. In this paper, we propose a hybrid mutual information-modified binary particle swarm optimization (MI-MBPSO) algorithm. This technique first constructs a network based on MI to improve the quality of the initial population, and then uses the decomposability of the scoring function to modify the BPSO algorithm. Experimental results show that, the proposed hybrid algorithm outperforms various other state-of-the-art structure learning algorithms.

Faster-than-Nyquist (FTN) signaling is investigated for quasi-static flat fading massive multiple-input multiple-output (MIMO) systems. In FTN signaling, pulse trains are sent at a symbol rate higher than the Nyquist rate to increase the transmission rate. As a result, inter-symbol interference occurs inevitably for flat fading channels. This paper assesses the information-theoretically achievable rate of MIMO FTN signaling based on the optimum joint equalization and multiuser detection. The replica method developed in statistical physics is used to evaluate the achievable rate in the large-system limit, where the dimensions of input and output signals tend to infinity at the same rate. An analytical expression of the achievable rate is derived for general modulation schemes in the large-system limit. It is shown that FTN signaling does not improve the channel capacity of massive MIMO systems, and that FTN signaling with quadrature phase-shift keying achieves the channel capacity for all signal-to-noise ratios as the symbol period tends to zero.

This paper presents a method for registering retinal images. Retinal image registration is crucial for the diagnoses and treatments of various eye conditions and diseases such as myopia and diabetic retinopathy. Retinal image registration is challenging because the images have non-uniform contrasts and intensity distributions, as well as having large homogeneous non-vascular regions. This paper provides a new retinal image registration method by effectively combining expectation maximization principal component analysis based mutual information (EMPCA-MI) with salient features. Experimental results show that our method is more efficient and robust than the conventional EMPCA-MI method.

This letter proposes an image fusion method which adopts a union of multiple directional lapped orthogonal transforms (DirLOTs). DirLOTs are used to generate symmetric orthogonal discrete wavelet transforms and then to construct a union of unitary transforms as a redundant dictionary with a multiple directional property. The multiple DirLOTs can overcome a disadvantage of separable wavelets to represent images which contain slant textures and edges. We analyse the characteristic of local luminance contrast, and propose a fusion rule based on interscale relation of wavelet coefficients. Relying on the above, a novel image fusion method is proposed. Some experimental results show that the proposed method is able to significantly improve the fusion performance from those with the conventional discrete wavelet transforms.

This paper presents the performance of outer-loop control for selecting the best modulation and coding scheme (MCS) based on mutual information (MI) for orthogonal frequency division multiplexing (OFDM) multiple-input multiple-output (MIMO) spatial division multiplexing (SDM). We propose an outer-loop control scheme that updates the measured MI per information bit value for selecting the best MCS from a mapping table that associates the block error rate (BLER) and MI per bit instead of directly updating the MCS selection threshold so that the required BLER is satisfied. The proposed outer-loop control is applicable to continuous data transmission including intermittent transmission with a short blank period. Moreover, we compare the measured BLER and throughput performance for two types of outer-loop control methods: instantaneous block error detection and moving-average BLER detection. In the paper, we use maximum likelihood detection (MLD) for MIMO SDM. Computer simulation results optimize the step size for the respective outer-loop control schemes for selecting the best MCS that achieves the higher throughput and the target BLER simultaneously. Computer simulation results also show that by using the most appropriate step size, the outer-loop control method based on the instantaneous block error detection of each physical resource block is more appropriate than that based on the moving-average BLER detection from the viewpoints of achieving the target BLER more accurately and higher throughput.

Texture feature descriptors such as local binary patterns (LBP) have proven effective for ground-based cloud classification. Traditionally, these texture feature descriptors are predefined in a handcrafted way. In this paper, we propose a novel method which automatically learns discriminative features from labeled samples for ground-based cloud classification. Our key idea is to learn these features through mutual information maximization which learns a transformation matrix for local difference vectors of LBP. The experimental results show that our learned features greatly improves the performance of ground-based cloud classification when compared to the other state-of-the-art methods.

The goal of dimension reduction is to represent high-dimensional data in a lower-dimensional subspace, while intrinsic properties of the original data are kept as much as possible. An important challenge in unsupervised dimension reduction is the choice of tuning parameters, because no supervised information is available and thus parameter selection tends to be subjective and heuristic. In this paper, we propose an information-theoretic approach to unsupervised dimension reduction that allows objective tuning parameter selection. We employ quadratic mutual information (QMI) as our information measure, which is known to be less sensitive to outliers than ordinary mutual information, and QMI is estimated analytically by a least-squares method in a computationally efficient way. Then, we provide an eigenvector-based efficient implementation for performing unsupervised dimension reduction based on the QMI estimator. The usefulness of the proposed method is demonstrated through experiments.

Energy harvesting technology was introduced into wireless sensor networks (WSNs) to solve the problem of the short lifetimes of sensor nodes. The technology gives sensor nodes the ability to convert environmental energy into electricity. Sufficient electrical energy can lengthen the lifetime and improve the quality of service of a WSN. This paper proposes a novel use of mutual information to evaluate data transmission behavior in the energy harvesting WSNs. Data at a sink for a node deteriorates over time until the next periodic transmission from the node is received. In this paper, we suggest an optimized intermittent transmission method for WSNs that harvest energy. Our method overcomes the problem of information deterioration without increasing energy cost. We show that by using spatial correlation between different sensor nodes, our proposed method can mitigate information deterioration significantly at the sink.

Squared-loss mutual information (SMI) is a robust measure of the statistical dependence between random variables. The sample-based SMI approximator called least-squares mutual information (LSMI) was demonstrated to be useful in performing various machine learning tasks such as dimension reduction, clustering, and causal inference. The original LSMI approximates the pointwise mutual information by using the kernel model, which is a linear combination of kernel basis functions located on paired data samples. Although LSMI was proved to achieve the optimal approximation accuracy asymptotically, its approximation capability is limited when the sample size is small due to an insufficient number of kernel basis functions. Increasing the number of kernel basis functions can mitigate this weakness, but a naive implementation of this idea significantly increases the computation costs. In this article, we show that the computational complexity of LSMI with the multiplicative kernel model, which locates kernel basis functions on unpaired data samples and thus the number of kernel basis functions is the sample size squared, is the same as that for the plain kernel model. We experimentally demonstrate that LSMI with the multiplicative kernel model is more accurate than that with plain kernel models in small sample cases, with only mild increase in computation time.

Localization in wireless sensor networks is the problem of estimating the geographical locations of wireless sensor nodes. We propose a framework to utilizing multiple data sources for localization scheme based on support vector machines. The framework can be used with both classification and regression formulation of support vector machines. The proposed method uses only connectivity information. Multiple hop count data sources can be generated by adjusting the transmission power of sensor nodes to change the communication ranges. The optimal choice of communication ranges can be determined by evaluating mutual information. We consider two methods for integrating multiple data sources together; unif method and align method. The improved localization accuracy of the proposed framework is verified by simulation study.

Mutual information (MI) is a standard measure of statistical dependence of random variables. However, due to the log function and the ratio of probability densities included in MI, it is sensitive to outliers. On the other hand, the L2-distance variant of MI called quadratic MI (QMI) tends to be robust against outliers because QMI is just the integral of the squared difference between the joint density and the product of marginals. In this paper, we propose a kernel least-squares QMI estimator called least-squares QMI (LSQMI) that directly estimates the density difference without estimating each density. A notable advantage of LSQMI is that its solution can be analytically and efficiently computed just by solving a system of linear equations. We then apply LSQMI to dependence-maximization clustering, and demonstrate its usefulness experimentally.

Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As an attempt to take feature interaction into account, we propose 1-LSMI, an 1-regularization based algorithm that maximizes a squared-loss variant of mutual information between selected features and outputs. Numerical results show that 1-LSMI performs well in handling redundancy, detecting non-linear dependency, and considering feature interaction.

This paper presents comprehensive comparisons of the achievable throughput between the 32-/64-ary amplitude and phase shift keying (APSK) and cross 32QAM/square 64QAM schemes based on mutual information (MI) considering the peak-to-average power ratio (PAPR) of the modulated signal. As a PAPR criterion, we use a cubic metric (CM) that directly corresponds to the transmission back-off of a power amplifier. In the analysis, we present the best ring ratio for the 32 or 64APSK scheme from the viewpoint of minimizing the required received signal-to-noise power ratio (SNR) considering the CM that achieves the peak throughput, i.e., maximum error-free transmission rate. We show that the required received SNR considering the CM at the peak throughput is minimized with the number of rings of M = 3 and 4 for 32-ary APSK and 64-asry APSK, respectively. Then, we show with the best ring ratios that the (4, 12, 16) 32APSK scheme with M = 3 achieves a lower required received SNR considering the CM compared to that for the cross 32QAM scheme. Similarly, we show that the (4, 12, 20, 28) 64APSK scheme with M = 4 achieves almost the same required received SNR considering the CM as that for the square 64QAM scheme.