In machine learning, data augmentation (DA) is a technique for improving the generalization performance of models. In this paper, we mainly consider gradient descent of linear regression under DA using noisy copies of datasets, in which noise is injected into inputs. We analyze the situation where noisy copies are newly generated and injected into inputs at each epoch, i.e., the case of using on-line noisy copies. Therefore, this article can also be viewed as an analysis on a method using noise injection into a training process by DA. We considered the training process under three training situations which are the full-batch training under the sum of squared errors, and full-batch and mini-batch training under the mean squared error. We showed that, in all cases, training for DA with on-line copies is approximately equivalent to the l2 regularization training for which variance of injected noise is important, whereas the number of copies is not. Moreover, we showed that DA with on-line copies apparently leads to an increase of learning rate in full-batch condition under the sum of squared errors and the mini-batch condition under the mean squared error. The apparent increase in learning rate and regularization effect can be attributed to the original input and additive noise in noisy copies, respectively. These results are confirmed in a numerical experiment in which we found that our result can be applied to usual off-line DA in an under-parameterization scenario and can not in an over-parametrization scenario. Moreover, we experimentally investigated the training process of neural networks under DA with off-line noisy copies and found that our analysis on linear regression can be qualitatively applied to neural networks.

Backscatter communication is an emerging wireless access technology to realize ultra-low power terminals exploiting the modulated reflection of incident radio wave. This paper proposes a method to measure the phase angle of backscatter link using principal component analysis (PCA). The phase angle measurement of backscatter link at the receiver is essential to maximize the signal quality for subsequent demodulation and to measure the distance and the angle of arrival. The drawback of popular phase angle measurement with naive phase averaging and linear regression analysis is to produce erroneous phase angle, where the phase angle is close to $pmrac{pi}{2}$ radian and the signal quality is poor. The advantage of the proposal is quantified with a computer simulation, a conducted experiment and radio propagation experiments.

Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.

Software maintenance is an important activity in the software lifecycle. Software maintenance does not only mean removing faults found after software release. Software needs extensions or modifications of its functions owing to changes in the business environment and software maintenance also refers to them. To help users and service suppliers benchmark work efficiency for software maintenance, and to clarify the relationships between software quality, work efficiency, and unit cost of staff, we used a dataset that includes 134 data points collected by the Economic Research Association in 2012, and analyzed the factors that affected the work efficiency of software maintenance. In the analysis, using a multiple regression model, we clarified the relationships between work efficiency and programming language and productivity factors. To analyze the influence to the quality, relationships of fault ratio was analyzed using correlation coefficients. The programming language and productivity factors affect work efficiency. Higher work efficiency and higher unit cost of staff do not affect the quality of software maintenance.

When applying estimation methods, the issue of outliers is inevitable. The extent of their influence has not been clarified, though several studies have evaluated outlier elimination methods. It is unclear whether we should always be sensitive to outliers, whether outliers should always be removed before estimation, and what amount of precaution is required for collecting project data. Therefore, the goal of this study is to illustrate a guideline that suggests how sensitively we should handle outliers. In the analysis, we experimentally add outliers to three datasets, to analyze their influence. We modified the percentage of outliers, their extent (e.g., we varied the actual effort from 100 to 200 person-hours when the extent was 100%), the variables including outliers (e.g., adding outliers to function points or effort), and the locations of outliers in a dataset. Next, the effort was estimated using these datasets. We used multiple linear regression analysis and analogy based estimation to estimate the development effort. The experimental results indicate that the influence of outliers on the estimation accuracy is non-trivial when the extent or percentage of outliers is considerable (i.e., 100% and 20%, respectively). In contrast, their influence is negligible when the extent and percentage are small (i.e., 50% and 10%, respectively). Moreover, in some cases, the linear regression analysis was less affected by outliers than analogy based estimation.

This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.

Surveillance with multiple cameras and microphones is promising to trace activities of suspicious persons for security purposes. When these sensors are connected to the Internet, they might also jeopardize innocent people's privacy because, as a result of human error, signals from sensors might allow eavesdropping by malicious persons. This paper presents a proposal for exploiting super-resolution to address this problem. Super-resolution is a signal processing technique by which a high-resolution version of a signal can be reproduced from a low-resolution version of the same signal source. Because of this property, an intelligible speech signal is reconstructed from multiple sensor signals, each of which is completely unintelligible because of its sufficiently low sampling rate. A method based on Bayesian linear regression is proposed in comparison with one based on maximum likelihood. Computer simulations using a simple sinusoidal input demonstrate that the methods restore the original signal from those which are actually measured. Moreover, results show that the method based on Bayesian linear regression is more robust than maximum likelihood under various microphone configurations in noisy environments and that this advantage is remarkable when the number of microphones enrolled in the process is as small as the minimum required. Finally, listening tests using speech signals confirmed that mean opinion score (MOS) of the reconstructed signal reach 3, while those of the original signal captured at each single microphone are almost 1.

We build a privacy-preserving system of linear regression protecting both input data secrecy and output privacy. Our system achieves those goals simultaneously via a novel combination of homomorphic encryption and differential privacy dedicated to linear regression and its variants (ridge, LASSO). Our system is proved scalable over cloud servers, and its efficiency is extensively checked by careful experiments.

A non-linear extension of generalized hyperplane approximation (GHA) method is introduced in this letter. Although GHA achieved a high-confidence result in motion parameter estimation by utilizing the supervised learning scheme in histogram of oriented gradient (HOG) feature space, it still has unstable convergence range because it approximates the non-linear function of regression from the feature space to the motion parameter space as a linear plane. To extend GHA into a non-linear regression for larger convergence range, we derive theoretical equations and verify this extension's effectiveness and efficiency over GHA by experimental results.

In this paper, a method for designing of Incremental Granular Model (IGM) based on integration of Linear Regression (LR) and Linguistic Model (LM) with the aid of fuzzy granulation is proposed. Here, IGM is designed by the use of information granulation realized via Context-based Interval Type-2 Fuzzy C-Means (CIT2FCM) clustering. This clustering approach are used not only to estimate the cluster centers by preserving the homogeneity between the clustered patterns from linguistic contexts produced in the output space, but also deal with the uncertainty associated with fuzzification factor. Furthermore, IGM is developed by construction of a LR as a global model, refine it through the local fuzzy if-then rules that capture more localized nonlinearities of the system by LM. The experimental results on two examples reveal that the proposed method shows a good performance in comparison with the previous works.

With recent developments in machine learning technology, the predictions by systems incorporating machine learning can now have a significant impact on the lives and activities of individuals. In some cases, predictions made by machine learning can result unexpectedly in unfair treatments to individuals. For example, if the results are highly dependent on personal attributes, such as gender or ethnicity, hiring decisions might be discriminatory. This paper investigates the neutralization of a probabilistic model with respect to another probabilistic model, referred to as a viewpoint. We present a novel definition of neutrality for probabilistic models, η-neutrality, and introduce a systematic method that uses the maximum likelihood estimation to enforce the neutrality of a prediction model. Our method can be applied to various machine learning algorithms, as demonstrated by η-neutral logistic regression and η-neutral linear regression.

In this paper, we propose a framework for the real-time estimation of a multidimensional QoE of Multi-View Video and Audio (MVV-A) IP transmission. The framework utilizes linear multiple regression analysis with application-level and transport-level QoS parameters which can be measured in real time. In order to cope with a variety of MVV-A usage-situations, we introduce the concept of usage-situation type for grouping usage-situations with similar features to apply a representative regression line. We deal with two contents, two camera arrangements, and two user interfaces for viewpoint change as representative examples of the usage-situations. We assess multidimensional QoE of MVV-A with various types of average load, playout buffering time, and delay in the network. We then conduct the multiple regression analysis for the multidimensional QoE values represented by a psychological scale. From the comparison of measured values and estimated ones, we notice that real-time estimation of QoE is feasible in MVV-A IP transmission.

A smoothing method for minimum phone error linear regression (MPELR) is proposed in this paper. We show that the objective function for minimum phone error (MPE) can be combined with a prior mean distribution. When the prior mean distribution is based on maximum likelihood (ML) estimates, the proposed method is the same as the previous smoothing technique for MPELR. Instead of ML estimates, maximum a posteriori (MAP) parameter estimate is used to define the mode of prior mean distribution to improve the performance of MPELR. Experiments on a large vocabulary speech recognition task show that the proposed method can obtain 8.4% relative reduction in word error rate when the amount of data is limited, while retaining the same asymptotic performance as conventional MPELR. When compared with discriminative maximum a posteriori linear regression (DMAPLR), the proposed method shows improvement except for the case of limited adaptation data for supervised adaptation.

In recent years, computation offloading, through which applications on a mobile device can offload their computations onto more resource-rich clouds, has emerged as a promising technique to reduce battery consumption as well as augment the devices' limited computation and memory capabilities. In order for computation offloading to be energy-efficient, an accurate estimate of battery consumption is required to decide between local processing and computation offloading. In this paper, we propose a novel technique for estimating battery consumption without requiring detailed information about the mobile application's internal structure or its execution behavior. In our approach, the relationship is derived between variables that affect battery consumption (i.e., the input to the application, the transmitted data, and resource status) and the actual consumed energy from the application's past run history. We evaluated the performance of the proposed technique using two different types of mobile applications over different wireless network environments such as 3G, Wi-Fi, and LTE. The experimental results show that our technique can provide tolerable estimation accuracy and thus make correct decisions between local processing and computation offloading.

This study proposes a variable selection linear regression (VSLR) adaptation framework to improve the accuracy of automatic speech recognition (ASR) with only limited and unlabeled adaptation data. The proposed framework can be divided into three phases. The first phase prepares multiple variable subsets by applying a ranking filter to the original regression variable set. The second phase determines the best variable subset based on a pre-determined performance evaluation criterion and computes a linear regression (LR) mapping function based on the determined subset. The third phase performs adaptation in either model or feature spaces. The three phases can select the optimal components and remove redundancies in the LR mapping function effectively and thus enable VSLR to provide satisfactory adaptation performance even with a very limited number of adaptation statistics. We formulate model space VSLR and feature space VSLR by integrating the VS techniques into the conventional LR adaptation systems. Experimental results on the Aurora-4 task show that model space VSLR and feature space VSLR, respectively, outperform standard maximum likelihood linear regression (MLLR) and feature space MLLR (fMLLR) and their extensions, with notable word error rate (WER) reductions in a per-utterance unsupervised adaptation manner.

We present an acoustic model adaptation method where the transformation matrix for a new speaker is given by the product of bases and a weight matrix. The bases are built from the parallel factor analysis 2 (PARAFAC2) of training speakers' transformation matrices. We perform continuous speech recognition experiments using the WSJ0 corpus.

Context-based Adaptive Binary Arithmetic Coding (CA-BAC) is adopted as an entropy coding tool for main profile of the video coding standard H.264/AVC. CABAC achieves higher degree of redundancy reduction by estimating the conditional probability of each binary symbol which is the input to the arithmetic coder. This paper presents an entropy coding method based on CABAC. In the proposed method, the binary symbol is coded using more precisely estimated conditional probability, thereby leading to performance improvement. We apply our method to the standard and evaluate its performance for different video sources and various quantization parameters (QP). Experiment results show that our method outperforms the original CABAC in term of coding efficiency, and the average bit-rate savings are up to 1.2%.

In speaker adaptation for speech synthesis, it is desirable to convert both voice characteristics and prosodic features such as F0 and phone duration. For simultaneous adaptation of spectrum, F0 and phone duration within the HMM framework, we need to transform not only the state output distributions corresponding to spectrum and F0 but also the duration distributions corresponding to phone duration. However, it is not straightforward to adapt the state duration because the original HMM does not have explicit duration distributions. Therefore, we utilize the framework of the hidden semi-Markov model (HSMM), which is an HMM having explicit state duration distributions, and we apply an HSMM-based model adaptation algorithm to simultaneously transform both the state output and state duration distributions. Furthermore, we propose an HSMM-based adaptive training algorithm to simultaneously normalize the state output and state duration distributions of the average voice model. We incorporate these techniques into our HSMM-based speech synthesis system, and show their effectiveness from the results of subjective and objective evaluation tests.

Magnetoencephalography (MEG) is a method to measure a magnetic field generated by electrical neural activity in a brain, and it plays increasingly important role in clinical diagnoses and neurophysiological studies. However, in MEG analysis, the estimation of the brain activity, of the electric current density distribution in a brain which is represented by current dipoles, is problematic. A spatial filter and subsequent reconstruction of the current density distribution estimated by the spatial filter (spatial filtered reconstruction: SFR) are proposed. The spatial filter is designed to be used without prior or temporal information. The proposed spatial filter ensures that it concentrates the current distribution around the activated sources in the conductor. The current distribution estimated by the spatial filter is reconstructed by multiple linear regression. Redundant current dipoles are eliminated, and the current distribution is optimized in the sense of the Mallows Cp statistic. Numerical studies are demonstrated and show successful estimation by SFR in multiple-dipole cases. In single-dipole cases with SNRs of 101 and more, the location of the true dipole was successfully estimated for about 80% of the simulations. The reconstruction with multiple linear regression corrected the location of the maximum current density estimated by the proposed spatial filtering. The dipole on the correct position contributes to more than 70% of the total dipoles in the estimated current distribution in those cases. These results show that the current distribution is effectively localized by SFR. We also investigate the differences among SFR, the LCMV (linearly constrained minimum variance) beamformer and the SAM (synthetic aperture magnetometry), the representatives of spatial filters in MEG analyses. It is indicated that spatial resolution is improved by avoiding dependence on temporal information.

This paper proposes a technique for synthesizing speech with a desired speaking style and/or emotional expression, based on model adaptation in an HMM-based speech synthesis framework. Speaking styles and emotional expressions are characterized by many segmental and suprasegmental features in both spectral and prosodic features. Therefore, it is essential to take account of these features in the model adaptation. The proposed technique called style adaptation, deals with this issue. Firstly, the maximum likelihood linear regression (MLLR) algorithm, based on a framework of hidden semi-Markov model (HSMM) is presented to provide a mathematically rigorous and robust adaptation of state duration and to adapt both the spectral and prosodic features. Then, a novel tying method for the regression matrices of the MLLR algorithm is also presented to allow the incorporation of both the segmental and suprasegmental speech features into the style adaptation. The proposed tying method uses regression class trees with contextual information. From the results of several subjective tests, we show that these techniques can perform style adaptation while maintaining naturalness of the synthetic speech.