This paper delves into the utilisation of the subgradient method with geometrically decaying stepsize for Bilinear Form Identification. We introduce the iterative Wiener Filter, an l2 regression method, and highlight its limitations when confronted with noise, particularly heavy-tailed noise. To address these challenges, the paper suggests employing the l1 regression method with a subgradient method utilizing a geometrically decaying step size. The effectiveness of this approach is compared to existing methods, including the ALS algorithem. The study demonstrates that the l1 algorithm, especially when paired with the proposed subgradient method, excels in stability and accuracy under conditions of heavy-tailed noise. Additionally, the paper introduces the standard rounding procedure and the S-outlier bound as relaxations of traditional assumptions. Numerical experiments provide support and validation for the presented results.

Wafer-level performance prediction techniques have been increasingly gaining attention in production LSI testing due to their ability to reduce measurement costs without compromising test quality. Despite the availability of several efficient methods, the site-to-site variation commonly observed in multi-site testing for radio frequency circuits remains inadequately addressed. In this manuscript, we propose a wafer-level performance prediction approach for multi-site testing that takes into account the site-to-site variation. Our proposed method is built on the Gaussian process, a widely utilized wafer-level spatial correlation modeling technique, and enhances prediction accuracy by extending hierarchical modeling to leverage the test site information test engineers provide. Additionally, we propose a test-site sampling method that maximizes cost reduction while maintaining sufficient estimation accuracy. Our experimental results, which employ industrial production test data, demonstrate that our proposed method can decrease the estimation error to 1/19 of that a conventional method achieves. Furthermore, our sampling method can reduce the required measurements by 97% while ensuring satisfactory estimation accuracy.

Melanin, which is responsible for the appearance of spots and freckles, is an important indicator in evaluating skin condition. To assess the efficacy of cosmetics, skin condition scoring is performed by analyzing the distribution and amount of melanin from microscopic images of the stratum corneum cells. However, the current practice of diagnosing skin condition using stratum corneum cells images relies heavily on visual evaluation by experts. The goal of this study is to develop a quantitative evaluation system for skin condition based on melanin within unstained stratum corneum cells images. The proposed system utilizes principal component regression to perform five-level scoring, which is then compared with visual evaluation scores to assess the system’s usefulness. Additionally, we evaluated the impact of indicators related to melanin obtained from images on the scores, and verified which indicators are effective for evaluation. In conclusion, we confirmed that scoring is possible with an accuracy of more than 60% on a combination of several indicators, which is comparable to the accuracy of visual assessment.

Software development companies must consider employees' job satisfaction and turnover intentions. To explain the related factors, this study focused on future perspective index (FPI). FPI was assumed to relate positively to satisfaction and negatively to turnover. In the analysis, we compared the FPI with existing factors that are considered to be related to job satisfaction. We discovered that the FPI was promising for enhancing explanatory power, particularly when analyzing satisfaction.

Following the formulation of Support Vector Regression (SVR), we consider a regression analogue of soft margin optimization over the feature space indexed by a hypothesis class H. More specifically, the problem is to find a linear model w ∈ ℝH that minimizes the sum of ρ-insensitive losses over all training data for as small ρ as posssible, where the ρ-insensitive loss for a single data (xi, yi) is defined as max{|yi - ∑h whh(xi)| - ρ, 0}. Intuitively, the parameter ρ and the ρ-insensitive loss are defined analogously to the target margin and the hinge loss in soft margin optimization, respectively. The difference of our formulation from SVR is two-fold: (1) we consider L1-norm regularization instead of L2-norm regularization, and (2) the feature space is implicitly defined by a hypothesis class instead of a kernel. We propose a boosting-type algorithm for solving the problem with a theoretically guaranteed convergence rate under a natural assumption on the weak learnability.

Statistical wafer-level characteristic variation modeling offers an attractive method for reducing the measurement cost in large-scale integrated (LSI) circuit testing while maintaining test quality. In this method, the performance of unmeasured LSI circuits fabricated on a wafer is statistically predicted based on a few measured LSI circuits. Conventional statistical methods model spatially smooth variations in the wafers. However, actual wafers can exhibit discontinuous variations that are systematically caused by the manufacturing environment, such as shot dependence. In this paper, we propose a modeling method that considers discontinuous variations in wafer characteristics by applying the knowledge of manufacturing engineers to a model estimated using Gaussian process regression. In the proposed method, the process variation is decomposed into systematic discontinuous and global components to improve estimation accuracy. An evaluation performed using an industrial production test dataset indicates that the proposed method effectively reduces the estimation error for an entire wafer by over 36% compared with conventional methods.

With the emergence of a large quantity of data in science and industry, it is urgent to improve the prediction accuracy and reduce the high complexity of Gaussian process regression (GPR). However, the traditional global approximation and local approximation have corresponding shortcomings, such as global approximation tends to ignore local features, and local approximation has the problem of over-fitting. In order to solve these problems, a large-scale Gaussian process regression algorithm (RFFLT) combining random Fourier features (RFF) and local approximation is proposed. 1) In order to speed up the training time, we use the random Fourier feature map input data mapped to the random low-dimensional feature space for processing. The main innovation of the algorithm is to design features by using existing fast linear processing methods, so that the inner product of the transformed data is approximately equal to the inner product in the feature space of the shift invariant kernel specified by the user. 2) The generalized robust Bayesian committee machine (GRBCM) based on Tsallis mutual information method is used in local approximation, which enhances the flexibility of the model and generates a sparse representation of the expert weight distribution compared with previous work. The algorithm RFFLT was tested on six real data sets, which greatly shortened the time of regression prediction and improved the prediction accuracy.

In machine learning, data augmentation (DA) is a technique for improving the generalization performance of models. In this paper, we mainly consider gradient descent of linear regression under DA using noisy copies of datasets, in which noise is injected into inputs. We analyze the situation where noisy copies are newly generated and injected into inputs at each epoch, i.e., the case of using on-line noisy copies. Therefore, this article can also be viewed as an analysis on a method using noise injection into a training process by DA. We considered the training process under three training situations which are the full-batch training under the sum of squared errors, and full-batch and mini-batch training under the mean squared error. We showed that, in all cases, training for DA with on-line copies is approximately equivalent to the l2 regularization training for which variance of injected noise is important, whereas the number of copies is not. Moreover, we showed that DA with on-line copies apparently leads to an increase of learning rate in full-batch condition under the sum of squared errors and the mini-batch condition under the mean squared error. The apparent increase in learning rate and regularization effect can be attributed to the original input and additive noise in noisy copies, respectively. These results are confirmed in a numerical experiment in which we found that our result can be applied to usual off-line DA in an under-parameterization scenario and can not in an over-parametrization scenario. Moreover, we experimentally investigated the training process of neural networks under DA with off-line noisy copies and found that our analysis on linear regression can be qualitatively applied to neural networks.

Backscatter communication is an emerging wireless access technology to realize ultra-low power terminals exploiting the modulated reflection of incident radio wave. This paper proposes a method to measure the phase angle of backscatter link using principal component analysis (PCA). The phase angle measurement of backscatter link at the receiver is essential to maximize the signal quality for subsequent demodulation and to measure the distance and the angle of arrival. The drawback of popular phase angle measurement with naive phase averaging and linear regression analysis is to produce erroneous phase angle, where the phase angle is close to $pmrac{pi}{2}$ radian and the signal quality is poor. The advantage of the proposal is quantified with a computer simulation, a conducted experiment and radio propagation experiments.

As a key technology of 5G, NFV has attracted much attention. In addition, monitoring plays an important role, and can be widely used for virtual network function placement and resource optimisation. The existing monitoring methods focus on the monitoring load without considering they own resources needed. This raises a unique challenge: jointly optimising the NFV monitoring systems and minimising their monitoring load at runtime. The objective is to enhance the gain in real-time monitoring metrics at minimum monitoring costs. In this context, we propose a novel NFV monitoring solution, namely, iMon (Monitoring by inferring), that jointly optimises the monitoring process and reduces resource consumption. We formalise the monitoring process into a multitarget regression problem and propose three regression models. These models are implemented by a deep neural network, and an experimental platform is built to prove their availability and effectiveness. Finally, experiments also show that monitoring resource requirements are reduced, and the monitoring load is just 0.6% of that of the monitoring tool cAdvisor on our dataset.

Aiming at the problem of the deterioration of the contact performance caused by the wear debris generated during the fretting wear of the electrical connector, low-frequency fretting wear experiments were carried out on the contacts of electrical connectors, the accumulation and distribution of the wear debris were detected by the electrical capacitance tomography technology; the influence of fretting cycles, vibration direction, vibration frequency and vibration amplitude on the accumulation and distribution of wear debris were analyzed; the correlation between characteristic value of wear debris and contact resistance value was studied, and a performance degradation model based on the accumulation and distribution of wear debris was built. The results show that fretting wear and performance degradation are the most serious in axial vibration; the characteristic value of wear debris and contact resistance are positively correlated with the fretting cycles, vibration frequency and vibration amplitude; there is a strong correlation between the sum of characteristic value of wear debris and the contact resistance value; the prediction error of ABC-SVR model of fretting wear performance degradation of electrical connectors constructed by the characteristic value of wear debris is less than 6%. Therefore, the characteristic value of wear debris in contact subareas can quantitatively describe the degree of fretting wear and the process of performance degradation.

Ordinal regression is used to classify instances by considering ordinal relation between labels. Existing methods tend to decrease the accuracy when they adhere to the preservation of the ordinal relation. Therefore, we propose a distributional knowledge-based network (DK-net) that considers ordinal relation while maintaining high accuracy. DK-net focuses on image datasets. However, in industrial applications, one can find not only image data but also tabular data. In this study, we propose DK-neural oblivious decision ensemble (NODE), an improved version of DK-net for tabular data. DK-NODE uses NODE for feature extraction. In addition, we propose a method for adjusting the parameter that controls the degree of compliance with the ordinal relation. We experimented with three datasets: WineQuality, Abalone, and Eucalyptus dataset. The experiments showed that the proposed method achieved high accuracy and small MAE on three datasets. Notably, the proposed method had the smallest average MAE on all datasets.

Soil moisture sensor calibration based on the Multivariate Adaptive Regression Splines (MARSplines) model is studied in this paper. Different from the generic polynomial fitting methods, the MARSplines model is a non-parametric model, and it is able to model the complex relationship between the actual and measured soil moisture. Rao-1 algorithm is employed to tune the hyper-parameters of the calibration model and thus the performance of the proposed method is further improved. Data collected from four commercial soil moisture sensors is utilized to verify the effectiveness of the proposed method. To assess the calibration performance, the proposed model is compared with the model without using the temperature information. The numeric studies prove that it is promising to apply the proposed model for real applications.

In this article, a GUI system is proposed to support clinical cardiology examinations. The proposed system estimates “pulmonary artery wedge pressure” based on patients' chest radiographs using an explainable regression-based convolutional neural network. The GUI system was validated by performing an effectiveness survey with 23 cardiology physicians with medical licenses. The results indicated that many physicians considered the GUI system to be effective.

The mean, median, and mode are usually calculated from univariate observations as the most basic representative values of a random variable. To measure the spread of the distribution, the standard deviation, interquartile range, and modal interval are also calculated. When we analyze continuous relations between a pair of random variables from bivariate observations, regression analysis is often used. By minimizing appropriate costs evaluating regression errors, we estimate the conditional mean, median, and mode. The conditional standard deviation can be estimated if the bivariate observations are obtained from a Gaussian process. Moreover, the conditional interquartile range can be calculated for various distributions by the quantile regression that estimates any conditional quantile (percentile). Meanwhile, the study of the modal interval regression is relatively new, and spline regression models, known as flexible models having the optimality on the smoothness for bivariate data, are not yet used. In this paper, we propose a modal interval regression method based on spline quantile regression. The proposed method consists of two steps. In the first step, we divide the bivariate observations into bins for one random variable, then detect the modal interval for the other random variable as the lower and upper quantiles in each bin. In the second step, we estimate the conditional modal interval by constructing both lower and upper quantile curves as spline functions. By using the spline quantile regression, the proposed method is widely applicable to various distributions and formulated as a convex optimization problem on the coefficient vectors of the lower and upper spline functions. Extensive experiments, including settings of the bin width, the smoothing parameter and weights in the cost function, show the effectiveness of the proposed modal interval regression in terms of accuracy and visual shape for synthetic data generated from various distributions. Experiments for real-world meteorological data also demonstrate a good performance of the proposed method.

The colors of objects in natural images are affected by the color of lighting, and accurately estimating an illuminant's color is indispensable in analyzing scenes lit by colored lightings. Recent lighting environments enhance colorfulness due to the spread of light-emitting diode (LED) lightings whose colors are flexibly controlled in a full visible spectrum. However, existing color estimations mainly focus on the single illuminant of normal color ranges. The estimation of multiple illuminants of unusual color settings, such as blue or red of high chroma, has not been studied yet. Therefore, new color estimations should be developed for multiple illuminants of various colors. In this article, we propose a color estimation for LED lightings using Color Line features, which regards the color distribution as a straight line in a local area. This local estimate is suitable for estimating various colors of multiple illuminants. The features are sampled at many small regions in an image and aggregated to estimate a few global colors using supervised learning with a convolutional neural network. We demonstrate the higher accuracy of our method over existing ones for such colorful lighting environments by producing the image dataset lit by multiple LED lightings in a full-color range.

Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.

In this letter, we propose a novel transferable sparse regression (TSR) method, for cross-database facial expression recognition (FER). In TSR, we firstly present a novel regression function to regress the data into a latent representation space instead of a strict binary label space. To further alleviate the influence of outliers and overfitting, we impose a row sparsity constraint on the regression term. And a pairwise relation term is introduced to guide the feature transfer learning. Secondly, we design a global graph to transfer knowledge, which can well preserve the cross-database manifold structure. Moreover, we introduce a low-rank constraint on the graph regularization term to uncover additional structural information. Finally, several experiments are conducted on three popular facial expression databases, and the results validate that the proposed TSR method is superior to other non-deep and deep transfer learning methods.

The solution of the ordinary kernel ridge regression, based on the squared loss function and the squared norm-based regularizer, can be easily interpreted as a stochastic linear estimator by considering the autocorrelation prior for an unknown true function. As is well known, a stochastic affine estimator is one of the simplest extensions of the stochastic linear estimator. However, its corresponding kernel regression problem is not revealed so far. In this paper, we give a formulation of the kernel regression problem, whose solution is reduced to a stochastic affine estimator, and also give interpretations of the formulation.

Recently the data-driven learning of dynamic systems has become a promising approach because no physical knowledge is needed. Pure machine learning approaches such as Gaussian process regression (GPR) learns a dynamic model from data, with all physical knowledge about the system discarded. This goes from one extreme, namely methods based on optimizing parametric physical models derived from physical laws, to the other. GPR has high flexibility and is able to model any dynamics as long as they are locally smooth, but can not generalize well to unexplored areas with little or no training data. The analytic physical model derived under assumptions is an abstract approximation of the true system, but has global generalization ability. Hence the optimal learning strategy is to combine GPR with the analytic physical model. This paper proposes a method to learn dynamic systems using GPR with analytic ordinary differential equations (ODEs) as prior information. The one-time-step integration of analytic ODEs is used as the mean function of the Gaussian process prior. The total parameters to be trained include physical parameters of analytic ODEs and parameters of GPR. A novel method is proposed to simultaneously learn all parameters, which is realized by the fully Bayesian GPR and more promising to learn an optimal model. The standard Gaussian process regression, the ODE method and the existing method in the literature are chosen as baselines to verify the benefit of the proposed method. The predictive performance is evaluated by both one-time-step prediction and long-term prediction. By simulation of the cart-pole system, it is demonstrated that the proposed method has better predictive performances.