The existing target-dependent scalable image compression network can control the target of the compressed images between the human visual system and the deep learning based classification task. However, in its RNN based structure controls the bit-rate through the number of iterations, where each iteration generates a fixed size of the bit stream. Therefore, a large number of iterations are required at the high BPP, and fine-grained image quality control is not supported at the low BPP. In this paper, we propose a novel RNN-based image compression model that can schedule the channel size per iteration, to reduce the number of iterations at the high BPP and fine-grained bit-rate control at the low BPP. To further enhance the efficiency, multiple network models for various channel sizes are combined into a single model using the slimmable network architecture. The experimental results show that the proposed method achieves comparable performance to the existing method with finer BPP adjustment, increases parameters by only 0.15% and reduces the average amount of computation by 40.4%.

The neural network has been one of the most useful techniques in the area of speech recognition, language translation and image analysis in recent years. Long Short-Term Memory (LSTM), a popular type of recurrent neural networks (RNNs), has been widely implemented on CPUs and GPUs. However, those software implementations offer a poor parallelism while the existing hardware implementations lack in configurability. In order to make up for this gap, a highly configurable 7.62 GOP/s hardware implementation for LSTM is proposed in this paper. To achieve the goal, the work flow is carefully arranged to make the design compact and high-throughput; the structure is carefully organized to make the design configurable; the data buffering and compression strategy is carefully chosen to lower the bandwidth without increasing the complexity of structure; the data type, logistic sigmoid (σ) function and hyperbolic tangent (tanh) function is carefully optimized to balance the hardware cost and accuracy. This work achieves a performance of 7.62 GOP/s @ 238 MHz on XCZU6EG FPGA, which takes only 3K look-up table (LUT). Compared with the implementation on Intel Xeon E5-2620 CPU @ 2.10GHz, this work achieves about 90× speedup for small networks and 25× speed-up for large ones. The consumption of resources is also much less than that of the state-of-the-art works.

Most existing methods of effort estimations in software development are manual, labor-intensive and subjective, resulting in overestimation with bidding fail, and underestimation with money loss. This paper investigates effectiveness of sequence models on estimating development effort, in the form of man-months, from software project data. Four architectures; (1) Average word-vector with Multi-layer Perceptron (MLP), (2) Average word-vector with Support Vector Regression (SVR), (3) Gated Recurrent Unit (GRU) sequence model, and (4) Long short-term memory (LSTM) sequence model are compared in terms of man-months difference. The approach is evaluated using two datasets; ISEM (1,573 English software project descriptions) and ISBSG (9,100 software projects data), where the former is a raw text and the latter is a structured data table explained the characteristic of a software project. The LSTM sequence model achieves the lowest and the second lowest mean absolute errors, which are 0.705 and 14.077 man-months for ISEM and ISBSG datasets respectively. The MLP model achieves the lowest mean absolute errors which is 14.069 for ISBSG datasets.

We develop a deep neural network (DNN) for detecting driver drowsiness in videos. The proposed DNN model that receives driver's faces extracted from video frames as inputs consists of three components - a convolutional neural network (CNN), a convolutional control gate-based recurrent neural network (ConvCGRNN), and a voting layer. The CNN is to learn facial representations from global faces which are then fed to the ConvCGRNN to learn their temporal dependencies. The voting layer works like an ensemble of many sub-classifiers to predict drowsiness state. Experimental results on the NTHU-DDD dataset show that our model not only achieve a competitive accuracy of 84.81% without any post-processing but it can work in real-time with a high speed of about 100 fps.

Very few existing works about inertial sensor based air-writing focused on writing constraints' effects on recognition performance. We proposed a LSTM-based system and made several quantitative analyses under different constraints settings against CHMM, DTW-AP and CNN. The proposed system shows its advantages in accuracy, real-time performance and flexibility.

The-state-of-the-art neural quality estimation (QE) of machine translation model consists of two sub-networks that are tuned separately, a bidirectional recurrent neural network (RNN) encoder-decoder trained for neural machine translation, called the predictor, and an RNN trained for sentence-level QE tasks, called the estimator. We propose to combine the two sub-networks into a whole neural network, called the unified neural network. When training, the bidirectional RNN encoder-decoder are initialized and pre-trained with the bilingual parallel corpus, and then, the networks are trained jointly to minimize the mean absolute error over the QE training samples. Compared with the predictor and estimator approach, the use of a unified neural network helps to train the parameters of the neural networks that are more suitable for the QE task. Experimental results on the benchmark data set of the WMT17 sentence-level QE shared task show that the proposed unified neural network approach consistently outperforms the predictor and estimator approach and significantly outperforms the other baseline QE approaches.

This paper proposes a deep learning-based non-intrusive objective speech intelligibility estimation method based on recurrent neural network (RNN) with long short-term memory (LSTM) structure. Conventional non-intrusive estimation methods such as standard P.563 have poor estimation performance and lack of consistency, especially, in various noise and reverberation environments. The proposed method trains the LSTM RNN model parameters by utilizing the STOI that is the standard intrusive intelligibility estimation method with reference speech signal. The input and output of the LSTM RNN are the MFCC vector and the frame-wise STOI value, respectively. Experimental results show that the proposed objective intelligibility estimation method outperforms the conventional standard P.563 in various noisy and reverberant environments.

As advances in networking technology help to connect industrial control networks with the Internet, the threat from spammers, attackers and criminal enterprises has also grown accordingly. However, traditional Network Intrusion Detection System makes significant use of pattern matching to identify malicious behaviors and have bad performance on detecting zero-day exploits in which a new attack is employed. In this paper, a novel method of anomaly detection in industrial control network is proposed based on RNN-GBRBM feature decoder. The method employ network packets and extract high-quality features from raw features which is selected manually. A modified RNN-RBM is trained using the normal traffic in order to learn feature patterns of the normal network behaviors. Then the test traffic is analyzed against the learned normal feature pattern by using osPCA to measure the extent to which the test traffic resembles the learned feature pattern. Moreover, we design a semi-supervised incremental updating algorithm in order to improve the performance of the model continuously. Experiments show that our method is more efficient in anomaly detection than other traditional approaches for industrial control network.

Recurrent neural networks (RNNs) are a powerful model for sequential data. RNNs that use long short-term memory (LSTM) cells have proven effective in handwriting recognition, language modeling, speech recognition, and language comprehension tasks. In this study, we propose LSTM conditional random fields (LSTM-CRF); it is an LSTM-based RNN model that uses output-label dependencies with transition features and a CRF-like sequence-level objective function. We also propose variations to the LSTM-CRF model using a gate recurrent unit (GRU) and structurally constrained recurrent network (SCRN). Empirical results reveal that our proposed models attain state-of-the-art performance for named entity recognition.

GPS receivers are susceptible to jamming by interference. This paper proposes a recurrent neural network (RNN) predictor for new application in GPS anti-jamming systems. Five types of narrowband jammers, i. e. AR process, continuous wave interference (CWI), multi-tone CWI, swept CWI, and pulsed CWI, are considered in order to emulate realistic conditions. As the observation noise of received signals is highly non-Gaussian, an RNN estimator with a nonlinear structure is employed to accurately predict the narrowband signals based on a real-time learning method. The node decoupled extended Kalman filter (NDEKF) algorithm is adopted to achieve better performance in terms of convergence rate and quality of solution while requiring less computation time and memory. We analyze the computational complexity and memory requirements of the NDEKF approach and compare them to the global extended Kalman filter (GEKF) training paradigm. Simulation results show that our proposed scheme achieves a superior performance to conventional linear/nonlinear predictors in terms of SNR improvement and mean squared prediction error (MSPE) while providing inherent protection against a broad class of interference environments.

Recently, neural networks (NNs) have been extensively applied to many signal processing problem due to their robust abilities to form complex decision regions. In particular, neural networks add flexibility to the design of equalizers for digital communication systems. Recurrent neural network (RNN) is a kind of neural network with one or more feedback loops, whereas self-organizing map (SOM) is characterized by the formation of a topographic map of the input patterns in which the spatial locations (i.e., coordinates) of the neurons in the lattice are indicative of intrinsic statistical features contained in the input patterns. In this paper, we propose a novel receiver structure by combining adaptive RNN equalizer with a SOM detector under serious ISI and nonlinear distortion in QAM system. According to the theoretical analysis and computer simulation results, the performance of the proposed scheme is shown to be quite effective in channel equalization under nonlinear distortion.

A number of schemes have been proposed for communication using chaos over the past years. Regardless of the exact modulation method used, the transmitted signal must go through a physical channel which undesirably introduces distortion to the signal and adds noise to it. The problem is particularly serious when coherent-based demodulation is used because the necessary process of chaos synchronization is difficult to implement in practice. This paper addresses the channel distortion problem and proposes a technique for channel equalization in chaos-based communication systems. The proposed equalization is realized by a modified recurrent neural network (RNN) incorporating a specific training (equalizing) algorithm. Computer simulations are used to demonstrate the performance of the proposed equalizer in chaos-based communication systems. The Henon map and Chua's circuit are used to generate chaotic signals. It is shown that the proposed RNN-based equalizer outperforms conventional equalizers.

Conventional system models such as the finite impulse response (FIR) model, autoregressive external input (ARX) model, time delay neural network (TDNN), and recurrent neural network (RNN) depend on short-term memory when modeling a discrete time system. However, short-term memory can be inefficient with a varying appearance speed of I/O data. This inefficiency is referred to herein as the Varying Appearance Speed Problem (VASP) and demonstrated by analyzing impulse and frequency responses. Simulation results indicate that the varying appearance speed leads to asymmetrical cycles. Unable to prevent the memory effect from extensively disturbing the next output cycle, conventional models simulate the systems inaccurately. A solution using rate independent memory is then proposed. Only concerned with the previous extreme inputs, rate independent memory differs from short-term memory and potentially prevents a system model from the impact of varying appearance speeds. To demonstrate the VASP and verify the proposed model, this study conducts three experiments, i.e. (a) learning random step trajectories of circular and trefoil shapes, (b) modeling the relationship between the economic leading and coincident indexes, (c) simulating the connection between the ground-water level and land subsidence. In contrast to conventional models, the model presented here performs better in terms of mean square errors.