With the rapid development of Internet technology, the type and quantity of network traffic data have increased accordingly, and network traffic classification has become an important research task. In previous research, there are methods based on traditional machine learning and deep learning; compared to machine learning, deep learning can obtain good results by converting network traffic into two-dimensional images and utilizing deep learning classification models. However, all of these methods have some limitations: the trained models cannot learn sustainably, and the generalization ability of the models is limited. In order to solve this problem, we propose a network traffic classification methods based on incremental learning and Mixup, which is based on generative adversarial networks. First, the network traffic is converted into a 2D image, the original database is linearly interpolated using Mixup to reduce the overfitting tendency of the model and improve the generalization ability, and the traffic is classified using the ability of deep learning on the image. Secondly, we improve the traditional incremental learning algorithm. To effectively address the imbalance between old and new categories in incremental learning. The experimental results show that the model performs well in classification experiments, reaching 92.26% and 93.86% accuracy on the ISCXVPN2016 and USTC datasets, respectively, and we can maintain a high accuracy rate with limited storage space in the process of increasing new categories.

Quantization is commonly used to reduce the inference time of convolutional neural networks (CNNs). To reduce the inference time without drastically reducing accuracy, optimal bit widths need to be allocated for each layer or filter of the CNN. In conventional methods, the optimal bit allocation is obtained by using the gradient descent algorithm while minimizing the model size. However, the model size has little to no correlation with the inference time. In this paper, we present a computational-complexity metric called MAC×bit that is strongly correlated with the inference time of quantized CNNs. We propose a gradient descent-based regularization method that uses this metric for optimal bit allocation of a quantized CNN to improve the recognition accuracy and reduce the inference time. In experiments, the proposed method reduced the inference time of a quantized ResNet-18 model by 21.0% compared with the conventional regularization method based on model size while maintaining comparable recognition accuracy.

Federated Learning (FL) facilitates deep learning model training across distributed networks while ensuring data privacy. When deployed on edge devices, network pruning becomes essential due to the constraints of computational resources. However, traditional FL pruning methods face bias issues arising from the varied distribution of local data, which poses a significant challenge. To address this, we propose DDPruneFL, an innovative FL pruning framework that utilizes Discriminative Data (DD). Specifically, we utilize minimally pre-trained local models, allowing each client to extract semantic concepts as DD, which then inform an iterative pruning process. As a result, DDPruneFL significantly outperforms existing methods on four benchmark datasets, adeptly handling both IID and non-IID distributions and Client Selection scenarios. This model achieves state-of-the-art (SOTA) performance in this field. Moreover, our studies comprehensively validate the effectiveness of DD. Furthermore, a detailed computational complexity analysis focused on Floating-point Operations (FLOPs) is also conducted. The FLOPs analysis reveals that DDPruneFL significantly improves performance during inference while only marginally increasing training costs. Additionally, it exhibits a cost advantage in inference when compared to other pruning FL methods of the same type, further emphasizing its cost-effectiveness and practicality.

The relevant model based on convolutional neural networks (CNNs) has been proven to be an effective solution in speech enhancement algorithms. However, there needs to be more research on CNNs based on microphone arrays, especially in exploring the correlation between networks associated with different microphones. In this paper, we proposed a CNN-based feature integration network for speech enhancement in microphone arrays. The input of CNN is composed of short-time Fourier transform (STFT) from different microphones. CNN includes the encoding layer, decoding layer, and skip structure. In addition, the designed feature integration layer enables information exchange between different microphones, and the designed feature fusion layer integrates additional information. The experiment proved the superiority of the designed structure.

Accurate water level prediction systems improve safety and quality of life. This study introduces a method that uses clustering and deep learning of multisite data to enhance the water level prediction of the Three Gorges Dam. The results show that Cluster-GRU-based can provide accurate forecasts for up to seven days.

As one of electrical components in transmission lines, vibration damper plays a role in preventing the power lines dancing, and its recognition is an important task for intelligent inspection. However, due to the complex background interference in aerial images, current deep learning algorithms for vibration damper detection often lack accuracy and robustness. To achieve vibration damper detection more accurately, in this study, improved You Only Look Once (YOLO) model is proposed for performing damper detection. Firstly, a damper dataset containing 1900 samples with different scenarios was created. Secondly, the backbone network of YOLOv4 was improved by combining the Res2Net module and Dense blocks, reducing computational consumption and improving training speed. Then, an improved path aggregation network (PANet) structure was introduced in YOLOv4, combined with top-down and bottom-up feature fusion strategies to achieve feature enhancement. Finally, the proposed YOLO model and comparative model were trained and tested on the damper dataset. The experimental results and analysis indicate that the proposed model is more effective and robust than the comparative models. More importantly, the average precision (AP) of this model can reach 98.8%, which is 6.2% higher than that of original YOLOv4 model; and the prediction speed of this model is 62 frames per second (FPS), which is 5 FPS faster than that of YOLOv4 model.

Accurately simulating physical systems is essential in various fields. In recent years, deep learning has been used to automatically build models of such systems by learning from data. One such method is the neural ordinary differential equation (neural ODE), which treats the output of a neural network as the time derivative of the system states. However, while this and related methods have shown promise, their training strategies still require further development. Inspired by error analysis techniques in numerical analysis while replacing numerical errors with modeling errors, we propose the error-analytic strategy to address this issue. Therefore, our strategy can capture long-term errors and thus improve the accuracy of long-term predictions.

Ground penetrating radar (GPR) has the advantage of non-destructively and quickly inspecting internal structures such as voids and buried pipes under roads. However, it is necessary to estimate the internal structures from the GPR images. Recently, recognition and detection methods for GPR images using deep learning have been studied. This paper examines a data augmentation method using a cutout method necessary to estimate GPR images with deep learning accurately. We find that the cutout augmentation exhibits higher detection rates for all objects used in this study than a commonly used horizontal shift augmentation.

Using deep learning (DL) to achieve direction-of-arrival (DOA) estimation is an open and meaningful exploration. Existing DL-based methods achieve DOA estimation by spectrum regression or multi-label classification task. While, both of them face the problem of off-grid errors. In this paper, we proposed a cascaded deep neural network (DNN) framework named as off-grid network (OGNet) to provide accurate DOA estimation in the case of off-grid. The OGNet is composed of an autoencoder consisted by fully connected (FC) layers and a deep convolutional neural network (CNN) with 2-dimensional convolutional layers. In the proposed OGNet, the off-grid error is modeled into labels to achieve off-grid DOA estimation based on its sparsity. As compared to the state-of-the-art grid-based methods, the OGNet shows advantages in terms of precision and resolution. The effectiveness and superiority of the OGNet are demonstrated by extensive simulation experiments in different experimental conditions.

Based on the framework of a multi-stage key recovery attack for a large block cipher, 2 and 3-round differential-neural distinguishers were trained for AES using partial ciphertext bits. The study introduces the differential characteristics employed for the 2-round ciphertext pairs and explores the reasons behind the near 100% accuracy of the 2-round differential neural distinguisher. Utilizing the trained 2-round distinguisher, the 3-round subkey of AES is successfully recovered through a multi-stage key guessing. Additionally, a complexity analysis of the attack is provided, validating the effectiveness of the proposed method.

Single-image dehazing is a challenging task in computer vision research. Aiming at the limitations of traditional convolutional neural network representation capabilities and the high computational overhead of the self-attention mechanism in recent years, we proposed image attention and designed a single image dehazing network based on the image attention: IAD-Net. The proposed image attention is a plug-and-play module with the ability of global modeling. IAD-Net is a parallel network structure that combines the global modeling ability of image attention and the local modeling ability of convolution, so that the network can learn global and local features. The proposed network model has excellent feature learning ability and feature expression ability, has low computational overhead, and also improves the detail information of hazy images. Experiments verify the effectiveness of the image attention module and the competitiveness of IAD-Net with state-of-the-art methods.

To improve the recognition rate of the end-to-end modulation recognition method based on deep learning, a modulation recognition method of communication signals based on a cascade network is proposed, which is composed of two networks: Stacked Denoising Auto Encoder (SDAE) network and DCELDNN (Dilated Convolution, ECA Mechanism, Long Short-Term Memory, Deep Neural Networks) network. SDAE network is used to denoise the data, reconstruct the input data through encoding and decoding, and extract deep information from the data. DCELDNN network is constructed based on the CLDNN (Convolutional, Long Short-Term Memory, Fully Connected Deep Neural Networks) network. In the DCELDNN network, dilated convolution is used instead of normal convolution to enlarge the receptive field and extract signal features, the Efficient Channel Attention (ECA) mechanism is introduced to enhance the expression ability of the features, the feature vector information is integrated by a Global Average Pooling (GAP) layer, and signal features are extracted by the DCELDNN network efficiently. Finally, end-to-end classification recognition of communication signals is realized. The test results on the RadioML2018.01a dataset show that the average recognition accuracy of the proposed method reaches 63.1% at SNR of -10 to 15 dB, compared with CNN, LSTM, and CLDNN models, the recognition accuracy is improved by 25.8%, 12.3%, and 4.8% respectively at 10 dB SNR.

Software refactoring is an important process in software development. During software refactoring, code smell is a popular research topic that refers to design or implementation flaws in the software. Large class is one of the most concerning code smells in software refactoring. Detecting and refactoring such problem has a profound impact on software quality. In past years, software metrics and clustering techniques have commonly been used for the large class detection. However, deep-learning-based approaches have also received considerable attention in recent studies. In this study, we apply graph neural networks (GNNs), an important division of deep learning, to address the problem of large class detection. First, to support the extensive data requirements of the deep learning task, we apply a semiautomatic approach to generate a substantial number of data samples. Next, we design a new type of directed heterogeneous graph (DHG) as an input graph using the methods similarity matrix and software metrics. We construct an input graph for each class sample and make the graph classification with GNNs to identify the smelly classes. In our experiments, we apply three typical GNN model architectures for large class detection and compare the results with those of previous studies. The results show that the proposed approach can achieve more accurate and stable detection performance.

Medical imaging plays an indispensable role in precise patient diagnosis. The integration of deep learning into medical diagnostics is becoming increasingly common. However, existing deep learning models face performance and efficiency challenges, especially in resource-constrained scenarios. To overcome these challenges, we introduce a novel dendritic neural efficientnet model called DEN, inspired by the function of brain neurons, which efficiently extracts image features and enhances image classification performance. Assessments on a diabetic retinopathy fundus image dataset reveal DEN’s superior performance compared to EfficientNet and other classical neural network models.

Terahertz (THz) ultra-massive multiple-input multiple-output (UM-MIMO) is envisioned as a key enabling technology of 6G wireless communication. In UM-MIMO systems, downlink channel state information (CSI) has to be fed to the base station for beamforming. However, the feedback overhead becomes unacceptable because of the large antenna array. In this letter, the characteristic of CSI is explored from the perspective of data distribution. Based on this characteristic, a novel network named Attention-GRU Net (AGNet) is proposed for CSI feedback. Simulation results show that the proposed AGNet outperforms other advanced methods in the quality of CSI feedback in UM-MIMO systems.

The development of energy-efficient neural network hardware using magnetic tunnel junction (MTJ) devices has been widely investigated. One of the issues in the use of MTJ devices is large write energy. Since MTJ devices show stochastic behaviors, a large write current with enough time length is required to guarantee the certainty of the information held in MTJ devices. This paper demonstrates that quantized neural networks (QNNs) exhibit high tolerance to bit errors in weights and an output feature map. Since probabilistic switching errors in MTJ devices do not have always a serious effect on the performance of QNNs, large write energy is not required for reliable switching operations of MTJ devices. Based on the evaluation results, we achieve about 80% write-energy reduction on buffer memory compared to the conventional method. In addition, it is demonstrated that binary representation exhibits higher bit-error tolerance than the other data representations in the range of large error rates.

At Crypto 2019, Gohr first adopted the neural distinguisher for differential cryptanalysis, and since then, this work received increasing attention. However, most of the existing work focuses on improving and applying the neural distinguisher, the studies delving into the intrinsic principles of neural distinguishers are finite. At Eurocrypt 2021, Benamira et al. conducted a study on Gohr’s neural distinguisher. But for the neural distinguishers proposed later, such as the r-round neural distinguishers trained with k ciphertext pairs or ciphertext differences, denoted as NDcpk_r (Gohr’s neural distinguisher is the special NDcpk_r with K = 1) and NDcdk_r , such research is lacking. In this work, we devote ourselves to study the intrinsic principles and relationship between NDcdk_r and NDcpk_r. Firstly, we explore the working principle of NDcd1_r through a series of experiments and find that it strongly relies on the probability distribution of ciphertext differences. Its operational mechanism bears a strong resemblance to that of NDcp1_r given by Benamira et al.. Therefore, we further compare them from the perspective of differential cryptanalysis and sample features, demonstrating the superior performance of NDcp1_r can be attributed to the relationships between certain ciphertext bits, especially the significant bits. We then extend our investigation to NDcpk_r, and show that its ability to recognize samples heavily relies on the average differential probability of k ciphertext pairs and some relationships in the ciphertext itself, but the reliance between k ciphertext pairs is very weak. Finally, in light of the findings of our research, we introduce a strategy to enhance the accuracy of the neural distinguisher by using a fixed difference to generate the negative samples instead of the random one. Through the implementation of this approach, we manage to improve the accuracy of the neural distinguishers by approximately 2% to 8% for 7-round Speck32/64 and 9-round Simon32/64.

Currently, a significant portion of acoustic scene categorization (ASC) research is centered around utilizing Convolutional Neural Network (CNN) models. This preference is primarily due to CNN’s ability to effectively extract time-frequency information from audio recordings of scenes by employing spectrum data as input. The expression of many dimensions can be achieved by utilizing 2D spectrum characteristics. Nevertheless, the diverse interpretations of the same object’s existence in different positions on the spectrum map can be attributed to the discrepancies between spectrum properties and picture qualities. The lack of distinction between different aspects of input information in ASC-based CNN networks may result in a decline in system performance. Considering this, a feature pyramid segmentation (FPS) approach based on CNN is proposed. The proposed approach involves utilizing spectrum features as the input for the model. These features are split based on a preset scale, and each segment-level feature is then fed into the CNN network for learning. The SoftMax classifier will receive the output of all feature scales, and these high-level features will be fused and fed to it to categorize different scenarios. The experiment provides evidence to support the efficacy of the FPS strategy and its potential to enhance the performance of the ASC system.

Amodal Instance Segmentation (AIS) aims to segment the regions of both visible and invisible parts of overlapping objects. The mainstream Mask R-CNN-based methods are unsuitable for thin objects with large overlaps because of their object proposal features with bounding boxes for three reasons. First, capturing the entire shapes of overlapping thin objects is difficult. Second, the bounding boxes of close objects are almost identical. Third, a bounding box contains many objects in most cases. In this paper, we propose a box-free AIS method, Seed-to-Mask, for thin objects with large overlaps. The method specifies a target object using a seed and iteratively extends the segmented region. We have achieved better performance in experiments on artificial data consisting only of thin objects.

The purpose of Facial Beauty Prediction (FBP) is to automatically assess facial attractiveness based on human aesthetics. Most neural network-based prediction methods do not consider the ranking information in the task. For scoring tasks like facial beauty prediction, there is abundant ranking information both between images and within images. Reasonable utilization of these information during training can greatly improve the performance of the model. In this paper, we propose a novel end-to-end Convolutional Neural Network (CNN) model based on ranking information of images, incorporating a Rank Module and an Adaptive Weight Module. We also design pairwise ranking loss functions to fully leverage the ranking information of images. Considering training efficiency and model inference capability, we choose ResNet-50 as the backbone network. We conduct experiments on the SCUT-FBP5500 dataset and the results show that our model achieves a new state-of-the-art performance. Furthermore, ablation experiments show that our approach greatly contributes to improving the model performance. Finally, the Rank Module with the corresponding ranking loss is plug-and-play and can be extended to any CNN model and any task with ranking information. Code is available at https://github.com/nehcoah/Rank-Info-Net.