Noise cancelation and system identification have been studied for many years, and adaptive filters have proved to be a good means for solving such problems. Some neural networks can be treated as nonlinear adaptive filters, and are thus expected to be more powerful than traditional adaptive filters when dealing with nonlinear system problems. In this paper, two new heterogeneous recurrent neural network (HRNN) architectures will be proposed to identify some nonlinear systems and to extract a fetal electrocardiogram (ECG), which is corrupted by a much larger noise signal, Mother's ECG. The main difference between a heterogeneous recurrent neural network (HRNN) and a recurrent neural network (RNN) is that a complete neural network is used for the feedback path along with an error back-propagation (BP) neural network as the feedforward one. Different feedback neural networks can be used to provide different feedback capabilities. In this paper, a BP neural network is used as the feedback network in the architecture we proposed. And a self-organizing feature mapping (SOFM) network is used next as an alternative feedback network to form another heterogeneous recurrent neural network (HRNN). The heterogeneous recurrent neural networks (HRNN) successfully solve these two problems and prove their superiority to traditional adaptive filters and BP neural networks.